Import customer list without creating duplicates

I am preparing to import approximately 12,000 customer records from our legacy CRM into FieldPulse. I have reviewed the bulk import documentation and have a concern regarding duplicate prevention.

Our legacy system contains multiple entries for the same customer due to historical data quality issues. I have confirmed that:

  1. Approximately 2,300 records share phone numbers with at least one other record
  2. Approximately 1,800 records share email addresses with at least one other record
  3. Approximately 890 records have matching first name, last name, and street address but different contact methods

My questions are as follows:

  1. What match key(s) does the FieldPulse import process use to identify potential duplicates?
  2. Is there a configurable merge rule set, or is deduplication handled automatically according to a fixed hierarchy?
  3. Can I specify which record should be considered "master" when duplicates are detected (e.g., most recent updated date, most complete data)?
  4. Does the import provide a preview or dry-run option showing which records would be merged versus created as new?
  5. Are there any constraints on which fields can be used as custom match keys?

I have noted that the import template includes columns for "Customer ID (External)" — it is not clear whether this field participates in deduplication logic or is strictly for reference.

Prior to executing this import, I require a reproducible process that does not create duplicate customer records, as remediation would be operationally costly.

Environment details: I will be using the web interface bulk import function, not the API, for this operation. Our account is on the Enterprise tier.

  • Hey Anita — good questions, and yeah, 12K records with that much duplication potential definitely warrants a careful approach.

    Short answer: the web import uses a fixed match key hierarchy, not configurable rules. Here's the actual order it checks:

    1. Email address (exact match, case-insensitive)
    2. Phone number (normalized, so "(555) 123-4567" and "5551234567" match)
    3. First name + Last name + Street address (all three must match)

    The "Customer ID (External)" field is reference only — it does not participate in matching. Useful for your own tracking, not for dedup logic.

    There's no native dry-run in the web UI, but here's what I do:

    // Quick script to preview what would merge
    // Run this on your CSV before import
    
    const seen = new Map();
    const duplicates = [];
    
    records.forEach(r => {
      const key = r.email?.toLowerCase() || 
                  r.phone?.replace(/\D/g, '') ||
                  `${r.first_name}|${r.last_name}|${r.street}`.toLowerCase();
      
      if (seen.has(key)) {
        duplicates.push({
          row: r.row_number,
          conflicts_with: seen.get(key),
          matched_on: r.email ? 'email' : r.phone ? 'phone' : 'name+address'
        });
      } else {
        seen.set(key, r.row_number);
      }
    });

    When duplicates are found, the first encountered record in the CSV wins and becomes the master. So sort your export by "last updated" descending if you want newer data to take precedence.

    The actual merge behavior preserves the most complete individual fields (non-empty values from both records), not a strict winner-take-all. Phone numbers and emails accumulate rather than replace.

    If you need deterministic master selection (e.g., always prefer the record with more populated fields), you'd need to pre-process externally or use the API with explicit update logic. The web import doesn't expose that level of control.

    Want me to share a Python script for that pre-merge analysis? Happy to adapt it for your specific priority rules.

  • +1 to Eli's hierarchy — ran into this exact scenario last quarter with ~8K records.

    Heads up on the phone normalization: it strips all non-digits, so extensions get borked. "555-123-4567 ext 204" becomes "5551234567204" and might collide weirdly. We pre-cleaned extensions into a separate custom field.

    Also worth noting: the "first in CSV wins" rule applies within a single import batch. If you're doing this in chunks (which I'd recommend at 12K), the match logic does not look across previously imported batches. So split your CSV by some natural grouping and sort each chunk carefully.

    YMMV on the field accumulation — we saw some inconsistent behavior with custom fields where the second record's empty values overwrote the first record's populated custom fields. Couldn't reproduce reliably though, so might have been user error on our end.

  • Thank you, Eli and Devon. The extension stripping behavior is noted — I will sanitize phone numbers accordingly.

    Regarding custom field accumulation: I have confirmed in a test environment with 50 records that empty values in the second record do not overwrite populated values in the first. However, I cannot guarantee this behavior at scale.

    I have one follow-up question: Is there any mechanism to receive a report post-import indicating which records were merged versus created? The Audit Log does not appear to capture import-level operations, only individual record changes.

  • The import process does not generate a merge report. Recommended approach: export your customer list immediately before import, then compare post-import using the "Created At" timestamp and "Updated At" values. Records with identical timestamps were created during import; those with older "Created At" but recent "Updated At" were likely merged.

    For Enterprise accounts, you may also request a data operation summary from your Customer Success Manager within 48 hours of import completion. This is not self-service.

  • I have completed the import using the following process:

    1. Pre-processed CSV to normalize phone numbers (removed extensions, stored separately)
    2. Sorted records by "Last Modified Date" descending within each logical group
    3. Executed import in four batches of approximately 3,000 records
    4. Captured pre-import customer export for comparison

    Result: 11,847 customer records created, 153 merges identified via timestamp comparison. This aligns with my deduplication estimate.

    One anomaly observed: three records with identical email addresses were not merged despite appearing in the same batch. I have confirmed the email addresses match character-for-character. Ticket #FP-28473 opened with support.

    Process documented for future imports. Thank you for the guidance.