Identity Resolution

By default, each distinct surface form gets its own placeholder — "Sarah Chen", "Chen", and "Dr. Chen" each become [PERSON_1], [PERSON_2], [PERSON_3]. Often you want them to share one placeholder, because they all refer to the same person. entity_collapse turns on that grouping.

Quick example

SELECT app_public.mask(
  discharge_summary,
  OBJECT_CONSTRUCT('entity_collapse', 0.5)
) AS r
FROM app_public.sample_records
WHERE record_id = 2;

Without collapse:

[PERSON_1] was admitted. [PERSON_2] reviewed the chart. [PERSON_3] signed off.

With entity_collapse: 0.5 when all three refer to the same clinician:

[PERSON_1] was admitted. [PERSON_1] reviewed the chart. [PERSON_1] signed off.

When to enable it

• Long-form text where the same person, place, or organization is mentioned multiple ways: clinical notes, incident reports, legal filings, discharge summaries.
• Analytics that need a stable count of unique people post-masking — without collapse, variants inflate the cardinality.
• Downstream consumers that expect one placeholder per real-world entity.

When to leave it off

• Short, single-sentence inputs where there’s nothing to collapse. The default -1 (off) is faster.
• Highly structured records where every row is already one person and variants don’t occur.
• Latency-sensitive workloads. Collapse runs AI-based resolution for unstructured types, which adds time per row.

The aggressiveness scale

Value	Behavior
-1 (default)	Disabled. Each unique surface form gets its own placeholder.
0.0	Enabled, most conservative. Only merges near-certain matches (e.g., "Sarah Chen" + "Dr. Sarah Chen").
0.5	Balanced. Merges clear variants ("Sarah Chen" + "Chen" + "Dr. Chen") when context supports it.
1.0	Most aggressive. Merges looser references ("Sarah" + "Ms. Chen" + "the clinician") — more likely to group variants, with higher risk of merging distinct people.

Start at 0.5 and adjust: bump up if variants aren’t collapsing; bump down if distinct people are merging.

Identity resolution is probabilistic: ambiguous references can stay split, and distinct entities can merge if the setting is too aggressive.

Which types collapse

Collapse applies to types where variant mentions are common:

• Unstructured types — PERSON, LOCATION, ORGANIZATION. These use AI-based resolution that reads surrounding context.
• Structured types — PHONE_NUMBER, DATE_TIME, CREDIT_CARD, US_SSN, IBAN_CODE, MEDICARE_ID, EMAIL_ADDRESS. These use deterministic normalization (e.g., (415) 555-0100 and +1-415-555-0100 collapse to the same placeholder regardless of the setting — this is always on for structured types).

Pattern-only types without a natural standard form (e.g., URL, IP_ADDRESS) are unaffected by entity_collapse.

Interaction with operators

Different operators show collapsed groups differently. Within a collapsed group:

• type_numbered — all variants share one placeholder ([PERSON_1]).
• faker — all variants get the same synthetic name. "Sarah Chen", "Chen", and "Dr. Chen" might all become "Patricia Johnson" across the text.
• hash — formatting variants of the same detected value are normalized before hashing. Set operator_params.entity_collapse to true to make aliases in a collapsed group hash to the same digest.
• encrypt — formatting variants of the same detected value are normalized before encryption. Set operator_params.entity_collapse to true to make aliases in a collapsed group encrypt to the same ciphertext.
• constant — every variant is replaced with the configured value; collapse has no visible effect.

Scope

Collapse operates per row of a mask() call — variants inside one document or one text cell collapse together. It does not carry state across rows or across separate mask() calls; two different records each get their own collapsed groups.

The same per-row scoping applies to replacements: faker values are chosen per row, so the bare name "John" in two different rows can receive two different fakes because they may be two different people. Within one request, Agent Mask avoids assigning the same faker output to two different detected identities. For continuity across separate requests, use ledger_seed.

Reading the ledger after collapse

When the entity ledger is included, merged groups are visible in entity_ledger: alias entries carry a canonical_original field pointing at the group’s canonical value, and share the canonical’s placeholder and replacement verbatim.

"entity_ledger": [
  {"entity_type": "PERSON", "original": "John Smith",
   "placeholder": "[REDACTED_PERSON_1]", "replacement": "Marcus Webb", "seeded": true},
  {"entity_type": "PERSON", "original": "John",
   "placeholder": "[REDACTED_PERSON_1]", "replacement": "Marcus Webb",
   "canonical_original": "John Smith"}
]

To map a replacement back to the original value, use an exact lookup: replacement → canonical_original (falling back to original for unmerged entries). You do not need to parse names or split strings.

Seeding across requests (ledger_seed)

To carry identity bindings across requests, such as chunked documents, pass the previous response’s canonical entries back via ledger_seed.

Seeded entries affect output in these ways:

• Seeded placeholders stay stable — existing seeded placeholders are not renumbered; new detections number above them.
• Seeded replacements are reserved — later faker draws in that request do not reuse them for a different identity.
• Seeded ledger entries are marked when used — entries whose binding came from the seed carry "seeded": true in the emitted ledger when detected. Seeded entries that never appear in the row are omitted from the response; because you already hold the seed, you can remove unused entries by comparing your seed to the response.
• In-row context can override an ambiguous seed — if collapse links a seeded value to a different in-row identity, the row returns a structured warning instead of silently changing the seeded binding:

"warnings": [
  {"code": "SEED_REMAPPED_BY_COLLAPSE", "entity_type": "PERSON", "original": "John",
   "seeded_placeholder": "[REDACTED_PERSON_7]", "new_placeholder": "[REDACTED_PERSON_2]",
   "canonical_original": "John Looper"}
]

Seed only strong identifiers — multi-token names, emails, IDs — not bare given names. A seeded bare "John" can point to the wrong person when a later row mentions a different John. Use the warning to review those cases, and prefer seeds that are specific enough to identify one real-world entity.

Language support

Collapse works across all supported languages, though AI-based resolution for PERSON / LOCATION / ORGANIZATION is most accurate for English. For non-English text, consider starting at a lower aggressiveness (0.3) and tuning up.

Tuning checklist

1. Start at 0.5.
2. If variants of the same entity still get different placeholders → bump up (0.7, 0.9).
3. If two different real-world entities merge into one → bump down (0.3, 0.1).
4. If you see latency pressure on large batches → leave off for short rows; enable only on long-form fields.

See also

• type_numbered — placeholders that benefit most from collapse.
• faker — synthetic-name consistency across variants.
• Request Parameters › entity_collapse