Data Retention

This page covers Snowflake Native App response retention. For Docker deployments, see Self-hosted security.

Request processing

Text and document bytes passed to mask() are processed on the app compute node for that request. Agent Mask does not create a separate durable input table from the raw request payload.

Request processing is separate from response retention. The response returned by mask() can include de-identified text, extracted document text, detected source values, redacted document bytes, and review metadata depending on request parameters. Temporary response records store that returned payload for long-running and multi-batch requests, so they can contain sensitive content even when the original request payload is not stored as a separate input record.

Temporary response records

To support long-running requests and multi-batch responses, the app stores temporary response records in an app-managed table. Each record contains the compressed response payload that mask() returns to the caller.

Default retention: 24 hours. Records older than the retention window are purged by an hourly cleanup task.

Configure retention

-- Set to 48 hours
CALL app_public.set_retention(48);

-- Check current value
CALL app_public.get_retention();

Retention is enforced by the cleanup task running on the app’s warehouse. The task is created automatically during install and runs every 60 minutes.

Trade-offs

Retention	Best for
Short (1–4h)	Interactive usage, shorter persistence window.
Default (24h)	Mixed usage — enables retry / result retrieval during a work day.
Long (48–168h)	Long-running ETL pipelines where results are fetched asynchronously.

Lower retention keeps response records for less time. Higher retention gives callers more time to retrieve asynchronous or retried results, but leaves response payloads in storage longer.

Minimize sensitive response content

Treat temporary response records as sensitive. The stored response payload is the same JSON returned to the caller, so it can include:

columns[].deidentified_text — de-identified output. With the keep operator, or when text is not detected as an entity, this can contain source text.
columns[].original_text — extracted source text for document inputs.
entity_ledger[].original and entity_ledger[].canonical_original — detected source values when the entity ledger is enabled.
columns[].analyzer_results — span and score metadata for detections.
columns[].redacted_document, columns[].document_metadata, and columns[].redaction_metadata when requested or applicable.

Response metadata can also be sensitive because it connects processing activity to Snowflake queries, batches, and timestamps.

Use the shortest retention window your workflow can tolerate. When you do not need review evidence, set include_entity_ledger=false and include_analyzer_results=false. Avoid keep for workflows where temporary response storage should not contain unmodified sensitive values.

Clean up task

Check status

SHOW TASKS IN APPLICATION agent_mask_en;

Manually trigger purge

Wait for the next hourly run, or call start_cleanup_task() to recreate and resume the task if it was stopped.

Pause / resume

CALL app_public.stop_cleanup_task();
CALL app_public.start_cleanup_task();

Stopping the task means expired records accumulate. Restart it to resume hourly purging.

What’s in retained response records

Agent Mask stores:

query_id, batch_id, source — result lookup keys.
status — request state.
response — the compressed, base64-encoded response payload returned to the caller. This may include de-identified text, extracted document text, detected original values in the entity ledger, redacted files, and review metadata depending on request parameters.
insert_time, completed_time — used by the retention cleanup.

Retention applies to these app-managed response records. It does not control copies you write to your own tables, saved query results, application logs, client logs, downstream systems, or other Snowflake-managed history outside Agent Mask’s app tables.

If your Snowflake query history or worksheet history is in scope for your data-handling policy, avoid embedding raw sensitive values directly in SQL text. Pass values from governed tables, stages, or application parameters according to your Snowflake controls.

Teardown

When you run app_public.teardown_app(), app-managed response records are dropped along with the rest of the service resources.