Administration → Data Retention → /data-retention
Some tables grow forever. Delivery logs, webhook events, poll failures, meter readings and audit streams accumulate at a rate set by the size of the portfolio, not by anything a user does. Data retention is the engine that ages those rows out on a policy, so storage stays bounded and queries stay fast.
It is deliberately hard to arm. Two surfaces:
| Surface | What it is |
|---|---|
| Retention Policies | What ages out of which table, after how long, and whether the policy is armed. |
| Retention Runs | Every sweep, with what it removed — or in shadow mode, what it would have removed. |
The safety model
There is no arm/disarm button in the app, and this is intentional. Policies are authored and armed by migration or explicit SQL, where the change leaves a reviewable trail. A browser toggle would collapse a three-switch safety model into one click.
Reading a policy
| Field | Meaning |
|---|---|
| Policy key | The policy's stable identifier. |
| MySQL table | The table it ages. |
| Mode | Shadow or armed — the effective mode, after all three switches are resolved. |
| Evaluated by the sweeper | Whether the sweeper considers this policy at all. |
| Strategy | Delete rows removes whole rows; Blank columns keeps the row and nulls named columns. |
| Keep (days) | Rows older than this are in scope. |
| Aged on column | The date column the age is measured from. |
| Columns blanked | For the blank-columns strategy, which columns are cleared. |
| Extra filter | An additional predicate narrowing what is in scope. |
| Rows per statement | Chunk size — how many rows one statement touches. |
| Max statements per sweep | Ceiling on work per run, so a sweep cannot monopolise the database. |
| Swept / Result | When the policy last ran and how it ended. |
Blank columns exists for tables where the row itself is still needed — a record of the event — but the payload inside it is not. It shrinks the table without breaking references.
Reading a run
Each sweep produces a run record per policy.
| Field | Meaning |
|---|---|
| Mode | Shadow or armed for this run. |
| Result | Succeeded, failed, or refused. |
| Rows in scope | What matched the policy. In shadow mode this is what would have been removed. |
| Rows removed | What was actually removed. Always 0 in shadow mode. |
| Chunks executed | How many statements ran. |
| Backlog remaining | The chunk ceiling was hit before the backlog cleared — more remains for the next sweep. |
| Cutoff | The computed date; everything older was in scope. |
| Error / refusal reason | Why a run failed or refused to proceed. |
A refused run is not a failure. It means a guard fired — the policy was misconfigured, the scope was implausibly large, or a switch was off — and the engine declined rather than guessing.
Bringing a policy into service
Author it in shadow
The policy is created disarmed. Nothing is deleted.
Let it sweep and read the runs
Check rows in scope against what you expect. A count far larger than anticipated usually means the wrong date column or a missing filter.
Check the cutoff date
Confirm the cutoff lands where the keep-days value implies.
Confirm the data is genuinely disposable
Check nothing reports off it, and that any regulatory retention period is longer than the keep-days value, not shorter.
Arm deliberately
By migration, with the shadow evidence in the change description.
Watch the first armed runs
Expect a backlog over several sweeps rather than one large deletion.
Common problems
| Symptom | Cause |
|---|---|
| Runs succeed but remove nothing | Shadow mode. Check all three switches. |
| Rows in scope is zero | Wrong date column, or the extra filter excludes everything. |
| Backlog never clears | Chunk ceiling too low for the table's growth rate. Raise max statements per sweep. |
| Run refused | Read the refusal reason — a guard fired on purpose. |
| Table size unchanged after a large removal | Space is reclaimed internally first; the file shrinks later. |
| A policy is not listed | It is not enabled, so the sweeper never evaluates it. |