Why your automation workflow reposts old content (and the state-layer fix)

Every automation builder eventually meets The Duplicate. You built a pipeline that drafts a tweet whenever you publish. It worked beautifully on day one. Then it drafted the same tweet four times, and your audience noticed before you did.

Root cause: triggers are not memory

A cron trigger answers "when should I run?" — never "what have I already done?". Polling APIs return the same items on every call. Without persistent state between executions, your workflow is amnesiac: every run is its first run.

The fix is a state layer: a record of processed item IDs that survives between runs, updated only when the full pipeline succeeds.

The three implementation options in n8n

1. Workflow static data — zero infrastructure. Persisted automatically on successful executions. Perfect below ~1000 IDs:

const staticData = $getWorkflowStaticData('global');
staticData.processedIds = staticData.processedIds || [];
const fresh = items.filter(i =>
  !staticData.processedIds.includes(i.json.id));
// ... do work ...
// LAST node only:
for (const item of $('Filter New').all())
  staticData.processedIds.push(item.json.id);

• A "state" sheet or table — one row per processed item with timestamps. Auditable, survives workflow deletion, works across multiple workflows.
• Idempotency keys at the destination — hash the content and check-before-write wherever it lands. The strongest guarantee; protects even against manual double-runs.

Three rules that prevent 90% of duplicate bugs

1. Mark state last. Update the processed set after the final node succeeds, not right after filtering. If generation fails mid-run, unprocessed items retry cleanly next cycle.
2. Clean up on a schedule. Trim the ID set periodically (slice(-200) pattern) so lookups stay fast forever.
3. First run = seed run. Expect the first execution to process nothing (or everything) by design; document which, so users don't panic.

The same discipline applies outside n8n — any polling system without durable state will eventually embarrass you publicly, because duplicates are public by nature.