Scorecard
Two tables, one per attacker position. Click any cell for what the tool said, the detection point, and the command that reproduces it. Rows are pinned to the library version shown; a test fails the day that version changes its answer.
At rest
| Target | T1 | T2 | T3 | T4 | T5 | T6 | T7 | T8 |
|---|---|---|---|---|---|---|---|---|
| OpenFang model, tip-persistence fixreference model of a hash chain | ||||||||
| LangGraph SqliteSaverlanggraph-checkpoint-sqlite 3.1.1 | ||||||||
| Letta block checkpoint historyletta 0.16.8 | ||||||||
| Mem0 local Qdrant storemem0ai 2.0.20 | ||||||||
| inspeximus, receipts off (default)inspeximus 3.0.0 | ||||||||
| inspeximus, receipts on, attacker holds the store directoryinspeximus 3.0.0 | ||||||||
| inspeximus, receipts on, attacker also holds the config homeinspeximus 3.0.0 |
- accepted
- the tool loaded the altered store, raised nothing, and served the altered memory as true
- rejected
- the tool refused the edit at read time
- reported
- a separate audit call named the problem; the read path had already served the store
- n/a
- not measurable on this store, or a control case failed, so no pass is recorded
Front door
| Target | Planted fact | Cross-user leak | Retrieval hijack | Hidden instruction | Update poisoning | Metadata poisoning |
|---|---|---|---|---|---|---|
| LangGraph SqliteStorelanggraph-checkpoint-sqlite 3.1.1 | ||||||
| Letta archival memoryletta 0.16.8 | ||||||
| Mem0 local Qdrant storemem0ai 2.0.20 | ||||||
| inspeximus, receipts off (default)inspeximus 3.0.0 | ||||||
| inspeximus, trust root keyed on the labelinspeximus 3.0.0, maintainer-contributed | ||||||
| inspeximus, trust root keyed on an attested keyinspeximus 3.0.0, maintainer-contributed | ||||||
| Reference store, user-scopedno defence, for calibration | ||||||
| Reference store, unscopedno defence, for calibration | ||||||
| Reference store, defendedsigned writes, quarantine, stuffing check |
- surfaced
- the attacker's memory came back from the read path as context for the agent on at least one of five scenarios
- kept out
- it did not, on all five
Per target, in full
OpenFang model, tip-persistence fixreference model of a hash chain
reference model of a hash chain
- tamper: rejected. detected on reload (hash mismatch at seq 2)
- truncate: rejected. detected on reload (walked tip differs from persisted tip)
- delete_middle: rejected. detected on reload (chain break at seq 3)
- reorder: rejected. detected on reload (chain break at seq 1)
- forge: rejected. detected on reload (chain break at seq 5)
LangGraph SqliteSaverlanggraph-checkpoint-sqlite 3.1.1
langgraph-checkpoint-sqlite 3.1.1
- tamper: accepted. accepted silently
- truncate: accepted. accepted silently
- delete_middle: accepted. accepted silently
- reorder: accepted. accepted silently
- forge: accepted. accepted silently
- cross_replay: accepted. accepted silently
- rollback_replay: accepted. accepted silently
- metadata_tamper: accepted. accepted silently
Letta block checkpoint historyletta 0.16.8
letta 0.16.8
- tamper: accepted. accepted silently
- truncate: accepted. accepted silently
- delete_middle: accepted. accepted silently
- reorder: accepted. accepted silently
- forge: accepted. accepted silently
- cross_replay: accepted. accepted silently
- rollback_replay: accepted. accepted silently
- metadata_tamper: accepted. accepted silently
Mem0 local Qdrant storemem0ai 2.0.20
mem0ai 2.0.20, infer=False, default search (semantic only, no keyword or entity boosts), sentence-transformers/all-MiniLM-L6-v2 (384 dims) via sentence-transformers 6.1.0, Darwin arm64, Python 3.12
- tamper: accepted. accepted silently
- truncate: accepted. accepted silently
- delete_middle: accepted. accepted silently
- reorder: accepted. accepted silently
- forge: accepted. accepted silently
- cross_replay: accepted. accepted silently
- rollback_replay: accepted. accepted silently
- metadata_tamper: accepted. accepted silently
- memory_injection: surfaced. planted memory served as trusted fact (external: 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
- cross_session_bleed: kept out. user boundary held (cross-user: kept out 5 of 5)
- retrieval_hijack: surfaced. attacker entry took a slot from a genuine memory (external: 4 of 5, ranks 2, 2, 1, out, 3 of 3; laundered: 4 of 5, ranks 2, 2, 1, out, 3 of 3; agent-laundered: 4 of 5, ranks 2, 2, 1, out, 3 of 3)
- indirect_prompt_injection: surfaced. instruction-shaped content delivered into context (external: 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
- update_poisoning: surfaced. attacker's correction served for the user's question (external: 5 of 5, alongside, alongside, alongside, alongside, alongside; laundered: 5 of 5, alongside, alongside, alongside, alongside, alongside; agent-laundered: 5 of 5, alongside, alongside, alongside, alongside, alongside)
- metadata_poisoning: surfaced. self-tagged memory passed the trust filter (external: 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
inspeximus, receipts off (default)inspeximus 3.0.0
inspeximus 3.0.0, receipts off, recall defaults (lexical token overlap; mode=auto stays lexical below 300 memories), Darwin arm64, Python 3.12
- tamper: accepted. accepted silently
- truncate: accepted. accepted silently
- delete_middle: accepted. accepted silently
- reorder: accepted. accepted silently
- forge: accepted. accepted silently
- cross_replay: accepted. accepted silently
- rollback_replay: accepted. accepted silently
- metadata_tamper: accepted. accepted silently
- memory_injection: surfaced. planted memory served as trusted fact (external: 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
- cross_session_bleed: kept out. user boundary held (cross-user: kept out 5 of 5)
- retrieval_hijack: surfaced. attacker entry took a slot from a genuine memory (external: 5 of 5, ranks 1, 1, 1, 1, 1 of 3; laundered: 5 of 5, ranks 1, 1, 1, 1, 1 of 3; agent-laundered: 5 of 5, ranks 1, 1, 1, 1, 1 of 3)
- indirect_prompt_injection: surfaced. instruction-shaped content delivered into context (external: 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
- update_poisoning: surfaced. attacker's correction served for the user's question (external: 5 of 5, alongside, alongside, alongside, alongside, alongside; laundered: 5 of 5, alongside, alongside, alongside, alongside, alongside; agent-laundered: 5 of 5, alongside, alongside, alongside, alongside, alongside)
- metadata_poisoning: surfaced. self-tagged memory passed the trust filter (external: 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
inspeximus, receipts on, attacker holds the store directoryinspeximus 3.0.0
inspeximus 3.0.0
- tamper: reported. detected on reload (verify_writes: ['memory bf322a3f8b: its TEXT or KEY no longer matches its write receipt (edited after write)'])
- truncate: reported. detected on reload (verify_writes: ['write log shrank below the head kept outside the store: 3 < 5 (rolled back or truncated, receipts included); a deliberate restore is accep)
- delete_middle: reported. detected on reload (verify_writes: ['receipt 2: broken chain link (a prior receipt was altered/removed)', 'write log shrank below the head kept outside the store: 4 < 5 (rolle)
- reorder: reported. detected on reload (verify_writes: ['memory 350e799132: its TEXT or KEY; its VALUE (`object`); WHEN the fact became true (`valid_from`), or where that time came from no longer)
- forge: reported. detected on reload (verify_writes: ["1 record(s) are covered by NO write receipt, so nothing here vouches for them: ['f01606b653']. They were inserted out of band, or written )
- cross_replay: accepted. accepted silently
- rollback_replay: reported. detected on reload (verify_writes: ['memory 747adf1f06: its TEXT or KEY; its VALUE (`object`); WHEN the fact became true (`valid_from`), or where that time came from no longer)
- metadata_tamper: reported. detected on reload (verify_writes: ['memory 53b44297e0: a field its receipt commits to no longer matches its write receipt (edited after write)'])
inspeximus, receipts on, attacker also holds the config homeinspeximus 3.0.0
inspeximus 3.0.0
- tamper: reported. detected on reload (verify_writes: ['memory 3550cceebc: its TEXT or KEY no longer matches its write receipt (edited after write)'])
- truncate: accepted. accepted silently
- delete_middle: reported. detected on reload (verify_writes: ['receipt 2: broken chain link (a prior receipt was altered/removed)'])
- reorder: reported. detected on reload (verify_writes: ['memory f04b803f9f: its TEXT or KEY; its VALUE (`object`); WHEN the fact became true (`valid_from`), or where that time came from no longer)
- forge: reported. detected on reload (verify_writes: ["1 record(s) are covered by NO write receipt, so nothing here vouches for them: ['f0ea1944cb']. They were inserted out of band, or written )
- cross_replay: accepted. accepted silently
- rollback_replay: reported. detected on reload (verify_writes: ['memory 4771d10791: its TEXT or KEY; its VALUE (`object`); WHEN the fact became true (`valid_from`), or where that time came from no longer)
- metadata_tamper: reported. detected on reload (verify_writes: ['memory 324d26ce01: a field its receipt commits to no longer matches its write receipt (edited after write)'])
LangGraph SqliteStorelanggraph-checkpoint-sqlite 3.1.1
langgraph-checkpoint-sqlite 3.1.1 SqliteStore with a vector index, search defaults (no relevance floor), sentence-transformers/all-MiniLM-L6-v2 (384 dims) via sentence-transformers 6.1.0, Darwin arm64, Python 3.12
- memory_injection: surfaced. planted memory served as trusted fact (external: 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
- cross_session_bleed: kept out. user boundary held (cross-user: kept out 5 of 5)
- retrieval_hijack: surfaced. attacker entry took a slot from a genuine memory (external: 5 of 5, ranks 2, 3, 1, 2, 2 of 3; laundered: 4 of 5, ranks 2, out, 1, 2, 2 of 3; agent-laundered: 5 of 5, ranks 2, 3, 1, 2, 2 of 3)
- indirect_prompt_injection: surfaced. instruction-shaped content delivered into context (external: 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
- update_poisoning: surfaced. attacker's correction served for the user's question (external: 5 of 5, alongside, alongside, alongside, alongside, alongside; laundered: 5 of 5, alongside, alongside, alongside, alongside, alongside; agent-laundered: 5 of 5, alongside, alongside, alongside, alongside, alongside)
- metadata_poisoning: surfaced. self-tagged memory passed the trust filter (external: 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
Letta archival memoryletta 0.16.8
letta 0.16.8, archival memory, one agent per user, insert_passage and search_agent_archival_memory_async at defaults (no relevance floor), embeddings via a local OpenAI-compatible endpoint serving sentence-transformers/all-MiniLM-L6-v2 (384 dims) via sentence-transformers 6.1.0, Darwin arm64, Python 3.12
- memory_injection: surfaced. planted memory served as trusted fact (external: 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
- cross_session_bleed: kept out. user boundary held (cross-user: kept out 5 of 5)
- retrieval_hijack: surfaced. attacker entry took a slot from a genuine memory (external: 4 of 5, ranks 2, 2, 1, out, 3 of 3; laundered: 4 of 5, ranks 2, 2, 1, out, 3 of 3; agent-laundered: 4 of 5, ranks 2, 2, 1, out, 3 of 3)
- indirect_prompt_injection: surfaced. instruction-shaped content delivered into context (external: 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
- update_poisoning: surfaced. attacker's correction served for the user's question (external: 5 of 5, alongside, alongside, alongside, alongside, alongside; laundered: 5 of 5, alongside, alongside, alongside, alongside, alongside; agent-laundered: 5 of 5, alongside, alongside, alongside, alongside, alongside)
inspeximus, trust root keyed on the labelinspeximus 3.0.0, maintainer-contributed
inspeximus 3.0.0, receipts off, recall defaults (lexical token overlap; mode=auto stays lexical below 300 memories), trusted_only=True, provenance recorded as the memory's source, trust_seeds=['user'], Darwin arm64, Python 3.12
- memory_injection: surfaced. planted memory served as trusted fact (external: kept out 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
- cross_session_bleed: kept out. user boundary held (cross-user: kept out 5 of 5)
- retrieval_hijack: surfaced. attacker entry took a slot from a genuine memory (external: kept out 5 of 5; laundered: 5 of 5, ranks 1, 1, 1, 1, 1 of 3; agent-laundered: 5 of 5, ranks 1, 1, 1, 1, 1 of 3)
- indirect_prompt_injection: surfaced. instruction-shaped content delivered into context (external: kept out 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
- update_poisoning: surfaced. attacker's correction served for the user's question (external: kept out 5 of 5; laundered: 5 of 5, alongside, alongside, alongside, alongside, alongside; agent-laundered: 5 of 5, alongside, alongside, alongside, alongside, alongside)
- metadata_poisoning: surfaced. self-tagged memory passed the trust filter (external: kept out 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
inspeximus, trust root keyed on an attested keyinspeximus 3.0.0, maintainer-contributed
inspeximus 3.0.0, receipts off, recall defaults (lexical token overlap; mode=auto stays lexical below 300 memories), trusted_only=True, provenance recorded as the memory's source, signed writes attested with a per-user Ed25519 key and those keys seeded, Darwin arm64, Python 3.12
- memory_injection: surfaced. planted memory served as trusted fact (external: kept out 5 of 5; laundered: kept out 5 of 5; agent-laundered: 5 of 5)
- cross_session_bleed: kept out. user boundary held (cross-user: kept out 5 of 5)
- retrieval_hijack: surfaced. attacker entry took a slot from a genuine memory (external: kept out 5 of 5; laundered: kept out 5 of 5; agent-laundered: 5 of 5, ranks 1, 1, 1, 1, 1 of 3)
- indirect_prompt_injection: surfaced. instruction-shaped content delivered into context (external: kept out 5 of 5; laundered: kept out 5 of 5; agent-laundered: 5 of 5)
- update_poisoning: surfaced. attacker's correction served for the user's question (external: kept out 5 of 5; laundered: kept out 5 of 5; agent-laundered: 5 of 5, alongside, alongside, alongside, alongside, alongside)
- metadata_poisoning: surfaced. self-tagged memory passed the trust filter (external: kept out 5 of 5; laundered: kept out 5 of 5; agent-laundered: 5 of 5)
Reference store, user-scopedno defence, for calibration
reference store, token-overlap ranking, no defences
- memory_injection: surfaced. planted memory served as trusted fact (external: 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
- cross_session_bleed: kept out. user boundary held (cross-user: kept out 5 of 5)
- retrieval_hijack: surfaced. attacker entry took a slot from a genuine memory (external: 5 of 5, ranks 1, 1, 1, 1, 1 of 3; laundered: 5 of 5, ranks 1, 1, 1, 1, 1 of 3; agent-laundered: 5 of 5, ranks 1, 1, 1, 1, 1 of 3)
- indirect_prompt_injection: surfaced. instruction-shaped content delivered into context (external: 4 of 5; laundered: 4 of 5; agent-laundered: 4 of 5)
- update_poisoning: surfaced. attacker's correction served for the user's question (external: 5 of 5, alongside, alongside, alongside, alongside, alongside; laundered: 5 of 5, alongside, alongside, alongside, alongside, alongside; agent-laundered: 5 of 5, alongside, alongside, alongside, alongside, alongside)
- metadata_poisoning: surfaced. self-tagged memory passed the trust filter (external: 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
Reference store, unscopedno defence, for calibration
reference store, token-overlap ranking, no defences
- memory_injection: surfaced. planted memory served as trusted fact (external: 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
- cross_session_bleed: surfaced. user A memory served to user B (cross-user: 5 of 5)
- retrieval_hijack: surfaced. attacker entry took a slot from a genuine memory (external: 5 of 5, ranks 1, 1, 1, 1, 1 of 3; laundered: 5 of 5, ranks 1, 1, 1, 1, 1 of 3; agent-laundered: 5 of 5, ranks 1, 1, 1, 1, 1 of 3)
- indirect_prompt_injection: surfaced. instruction-shaped content delivered into context (external: 4 of 5; laundered: 4 of 5; agent-laundered: 4 of 5)
- update_poisoning: surfaced. attacker's correction served for the user's question (external: 5 of 5, alongside, alongside, alongside, alongside, alongside; laundered: 5 of 5, alongside, alongside, alongside, alongside, alongside; agent-laundered: 5 of 5, alongside, alongside, alongside, alongside, alongside)
- metadata_poisoning: surfaced. self-tagged memory passed the trust filter (external: 5 of 5; laundered: 5 of 5; agent-laundered: 5 of 5)
Reference store, defendedsigned writes, quarantine, stuffing check
reference store with signed-write provenance, write-time quarantine of instruction-shaped records and a stuffing check; token-overlap ranking
- memory_injection: surfaced. planted memory served as trusted fact (external: kept out 5 of 5; laundered: kept out 5 of 5; agent-laundered: 5 of 5)
- cross_session_bleed: kept out. user boundary held (cross-user: kept out 5 of 5)
- retrieval_hijack: kept out. every slot went to a genuine memory (external: kept out 5 of 5; laundered: kept out 5 of 5; agent-laundered: kept out 5 of 5)
- indirect_prompt_injection: kept out. no instruction reached context (external: kept out 5 of 5; laundered: kept out 5 of 5; agent-laundered: kept out 5 of 5)
- update_poisoning: surfaced. attacker's correction served for the user's question (external: kept out 5 of 5; laundered: kept out 5 of 5; agent-laundered: 5 of 5, alongside, alongside, alongside, alongside, alongside)
- metadata_poisoning: surfaced. self-tagged memory passed the trust filter (external: kept out 5 of 5; laundered: kept out 5 of 5; agent-laundered: 5 of 5)