SulcusBench Suite
An open, reproducible benchmark evaluating AI agent memory layers across contradiction resolution, temporal ordering, cross-session persistence, long-horizon scale, and grounded calibration.
Memory Territory Gap
Raw prompt stuffing hits a hard ceiling at 57.9% because context windows vanish when sessions close and degrade under long-horizon noise.
Context Window Ceiling
Handles single-session calibration, local contradiction updates, and short-range temporal sequence checks while the prompt window remains open.
Dedicated Memory Territory
Requires persistent Apache AGE graphs, cross-session entity recall, thermodynamic salience decay, and O(1) Synapse Map pointer paging.
Overall Leaderboard
Inspect verified baseline runs across all twenty tasks or isolate any evaluation dimension.
Context Window Baseline
10 of 20 passed
Zero Memory Floor
0 of 20 passed
Execute Benchmark Suite
SulcusBench is open source and runs locally in Python. Submit verified adapter runs via pull request; suites include adversarial distractors for engineering credibility.
Install Runner
pip install "git+https://github.com/digitalforgeca/sulcus.git#subdirectory=sdks/python"
cd sulcus/packages/sulcusbenchExecute Baselines
python -m sulcusbench --adapter no-memory
python -m sulcusbench --adapter in-contextEvaluate Adapter
python -m sulcusbench --adapter sulcus --api-key sk-...
python -m sulcusbench --adapter mem0 --api-key ...
python -m sulcusbench --adapter sulcus --api-key sk-... --categories recall temporal