Vibgrate AI Context
Token-cost benchmarks: AI sessions with and without the code map
Every question you ask an AI assistant about your codebase starts with the assistant re-reading your codebase — and that re-reading is most of the token bill. Vibgrate AI Context replaces it with answers from a pre-built code map. This page publishes what that is worth, measured: the same tasks, the same model, with and without the map, at equal verified success — including the repo sizes where the map is not worth it.
Measured, in the open
~29% fewer tokens, same results
The same coding tasks, the same model, with and without the Vibgrate code map — on a ~300-file codebase. We count only the tasks both runs solved, so it is fewer tokens for the same result, never “cheaper because it gave up”.
Without the map771,704 tokens
With Vibgrate Graph549,341 tokens
Measured with a frontier coding model. On tiny repositories grep already finds the code and the map costs more — we publish that part of the curve too.
How the numbers are measured
Same model, twice
Each coding task runs twice with the same model on two fresh copies of the repo: once with generic file tools (list, search, read, write), once with Vibgrate AI Context (vg serve) for discovery.
Only discovery differs
Both runs can read and write files. The only difference is how the model finds the right code: walking and grepping the tree, or asking the pre-built code map.
Success is verified, not assumed
After each run, a deterministic verifier executes the edited code in the language’s real runtime and checks behaviour. A run that fails the verifier counts for nothing.
Savings only count when results match
Reduction is computed only across tasks both runs solved. The headline is “fewer tokens, same results” — never “cheaper because it gave up”. Tiers where the map saves nothing are published too.
The tasks are symptom-described — “the total is wrong after a promotion”, “refunds omit the tax” — the shape a real bug report takes, not “change function X”. They run against a ~300-file commerce codebase plus small fixtures, so the curve shows where a code map earns its keep and where grep already wins. The mix is fixed in advance (a written pre-registration) so the result can’t be tuned after the fact. Tokens are the model’s own reported usage summed across every step. The run is expensive (a live model over the whole corpus), so it is measured intermittently against a pinned CLI build and reviewed before publication — not on every release. The report below is pinned to the version it was last measured on.
Latest report — Vibgrate CLI v2026.814.2
Model gpt-5.4-mini · measured 2026-08-17 · step cap 30 · 35 of 38 tasks solved by both runs. This is one A/B run against the shipped build; the mix was run three times and the discovery-tier median held at +33.8% fewer tokens (per-task variance is large, so the median is the honest figure).
The curve — result by repository size
| Repo size | Tasks compared | File-tools tokens | AI Context tokens | Result |
|---|---|---|---|---|
| Small (3–5 files) | 4/6 | 9,985 | 23,663 | 137% more |
| Mid (~300 files) | 30/31 | 771,704 | 549,341 | 28.8% fewer |
| XL (~1,000 files) | 1/1 | 25,825 | 17,458 | 32.4% fewer |
A code map earns little on tiny repositories — any tool finds the code instantly, so the map’s fixed overhead shows as a cost. The saving appears as the repository grows and discovery becomes the dominant token cost. We publish the whole curve, not a single flattering number.
Every task, every repository
| Task | Repository | Language | Files | File tools | AI Context | Result |
|---|---|---|---|---|---|---|
| Fix order total to apply the discount | orders-api | JavaScript | 5 | 2,632 tok · 4 steps | 6,749 tok · 4 steps | 156.4% more |
| Fix the cents/dollars mixup in shipping quotes (mid repo) | meridian-commerce | JavaScript | 323 | 23,373 tok · 5 steps | 10,928 tok · 4 steps | 53.2% fewer |
| Stop billing shipping on packaging weight (mid repo) | meridian-commerce | JavaScript | 323 | 19,885 tok · 5 steps | 17,135 tok · 6 steps | 13.8% fewer |
| Implement the statement balance from its usage (mid repo) | meridian-commerce | JavaScript | 323 | 18,453 tok · 6 steps | 19,654 tok · 5 steps | 6.5% more |
| Stop reporting amounts from dropping cents (mid repo) | meridian-commerce | JavaScript | 323 | 93,850 tok · 8 steps | 25,702 tok · 6 steps | 72.6% fewer |
| Fix the tax-exemption boundary (mid repo) | meridian-commerce | JavaScript | 323 | 10,664 tok · 4 steps | 9,092 tok · 4 steps | 14.7% fewer |
| Apply the checkout promotion to invoice totals (Ruby) | ledger-app | Ruby | 116 | 21,644 tok · 5 steps | 13,255 tok · 5 steps | 38.8% fewer |
| Apply discount with README cents rounding | commerce-docs | JavaScript | 6 | 4,290 tok · 5 steps | did not finish | — |
| Make slugify lowercase and punctuation-safe | slug-lib | JavaScript | 4 | 2,305 tok · 4 steps | 7,047 tok · 5 steps | 205.7% more |
| Stop charging sales tax on the shipping fee (mid repo) | meridian-commerce | JavaScript | 323 | 40,278 tok · 7 steps | 10,342 tok · 4 steps | 74.3% fewer |
| Charge the card fee on the original captured amount (mid repo) | meridian-commerce | JavaScript | 323 | 43,834 tok · 7 steps | 13,184 tok · 5 steps | 69.9% fewer |
| Implement the late-fee calculator from its usage (mid repo) | meridian-commerce | JavaScript | 323 | 11,348 tok · 4 steps | 23,024 tok · 6 steps | 102.9% more |
| Raise the free-shipping minimum everywhere (mid repo) | meridian-commerce | JavaScript | 323 | 30,071 tok · 8 steps | 10,792 tok · 7 steps | 64.1% fewer |
| Apply the checkout promotion to invoice totals (XL repo) | meridian-commerce-xl | JavaScript | 963 | 25,825 tok · 4 steps | 17,458 tok · 6 steps | 32.4% fewer |
| Apply the checkout promotion to invoice totals (Java) | inventory-forge | Java | 88 | 14,514 tok · 5 steps | 15,035 tok · 5 steps | 3.6% more |
| Fix tiny-validate 1.2 call site from library docs | libdocs-mini | JavaScript | 4 | 2,304 tok · 4 steps | 4,905 tok · 4 steps | 112.9% more |
| Implement parseArgs from how it is used | argv-cli | JavaScript | 3 | did not finish | did not finish | — |
| Fix wholesale lines billing below the agreed price (mid repo) | meridian-commerce | JavaScript | 323 | 18,441 tok · 6 steps | 10,157 tok · 4 steps | 44.9% fewer |
| Free shipping should use the pre-discount subtotal (mid repo) | meridian-commerce | JavaScript | 323 | 24,456 tok · 6 steps | 44,100 tok · 11 steps | 80.3% more |
| Fix the monthly-to-annual rate conversion (mid repo) | meridian-commerce | JavaScript | 323 | 59,887 tok · 5 steps | 46,470 tok · 5 steps | 22.4% fewer |
| Make the two discount helpers agree (mid repo) | meridian-commerce | JavaScript | 323 | 19,715 tok · 4 steps | 10,200 tok · 4 steps | 48.3% fewer |
| Apply the checkout promotion to invoice totals (Python) | atlas-crm | Python | 137 | 18,337 tok · 5 steps | 10,516 tok · 4 steps | 42.7% fewer |
| Apply the checkout promotion to invoice totals (PHP) | storefront-cart | PHP | 106 | 14,618 tok · 5 steps | 10,952 tok · 4 steps | 25.1% fewer |
| Prefix normalizeId once; keep all callers on core | impact-mini | JavaScript | 5 | 2,744 tok · 4 steps | 4,962 tok · 4 steps | 80.8% more |
| Apply the checkout promotion to invoice totals (mid repo) | meridian-commerce | JavaScript | 323 | 22,723 tok · 5 steps | 17,185 tok · 6 steps | 24.4% fewer |
| Include the sales tax in return credits (mid repo) | meridian-commerce | JavaScript | 323 | 22,099 tok · 5 steps | 9,770 tok · 4 steps | 55.8% fewer |
| Fix subscription invoices billing 100× too high (mid repo) | meridian-commerce | JavaScript | 323 | 3,383 tok · 5 steps | 10,296 tok · 4 steps | 204.3% more |
| Fix the sign on account running balances (mid repo) | meridian-commerce | JavaScript | 323 | did not finish | did not finish | — |
| Round totals to the nearest cent, not truncate (mid repo) | meridian-commerce | JavaScript | 323 | 32,304 tok · 6 steps | 18,935 tok · 7 steps | 41.4% fewer |
| Apply the checkout promotion to invoice totals (TypeScript) | harbor-api | TypeScript | 128 | 6,464 tok · 4 steps | 10,492 tok · 4 steps | 62.3% more |
| Apply the checkout promotion to invoice totals (Bash) | ops-toolkit | Bash | 95 | 5,689 tok · 4 steps | 10,295 tok · 4 steps | 81% more |
| Implement the refund amount calculator (mid repo) | meridian-commerce | JavaScript | 323 | 35,854 tok · 7 steps | 39,829 tok · 9 steps | 11.1% more |
| Stop loyalty credits from reducing taxable amount (mid repo) | meridian-commerce | JavaScript | 323 | 29,188 tok · 6 steps | 18,127 tok · 6 steps | 37.9% fewer |
| Apply gift cards as a fixed dollar amount (mid repo) | meridian-commerce | JavaScript | 323 | 11,124 tok · 4 steps | 13,029 tok · 5 steps | 17.1% more |
| Raise the sales-tax rate everywhere it is applied (mid repo) | meridian-commerce | JavaScript | 323 | 49,061 tok · 8 steps | 21,680 tok · 6 steps | 55.8% fewer |
| Round refund amounts to cents (mid repo) | meridian-commerce | JavaScript | 323 | 58,355 tok · 6 steps | 50,534 tok · 9 steps | 13.4% fewer |
| Apply the checkout promotion to invoice totals (Go) | freight-routing | Go | 86 | 5,240 tok · 4 steps | 17,767 tok · 6 steps | 239.1% more |
| Apply the checkout promotion to invoice totals (C) | sensor-hub | C | 70 | 6,852 tok · 5 steps | 10,864 tok · 4 steps | 58.6% more |
“did not finish” means that run failed the behaviour verifier; such tasks are excluded from every aggregate on this page.
Previous reports
- v2026.722.240.7% fewer overall · 31/35 tasks comparable2026-07-22
- v2026.718.222.6% fewer overall · 33/35 tasks comparable2026-07-20
- v2026.711.233.6% fewer overall · 32/35 tasks comparable2026-07-13
- v2026.708.332.5% fewer overall · 32/35 tasks comparable2026-07-09
- v2026.704.122.9% fewer overall · 24/25 tasks comparable2026-07-04
These reports sit alongside the CLI release benchmarks (scan correctness, code-graph extraction, retrieval accuracy, performance) and follow the same rule: measured on a pinned corpus, reviewed by a human before anything appears here. Unlike those, which run with every release, this token-savings run is measured intermittently because it drives a live model.
Vibgrate Graph
See the code graph in action
A replay of the actual CLI querying the code map — ask a question in plain English, trace a change’s blast radius, follow a path between symbols. Nothing executes in your browser.
Replay
❯ npx @vibgrate/cli scan
╭──────────────────────────────────────────╮
│ Vibgrate Drift Report │
╰──────────────────────────────────────────╯
── node-turborepo (node) .
Runtime: >=18.0.0 (6 majors behind)
Frameworks:
Turbo: 1.13.4 → 2.10.11 (1 behind)
TypeScript: 5.9.3 → 7.0.2 (2 behind)
Dependencies:
1 current 1 1-behind 3 2+ behind 1 unknown
── @repo/admin (node) apps/admin
Frameworks:
TanStack Query: 5.101.4 → 5.101.4 (current)
React: 18.3.1 → 19.2.8 (1 behind)
React DOM: 18.3.1 → 19.2.8 (1 behind)
TypeScript: 5.9.3 → 7.0.2 (2 behind)
Vite: 5.4.21 → 8.2.1 (3 behind)
Dependencies:
3 current 9 1-behind 3 2+ behind 4 unknown
── @repo/api (node) apps/api
Frameworks:
Express: 4.22.2 → 5.2.1 (1 behind)
TypeScript: 5.9.3 → 7.0.2 (2 behind)
Vitest: 1.6.1 → 4.1.11 (3 behind)
Dependencies:
7 current 5 1-behind 3 2+ behind 4 unknown
── @repo/web (node) apps/web
Frameworks:
Next.js: 14.2.35 → 16.3.1 (2 behind)
React: 18.3.1 → 19.2.8 (1 behind)
React DOM: 18.3.1 → 19.2.8 (1 behind)
TypeScript: 5.9.3 → 7.0.2 (2 behind)
Dependencies:
2 current 6 1-behind 3 2+ behind 5 unknown
── @repo/config (node) packages/config
Frameworks:
TypeScript: 5.9.3 → 7.0.2 (2 behind)
Dependencies:
2 current 2 1-behind 5 2+ behind 0 unknown
── @repo/database (node) packages/database
Frameworks:
Prisma: 5.22.0 → 7.9.1 (2 behind)
TypeScript: 5.9.3 → 7.0.2 (2 behind)
Dependencies:
1 current 0 1-behind 3 2+ behind 1 unknown
── @repo/types (node) packages/types
Frameworks:
TypeScript: 5.9.3 → 7.0.2 (2 behind)
Dependencies:
0 current 0 1-behind 1 2+ behind 1 unknown
── @repo/ui (node) packages/ui
Frameworks:
React: 18.3.1 → 19.2.8 (1 behind)
TypeScript: 5.9.3 → 7.0.2 (2 behind)
React: 18.3.1 → 19.2.8 (1 behind)
Dependencies:
1 current 4 1-behind 1 2+ behind 1 unknown
── @repo/utils (node) packages/utils
Frameworks:
TypeScript: 5.9.3 → 7.0.2 (2 behind)
Vitest: 1.6.1 → 4.1.11 (3 behind)
Dependencies:
0 current 1 1-behind 2 2+ behind 1 unknown
Tech Stack
Frontend: React, React DOM
Meta-frameworks: Next.js
Bundlers: tsx, Turbo, Vite
CSS / UI: Autoprefixer, PostCSS, Tailwind CSS
Backend: Express
ORM / Database: Prisma, Prisma Client
Testing: Vitest
Lint & Format: ESLint, ESLint Prettier, ESLint React, Prettier, typescript-eslint
Services & Integrations
Auth: JWT 9.0.3
Databases: Prisma 5.22.0
TypeScript
v5.3.3 · strict ✔ · MIXED · target: ES2022
Build & Deploy
Package Managers: pnpm
Monorepo: npm-workspaces, pnpm-workspaces, turbo
Product Purpose Signals
Frameworks: react, nextjs
Evidence: 177
Top Signals:
- [heading] Dashboard (apps/admin/src/pages/Dashboard.tsx)
- [title] Revenue Overview (apps/admin/src/pages/Dashboard.tsx)
- [copy] workspace:* (packages/ui/package.json)
- [copy] ./dist (packages/ui/tsconfig.json)
- [copy] ./src/index.ts (packages/ui/package.json)
- [copy] @repo/config/tsconfig-base.json (packages/ui/tsconfig.json)
- [copy] @repo/ui (packages/ui/package.json)
- [copy] #3b82f6 (apps/admin/src/pages/Dashboard.tsx)
Unknowns:
- No pricing or billing evidence found.
- No integrations/connectors evidence found.
- No route structure evidence found.
Security Posture
Lockfile ✖ · .env ✔ · node_modules ✔
Platform
Native modules: turbo
Code Quality
Files: 36 · Functions: 183 · Avg complexity: 2.62 · Avg length: 21.13 lines
Max nesting: 2 · Circular deps: 0 · Dead code: 0%
God files: apps/admin/src/pages/Products (448 lines)
Database Schema
postgresql · 8 models · 1 enum
Models: Address, CartItem, Category, Order, OrderItem (+3 more)
Findings (16 errors, 11 warnings)
✖ Node.js runtime ">=18.0.0" reached end-of-life on 2025-04-30 (latest: 24.0.0).
vibgrate/runtime-eol in .
⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).
vibgrate/framework-major-lag in .
✖ 60% of dependencies are 2+ major versions behind in node-turborepo.
vibgrate/dependency-rot in .
✖ @types/node is 6 major versions behind (spec: ^20.11.0, latest: 26.2.0).
vibgrate/dependency-major-lag in .
⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).
vibgrate/framework-major-lag in apps/admin
✖ Vite is 3 major versions behind (current: 5.4.21, latest: 8.2.1).
vibgrate/framework-major-lag in apps/admin
✖ vite is 3 major versions behind (spec: ^5.0.12, latest: 8.2.1).
vibgrate/dependency-major-lag in apps/admin
⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).
vibgrate/framework-major-lag in apps/api
✖ Vitest is 3 major versions behind (current: 1.6.1, latest: 4.1.11).
vibgrate/framework-major-lag in apps/api
✖ @types/node is 6 major versions behind (spec: ^20.11.0, latest: 26.2.0).
vibgrate/dependency-major-lag in apps/api
✖ vitest is 3 major versions behind (spec: ^1.2.1, latest: 4.1.11).
vibgrate/dependency-major-lag in apps/api
⚠ Next.js is 2 major versions behind (current: 14.2.35, latest: 16.3.1).
vibgrate/framework-major-lag in apps/web
⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).
vibgrate/framework-major-lag in apps/web
✖ @types/node is 6 major versions behind (spec: ^20.11.0, latest: 26.2.0).
vibgrate/dependency-major-lag in apps/web
⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).
vibgrate/framework-major-lag in packages/config
✖ 56% of dependencies are 2+ major versions behind in @repo/config.
vibgrate/dependency-rot in packages/config
✖ eslint-plugin-react-hooks is 3 major versions behind (spec: ^4.6.0, latest: 7.1.1).
vibgrate/dependency-major-lag in packages/config
⚠ Prisma is 2 major versions behind (current: 5.22.0, latest: 7.9.1).
vibgrate/framework-major-lag in packages/database
⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).
vibgrate/framework-major-lag in packages/database
✖ 75% of dependencies are 2+ major versions behind in @repo/database.
vibgrate/dependency-rot in packages/database
⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).
vibgrate/framework-major-lag in packages/types
✖ 100% of dependencies are 2+ major versions behind in @repo/types.
vibgrate/dependency-rot in packages/types
⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).
vibgrate/framework-major-lag in packages/ui
⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).
vibgrate/framework-major-lag in packages/utils
✖ Vitest is 3 major versions behind (current: 1.6.1, latest: 4.1.11).
vibgrate/framework-major-lag in packages/utils
✖ 67% of dependencies are 2+ major versions behind in @repo/utils.
vibgrate/dependency-rot in packages/utils
✖ vitest is 3 major versions behind (spec: ^1.2.1, latest: 4.1.11).
vibgrate/dependency-major-lag in packages/utils
╭──────────────────────────────────────────╮
│ Top Priority Actions │
╰──────────────────────────────────────────╯
1. Upgrade EOL runtime in node-turborepo
End-of-life runtimes no longer receive security patches and block ecosystem upgrades.
./.
>=18.0.0 → 24.0.0 (6 majors behind)
Impact: −10 drift points (runtime & EOL)
2. Fix security posture: no lockfile found
Without a lockfile, installs are non-deterministic. Run the install command to generate one and commit it.
./
Missing: package-lock.json, pnpm-lock.yaml, or yarn.lock
3. Upgrade Vite 5.4.21 → 8.2.1 in @repo/admin (+2 more)
3 major versions behind. Major framework drift increases breaking change risk and blocks access to security fixes and performance improvements.
./apps/admin
Vite: 5.4.21 → 8.2.1 (3 majors behind)
./apps/api
Vitest: 1.6.1 → 4.1.11 (3 majors behind)
./packages/utils
Vitest: 1.6.1 → 4.1.11 (3 majors behind)
Impact: −5–15 drift points
4. Reduce dependency rot in @repo/types (100% severely outdated)
1 of 1 dependencies are 2+ majors behind. Run `npm outdated` and prioritise packages with known CVEs or breaking API changes.
./packages/types
typescript: 5.9.3 → 7.0.2 (2 majors behind)
Impact: −5–10 drift points
5. Reduce dependency rot in @repo/database (75% severely outdated)
3 of 4 dependencies are 2+ majors behind. Run `npm outdated` and prioritise packages with known CVEs or breaking API changes.
./packages/database
@prisma/client: 5.22.0 → 7.9.1 (2 majors behind)
prisma: 5.22.0 → 7.9.1 (2 majors behind)
typescript: 5.9.3 → 7.0.2 (2 majors behind)
Impact: −5–10 drift points
╭──────────────────────────────────────────╮
│ Architecture Layers │
╰──────────────────────────────────────────╯
Archetype: nextjs (80% confidence)
Files classified: 24 (11 unclassified)
Folders classified: 8
apps/admin/src presentation 100% 4 files
apps/admin/src/pages presentation 100% 2 files
apps/api/src/middleware middleware 100% 2 files
apps/api/src/routes routing 100% 2 files
apps/web/src/app presentation 100% 4 files
apps/web/src/app/products presentation 100% 2 files
apps/web/src/app/products/[id] presentation 100% 1 file
packages/ui/src presentation 100% 6 files
Unclassified source (sample): 11
presentation 15 files drift ████████████████████ 100 risk high
routing 4 files drift ████████████████████ 100 risk high
middleware 2 files drift ███████▍░░░░░░░░░░░░ 37 risk moderate
config 2 files drift ░░░░░░░░░░░░░░░░░░░░ 0 risk none
shared 1 file drift ████████████████████ 100 risk high
╭──────────────────────────────────────────╮
│ DriftScore Summary │
╰──────────────────────────────────────────╯
DriftScore: 66/100
Risk Level: HIGH
Projects: 9
Classified: 8 nano · 1 micro · 0 small · 0 standard
Billable: 0.42 · 9 detected → 0.42 billable projects (micro-project pricing)
0.1 micro · 0.32 nano
These fractions add up across repositories, then round down to whole billable projects.
Score Breakdown
Runtime: ████████████████████ 100
Frameworks: █████████▏░░░░░░░░░░ 46
Dependencies: ██████▏░░░░░░░░░░░░░ 31
EOL Risk: ████████████████████ 100
Scanned at 2026-08-19T10:20:40.993Z · 5.9s · 286 files scanned · 56 workspace files · 27 dirs
❯
Press Run to start.