Data Format Migration Checklist

A checklist for migrating data between serialization formats like CSV, JSON, Avro, Parquet, and ORC without losing fidelity. It focuses on type mapping, null and timestamp handling, sample round-trip validation, and reconciliation. Use it whenever changing how data is stored on disk.

Estimated Time

2-4 hours

Type

migration readiness

When to Use This Checklist

Use this checklist when migrating data between serialization formats: CSV to Parquet, JSON to Avro, or similar. Format choice affects storage cost, query speed, and schema enforcement. Row-based text formats like CSV and JSON are simple but verbose; columnar formats like Parquet and ORC compress well and accelerate analytics; Avro suits streaming with strong schema support. Converting between them risks subtle fidelity loss.

How to Use This Checklist

Document the source format, encoding, and schema precisely, then choose a target that fits how the data is read. The type-mapping and value-representation items are where fidelity is won or lost: nulls, timestamps, timezones, and decimal precision are the usual casualties. Validate a sample round-trip before converting everything, then reconcile counts and key fields. Keep source files until downstream readers confirm they handle the new format.

What Good Looks Like

A clean format migration preserves every value. The target format matches the access pattern, type mappings are explicit with no unresolved lossy conversions, and null, timestamp, timezone, and precision handling are verified. A sample round-trips without loss before the full conversion runs, counts reconcile afterward, and downstream readers are updated and tested. Source files are retained until adoption is confirmed, giving a clean rollback if a reader cannot parse the new format.

Common Pitfalls

The quiet killers are type and value conversions: floats losing precision, timestamps shifting timezone, and empty strings becoming nulls or vice versa. Choosing a columnar format for a row-by-row access pattern, or the reverse, hurts performance. Teams convert everything before validating a sample, then discover loss at scale. Forgetting to update downstream readers breaks consumers at cutover. Deleting source files too early removes the only recovery path.

Related Resources

Use schema-evolution-schema-registry and data-contracts to govern format and schema changes, and verify fidelity with data-quality-management. Trace conversions through data-lineage so consumers can follow the change.

See a real scan run

A replay of the actual CLI running against our test repositories — live progress, real findings, a genuine DriftScore. Nothing executes in your browser.

Replay

demo@vibgrate — bash

❯ npx @vibgrate/cli scan

╭──────────────────────────────────────────╮

│ Vibgrate Drift Report │

╰──────────────────────────────────────────╯

── node-turborepo (node) .

Runtime: >=18.0.0 (6 majors behind)

Frameworks:

Turbo: 1.13.4 → 2.10.12 (1 behind)

TypeScript: 5.9.3 → 7.0.2 (2 behind)

Dependencies:

1 current 1 1-behind 3 2+ behind 1 unknown

── @repo/admin (node) apps/admin

Frameworks:

TanStack Query: 5.102.5 → 5.102.5 (current)

React: 18.3.1 → 19.2.8 (1 behind)

React DOM: 18.3.1 → 19.2.8 (1 behind)

TypeScript: 5.9.3 → 7.0.2 (2 behind)

Vite: 5.4.21 → 8.2.2 (3 behind)

Dependencies:

3 current 9 1-behind 3 2+ behind 4 unknown

── @repo/api (node) apps/api

Frameworks:

Express: 4.22.2 → 5.2.1 (1 behind)

TypeScript: 5.9.3 → 7.0.2 (2 behind)

Vitest: 1.6.1 → 4.1.11 (3 behind)

Dependencies:

7 current 5 1-behind 3 2+ behind 4 unknown

── @repo/web (node) apps/web

Frameworks:

Next.js: 14.2.35 → 16.3.3 (2 behind)

React: 18.3.1 → 19.2.8 (1 behind)

React DOM: 18.3.1 → 19.2.8 (1 behind)

TypeScript: 5.9.3 → 7.0.2 (2 behind)

Dependencies:

2 current 6 1-behind 3 2+ behind 5 unknown

── @repo/config (node) packages/config

Frameworks:

TypeScript: 5.9.3 → 7.0.2 (2 behind)

Dependencies:

2 current 2 1-behind 5 2+ behind 0 unknown

── @repo/database (node) packages/database

Frameworks:

Prisma: 5.22.0 → 7.10.0 (2 behind)

TypeScript: 5.9.3 → 7.0.2 (2 behind)

Dependencies:

1 current 0 1-behind 3 2+ behind 1 unknown

── @repo/types (node) packages/types

Frameworks:

TypeScript: 5.9.3 → 7.0.2 (2 behind)

Dependencies:

0 current 0 1-behind 1 2+ behind 1 unknown

── @repo/ui (node) packages/ui

Frameworks:

React: 18.3.1 → 19.2.8 (1 behind)

TypeScript: 5.9.3 → 7.0.2 (2 behind)

React: 18.3.1 → 19.2.8 (1 behind)

Dependencies:

1 current 4 1-behind 1 2+ behind 1 unknown

── @repo/utils (node) packages/utils

Frameworks:

TypeScript: 5.9.3 → 7.0.2 (2 behind)

Vitest: 1.6.1 → 4.1.11 (3 behind)

Dependencies:

0 current 1 1-behind 2 2+ behind 1 unknown

Tech Stack

Frontend: React, React DOM

Meta-frameworks: Next.js

Bundlers: tsx, Turbo, Vite

CSS / UI: Autoprefixer, PostCSS, Tailwind CSS

Backend: Express

ORM / Database: Prisma, Prisma Client

Testing: Vitest

Lint & Format: ESLint, ESLint Prettier, ESLint React, Prettier, typescript-eslint

Services & Integrations

Auth: JWT 9.0.3

Databases: Prisma 5.22.0

TypeScript

v5.3.3 · strict ✔ · MIXED · target: ES2022

Build & Deploy

Package Managers: pnpm

Monorepo: npm-workspaces, pnpm-workspaces, turbo

Product Purpose Signals

Frameworks: react, nextjs

Evidence: 177

Top Signals:

- [heading] Dashboard (apps/admin/src/pages/Dashboard.tsx)

- [title] Revenue Overview (apps/admin/src/pages/Dashboard.tsx)

- [copy] workspace:* (packages/ui/package.json)

- [copy] ./dist (packages/ui/tsconfig.json)

- [copy] ./src/index.ts (packages/ui/package.json)

- [copy] @repo/config/tsconfig-base.json (packages/ui/tsconfig.json)

- [copy] @repo/ui (packages/ui/package.json)

- [copy] #3b82f6 (apps/admin/src/pages/Dashboard.tsx)

Unknowns:

- No pricing or billing evidence found.

- No integrations/connectors evidence found.

- No route structure evidence found.

Security Posture

Lockfile ✖ · .env ✔ · node_modules ✔

Platform

Native modules: turbo

Code Quality

Files: 36 · Functions: 183 · Avg complexity: 2.62 · Avg length: 21.13 lines

Max nesting: 2 · Circular deps: 0 · Dead code: 0%

God files: apps/admin/src/pages/Products (448 lines)

Database Schema

postgresql · 8 models · 1 enum

Models: Address, CartItem, Category, Order, OrderItem (+3 more)

Findings (16 errors, 11 warnings)

✖ Node.js runtime ">=18.0.0" reached end-of-life on 2025-04-30 (latest: 24.0.0).

vibgrate/runtime-eol in .

⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).

vibgrate/framework-major-lag in .

✖ 60% of dependencies are 2+ major versions behind in node-turborepo.

vibgrate/dependency-rot in .

✖ @types/node is 6 major versions behind (spec: ^20.11.0, latest: 26.3.0).

vibgrate/dependency-major-lag in .

⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).

vibgrate/framework-major-lag in apps/admin

✖ Vite is 3 major versions behind (current: 5.4.21, latest: 8.2.2).

vibgrate/framework-major-lag in apps/admin

✖ vite is 3 major versions behind (spec: ^5.0.12, latest: 8.2.2).

vibgrate/dependency-major-lag in apps/admin

⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).

vibgrate/framework-major-lag in apps/api

✖ Vitest is 3 major versions behind (current: 1.6.1, latest: 4.1.11).

vibgrate/framework-major-lag in apps/api

✖ @types/node is 6 major versions behind (spec: ^20.11.0, latest: 26.3.0).

vibgrate/dependency-major-lag in apps/api

✖ vitest is 3 major versions behind (spec: ^1.2.1, latest: 4.1.11).

vibgrate/dependency-major-lag in apps/api

⚠ Next.js is 2 major versions behind (current: 14.2.35, latest: 16.3.3).

vibgrate/framework-major-lag in apps/web

⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).

vibgrate/framework-major-lag in apps/web

✖ @types/node is 6 major versions behind (spec: ^20.11.0, latest: 26.3.0).

vibgrate/dependency-major-lag in apps/web

⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).

vibgrate/framework-major-lag in packages/config

✖ 56% of dependencies are 2+ major versions behind in @repo/config.

vibgrate/dependency-rot in packages/config

✖ eslint-plugin-react-hooks is 3 major versions behind (spec: ^4.6.0, latest: 7.1.1).

vibgrate/dependency-major-lag in packages/config

⚠ Prisma is 2 major versions behind (current: 5.22.0, latest: 7.10.0).

vibgrate/framework-major-lag in packages/database

⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).

vibgrate/framework-major-lag in packages/database

✖ 75% of dependencies are 2+ major versions behind in @repo/database.

vibgrate/dependency-rot in packages/database

⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).

vibgrate/framework-major-lag in packages/types

✖ 100% of dependencies are 2+ major versions behind in @repo/types.

vibgrate/dependency-rot in packages/types

⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).

vibgrate/framework-major-lag in packages/ui

⚠ TypeScript is 2 major versions behind (current: 5.9.3, latest: 7.0.2).

vibgrate/framework-major-lag in packages/utils

✖ Vitest is 3 major versions behind (current: 1.6.1, latest: 4.1.11).

vibgrate/framework-major-lag in packages/utils

✖ 67% of dependencies are 2+ major versions behind in @repo/utils.

vibgrate/dependency-rot in packages/utils

✖ vitest is 3 major versions behind (spec: ^1.2.1, latest: 4.1.11).

vibgrate/dependency-major-lag in packages/utils

╭──────────────────────────────────────────╮

│ Top Priority Actions │

╰──────────────────────────────────────────╯

1. Upgrade EOL runtime in node-turborepo

End-of-life runtimes no longer receive security patches and block ecosystem upgrades.

./.

>=18.0.0 → 24.0.0 (6 majors behind)

Impact: −10 drift points (runtime & EOL)

2. Fix security posture: no lockfile found

Without a lockfile, installs are non-deterministic. Run the install command to generate one and commit it.

Missing: package-lock.json, pnpm-lock.yaml, or yarn.lock

3. Upgrade Vite 5.4.21 → 8.2.2 in @repo/admin (+2 more)

3 major versions behind. Major framework drift increases breaking change risk and blocks access to security fixes and performance improvements.

./apps/admin

Vite: 5.4.21 → 8.2.2 (3 majors behind)

./apps/api

Vitest: 1.6.1 → 4.1.11 (3 majors behind)

./packages/utils

Vitest: 1.6.1 → 4.1.11 (3 majors behind)

Impact: −5–15 drift points

4. Reduce dependency rot in @repo/types (100% severely outdated)

1 of 1 dependencies are 2+ majors behind. Run `npm outdated` and prioritise packages with known CVEs or breaking API changes.

./packages/types

typescript: 5.9.3 → 7.0.2 (2 majors behind)

Impact: −5–10 drift points

5. Reduce dependency rot in @repo/database (75% severely outdated)

3 of 4 dependencies are 2+ majors behind. Run `npm outdated` and prioritise packages with known CVEs or breaking API changes.

./packages/database

@prisma/client: 5.22.0 → 7.10.0 (2 majors behind)

prisma: 5.22.0 → 7.10.0 (2 majors behind)

typescript: 5.9.3 → 7.0.2 (2 majors behind)

Impact: −5–10 drift points

╭──────────────────────────────────────────╮

│ Architecture Layers │

╰──────────────────────────────────────────╯

Archetype: nextjs (80% confidence)

Files classified: 24 (11 unclassified)

Folders classified: 8

apps/admin/src presentation 100% 4 files

apps/admin/src/pages presentation 100% 2 files

apps/api/src/middleware middleware 100% 2 files

apps/api/src/routes routing 100% 2 files

apps/web/src/app presentation 100% 4 files

apps/web/src/app/products presentation 100% 2 files

apps/web/src/app/products/[id] presentation 100% 1 file

packages/ui/src presentation 100% 6 files

Unclassified source (sample): 11

presentation 15 files drift ████████████████████ 100 risk high

routing 4 files drift ████████████████████ 100 risk high

middleware 2 files drift ███████▍░░░░░░░░░░░░ 37 risk moderate

config 2 files drift ░░░░░░░░░░░░░░░░░░░░ 0 risk none

shared 1 file drift ████████████████████ 100 risk high

╭──────────────────────────────────────────╮

│ DriftScore Summary │

╰──────────────────────────────────────────╯

DriftScore: 66/100

Risk Level: HIGH

Projects: 9

Classified: 8 nano · 1 micro · 0 small · 0 standard

Billable: 0.42 · 9 detected → 0.42 billable projects (micro-project pricing)

0.1 micro · 0.32 nano

These fractions add up across repositories, then round down to whole billable projects.

Score Breakdown

Runtime: ████████████████████ 100

Frameworks: █████████▏░░░░░░░░░░ 46

Dependencies: ██████▏░░░░░░░░░░░░░ 31

EOL Risk: ████████████████████ 100

Scanned at 2026-08-26T09:08:28.481Z · 7.1s · 286 files scanned · 56 workspace files · 27 dirs

❯

Press Run to start.

Explore the Vibgrate CLI

Data Format Migration Checklist

When to Use This Checklist

How to Use This Checklist

What Good Looks Like

Common Pitfalls

Related Resources

Tags

See a real scan run