About Data Migration Simulator

Simulate database migration scenarios with data transformation and conflict detection. Part of the DevTools Surf developer suite. Browse more tools in the Developer Utilities collection.

Use Cases

• Plan and test a database schema change before running in production
• Simulate data transformation during a platform migration (e.g., MySQL to PostgreSQL)
• Identify data conflicts or integrity violations in migration scripts
• Estimate migration duration based on table size and transformation complexity

Tips

• Define both source and target schema versions before running the simulation — the conflict detector compares field mappings and data type compatibility
• Use the 'batch size' setting to simulate migrating large tables incrementally — batch migration reduces lock contention versus single-transaction migrations
• The rollback simulation shows the steps needed to reverse the migration if validation fails — plan this before starting any production migration

Fun Facts

• Database migrations are one of the most failure-prone operations in software engineering. A 2022 survey by Flyway found that 41% of organizations had experienced a failed migration in production in the past 12 months, with an average downtime of 4.5 hours.
• The 'expand-contract' pattern (also called 'parallel change') was developed to enable zero-downtime schema migrations: add the new column/table, deploy code that writes to both old and new, migrate data, deploy code that reads from new only, then drop old. It avoids locking and failed rollbacks.
• Flyway and Liquibase, the two most popular database migration tools, both maintain a migration version history table in the target database. This table is the source of truth for which migrations have been applied — similar to how Git tracks commits.

FAQ

What is a zero-downtime database migration?

A migration that runs while the application serves live traffic. Achieved by: making changes backwards-compatible (add columns with defaults, don't drop columns immediately), using the expand-contract pattern, and applying changes in multiple deploys.

What is the difference between a schema migration and a data migration?

Schema migration changes the database structure (add/remove columns, change types, add indexes). Data migration moves or transforms actual data (backfill new columns, normalize records, dedup entries). Complex changes require both in a coordinated sequence.

How do I handle large table migrations (>100M rows)?

Use batched background jobs rather than a single transaction. Process 1,000-10,000 rows at a time with delays between batches to avoid locking and I/O spikes. Tools like pt-online-schema-change (Percona) and gh-ost (GitHub) automate this for MySQL.