Standardize Database Migrations Using Flyway for All Sztab Releases (SZ-49)
rk@tigase.net opened 1 week ago

Background

Sztab’s database schema is now evolving alongside the application. New features routinely introduce changes to tables, constraints, indexes, and seed data. Until now, schema changes have been handled informally during development, and the risk is growing:

  • Pulling a new branch may require manual DB fixes.
  • Local development environments drift from one another.
  • CI integration tests occasionally fail due to schema mismatches.
  • Release builds require manual coordination to apply schema updates.

As Sztab matures, we need a consistent, deterministic approach to database evolution for both day-to-day development and formal customer releases.

Problem Statement

Sztab currently lacks a unified mechanism to version, apply, and promote database schema changes. Hibernate auto-DDL is unsuitable for production and unreliable for repeatable evolution. Manual SQL execution is error-prone and does not scale across environments (dev => test => staging => prod). Without a migration system, schema drift becomes inevitable.

Proposal

Adopt Flyway as the standard tool for all Sztab database migrations.

Flyway provides:

  • Versioned SQL files checked into source control
  • Deterministic, forward-only schema evolution
  • Automatic migrations on startup in development and CI
  • Controlled migrations through pipelines for customer deployments
  • Portable, readable SQL that fully exposes PostgreSQL features

This approach ensures every environment—developer laptops, automated tests, internal deployments, and external customers—runs the exact same sequence of schema updates.

Scope

This issue proposes adopting Flyway for two use cases:

1. Major Customer Releases

For releases shipped to clients or external environments:

  • Each release includes a set of new migration files (e.g., V104__add_review_requirements.sql).
  • Deploy pipelines apply migrations before or during application rollout.
  • Schema and code remain synchronized across all customer deployments.
  • Release notes list migration versions included in the build.

This provides predictable upgrades and eliminates schema-related outages.

2. Incremental Development Releases

During active team development:

  • Every schema-affecting change requires a corresponding Flyway migration file.
  • Local application startup automatically applies migrations.
  • Switching branches becomes reliable, since Flyway tracks executed versions.
  • CI integration tests always run against the correct schema.
  • Developers avoid manual database modifications or Hibernate auto-DDL.

This keeps everyone on a consistent schema and significantly reduces debugging time.

Implementation Plan

  1. Add Flyway dependency and configuration to the backend module.
  2. Create a src/main/resources/db/migration/ folder in the backend project.
  3. Disable Hibernate auto-DDL in all environments.
  4. Convert current schema definition into an initial baseline migration.
  5. Begin adding new migrations for all subsequent schema changes.
  6. Document migration workflow for feature branches and releases.
  7. Integrate Flyway execution into CI and deployment pipelines.

Expected Outcomes

  • Stable and reproducible database evolution across all environments.
  • Fewer integration test failures related to schema drift.
  • Clear visibility into when and how schema changes occur.
  • Safer customer upgrades and rollouts.
  • Reduced confusion during feature development and branching.
  • rk@tigase.net commented 1 week ago

    Work Log: Flyway Adoption for Sztab Database Migrations

    Scope

    Introduce Flyway as the standard mechanism for versioned PostgreSQL schema migrations across all Sztab environments: developer machines, CI, internal test deployments, and customer releases.

    Estimated Time: 8–10 hours


    Task Breakdown

    1. Add Flyway Dependencies and Configuration (0.5h)

    • Add Flyway starter to backend pom.xml.
    • Configure spring.flyway.* properties for dev/test/prod.
    • Verify Flyway auto-detection of migration folder.

    2. Generate and Normalize Baseline Migration (2h)

    • Extract current PostgreSQL schema using pg_dump (schema-only).
    • Clean and reorder SQL for deterministic Flyway application.
    • Save as V1__baseline.sql under src/main/resources/db/migration/.
    • Validate that the baseline matches existing live schemas.

    3. Convert Existing Bootstrapping SQL (1h)

    • Move any schema creation logic from schema.sql, test scaffolding, or initializer code into structured Flyway migration files.
    • Remove redundant manual schema creation steps.

    4. Disable Hibernate Auto-DDL (0.5h)

    • Set spring.jpa.hibernate.ddl-auto=none across all profiles.
    • Verify application boot without schema-generation warnings.

    5. Update Integration Tests to Use Flyway (1h)

    • Ensure Docker Compose PostgreSQL container is initialized via Flyway.
    • Confirm migrations run before Spring Boot context loads in tests.
    • Fix tests that implicitly relied on auto-DDL behavior.

    6. Validate Full Workflow (2h)

    • Local run: wipe DB, start app, verify migrations applied.
    • CI run: migrations applied before backend tests.
    • Deployment run: verify migrations executed in controlled order.

    7. Write Team Migration Workflow Documentation (1h)

    • How to create VXXX__description.sql migrations.
    • Naming conventions and versioning rules.
    • Branching model: how to handle parallel features requiring schema changes.
    • Guidelines for SQL quality (idempotency, constraints, index naming, etc.).

    Expected Outcomes

    • Full elimination of schema drift during development.
    • Deterministic, version-controlled DB evolution across environments.
    • Cleaner CI runs with reproducible schema setup.
    • Predictable customer upgrades with packaged migration sets.
    • Removal of Hibernate auto-DDL reliance.

    Notes

    • Migration workflow must become a mandatory step for all schema-affecting changes.
    • Future releases will include corresponding Flyway version ranges in release notes.
    • Team should update integration test harness to ensure Flyway is always executed cleanly.
  • rk@tigase.net changed state to 'In Progress' 6 days ago
    Previous Value Current Value
    Open
    In Progress
issue 1 of 1
Type
New Feature
Priority
Normal
Assignee
Version
1.0
Sprints
n/a
Customer
n/a
Issue Votes (0)
Watchers (3)
Reference
SZ-49
Please wait...
Page is in error, reload to recover