Preskočiť na obsah

Databázové migrácie

Sprievodca správou databázových migrácií pomocou Alembic.

Prehľad

graph LR
    DEV["Vývojár"] --> |alembic revision| NEW["🆕 Nová migrácia"]
    NEW --> |alembic upgrade| DB[("Databáza")]
    DB --> |alembic downgrade| PREV["⏪ Predchádzajúci stav"]

    style NEW fill:#f9f,stroke:#333
    style DB fill:#bbf,stroke:#333

Konfigurácia

alembic.ini

INI
# alembic.ini
[alembic]
script_location = alembic
prepend_sys_path = src

# Database URL z environment variable
sqlalchemy.url = 

[post_write_hooks]
hooks = ruff
ruff.type = exec
ruff.executable = ruff
ruff.options = format REVISION_SCRIPT_FILENAME

env.py

Python
# alembic/env.py
import asyncio
from logging.config import fileConfig

from sqlalchemy import pool
from sqlalchemy.ext.asyncio import async_engine_from_config
from alembic import context

from app.config import settings
from app.models.base import Base

# Importovať všetky modely
from app.models import *  # noqa

config = context.config
config.set_main_option("sqlalchemy.url", settings.DATABASE_URL)

if config.config_file_name is not None:
    fileConfig(config.config_file_name)

target_metadata = Base.metadata


def run_migrations_offline() -> None:
    """Run migrations in 'offline' mode."""
    url = config.get_main_option("sqlalchemy.url")
    context.configure(
        url=url,
        target_metadata=target_metadata,
        literal_binds=True,
        dialect_opts={"paramstyle": "named"},
    )

    with context.begin_transaction():
        context.run_migrations()


def do_run_migrations(connection):
    context.configure(connection=connection, target_metadata=target_metadata)
    with context.begin_transaction():
        context.run_migrations()


async def run_async_migrations():
    connectable = async_engine_from_config(
        config.get_section(config.config_ini_section, {}),
        prefix="sqlalchemy.",
        poolclass=pool.NullPool,
    )

    async with connectable.connect() as connection:
        await connection.run_sync(do_run_migrations)

    await connectable.dispose()


def run_migrations_online() -> None:
    """Run migrations in 'online' mode."""
    asyncio.run(run_async_migrations())


if context.is_offline_mode():
    run_migrations_offline()
else:
    run_migrations_online()

Základné príkazy

Vytvorenie migrácie

Bash
1
2
3
4
5
# Automatická detekcia zmien v modeloch
alembic revision --autogenerate -m "add_team_description"

# Prázdna migrácia (manuálna)
alembic revision -m "custom_data_migration"

Aplikovanie migrácií

Bash
1
2
3
4
5
6
7
8
# Najnovšia verzia
alembic upgrade head

# Konkrétna verzia
alembic upgrade 003_multi_tenant

# Relatívne (2 kroky dopredu)
alembic upgrade +2

Rollback

Bash
1
2
3
4
5
6
7
8
# O jeden krok späť
alembic downgrade -1

# Na konkrétnu verziu
alembic downgrade 002_add_communication

# Na začiatok (pozor!)
alembic downgrade base

Informácie

Bash
1
2
3
4
5
6
7
8
# Aktuálna verzia
alembic current

# História migrácií
alembic history --verbose

# Pending migrácie
alembic heads

Existujúce migrácie

Migrácia Popis
001_core_schema Základné tabuľky (16 tabuliek, courses, teams, users, events)
002_teaching_tools Komunikačné metriky, mrs_template_compliant
003_scale_and_telemetry Scale optimalizácie, pilot telemetria, partitioned webhook tabuľka
004_user_avatar Stĺpec avatar_url v tabuľke users
005_membership_sync Lifecycle stĺpce pre team_members (is_active, left_at, membership_source, last_synced_at)
006_team_member_instructor Stĺpec is_instructor v team_members
007_check_configs JSONB stĺpec check_configs v course_settings
008_custom_checks Tabuľka custom_checks pre dynamické kontroly
009_user_attribution Stĺpce triggered_by_username, created_by_username
010_dynamic_feedback_forms Tabuľka feedback_forms, form_id/responses v pilot_feedback
011_add_missing_indexes Chýbajúce indexy pre výkon
011_bg_job_fk_cascade FK cascade pre background jobs
012_dynamic_docker_ci_config Docker lint a CI konfigurácia v course_settings
013_ci_auto_rollback Auto-rollback konfigurácia CI
014_ci_scenario_resolution Scenáre riešenia CI problémov
015_course_memberships Tabuľka course_memberships pre členstvo v kurzoch
016_fix_weight_and_nullable Oprava váh a nullable stĺpcov
017_hash_webhook_secret_and_cleanup Hashované webhook secrety, cleanup
018_mr_note_type_position Typ a pozícia MR poznámok
019_activity_monitor_indexes Indexy pre activity monitor
020_fix_schema_drift Oprava schema driftu
021_llm_course_settings LLM nastavenia kurzov
022_llm_custom_prompt Vlastné LLM prompty
023_user_role_locked Uzamknutie rolí používateľov
024_member_exclusion Vylúčenie členov z hodnotenia
025_relax_default_checks Uvoľnenie predvolených kontrol
026_add_r13_commit_quality Pravidlo R13 - kvalita commitov
027_secret_rotation_and_user_tokens Rotácia secretov, používateľské tokeny
028_unify_roles_drop_legacy Unifikácia rolí, odstránenie legacy
029_rename_is_instructor_to_is_teacher Premenovanie is_instructor -> is_teacher
030_feedback_links Odkazy na spätnú väzbu
031_support_ticket_enhancements Vylepšenia support ticketov
032_submitted_by Stĺpec submitted_by
033_email_log Logovanie emailov
034_is_mentor Stĺpec is_mentor v team_members
035_semester_end_date Semester-end auto-archive nastavenia
036_skip_auto_archive Manuálny override pre auto-archive
037_webhook_secret_lookup HMAC lookup stĺpce pre webhook secrety
038_relax_branch_pattern_default Uvoľnený predvolený branch pattern
039_lower_min_review_words Znížený default min_review_word_count
040_group_webhook_audit Audit políčka pre group webhook
041_webhook_configurations Multi-scope webhook konfigurácie
042_user_is_bot users.is_bot pre service accounty
043_backfill_email_alias_bots Backfill botov podľa email plus-aliasov
044_source_branch_deleted source_branch_deleted pre merge requests
045_user_tour_seen Per-user stav absolvovaných tour
046_drop_weight_commit_references Odstránenie legacy weight_commit_references

Písanie migrácií

Autogenerated migrácia

Python
# alembic/versions/011_add_user_preferences.py
"""add user preferences

Revision ID: 011_add_user_preferences
Revises: 010_ci_failure_generator
Create Date: 2024-11-15 10:00:00.000000

"""
from typing import Sequence

from alembic import op
import sqlalchemy as sa


# revision identifiers
revision: str = '011_add_user_preferences'
down_revision: str | None = '010_ci_failure_generator'
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None


def upgrade() -> None:
    # ### autogenerated ###
    op.create_table(
        'user_preferences',
        sa.Column('id', sa.Integer(), nullable=False),
        sa.Column('user_id', sa.Integer(), nullable=False),
        sa.Column('theme', sa.String(20), nullable=False, default='light'),
        sa.Column('notifications_enabled', sa.Boolean(), default=True),
        sa.Column('created_at', sa.DateTime(), nullable=False),
        sa.Column('updated_at', sa.DateTime(), nullable=False),
        sa.ForeignKeyConstraint(['user_id'], ['users.id'], ondelete='CASCADE'),
        sa.PrimaryKeyConstraint('id')
    )
    op.create_index('ix_user_preferences_user_id', 'user_preferences', ['user_id'])
    # ### end autogenerated ###


def downgrade() -> None:
    # ### autogenerated ###
    op.drop_index('ix_user_preferences_user_id', 'user_preferences')
    op.drop_table('user_preferences')
    # ### end autogenerated ###

Data migrácia

Python
# alembic/versions/012_populate_default_rubrics.py
"""populate default rubrics

Revision ID: 012_populate_default_rubrics
Revises: 011_add_user_preferences
Create Date: 2024-11-15 11:00:00.000000

"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy.sql import table, column


revision = '012_populate_default_rubrics'
down_revision = '011_add_user_preferences'


def upgrade() -> None:
    # Definícia tabuľky pre insert
    rubrics = table(
        'rubrics',
        column('id', sa.Integer),
        column('name', sa.String),
        column('description', sa.Text),
        column('is_default', sa.Boolean),
    )

    # Batch insert
    op.bulk_insert(rubrics, [
        {
            'id': 1,
            'name': 'Default DevOps Rubric',
            'description': 'Standard rubric for DevOps course',
            'is_default': True,
        },
        {
            'id': 2,
            'name': 'Advanced Rubric',
            'description': 'Rubric with gaming detection',
            'is_default': False,
        },
    ])


def downgrade() -> None:
    op.execute("DELETE FROM rubrics WHERE id IN (1, 2)")

Komplexná migrácia

Python
# alembic/versions/013_split_events_table.py
"""split events table for better performance

Revision ID: 013_split_events_table
Revises: 012_populate_default_rubrics
Create Date: 2024-11-15 12:00:00.000000

"""
from alembic import op
import sqlalchemy as sa


revision = '013_split_events_table'
down_revision = '012_populate_default_rubrics'


def upgrade() -> None:
    # 1. Vytvoriť novú tabuľku
    op.create_table(
        'commit_events',
        sa.Column('id', sa.Integer(), primary_key=True),
        sa.Column('project_id', sa.Integer(), sa.ForeignKey('projects.id')),
        sa.Column('sha', sa.String(40), nullable=False),
        sa.Column('message', sa.Text()),
        sa.Column('author_id', sa.Integer(), sa.ForeignKey('users.id')),
        sa.Column('committed_at', sa.DateTime(), nullable=False),
        sa.Column('lines_added', sa.Integer(), default=0),
        sa.Column('lines_removed', sa.Integer(), default=0),
    )

    # 2. Migrovať dáta
    op.execute("""
        INSERT INTO commit_events (id, project_id, sha, message, author_id, committed_at, lines_added, lines_removed)
        SELECT id, project_id, data->>'sha', data->>'message', author_id, created_at, 
               COALESCE((data->>'additions')::int, 0),
               COALESCE((data->>'deletions')::int, 0)
        FROM events
        WHERE event_type = 'commit'
    """)

    # 3. Vytvoriť index
    op.create_index('ix_commit_events_project_committed', 'commit_events', ['project_id', 'committed_at'])

    # 4. Odstrániť staré dáta (voliteľné)
    # op.execute("DELETE FROM events WHERE event_type = 'commit'")


def downgrade() -> None:
    # Migrovať dáta späť
    op.execute("""
        INSERT INTO events (project_id, event_type, data, author_id, created_at)
        SELECT project_id, 'commit', 
               jsonb_build_object('sha', sha, 'message', message, 'additions', lines_added, 'deletions', lines_removed),
               author_id, committed_at
        FROM commit_events
        WHERE NOT EXISTS (
            SELECT 1 FROM events e WHERE e.id = commit_events.id
        )
    """)

    op.drop_table('commit_events')

Best practices

Odporúčania

  1. Vždy testovať up aj down

    Bash
    1
    2
    3
    alembic upgrade head
    alembic downgrade -1
    alembic upgrade head
    

  2. Atomické zmeny

    Python
    # Jedna migrácia = jedna logická zmena
    # Nie: pridať tabuľku + zmeniť inú + migrovať dáta
    

  3. Záloha pred produkčnou migráciou

    Bash
    pg_dump -Fc gitpulse > backup_before_migration.dump
    alembic upgrade head
    

  4. Review autogenerated kód

    Python
    1
    2
    3
    # Alembic nedetekuje všetko správne
    # - Premenovanie stĺpca -> drop + add
    # - Zmena typu -> môže stratiť dáta
    

Anti-patterns

Python
# 1. Hardcoded dáta v upgrade
def upgrade():
    op.execute("INSERT INTO users VALUES (1, 'admin', 'admin@example.com')")
    # Čo ak user už existuje?

# 2. DROP bez zálohy dát
def upgrade():
    op.drop_column('users', 'legacy_field')  # Dáta stratené navždy

# 3. Chýbajúci downgrade
def downgrade():
    pass  # Nemožné vrátiť späť!

Produkčný deployment

Migrácia v CI/CD

YAML
1
2
3
4
5
6
7
8
# .gitlab-ci.yml
migrate:
  stage: deploy
  script:
    - alembic upgrade head
  only:
    - main
  when: manual  # Manuálne spustenie pre bezpečnosť

Docker entrypoint

Bash
#!/bin/bash
# docker-entrypoint.sh

# Čakať na databázu
while ! pg_isready -h $DB_HOST -p $DB_PORT; do
    echo "Waiting for database..."
    sleep 2
done

# Spustiť migrácie
alembic upgrade head

# Spustiť aplikáciu
exec uvicorn app.main:app --host 0.0.0.0 --port 8000

Troubleshooting

"Target database is not up to date"

Bash
1
2
3
4
5
# Označiť aktuálny stav ako migrated
alembic stamp head

# Alebo špecifická verzia
alembic stamp 010_ci_failure_generator

"Can't locate revision"

Bash
1
2
3
4
5
6
# Zobraziť históriu
alembic history

# Opraviť broken chain
alembic stamp --purge
alembic stamp head

Konfliktné migrácie

Bash
1
2
3
# Ak dva vývojári vytvorili migrácie súčasne:
# 1. Premenovať jednu (zmeniť revision ID)
# 2. Upraviť down_revision reťaz

Ďalšie čítanie