Version: v2

Migrations

Migrating content from a legacy site means Drupal has to read that site's database. Vortex can run it for you as an optional second database, next to the primary one. When you enable it, a database2 Docker service runs alongside the primary database, and a settings.migration.php file registers the $databases['migrate'] connection that Drupal's Migrate API uses by default. A deploy step then imports the legacy dump and runs your migrations in every environment except production.

This page is for developers bringing content from a legacy site into a project built from Vortex. It shows how to turn the migration database on, fetch the legacy dump, and run your migrations during provisioning. It also lists every variable and command involved. For writing the migrations themselves, see the Migrate API overview in the Drupal documentation.

Enabling the migration database

Via the CLI

When you run vortex install, answer Yes at the Use a second database for migrations? prompt. Then choose where the migration dump comes from at the Migration database source prompt. The source defaults to your hosting provider, acquia or lagoon, and to url without one.

Manual setup

To add migration support to an existing Vortex project, copy the following pieces from a fresh Vortex install with migration enabled:

In docker-compose.yml, add the database2 service, the DATABASE2_* environment variables, and the database2 entries in the wait-for-dependencies service.
In .env, add VORTEX_FETCH_DB2_FILE, VORTEX_FETCH_DB2_SOURCE, and the variables for your fetch source, such as VORTEX_FETCH_DB2_URL.
Create web/sites/default/settings.migration.php (see Drupal settings).
In web/sites/default/settings.php, add the block that loads settings.migration.php, between the per-module overrides and the local overrides.
Add the migrate connection to the expected values in tests/phpunit/Drupal/DatabaseSettingsTest.php and tests/phpunit/Drupal/EnvironmentSettingsTest.php, and the DATABASE2_ prefix to tests/phpunit/Drupal/SettingsTestCase.php. Without them, the drupal_settings tests fail on the extra connection.
Add the fetch-db2, reload-db2, and db2 commands to .ahoy.yml.
Require drupal/migrate_plus and drupal/migrate_tools, and add the ys_migrate module with its MigrateContentDeployStep plugin.
Add the "Fetch migration DB" step to the database job and the dump copy to the build job of your CI configuration. On Lagoon, also add the "Download migration database" task to .lagoon.yml.

Confirm that ahoy fetch-db2 saves the dump as .data/db2.sql.

Example workflow

The following steps use the url fetch source on a project with the migration database enabled:

Set VORTEX_FETCH_DB2_URL in .env (or .env.local):

VORTEX_FETCH_DB2_URL=https://example.com/legacy-database.sql

Fetch the migration database:
```
ahoy fetch-db2
```
Build and provision:
```
ahoy build
```
Define the migration in the ys_migrate module, as a migrations/*.yml file or migrate_plus configuration. The MigrateContentDeployStep plugin runs migrate:import --all, so it imports every migration that an enabled module provides.
Re-provision to run migrations:
```
ahoy provision
```

Confirm that ahoy drush migrate:status lists your migration with its imported items.

Environment variables

The following table lists the variables that configure the migration database:

Variable	Default	Description
`VORTEX_FETCH_DB2_FILE`	`db2.sql`	Migration database dump filename
`VORTEX_FETCH_DB2_SOURCE`	`url`	Fetch source (`url`, `ftp`, `acquia`, `lagoon`, `container_registry`, `s3`)
`VORTEX_FETCH_DB2_URL`	None	URL to fetch the migration database dump
`VORTEX_DB2_IMAGE`	None	Migration database container image, for the `database2` service and the `container_registry` source
`DATABASE2_HOST`	`database2`	Migration database host
`DATABASE2_NAME`	`drupal`	Migration database name
`DATABASE2_USERNAME`	`drupal`	Migration database user
`DATABASE2_PASSWORD`	`drupal`	Migration database password
`DATABASE2_PORT`	`3306`	Migration database port

The DATABASE2_* defaults are the values docker-compose.yml sets for the local stack. When a variable isn't set at all, settings.migration.php falls back to localhost for the host and an empty port. It also accepts DATABASE2_DATABASE in place of DATABASE2_NAME.

Each fetch source reads its own settings under the same VORTEX_FETCH_DB2_ prefix, such as VORTEX_FETCH_DB2_FTP_HOST or VORTEX_FETCH_DB2_S3_BUCKET. The CLI adds the ones for the source you pick to .env.

Docker service

By default, the database2 service runs the same uselagoon/mysql-8.4 image as the primary database. Set VORTEX_DB2_IMAGE to run a different image, for example one with the migration database already baked in (a database-in-image seed):

database2:
  image: "${VORTEX_DB2_IMAGE:-uselagoon/mysql-8.4:26.10.0}"
  environment:
    <<: *default-environment
    MYSQL_DATABASE: drupal
    MYSQL_USER: drupal
    MYSQL_PASSWORD: drupal
  <<: *default-user
  ports:
    - '3306'
  labels:
    lagoon.type: mariadb

The image tag tracks the primary database image. Check the shipped docker-compose.yml for the current version.

Drupal settings

Drupal reads the migration database connection from web/sites/default/settings.migration.php. settings.php loads it between the per-module overrides and the local overrides.

`settings.migration.php`

The file registers the migrate connection from the DATABASE2_* variables:

<?php

/**
 * @file
 * Migration settings.
 *
 * @todo Migration. Remove when migration configuration is no longer required.
 */

declare(strict_types=1);

// Migration DB settings.
$databases['migrate']['default'] = [
  'database' => getenv('DATABASE2_NAME') ?: getenv('DATABASE2_DATABASE') ?: 'drupal',
  'username' => getenv('DATABASE2_USERNAME') ?: 'drupal',
  'password' => getenv('DATABASE2_PASSWORD') ?: 'drupal',
  'host' => getenv('DATABASE2_HOST') ?: 'localhost',
  'port' => getenv('DATABASE2_PORT') ?: '',
  'prefix' => '',
  'driver' => 'mysql',
];

Source plugins built on the Migrate API's SqlBase class use key: migrate by default, which resolves to this connection.

Ahoy commands

The migration feature adds the following Ahoy commands:

Command	Description
`ahoy fetch-db2`	Fetch the migration database dump
`ahoy fetch-db2 --fresh`	Force a fresh fetch
`ahoy reload-db2`	Recreate the `database2` container from its image, discarding its data
`ahoy db2`	Open the migration database in Sequel Ace (macOS)

With the default image, ahoy reload-db2 brings the database back empty. The next ahoy provision finds the probe table missing and imports the dump again (see Corruption detection).

ahoy fetch-db2 runs the same vendor/bin/vortex-fetch-db script as ahoy fetch-db, with VORTEX_DB_INDEX=2. The index makes the fetch script and its source scripts read indexed variable names (for example, VORTEX_FETCH_DB2_SOURCE instead of VORTEX_FETCH_DB_SOURCE, and VORTEX_DB2_IMAGE instead of VORTEX_DB_IMAGE). As a result, every fetch source (URL, FTP, Acquia, Lagoon, container registry, and S3) works for the migration database too.

Migration deploy step

The migration runs as the MigrateContentDeployStep plugin in the ys_migrate module (src/Plugin/DeployStep/MigrateContentDeployStep.php). Provisioning installs every *_migrate custom module, and the plugin then runs in the post-update phase of every drush deploy:hook. As a result, it runs during ahoy provision and on every deployment. See Provision for how deploy step plugins work.

The plugin skips itself in these cases:

In the production environment, so migrations never run in production
When DRUPAL_MIGRATION_SKIP is 1
When the migrate_tools module isn't enabled

Otherwise, the plugin does the following, in order:

Imports the dump into the migrate connection, when the import is due or the probe table is missing (see Corruption detection)
Verifies the source database with the probe
Disables the solr search server, on projects with the Solr service
Runs drush migrate:import --all, with the limit and update options from its variables
Enables the solr search server and all search indexes again

migrate:import runs as a Drupal batch that Drush processes in restarting subprocesses, so a large import stays within memory bounds. A repeated run imports only what's missing.

The plugin needs the dump whenever it has to import the source database. Locally, ahoy fetch-db2 downloads it. CI fetches it in the database job (see CI integration), and on Lagoon .lagoon.yml fetches it before the provision step. In any other environment, provide the dump yourself or set DRUPAL_MIGRATION_SKIP=1.

Disabling the search server stops Search API from indexing migrated entities one at a time. The RebuildSearchIndex deploy step in ys_search rebuilds the index straight after. To leave the search server running and index entities as they're migrated, set DRUPAL_MIGRATION_SEARCH_DISABLE=0.

Configuration variables

The plugin reads the following variables. Set them in .env, in .env.local on your machine, or in your hosting provider's environment variables: vortex-provision loads .env and .env.local and passes the values on to Drush.

Variable	Default	Description
`DRUPAL_MIGRATION_SKIP`	`0`	Skip all migration operations
`DRUPAL_MIGRATION_IMPORT_LIMIT`	`50`	Limit entities per migration (`0` for no limit)
`DRUPAL_MIGRATION_UPDATE`	`0`	Update already imported entities
`DRUPAL_MIGRATION_SOURCE_DB_IMPORT`	`$VORTEX_PROVISION_OVERRIDE_DB`	Import source database (`1` to import, `0` to skip)
`DRUPAL_MIGRATION_SEARCH_DISABLE`	`1`	Disable the Solr search server while migrations run (`0` to keep indexing)

Provisioning sets VORTEX_PROVISION_OVERRIDE_DB=1 whenever the primary database is fresh, so by default the source database is imported again along with it. The plugin reads the dump from VORTEX_DB_DIR (default ./.data) and VORTEX_FETCH_DB2_FILE (default db2.sql), where ahoy fetch-db2 saves it.

Adding migrations

Each migration is a YAML definition in a module's migrations directory. The demo one is web/modules/custom/ys_migrate/migrations/ys_migrate_categories.yml. Drupal finds the definitions of enabled modules only. Put your definitions in ys_migrate, which provisioning installs, or in another module that you enable.

The plugin runs migrate:import --all, so a new definition needs no change to the plugin. The per-migration entity limit comes from DRUPAL_MIGRATION_IMPORT_LIMIT, and DRUPAL_MIGRATION_UPDATE=1 adds --update.

Corruption detection

If DRUPAL_MIGRATION_SOURCE_DB_IMPORT isn't 1, the plugin probes the source database for the categories table. If the table is missing (corrupted or empty database), the plugin re-imports the dump file.

Before the migrations run, the plugin probes again, with or without an import. If the table is still missing, it fails with Migration source database is corrupted or empty. A missing dump file stops it too, with a reminder to run ahoy fetch-db2.

The categories probe table belongs to the demo source database, and the plugin sets it in its PROBE_TABLE constant. For a real source, change the constant to a table that source holds, or the plugin fails at this check and stops the provisioning.

Demo migration module

Vortex ships with a demo migration module ys_migrate in web/modules/custom/ys_migrate/. It shows the full migration workflow by migrating categories from the source database into Drupal's tags taxonomy vocabulary.

Migration: `ys_migrate_categories`

The following table describes the demo migration:

Property	Value
Source plugin	`table` from Migrate Plus (reads the `categories` table, keyed by `id`)
Source key	`migrate` (the `$databases['migrate']` connection)
Destination	`entity:taxonomy_term`
Vocabulary	`tags`
Migration tag	`ys_migrate`

The migration maps the following source fields:

name → term name
description → term description (plain text format)

Dependencies

The module depends on deploy_steps, on core's Migrate module, and on drupal/migrate_plus and drupal/migrate_tools, which stay in composer.json when the migration feature is enabled. Migrate Plus provides the table source plugin, and Migrate Tools provides the drush migrate:* commands the plugin runs.

CI integration

Both CI systems (GitHub Actions and CircleCI) include a migration database fetch step in the existing database job. Both database files are cached together in the same .data directory, so the migration dump follows the same daily cache as the primary one.

The build job copies the migration database file into the CLI container alongside the primary database before running provision.

➡️ See Provision for how deploy steps run during provisioning, and Contributed modules for the shipped migration modules.

Enabling the migration database​

Via the CLI​

Manual setup​

Example workflow​

Environment variables​

Docker service​

Drupal settings​

settings.migration.php​

Ahoy commands​

Migration deploy step​

Configuration variables​

Adding migrations​

Corruption detection​

Demo migration module​

Migration: ys_migrate_categories​

Dependencies​

CI integration​