Skip to content

Commit

Permalink
Merge pull request #16 from axelerant/ddev-support
Browse files Browse the repository at this point in the history
Add support for DDEV
  • Loading branch information
hussainweb authored May 27, 2021
2 parents 5137a92 + 00ed16a commit e28a082
Show file tree
Hide file tree
Showing 8 changed files with 271 additions and 14 deletions.
33 changes: 29 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ The plugin also supports configuration via composer.json `extra` section.
"docker-image-name": "auto",
"docker-tag": "auto",
"docker-base": {
"base-flavor": "bitnami",
"image": "bitnami/mariadb:10.4",
"user": "drupal8",
"password": "drupal8",
Expand All @@ -117,6 +118,28 @@ You can use the `docker-base` configuration to specify the base image to use to

The base image details cannot be customized from the command line. They must be specified via the `composer.json`'s extra section.

### DDEV base image

The defaults for `docker-base` depend on the value of `base-flavor` setting. By default, the `base-flavor` is set to `bitnami` and the defaults are set as above. However, if `base-flavor` is `ddev`, the defaults change to the following:

```json
{
"extra": {
"dbdocker": {
"docker-base": {
"base-flavor": "ddev",
"image": "drud/ddev-dbserver-mariadb-10.4:v1.17.0",
"user": "db",
"password": "db",
"database": "db"
}
}
}
}
```

These are the defaults required for building database images compatible for DDEV.

## Default options

The plugin tries to guess most values for input to correctly select the source, build the image, and push it.
Expand All @@ -135,13 +158,15 @@ The image tag, unless specified with the `--docker-tag` option, is assumed to be

### Determining the database source

Three database sources are supported: `file`, `lando`, and `drush`. The source can be explicitly specified using the `--db-source` option. If not specified, the following rules are used to determine the source.
These database sources are supported: `file`, `lando`, `ddev`, and `drush`. The source can be explicitly specified using the `--db-source` option. If not specified, the following rules are used to determine the source.
* If the `--db-file` option is present, then the source is set as `file`.
* If a file called `.lando.yml` is present, then the source is set as `lando`.
* As an exception to above, the plugin attempts to detect if it is running inside a lando container. If so, the source is set to `drush`.
* If the above two conditions fail, then the source is assumed to be `drush`.
* As an exception to above, the plugin attempts to detect if it is running inside a lando container. If so, the source is set to `drush`.
* If a directory called `.ddev` is present, then the source is set as `ddev`.
* As an exception to above, the plugin attempts to detect if it is running inside a DDEV container. If so, the source is set to `drush`.
* If the above conditions fail, then the source is assumed to be `drush`.

In case the source is `lando` or `drush`, the `drush sql:dump` command is used to obtain the SQL file. If the source is `lando`, then the drush command is executed inside of the Lando container like so: `lando drush ...`.
In case the source is `lando`, `ddev`, or `drush`, the `drush sql:dump` command is used to obtain the SQL file. If the source is `lando` or `ddev`, then the drush command is executed inside the relevant container like so: `lando drush ...` or `ddev drush ...`.

## Reporting problems

Expand Down
25 changes: 25 additions & 0 deletions assets/dockerize-db-ddev/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# This image is used to copy and install the databases.
ARG BASE_IMAGE=drud/ddev-dbserver-mariadb-10.4:v1.17.0
FROM ${BASE_IMAGE} AS build
ARG BASE_IMAGE_USER
ARG BASE_IMAGE_PASSWORD
ARG BASE_IMAGE_DATABASE

ENV ALLOW_EMPTY_PASSWORD=yes
ENV MARIADB_USER=${BASE_IMAGE_USER:-db}
ENV MARIADB_PASSWORD=${BASE_IMAGE_PASSWORD:-db}
ENV MARIADB_DATABASE=${BASE_IMAGE_DATABASE:-db}

COPY create_init_db.sh /

COPY dumps/ /docker-entrypoint-initdb.d/
COPY zzzz-truncate-caches.sql /docker-entrypoint-initdb.d/

RUN /create_init_db.sh
RUN rm -rf /var/lib/mysql/*
RUN chmod -R ugo+rw /mysqlbase && find /mysqlbase -type d | xargs chmod ugo+rwx

# This image is used to copy the installed databases and configure MySQL.
FROM ${BASE_IMAGE}

COPY --from=build /mysqlbase /mysqlbase
3 changes: 3 additions & 0 deletions assets/dockerize-db-ddev/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Dockerize your Database

These files are used in building an image from a SQL file. This was originally started in a separate internal repository but has been moved here because it is simpler to maintain and we don't need an additional dependency.
99 changes: 99 additions & 0 deletions assets/dockerize-db-ddev/create_init_db.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
#!/bin/bash

set -e
set -o pipefail

SOCKET=/var/tmp/mysql.sock
OUTDIR=/mysqlbase

mkdir -p ${OUTDIR}
chown -R "$(id -u):$(id -g)" $OUTDIR

chmod ugo+w /var/tmp
mkdir -p /var/lib/mysql /mnt/ddev_config/mysql && rm -f /var/lib/mysql/* && chmod -R ugo+w /var/lib/mysql

echo 'Initializing mysql'
mysqld --version
mysqld_version=$(mysqld --version | awk '{ gsub(/-log/, ""); gsub(/\.[0-9]+$/, "", $3); print $3}')
echo version=$mysqld_version
# Oracle mysql 5.7+ deprecates mysql_install_db
if [ "${mysqld_version}" = "5.7" ] || [ "${mysqld_version%%%.*}" = "8.0" ]; then
mysqld --initialize-insecure --datadir=/var/lib/mysql --server-id=0
else
# mysql 5.5 requires running mysql_install_db in /usr/local/mysql
if command -v mysqld | grep usr.local; then
cd /usr/local/mysql
fi
mysql_install_db --force --datadir=/var/lib/mysql
fi
echo "Starting mysqld --skip-networking --socket=${SOCKET}"
mysqld --user=root --socket=$SOCKET --innodb_log_file_size=48M --skip-networking --datadir=/var/lib/mysql --server-id=0 --skip-log-bin &
pid="$!"

# Wait for the server to respond to mysqladmin ping, or fail if it never does,
# or if the process dies.
for i in {90..0}; do
if mysqladmin ping -uroot --socket=$SOCKET 2>/dev/null; then
break
fi
# Test to make sure we got it started in the first place. kill -s 0 just tests to see if process exists.
if ! kill -s 0 $pid 2>/dev/null; then
echo "MariaDB initialization startup failed"
exit 3
fi
sleep 1
done
if [ "$i" -eq 0 ]; then
echo 'MariaDB initialization startup process timed out.'
exit 4
fi


mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -uroot mysql

mysql -uroot <<EOF
CREATE DATABASE IF NOT EXISTS $MYSQL_DATABASE;
CREATE USER '$MYSQL_USER'@'%' IDENTIFIED BY '$MYSQL_PASSWORD';
CREATE USER '$MYSQL_USER'@'localhost' IDENTIFIED BY '$MYSQL_PASSWORD';
GRANT ALL ON $MYSQL_DATABASE.* TO '$MYSQL_USER'@'%';
GRANT ALL ON $MYSQL_DATABASE.* TO '$MYSQL_USER'@'localhost';
CREATE USER 'root'@'%' IDENTIFIED BY '$MYSQL_ROOT_PASSWORD';
GRANT ALL ON *.* TO 'root'@'%' WITH GRANT OPTION;
FLUSH PRIVILEGES;
FLUSH TABLES;
EOF

mysqladmin -uroot password root

if [ "${mysqld_version%%%.*}" = "8.0" ]; then
mysql -uroot -proot <<EOF
ALTER USER 'db'@'%' IDENTIFIED WITH mysql_native_password BY '$MYSQL_PASSWORD';
ALTER USER 'root'@'%' IDENTIFIED WITH mysql_native_password BY '$MYSQL_ROOT_PASSWORD';
ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY '$MYSQL_ROOT_PASSWORD';
EOF
fi

set +u
source /usr/local/bin/docker-entrypoint.sh
docker_process_init_files /docker-entrypoint-initdb.d/*
set -u


rm -rf $OUTDIR/*

backuptool=mariabackup
if command -v xtrabackup; then backuptool="xtrabackup --datadir=/var/lib/mysql"; fi
${backuptool} --backup --target-dir=$OUTDIR --user=root --password=root --socket=$SOCKET

# Initialize with current mariadb_version
my_mariadb_version=$(PATH=$PATH:/usr/sbin:/usr/local/bin:/usr/local/mysql/bin mysqld -V 2>/dev/null | awk '{sub( /\.[0-9]+(-.*)?$/, "", $3); print $3 }')
echo $my_mariadb_version >$OUTDIR/db_mariadb_version.txt

if ! kill -s TERM "$pid" || ! wait "$pid"; then
echo >&2 'Database initialization process failed.'
exit 5
fi

echo "The startup database files (in mariabackup/xtradb format) are now in $OUTDIR"
44 changes: 44 additions & 0 deletions assets/dockerize-db-ddev/zzzz-truncate-caches.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
-- From https://stackoverflow.com/a/52370521/124844

DROP PROCEDURE IF EXISTS truncate_tables;

DELIMITER $$
CREATE PROCEDURE truncate_tables()
BEGIN
DECLARE tblName CHAR(64);
DECLARE done INT DEFAULT FALSE;
DECLARE dbTables CURSOR FOR
SELECT table_name
FROM information_schema.tables
WHERE table_schema = (SELECT DATABASE()) AND
(table_name LIKE 'cache%' OR
table_name LIKE 'search_%' OR
table_name LIKE 'old_%' OR
table_name IN ('flood', 'batch', 'queue', 'sessions', 'semaphore'));
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;

OPEN dbTables;
SET FOREIGN_KEY_CHECKS = 0;

read_loop: LOOP
FETCH dbTables INTO tblName;
IF done THEN
LEAVE read_loop;
END IF;

PREPARE stmt FROM CONCAT('TRUNCATE ', tblName);
EXECUTE stmt;
DEALLOCATE PREPARE stmt;

PREPARE stmt FROM CONCAT('OPTIMIZE TABLE ', tblName);
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END LOOP read_loop;

CLOSE dbTables;
SET FOREIGN_KEY_CHECKS = 1;
END
$$

CALL truncate_tables();
DROP PROCEDURE IF EXISTS truncate_tables;
33 changes: 29 additions & 4 deletions src/DbDockerCommand.php
Original file line number Diff line number Diff line change
Expand Up @@ -164,8 +164,9 @@ protected function getImageNameFromRepoUrl(string $url): string
protected function getDbFile(): string
{
$src = $this->options->getDbSource() ?: $this->guessSource();
if ($src != 'lando' && $src != 'drush' && $src != 'file') {
throw new InvalidOptionException("db-source can only be 'lando', 'drush', or 'file'");
$validOptions = ['lando', 'ddev', 'drush', 'file'];
if (!in_array($src, $validOptions)) {
throw new InvalidOptionException("db-source can only be one of 'lando', 'ddev', 'drush', or 'file'");
}

$this->output->writeln("<info>Getting SQL file from source '{$src}'</info>");
Expand All @@ -183,6 +184,9 @@ protected function getDbFile(): string
if ($src == 'lando') {
$drushCmd = 'lando ' . $drushCmd;
}
if ($src == 'ddev') {
$drushCmd = 'ddev ' . $drushCmd;
}

$this->execCmd($drushCmd);
return $sqlFileName;
Expand All @@ -199,6 +203,12 @@ protected function guessSource(): string
// If we are running inside Lando, just use 'drush'.
return getenv('LANDO') == 'ON' ? 'drush' : 'lando';
}
// Similarly, if there is a directory called `.ddev`, we are probably
// running ddev.
if (file_exists('.ddev')) {
// If we are running inside DDEV, just use 'drush'.
return getenv('IS_DDEV_PROJECT') ? 'drush' : 'ddev';
}

return 'drush';
}
Expand All @@ -216,7 +226,14 @@ protected function verifyBaseImage(array $baseImageDetails): void
}

$imageName = $matches[1];
$allowedImages = ['bitnami/mariadb', 'mariadb'];
$allowedImages = [
'bitnami/mariadb',
'mariadb',
'drud/ddev-dbserver-mariadb-10.2',
'drud/ddev-dbserver-mariadb-10.3',
'drud/ddev-dbserver-mariadb-10.4',
'drud/ddev-dbserver-mariadb-10.5',
];
if (!in_array(strtolower($imageName), $allowedImages)) {
$this->output->writeln(
"<comment>Cannot recognize image name '{$imageName}'. Use at your own risk.</comment>"
Expand All @@ -235,14 +252,22 @@ protected function buildImage(string $imageId, string $sqlFile, array $baseDetai
{
$tempDir = realpath(sys_get_temp_dir());
$tempPath = sprintf('%s%s%s', $tempDir, DIRECTORY_SEPARATOR, sha1(uniqid())) . '/';
$assetPath = realpath(__DIR__ . '/../assets/dockerize-db') . '/';

$dockerfileName = $baseDetails['base-flavor'] == 'ddev' ? 'dockerize-db-ddev' : 'dockerize-db';
$assetPath = realpath(__DIR__ . '/../assets/' . $dockerfileName) . '/';

mkdir($tempPath);
mkdir($tempPath . 'dumps');
copy($assetPath . 'Dockerfile', $tempPath . 'Dockerfile');
copy($sqlFile, $tempPath . "/dumps/db.sql");
copy($assetPath . "zzzz-truncate-caches.sql", $tempPath . "zzzz-truncate-caches.sql");

// Little hacky but this works for now
if (file_exists($assetPath . 'create_init_db.sh')) {
copy($assetPath . 'create_init_db.sh', $tempPath . 'create_init_db.sh');
chmod($tempPath . 'create_init_db.sh', 0755);
}

$this->output->writeln("<info>Building image '{$imageId}'</info>");
$dockerCmd = ['docker', 'build', '-t', $imageId];
$dockerCmd[] = '--build-arg';
Expand Down
14 changes: 13 additions & 1 deletion src/OptionsProvider.php
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,24 @@ public function __construct(InputInterface $input, RootPackageInterface $package
'db-source' => '',
'no-push' => false,
];
$this->packageConfig['docker-base'] += [

$baseImageDetailsDefault = [
'base-flavor' => 'bitnami',
'image' => 'bitnami/mariadb:10.4',
'user' => 'drupal8',
'password' => 'drupal8',
'database' => 'drupal8',
];
$baseFlavor = $this->packageConfig['docker-base']['base-flavor'] ?? 'default';
if ($baseFlavor == 'ddev') {
$baseImageDetailsDefault = [
'image' => 'drud/ddev-dbserver-mariadb-10.4:v1.17.0',
'user' => 'db',
'password' => 'db',
'database' => 'db',
];
}
$this->packageConfig['docker-base'] += $baseImageDetailsDefault;
}

public function getDockerTag(): string
Expand Down
34 changes: 29 additions & 5 deletions tests/OptionsProviderTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ public function dataOptions(): array
'docker-image-name' => 'auto',
'docker-tag' => 'auto',
'docker-base' => [
'base-flavor' => 'bitnami',
'image' => 'bitnami/mariadb:10.4',
'user' => 'drupal8',
'password' => 'drupal8',
Expand Down Expand Up @@ -178,15 +179,38 @@ public function dataOptions(): array
'db-source' => 'lando',
],
'expected' => [
'docker-tag' => 'latest',
'docker-base' => [
'image' => 'mariadb:latest',
'user' => 'drupal8',
'password' => 'drupal8',
'database' => 'drupal8',
],
'db-source' => 'lando',
] + $defaultExpected,
];

// Partial configuration with DDEV base flavor.
$cases[] = [
'input' => $defaultInput,
'package' => [
'docker-tag' => 'latest',
'docker-base' => [
'image' => 'mariadb:latest',
'user' => 'drupal8',
'password' => 'drupal8',
'database' => 'drupal8',
'base-flavor' => 'ddev',
],
'db-source' => 'lando',
] + $defaultExpected,
],
'expected' => [
'docker-tag' => 'latest',
'docker-base' => [
'base-flavor' => 'ddev',
'image' => 'drud/ddev-dbserver-mariadb-10.4:v1.17.0',
'user' => 'db',
'password' => 'db',
'database' => 'db',
],
'db-source' => 'lando',
] + $defaultExpected,
];

return $cases;
Expand Down

0 comments on commit e28a082

Please sign in to comment.