Multisite setup enhancements and bugfixes (#2997)

* Don't use drush to validate mysql connection.

* PHPCS cleanup.

* Fix localhost permissions for drupal user.

* Grant wildcard privs to default database.

* Dynamically update multisite database array, generate dependent drush and blt site config.

* Dynamically populate sites.php config for local multisite hosts from available blt.multisites values.

* Simplify acsf_site_name mapping to local db.

* Update multisite docs.

* Fix failing test due to incorrect drush alias.

* Fixing a typo in privileges that caused the new user to not have expected access.

docs/multisite.md

@@ -6,45 +6,20 @@ There are two parts to setting up a multisite instance on BLT: the local setup a
 
 1. Set up a single site on BLT, following the standard instructions, and ssh to the vm (`vagrant ssh`).
 1. Run `blt recipes:multisite:init`.
-    
+
     Running `blt recipes:multisite:init`...
-    
+
     * Sets up new a directory in your docroot/sites directory with the multisite name given with all the necessary files and subdirectories.
     * Sets up a new drush alias.
-    * Sets up a new vhost in the box/config.yml file. 
-
-    Running `blt recipes:multisite:init` currently **does not**...
-
-    * Set up a new MySQL user in the box/config.yml file.
-    * Add a multisite array to your blt/blt.yml file.
-    * Set up a sites.php file.
-    * Update the new site's database credentials.
-
-    Most likely you will want to do all these steps. Details for how to complete them are below. 
+    * Sets up a new vhost in the box/config.yml file.
+    * Grants necessary permissions to the MySQL user in the box/config.yml file.
+    * Generates sites.php file.
+    * Updates the new site's database credentials.
+    * Updates the new site's local drush configuration.
 
 1. If desired override any blt settings in the `docroot/sites/{newsite}/blt.yml` file.
 1. Once you've completed the above and any relevant manual steps, exit out of your virtual machine environment and update with the new configuration using `vagrant provision`.
 
-### Optional local setup steps
-
-#### Add a new MySQL user to the `box/config.yml` file.
-
-Edit your `box/config.yml` file and add a new MySQL user block in the existing `mysql_users` section. If your original database user was named 'drupal' (the BLT default) and during the `multisite:recipe:init` process you told it to use `newsite` for the password, user, and database of your new site, the completed mysql_users block would look like:
-
-```
-mysql_users:
-    -
-        name: drupal
-        host: '%'
-        password: drupal
-        priv: 'drupal.*:ALL'
-    -
-        name: newsite
-        host: '%'
-        password: newsite
-        priv: 'newsite.*:ALL'
-```
-
 
 #### Add a multisite array to `blt/blt.yml`
 
@@ -60,26 +35,21 @@ At this point you should have a functional multisite codebase that can be instal
 
 #### Set up a sites.php file.
 
-Creating a sites.php file in `docroot/sites/` allows your Drupal instance to direct incoming HTTP requests to the appropriate site. 
-
-Note that if you name your sites according to their domain names, and use a canonical approach to subdomains (local.example.com, dev.example.com, example.com), you don't need to modify sites.php at all--but the file does need to exist, even if it's empty.
+BLT creates a sites.php file in `docroot/sites/` to allow your Drupal instance to direct incoming HTTP requests to the appropriate site.
 
-Drupal core provides an `example.sites.php` file which can be copied, renamed, and modified as needed.
+This file must exist to install Drupal on a multisite in Drush 9. Drupal core provides an `example.sites.php` file which can be copied, renamed, and modified as needed. When running the `blt init:settings` task, BLT maps your local multisite canonical domains to their respective site directory in `docroot/sites/[sitename]`. If this file is blank or the `$sites[]` array does not map to a valid directory, then Drush and BLT will use the values in the default site at `docroot/sites/default`. This will likely also cause issues with multisite drush aliases using the incorrect site uri and database credentials.
 
-#### Update the new site's database credentials
-
-BLT does not currently set the new site's local database credentials in the `docroot/sites/{newsite}/settings/local.settings.php` file. To ensure your new site connects to the correct database, you'll need to edit these yourself.
 
 #### Override BLT variables in `docroot/sites/{newsite}/blt.yml`
 
 You may override BLT variables on a per-site basis by editing the `blt.yml` file in `docroot/sites/{newsite}/`. You may then run BLT with the `site` variable set at the command line to load the site's properties.
 
-For instance, if the `drush` aliases for your site in `docroot/sites/mysite` were `@mysite.local` and `@mysite.test`, you could define these in `docroot/sites/mysite/blt.yml` as:
+For instance, the `drush` aliases for your site in `docroot/sites/mysite` were `@mysite.local` and `@mysite.test`, you could define these in `docroot/sites/mysite/blt.yml` as:
 
 ```yaml
 drush:
   aliases:
-    local: mysite.local
+    local: self
     remote: mysite.test
 ```
 
@@ -100,6 +70,8 @@ Start by following the [Acquia Cloud multisite instructions](https://docs.acquia
 
 ### Drush aliases
 
-The default Drush site aliases provided by [Acquia Cloud](https://docs.acquia.com/acquia-cloud/drush/aliases) and [Club](https://github.com/acquia/club#usage) are not currently multisite-aware. They will connect to the first ("default") site / database on the subscription by default. You will need to create your own Drush aliases for each site.
+The default Drush site aliases provided by [Acquia Cloud](https://docs.acquia.com/acquia-cloud/drush/aliases) are not currently multisite-aware. They will connect to the first ("default") site / database on the subscription by default. You will need to create your own Drush aliases for each site. It's recommended to copy the alias file provided by the `blt aliases` command for each Acquia CLoud multisite into a separate alias file for each site. Simply modify the `uri` and `parent` keys for the aliases within each file to match the correct database / site to the Acquia Cloud environment.
+
+*Note that the aliases downloaded from Acquia Cloud through the link on your user's Profile > Credenditials > Drush integration page or through the Drush 8 `drush acquia-update` command are not supported on Acquia Cloud Site Factory subscriptions* BLT currently generates drush aliases for each of your Acquia Cloud Site Factory sites with the `blt aliases` command.
+
 
-It's recommended to copy the aliases file provided by Acquia Cloud or Club to create a separate aliases file for each site. Simply modify the `uri` and `parent` keys for the aliases within each file to match the correct database / site.

scripts/drupal-vm/config.yml

@@ -60,7 +60,12 @@ mysql_users:
   - name: drupal
     host: "%"
     password: drupal
-    priv: "drupal%.*:ALL"
+    priv: "*.*:ALL"
+  - name: drupal
+    host: "localhost"
+    password: drupal
+    priv: "*.*:ALL"
+
 
 # Set this to 'false' if you don't need to install drupal (using the drupal_*
 # settings below), but instead copy down a database (e.g., using drush sql-sync).

settings/default.local.settings.php

@@ -7,9 +7,10 @@
 
 use Drupal\Component\Assertion\Handle;
 
+// The local db name must match the site_name key for split mapping.
 $db_name = '${drupal.db.database}';
 if (isset($acsf_site_name)) {
-  $db_name .= '_' . $acsf_site_name;
+  $db_name = $acsf_site_name;
 }
 
 /**
@@ -108,10 +109,10 @@
 /**
  * Configure static caches.
  *
- * Note: you should test with the config, bootstrap, and discovery caches enabled to 
+ * Note: you should test with the config, bootstrap, and discovery caches enabled to
  * test that metadata is cached as expected. However, in the early stages of development,
- * you may want to disable them. Overrides to these bins must be explicitly set for each 
- * bin to change the default configuration provided by Drupal core in core.services.yml. 
+ * you may want to disable them. Overrides to these bins must be explicitly set for each
+ * bin to change the default configuration provided by Drupal core in core.services.yml.
  * See https://www.drupal.org/node/2754947
  */
 

src/Robo/Commands/Generate/MultisiteCommand.php

@@ -9,6 +9,7 @@
 use Robo\Contract\VerbosityThresholdInterface;
 use Symfony\Component\Console\Input\InputOption;
 use Symfony\Component\Yaml\Yaml;
+use Symfony\Component\Finder\Finder;
 
 /**
  * Defines commands in the "recipes:multisite:init" namespace.
@@ -37,7 +38,7 @@ public function generate($options = [
       throw new BltException("Cannot generate new multisite, $new_site_dir already exists!");
     }
 
-    $domain = $this->getNewSiteDoman($options, $site_name);
+    $domain = $this->getNewSiteDomain($options, $site_name);
     $url = parse_url($domain);
     // @todo Validate uri, ensure includes scheme.
 
@@ -51,7 +52,7 @@ public function generate($options = [
     $this->createNewSiteDir($default_site_dir, $new_site_dir);
 
     $remote_alias = $this->getNewSiteRemoteAlias($site_name, $options);
-    $this->createNewBltSiteYml($new_site_dir, $site_name, $url, $remote_alias);
+    $this->createNewBltSiteYml($new_site_dir, $site_name, $url, $remote_alias, $newDBSettings);
     $this->createNewSiteConfigDir($site_name);
     $this->createSiteDrushAlias($site_name);
     $this->resetMultisiteConfig();
@@ -102,7 +103,7 @@ protected function configureDrupalVm($url, $site_name, $newDBSettings) {
           'name' => $newDBSettings['username'],
           'host' => '%',
           'password' => $newDBSettings['password'],
-          'priv' => $newDBSettings['database'] . '*:ALL',
+          'priv' => $newDBSettings['database'] . '.*:ALL',
         ];
       }
       file_put_contents($this->projectDrupalVmConfigFile,
@@ -180,15 +181,18 @@ protected function createNewBltSiteYml(
     $new_site_dir,
     $site_name,
     $url,
-    $remote_alias
+    $remote_alias,
+    $newDBSettings
   ) {
     $site_yml_filename = $new_site_dir . '/blt.yml';
     $site_yml['project']['machine_name'] = $site_name;
     $site_yml['project']['human_name'] = $site_name;
     $site_yml['project']['local']['protocol'] = $url['scheme'];
     $site_yml['project']['local']['hostname'] = $url['host'];
-    $site_yml['drush']['aliases']['local'] = $site_name . ".local";
+    $site_yml['drush']['aliases']['local'] = "self";
     $site_yml['drush']['aliases']['remote'] = $remote_alias;
+    $site_yml['drupal']['db'] = $newDBSettings;
+    $site_yml['project']['local']['hostname'] = $url['host'];
     YamlMunge::mergeArrayIntoFile($site_yml, $site_yml_filename);
   }
 
@@ -199,14 +203,36 @@ protected function createNewBltSiteYml(
    * @throws \Acquia\Blt\Robo\Exceptions\BltException
    */
   protected function createNewSiteDir($default_site_dir, $new_site_dir) {
+
     $result = $this->taskCopyDir([
       $default_site_dir => $new_site_dir,
     ])
+      ->exclude(array('files'))
       ->setVerbosityThreshold(VerbosityThresholdInterface::VERBOSITY_VERBOSE)
       ->run();
+
     if (!$result->wasSuccessful()) {
       throw new BltException("Unable to create $new_site_dir.");
     }
+
+    $taskFilesystemStack = $this->taskFilesystemStack()
+      ->setVerbosityThreshold(VerbosityThresholdInterface::VERBOSITY_VERBOSE);
+    $finder = new Finder();
+    $files = $finder
+      ->in($new_site_dir)
+      ->files()
+      ->name('local.*');
+
+    foreach ($files->getIterator() as $item) {
+      $taskFilesystemStack->remove($item->getRealPath());
+    }
+
+    $result = $taskFilesystemStack->run();
+
+    if (!$result->wasSuccessful()) {
+      throw new BltException("Unable to remove default site configuration.");
+    }
+
   }
 
   /**
@@ -236,7 +262,7 @@ protected function resetMultisiteConfig() {
    *
    * @return string
    */
-  protected function getNewSiteDoman($options, $site_name) {
+  protected function getNewSiteDomain($options, $site_name) {
     if (empty($options['site-uri'])) {
       $domain = $this->askDefault("Local domain name",
         "http://local.$site_name.com");

src/Robo/Commands/Setup/BuildCommand.php

@@ -22,7 +22,6 @@ class BuildCommand extends BltTasks {
    * @interactGenerateSettingsFiles
    *
    * @validateDrushConfig
-   * @validateMySqlAvailable
    * @validateDocrootIsPresent
    * @executeInVm
    *

src/Robo/Commands/Setup/DrupalCommand.php

@@ -16,7 +16,6 @@ class DrupalCommand extends BltTasks {
    *
    * @command internal:drupal:install
    *
-   * @validateMySqlAvailable
    * @validateDrushConfig
    * @hidden
    *
@@ -26,6 +25,19 @@ class DrupalCommand extends BltTasks {
    */
   public function install() {
 
+    $status = $this->getInspector()->getStatus();
+    $connection = @mysqli_connect(
+        $status['db-hostname'],
+        $status['db-username'],
+        $status['db-password'],
+        '',
+        $status['db-port']
+      );
+    if (!$connection) {
+      throw new BltException("Unable to connect to database.");
+    }
+    $connection->query('CREATE DATABASE IF NOT EXISTS ' . $status['db-name']);
+
     // Generate a random, valid username.
     // @see \Drupal\user\Plugin\Validation\Constraint\UserNameConstraintValidator
     $username = RandomString::string(10, FALSE,

src/Robo/Commands/Setup/SettingsCommand.php

@@ -59,6 +59,10 @@ public function generateSiteConfigFiles() {
     $initial_site = $this->getConfigValue('site');
     $current_site = $initial_site;
 
+    // Accretion variable for holding the list of all the defined multisites.
+    // Used so that we can gather all the info to write the sites.php file.
+    $sites = [];
+
     foreach ($multisites as $multisite) {
       if ($current_site != $multisite) {
         $this->switchSiteContext($multisite);
@@ -80,6 +84,10 @@ public function generateSiteConfigFiles() {
       $default_local_drush_file = "$multisite_dir/default.local.drush.yml";
       $project_local_drush_file = "$multisite_dir/local.drush.yml";
 
+      // Populate the accretion variable.
+      $site_local_hostname = $this->getConfigValue('project.local.hostname');
+      $sites[$site_local_hostname] = $multisite;
+
       $copy_map = [
         $blt_local_settings_file => $default_local_settings_file,
         $default_local_settings_file => $project_local_settings_file,
@@ -149,6 +157,10 @@ public function generateSiteConfigFiles() {
       }
     }
 
+    // Generate sites.php for local multisite.
+    $contents = "<?php\n \$sites = " . var_export($sites, TRUE) . ";";
+    file_put_contents($this->getConfigValue('docroot') . "/sites/sites.php", $contents);
+
     if ($current_site != $initial_site) {
       $this->switchSiteContext($initial_site);
     }

tests/phpunit/BltProject/MultiSiteTest.php

@@ -42,7 +42,7 @@ public function testMultisiteGenerate() {
     $this->assertEquals("$this->site1Dir.clone", $site1_blt_yml['drush']['aliases']['remote']);
 
     $site2_blt_yml = YamlMunge::parseFile("$this->sandboxInstance/docroot/sites/$this->site2Dir/blt.yml");
-    $this->assertEquals("$this->site2Dir.local", $site2_blt_yml['drush']['aliases']['local']);
+    $this->assertEquals("self", $site2_blt_yml['drush']['aliases']['local']);
     $this->assertEquals("$this->site2Dir.clone", $site2_blt_yml['drush']['aliases']['remote']);
 
     // Clone.