Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated exec2 case in quickstart #40

Merged
merged 7 commits into from
Jun 27, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions packer/hashibox.pkr.hcl
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,11 @@ build {
"sudo mkdir /etc/consul.d/certs && cd /etc/consul.d/certs ; sudo consul tls ca create",
"sudo mkdir /etc/nomad.d/certs && cd /etc/nomad.d/certs ; sudo nomad tls ca create",

# Install exec2 driver and copy under /opt/nomad/data/plugins dir
"sudo dnf install -y nomad-driver-exec2 --enablerepo hashicorp-test",
"sudo mkdir /opt/nomad/data/plugins && sudo chown nomad:nomad /opt/nomad/data/plugins",
"sudo cp /usr/bin/nomad-driver-exec2 /opt/nomad/data/plugins/",

# Set permissions for the certs directory
"sudo chown consul:consul /etc/consul.d/certs",
"sudo chown nomad:nomad /etc/nomad.d/certs",
Expand Down
5 changes: 3 additions & 2 deletions scenarios/nomad-consul-quickstart/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Scenario: Nomad Consul Quickstart

This scenario deploys both Nomad and Consul with out any of the security configurations in place. This scenario is useful when you have to play around the features of Nomad (with or without Consul) and not worry about the security aspects.
This scenario deploys both Nomad and Consul with out any of the security configurations in place. This scenario is useful when you have to play around the features of Nomad (with or without Consul) and not worry about the security aspects.

## Prerequsites

Expand All @@ -10,6 +10,7 @@ This scenario has the following pre-requsites:
* Requires a base VM image built using packer (`../../packer/hashibox.pkr.hcl`)
* Uses `qemu` driver (you can use `vz` by modifying `vmType` in the template)
* If running enterprise binaries, the Consul and Nomad licenses should be passed as environment variable (shown in the example below)
* Packer is baked with `exec2` driver, and Nomad only utilizes this driver when version 1.8.x or higher is deployed.

### Usage

Expand Down Expand Up @@ -68,4 +69,4 @@ ID Node Pool DC Name Class Drain Eligibility Sta

```
$ shikari destroy -f -n murphy
```
```
133 changes: 74 additions & 59 deletions scenarios/nomad-consul-quickstart/hashibox.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,119 +2,131 @@ images:
# Try to use a local image first.
- location: ~/artifacts/qemu/c-1.18-n-1.7/c-1.18-n-1.7.qcow2


# disable port-mapping, mounts, containerd etc
plain: true

provision:
- mode: system # install Consul and Nomad Licenses if any
script: |
#!/bin/bash

if [[ -n $CONSUL_LICENSE ]]; then
echo "CONSUL_LICENSE=$CONSUL_LICENSE" > /etc/consul.d/consul.env
echo "CONSUL_LICENSE=$CONSUL_LICENSE" > /etc/consul.d/consul.env
fi

if [[ -n $NOMAD_LICENSE ]]; then
echo "NOMAD_LICENSE=$NOMAD_LICENSE" > /etc/nomad.d/nomad.env
echo "NOMAD_LICENSE=$NOMAD_LICENSE" > /etc/nomad.d/nomad.env
fi

- mode: system # Configure Consul common settings
- mode: system # Configure Consul common settings
script: |
#!/bin/bash

# common config for Server and Client
cat <<-EOF > /etc/consul.d/consul.hcl
data_dir = "/opt/consul/data"
log_level = "INFO"
bind_addr = {{ "\"{{ GetInterfaceIP \\\"lima0\\\"}}\"" }}
client_addr = "0.0.0.0"
retry_join = ["lima-$CLUSTER-srv-01.local"]
datacenter = "$CLUSTER"

ui_config {
enabled = true
}
data_dir = "/opt/consul/data"
log_level = "INFO"
bind_addr = {{ "\"{{ GetInterfaceIP \\\"lima0\\\"}}\"" }}
client_addr = "0.0.0.0"
retry_join = ["lima-$CLUSTER-srv-01.local"]
datacenter = "$CLUSTER"

ui_config {
enabled = true
}
EOF

- mode: system # Configure Consul server settings
script: |
#!/bin/bash

if [[ $MODE == "server" ]]; then
cat <<-EOF > /etc/consul.d/server.hcl
connect {
enabled = true
}

server = true
bootstrap_expect = $BOOTSTRAP_EXPECT
connect {
enabled = true
}
server = true
bootstrap_expect = $BOOTSTRAP_EXPECT
EOF
fi

- mode: system # Configure Consul client settings
script: |
#!/bin/bash

if [[ $MODE == "client" ]]; then
cat <<-EOF > /etc/consul.d/client.hcl

recursors = ["1.1.1.1", "8.8.8.8"]

ports {
grpc = 8502
}
recursors = ["1.1.1.1", "8.8.8.8"]
ports {
grpc = 8502
}
EOF
fi

- mode: system # Configure Nomad common settings
script: |
#!/bin/bash
cat <<-EOF > /etc/nomad.d/nomad.hcl
data_dir = "/opt/nomad/data"
bind_addr = "0.0.0.0"
datacenter = "$CLUSTER"
log_level = "DEBUG"

advertise {
http = {{ "\"{{ GetInterfaceIP \\\"lima0\\\"}}\"" }}
rpc = {{ "\"{{ GetInterfaceIP \\\"lima0\\\"}}\"" }}
serf = {{ "\"{{ GetInterfaceIP \\\"lima0\\\"}}\"" }}
}
data_dir = "/opt/nomad/data"
bind_addr = "0.0.0.0"
datacenter = "$CLUSTER"
log_level = "DEBUG"
advertise {
http = {{ "\"{{ GetInterfaceIP \\\"lima0\\\"}}\"" }}
rpc = {{ "\"{{ GetInterfaceIP \\\"lima0\\\"}}\"" }}
serf = {{ "\"{{ GetInterfaceIP \\\"lima0\\\"}}\"" }}
}
EOF

- mode: system # configure Nomad server settings
script: |
#!/bin/bash

if [[ $MODE == "server" ]]; then
cat <<-EOF > /etc/nomad.d/server.hcl
server {
#license_path = "/etc/nomad.d/license.hclic"
enabled = true
bootstrap_expect = $BOOTSTRAP_EXPECT

server_join {
retry_join = ["lima-$CLUSTER-srv-01.local"]
}
server {
#license_path = "/etc/nomad.d/license.hclic"
enabled = true
bootstrap_expect = $BOOTSTRAP_EXPECT
server_join {
retry_join = ["lima-$CLUSTER-srv-01.local"]
}
}
EOF
fi

- mode: system # configure Nomad client settings
script: |
#!/bin/bash

if [[ $MODE == "client" ]]; then
cat <<-EOF > /etc/nomad.d/client.hcl
client {
enabled = true
servers = ["lima-$CLUSTER-srv-01.local"]

network_interface = "lima0"
client {
enabled = true
servers = ["lima-$CLUSTER-srv-01.local"]
network_interface = "lima0"
}
EOF
fi
# Get Nomad version
nomad_version=$(nomad --version | sed -n 's/^Nomad v\([0-9]\+\.[0-9]\+\).*/\1/p')
nomad_required_version="1.8"
if awk 'BEGIN { exit !('"${nomad_version}"' >= '"${nomad_required_version}"') }'; then
# Check if Nomad version is 1.8 or higher
echo "Nomad version $nomad_version is equal or higher than 1.8. Pushing config for nomad-driver-exec2..."
cat <<-EOF > /etc/nomad.d/exec2.hcl
plugin "nomad-driver-exec2" {
config {
unveil_defaults = true
unveil_paths = []
unveil_by_task = true
}
}
EOF
fi
package_name="nomad-driver-exec2"
existing_package_location="/usr/bin"

# Check if the directory exists
if [ ! -d "$NOMAD_PLUGINSDIR" ]; then
mkdir -p "$NOMAD_PLUGINSDIR"
fi

# Check if the plugin exists.
if rpm -q "$package_name" >/dev/null 2>&1; then
cp "$existing_package_location/$package_name" "$NOMAD_PLUGINSDIR"
fi

- mode:
script: |
systemctl enable --now docker
Expand All @@ -127,3 +139,6 @@ provision:
networks:
- lima: shared
vmType: qemu

env:
NOMAD_PLUGINSDIR: /opt/nomad/data/plugins