Skip to content

epomatti/aws-ubuntu-landscape

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

83 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Ubuntu Landscape

Create the AWS resources

Securely connect to your profile with SSO:

export AWS_PROFILE="<PROFILE>"
aws sso login

Create the .auto.tfvars variables file:

cp config/template.tfvars .auto.tfvars

Choose a current AMI from the Canonical SSM index. Example:

aws ssm get-parameters --names \
   /aws/service/canonical/ubuntu/pro-server/24.04/stable/current/arm64/hvm/ebs-gp3/ami-id

Set the required variables:

  • landscape_server_fqdn - Internet FQDN for the Landscape server.
  • landscape_certbot_email - Required while setting certbot certificates.

Create the infrastructure:

terraform init
terraform apply -auto-approve

Setup the Ubuntu Landscape server

This project currently uses the quickstart mode for installation. The following are references for such configuration:

Elevate your privileges in the server session:

sudo su -

Make sure cloud init executed properly

cloud-init status

Important

Make sure you don't skip the next step. Change the "example.com" to your owned domain of choice.

Setup your domain registry for the landscape.example.com to the public IP or AWS EC2 instance name.

Note

The project is created with an Elastic IP, therefore the public IP address will not change.

Make sure that the DNS has replicated successfully before continuing:

dig landscape.example.com

Attach the server to your Ubuntu Pro subscription. You're currently allowed 5 machines in the Personal free subscription.

Tip

Don't need to attach again when using cloud Ubuntu Pro image

pro attach

Set the session variables:

export FQDN=$(aws ssm get-parameter --name "landscape-server-fqdn" --query "Parameter.Value" --output text)
export CERTBOT_EMAIL=$(aws ssm get-parameter --name "landscape-server-certbot-email" --query "Parameter.Value" --output text)

Set your hostname using variables:

hostnamectl set-hostname "$FQDN"

It would be a good idea to reboot after changing the hostname:

reboot

Install Landscape:

apt update && DEBIAN_FRONTEND=noninteractive apt-get install -y landscape-server-quickstart

Install your certificate:

Important

Make sure that after these steps the certificate is correctly issued and installed. This will required by the landscape client configuration later

Tip

If you rebooted, make sure to re-enter the required variables

certbot --non-interactive --apache --no-redirect --agree-tos --email $CERTBOT_EMAIL --domains $FQDN

Your server should be ready for use at https://landscape.example.com.

Access the server and create your administrator account.

In case this come in handy, these are the commands to manage the Landscape Server CTL:

lsctl status
lsctl restart

Ubuntu Pro Server

Note

As of this writing, USG is not yet available for 24.04.

If you want to enable Ubuntu Pro for the created instance, just flip the variable switch:

create_ubuntu_pro_server = true

Check the pro licensing status:

Note

USG and the Landscape client should already be installed. USG should already be enabled.

Warning

Livepatch is currently not supported for ARM64 systems

sudo pro status --all

Link the instance to Landscape SaaS and then approve the registration:

# sudo is required to read /etc/landscape/client.conf
sudo landscape-config --computer-title "aws-ubuntu-pro-server" \
  --account-name "<ACCOUNT>" \
  --http-proxy="" \
  --https-proxy="" \
  --script-users="root,landscape,nobody" \
  --tags="server,aws"

Enable script execution:

sudo landscape-config --include-manager-plugins=ScriptExecution --script-users=root,landscape,nobody

Restart the service:

sudo service landscape-client restart

Apply a profile:

sudo usg fix cis_level1_server

Ubuntu Desktop

This project uses a locally deployed bear metal remote Ubuntu 22.04 Desktop. Create a Ubuntu image. The recommended burning tool is Balena.

Note

Using Ubuntu 22.04 as of the time of this project Ubuntu Pro des not support 24.04 USG, which is part of this scope of experimentation.

Make sure all packages are updated:

sudo apt update
sudo apt upgrade -y

Set up XRDP to manage your machine remotely. This project follows this Digital Ocean's article.

sudo apt install xfce4 xfce4-goodies -y
sudo apt install xrdp -y
sudo systemctl status xrdp

Once logged into your Ubuntu Desktop workstation, attach the machine for Ubuntu landscape management.

The ubuntu-pro-client should already be installed. Just make sure it is updated:

# This will update the client ot the latest version 
sudo apt install -y ubuntu-pro-client

Note

You should be using a Let's Encrypt certificated issued earlier in this documentation. In for some reason you're opting for a self-signed approach, check how to set it up in the client.

Attach the Ubuntu Desktop machine to a license:

sudo pro attach

Now, from your Ubuntu Landscape Server, follow the instructions on how to register a computer vi the menu. The path should be something like this:

https://landscape.example.com/account/standalone/how-to-register

These are examples of commands to be executed in the Ubuntu Desktop client machine:

sudo apt-get update
sudo apt-get install -y landscape-client
# Replace the domain
sudo landscape-config --computer-title "MyUbuntuDesktop" --account-name standalone  --url https://landscape.example.com/message-system --ping-url http://landscape.example.com/ping

Note

Make sure to check Ubuntu Pro status in Landscape to confirm it has been properly registered.

Enable script execution (administrator)

In case script execution needs to be enabled.

sudo -u landscape bash -x /opt/canonical/landscape/scripts/update_security_db.sh

Configuration file:

/etc/landscape/client.conf

Restarting the client:

sudo service landscape-client restart

Ubuntu Hardening

Ubuntu Pro supports Ubuntu Security Guide (USG). For quick guide, the tutorial can be o good starting point.

Following the installation guide:

Note

This is already covered by the previous steps already executed

  1. Install the UA client
  2. Attach the subscription

Enable and install USG:

sudo ua enable usg
sudo apt install usg

This project uses CIS benchmarks, for which there are different profiles:

Tip

Check the CIS Benchmark publications for in-depth details about each profile

  • Level 1: Balanced
  • Level 2: Restrictive

To apply the benchmark, select one of the profiles:

# Profiles: cis_level1_workstation, cis_level2_workstation
sudo usg fix <PROFILE>

A system reboot is required after this point.

Then, run the audit command:

Tip

Once the audit below is completed, access the HTML page for analysis

# Profiles: cis_level1_workstation, cis_level2_workstation
sudo usg audit <PROFILE>

To access the file from another user:

cp usg-report-xxx.html /home/user/Desktop/
chown <user>: /home/<user>/Desktop/usg-report-20241110.1302.html

To apply for a set of systems:

sudo usg generate-fix <PROFILE> --output fix.sh

If required, explore the customization options.

Other references include the Ubuntu engagement page, and the CIS Benchmark Ubuntu page.

VirtualBox

Simplest way might be to use VirtualBox with Vagrant:

Note

USG currently not supported for 24.04

mkdir -p vagrant/ubuntu-jammy
cd vagrant/ubuntu-jammy

vagrant init ubuntu/jammy64
vagrant up
vagrant ssh

To increase the VM performance, set custom values:

config.vm.provider "virtualbox" do |vb|
  vb.memory = "2048"
  vb.cpus = "2"
end

Setup Ubuntu Pro:

sudo apt update && sudo apt upgrade -y
sudo apt install -y ubuntu-advantage-tools

Attach to a subscription:

sudo pro attach

Enable and install USG, the Landscape Client:

sudo pro enable usg
sudo apt install -y usg landscape-client

Set up the configuration file /etc/landscape/client.conf:

Tip

The documentation have guidelines for CM tools such as Puppet or Ansible.

[client]
log_level = info 
url = https://{FQDN}/message-system
ping_url = http://{FQDN}/ping
data_path = /var/lib/landscape/client
registration_key = {REGISTRATION_KEY}
computer_title = {COMPUTER_TITLE}
account_name = {ACCOUNT_NAME}
include_manager_plugins = ScriptExecution
script_users = root,landscape,nobody

With the file set, register the machine:

sudo landscape-config

Warning

If you run into a "twisted.internet" error it might be due to this bug. Check the ownership of the /var/lib/landscape/client structure, it should have landscape ownership. If necessary, fix it by running sudo chown -R landscape:landscape /var/lib/landscape/client.

Apply a USG a profile:

sudo usg fix cis_level1_server

Tuning

The client can read several variables to adjust the behavior.

An example is provided in the repository, and this is a sample question.

Configuration can be changed in the client.conf file:

/etc/landscape/client.conf

For local debugging this might might be useful:

# The number of seconds between server exchanges
exchange_interval = 900 # 15 minutes

# The number of seconds between urgent exchanges with the server.
urgent_exchange_interval = 60 # 1 minute

# The number of seconds between pings.
ping_interval = 30

# The number of seconds between apt update calls.
apt_update_interval = 21600

# The number of seconds between package monitor runs.
package_monitor_interval = 1800

# The number of seconds between snap monitor runs.
snap_monitor_interval = 1800

Changing this configuration will require a client restart:

sudo service landscape-client restart

Troubleshooting

Make sure to run scripts with the right user landscape, or if root is used, apply the correct permissions.

sudo cat /etc/sudoers
groups landscape

An example with the docker list:

stat /etc/apt/sources.list.d/docker.list
chmod -v o+r /etc/apt/sources.list.d/docker.list

Monitoring

Here's a New Relic example setup with log forwarding:

Configuration is declared in the logging.yml file:

/etc/newrelic-infra/logging.d/logging.yml

Forward all the Landscape client logs:

logs:
  - name: landscape-client-logs
    file: /var/log/landscape/*.log
    attributes:
      logtype: landscape-client
      environment: sandbox

Quick commands to manage the agents:

sudo systemctl <start|stop|restart|status> newrelic-infra
sudo service landscape-client restart

Mirrors / Repositories

This section will cover mirror configuration. Check out the repository mirroring and mirror management articles.

EBS Modifications

Caution

Watch out for EBS modifications rate limit

Be mindful that there might be a 6 hour rate-limit for EBS.

If you reach the limit, then try this resolution procedure that involves attaching the instance to a new volume from a snapshot.

CloudWatch

The CloudWatch Agent has been configured and installed by Terraform. Check its status, and troubleshoot it if necessary:

sudo systemctl status amazon-cloudwatch-agent
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status

Mirror GPG Key Generation

Note

You have to give a real name and email so that he key is generated, otherwise it won't. Check this video for reference.

Important

Landscape mirror keys must not have passwords

# Install and run rngd to improve the efficiency of generating the GPG key
sudo apt-get install rng-tools && sudo rngd -r /dev/urandom

# Either 2 years, or never expire
gpg --gen-key 
gpg --full-gen-key

# List the public or secret keys
gpg -K
gpg --list-keys
gpg --list-secret-keys

# Export it
gpg -a --export-secret-keys {SECRET_KEY_ID} > mirror-key.asc

Just in case deleting a key is required:

gpg --delete-secret-key {SECRET_KEY_ID}
gpg --delete-key {SECRET_KEY_ID}

RabbitMQ Timeout

Tip

Consider changing the timeout for RabbitMQ

sudo touch /etc/rabbitmq/rabbitmq
# Add this (5 hours): consumer_timeout = 18000000
sudo vim /etc/rabbitmq/rabbitmq
sudo rabbitmq-diagnostics environment | grep consumer_timeout

Create and Sync the Mirror

Volume Space

Packages are going to be downloaded to /var/lib/landscape/landscape-repository/standalone/ by default.

Follow up the size with du:

du -h --max-depth=1 | sort -hr

Pockets

It's important to understand this bit of the documentation:

Landscape will take over the client's apt sources. The original sources.list file will be moved aside and only the ones enabled in Landscape will work.

Purpose for package management:

  • release - Official release
  • security - Critical security updates
  • updates - Bug fixes and stability improvements. Tested and approved before being released.

The pockets proposed and backports do not receive as much testing as updates.

This is a common recommendation:

deb http://archive.ubuntu.com/ubuntu jammy main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu jammy-updates main restricted universe multiverse
deb http://security.ubuntu.com/ubuntu jammy-security main restricted universe multiverse

Components

Components breakdown:

  • main - Officially supported free software.
  • restricted - Proprietary drivers and firmware supported by Ubuntu.
  • universe - Community-maintained open-source packages.
  • multiverse - Non-free software that Ubuntu does not officially support.

PostgreSQL Mirror

On Landscape, add the distribution and the mirror. Example for PostgreSQL:

Configuration Value
Name postgres
URI https://apt.postgresql.org/pub/repos/apt
Series name jammy-pgs
Pockets release
Components main
Architectures amd64

Now sync the mirror.

Create a repository profile and save to the clients:

sudo ls -l /etc/apt/sources.list.d/

PPA

To configure PPA repositories we need to deconstruct the configuration process.

PPA Key

Go to https://launchpad.net/ and find the repository in the format ppa:example/ppa.

This example will use ppa:graphics-drivers/ppa.

If the key is not visible in there, manually create the following URL:

https://launchpad.net/~graphics-drivers/+archive/ubuntu/ppa

Download the key manually. Use the 0x prefix with followed by the last 16 digits of the key, which are the key id:

curl -L "https://keyserver.ubuntu.com/pks/lookup?op=get&search=0xFCAE110B1118213C" -o graphics-drivers-ppa.asc

Dearmor the key and copy it:

Note

Either /etc/apt/keyrings/ or /etc/apt/trusted.gpg.d/ could be used based on security requirements.

sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/graphics-drivers-ppa.gpg graphics-drivers-ppa.asc
sudo chmod 644 /etc/apt/trusted.gpg.d/graphics-drivers-ppa.gpg

Confirm that the key is correctly installed:

sudo gpg --no-default-keyring --keyring /etc/apt/trusted.gpg.d/graphics-drivers-ppa.gpg --list-keys

PPA APT Sources

In the cause of graphics-drivers, these are the examples for APT sources.

When storing the keys in /etc/apt/trusted.gpg.d/, the key should be trusted:

deb http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu jammy main

If storing it under /etc/apt/keyrings/, then an explicit declaration is required:

deb [signed-by=/etc/apt/keyrings/graphics-drivers-ppa.asc] http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu jammy main

Testing

Check if the package is available:

apt-cache search nvidia-driver-535

Or installed if necessary:

apt install nvidia-driver-535

Issues with APT

sudo dpkg -r <package> # or sudo dpkg -P <program>
sudo apt purge <package>
sudo apt clean
sudo apt --fix-broken install
sudo apt autoremove

In some scenarios, a direct and / or forced removal might be necessary:

Note

This can happen for example with this PostgreSQL package conflict.

sudo dpkg -i --force-overwrite /var/cache/apt/archives/postgresql-client-common_272.pgdg22.04+1_all.deb
sudo apt --fix-broken install

Other commands that can be used (SO reference):

sudo dpkg --purge nodejs-legacy

Profiles

This YouTube video gives a run through the Profiles feature.

Demonstrate the usage of the Profiles feature:

  • Repository
  • Packages
  • Upgrade
  • Removal

API

Direct interaction with the API is possible via API Endpoints, such as with the Packages API.

Postfix Email

Following the official documentation but for Google's SMTP Relay.

The SMTP Host will be the Gmail URL:

smtp.gmail.com

To generate the credentials, one way is to use the App Password feature. Reference from linode.

For deep dive details and troubleshooting, check the Ubuntu Server documentation page.

Check for errors on both files:

sudo tail -f /var/log/mail.log
sudo tail -f /var/log/mail.err

You may also want to check the syslog:

sudo tail -f /var/log/syslog

Miscellaneous

General configurations of various types: https://ubuntu.com/landscape/docs/other-classic-web-portal-tasks