New version of the utility based on Plesk dist-upgrader

.buckconfig

@@ -1,6 +1,11 @@
 # Documentation: https://buckbuild.com/concept/buckconfig.html
 
-[python]
-    interpreter = python2.7
+[parser]
+    python_interpreter = /usr/bin/python2
+
 [python#py3]
-    interpreter = /usr/bin/python3.6
+    interpreter = /usr/bin/python3
+
+[repositories]
+    ubuntu18to20 = .
+    dist-upgrader = ./dist-upgrader

.github/workflows/build.yml

@@ -4,18 +4,23 @@ on: [push]
 
 jobs:
   build:
-    runs-on: ubuntu-20.04 # Not latest, because python3.6 is not available on latest
-                          # https://github.com/actions/setup-python/issues/544
+    # Not latest, because python3.6 is not available on latest
+    # (https://github.com/actions/setup-python/issues/544),
+    # and SandakovMM/build-with-buck@v2 action requires it
+    runs-on: ubuntu-20.04
     steps:
     - name: Checkout repository and submodules
-      uses: actions/checkout@v2
+      uses: actions/checkout@v4
       with:
         submodules: recursive
+        # We use tags to determine version, so fetch them
+        fetch-depth: 0
+        fetch-tags: true
     - name: Prepare artifact store
       run: mkdir -p ./buck-out/gen
     - name: Build ubuntu18to20
       id: build
-      uses: SandakovMM/build-with-buck@v2
+      uses: SandakovMM/build-with-buck@v3
       with:
         command: build
         target: :ubuntu18to20

.gitmodules

@@ -1,3 +1,3 @@
-[submodule "common"]
-	path = common
-	url = https://github.com/plesk/distro-conversion-base
+[submodule "dist-upgrader"]
+	path = dist-upgrader
+	url = https://github.com/plesk/dist-upgrader.git

BUCK

@@ -1,44 +1,23 @@
 # Copyright 1999-2023. Plesk International GmbH. All rights reserved.
 # vim:ft=python:
 
-PRODUCT_VERSION = '1.0.0'
-
-genrule(
-    name = 'version',
-    out = 'version.json',
-    bash = r"""echo "{\"version\": \"%s\", \"revision\": \"`git rev-parse HEAD`\"}" > $OUT""" % (PRODUCT_VERSION),
-)
-
-python_library(
-    name = 'actions.lib',
-    srcs = glob(['./actions/*.py']),
-)
-
-python_library(
-    name = 'ubuntu18to20.lib',
-    srcs = glob(['main.py', 'messages.py']),
-    deps = [
-        '//common:common.lib',
-        ':actions.lib',
-    ],
-    resources = [
-        ':version',
-    ],
-)
+include_defs('//product.defs.py')
 
 
 python_binary(
-    name = 'ubuntu18to20-script',
+    name = 'ubuntu18to20.pex',
     platform = 'py3',
-    main_module = 'main',
+    build_args = ['--python-shebang', '/usr/bin/env python3'],
+    main_module = 'ubuntu18to20.main',
     deps = [
-        ':ubuntu18to20.lib',
+        'dist-upgrader//pleskdistup:lib',
+        '//ubuntu18to20:lib',
     ],
 )
 
 genrule(
     name = 'ubuntu18to20',
-    srcs = [':ubuntu18to20-script'],
+    srcs = [':ubuntu18to20.pex'],
     out = 'ubuntu18to20',
-    cmd = 'cp $(location :ubuntu18to20-script) $OUT && chmod +x $OUT',
+    cmd = 'cp $(location :ubuntu18to20.pex) $OUT && chmod +x $OUT',
 )

README.md

@@ -1,47 +1,49 @@
-# The tool to distupgrade an Ubuntu 18 server with Plesk to to Ubuntu 20
-
-Ubuntu 18 to Ubuntu 20 distupgrade tool
+# The tool to dist-upgrade servers with Plesk from Ubuntu 18 to 20
 
 ## Introduction
-This script is the official tool for distupgrade an Ubuntu 18 server with Plesk to Ubuntu 20. The script is based on the official ubuntu distupgrade tool. The script includes additional repository and configuration support provided by Plesk.
+This utility is the official tool to dist-upgrade servers with Plesk from Ubuntu 18 to 20.
+
+The utility uses [Plesk dist-upgrader](https://github.com/plesk/dist-upgrader).
 
 ## Preparation
-To avoid downtime and data loss, make sure you have read and understood the following information before using the script:
+To avoid downtime and data loss, make sure you have read and understood the following information before using the utility:
 1. **Upgrade Plesk to the last version.**
 2. **Create a full server backup.** Before the upgrade, make a full server backup (which includes a full backup of all the databases).
-3. Notify the customers about upcoming downtime. Expected downtime is between 25 and 35 minutes.
+3. Notify the customers about upcoming downtime. Expected downtime is 20-30 minutes.
 4. **Remote management module must be installed on the server**.
 5. We strongly recommend that you **create a snapshot you can use as a recovery point** in case the conversion process fails.
 6. Read the [Known issues](#known-issues) section below for the list of known issues.
 
 ## Timing
-The conversion process should run between 25 and 35 minutes. **Plesk services, hosted websites, and emails will be unavailable during the entirety of the conversion process**.
+The conversion process should run between 20 and 30 minutes. **Plesk services, hosted websites, and e-mails will be unavailable during the entirety of the conversion process**.
 
 ## Known issues
 ### Blockers
-Do not use the script if any of the following is true:
-- **You are running an OS other than Ubuntu 18**. The script was designed to convert Ubuntu 18 servers only. Please, don't use it for other distributions.
-- **PHP 7.1 and earlier are not supported** in Ubuntu 20, and will not receive any updates after the conversion. These PHP versions are deprecated and may have security vulnerabilities. So we force to remove this versions before the conversion.
-- **Distupgrade inside containers (like Virtuozzo containers, Docker Containers, etc) are not supported**. 
+Do not use the utility if any of the following is true:
+- **Your system is in a container (like Virtuozzo containers, Docker Containers, etc).**
 
 ## Requirements
 - Last Plesk version.
-- Ubuntu 18
-- grub is installed
 - At least 5 GB of free disk space.
 - At least 2 GB of RAM.
 
-## Using the script
-To retrieve the latest available version of the tool, please navigate to the "Releases" section. Once there, locate the most recent version of the tool and download the zip archive. The zip archive will contain the ubuntu18to20 tool binary.
+## Conversion phases
+The conversion process consists of two phases:
+1. The "convert" phase contains preparation and upgrading actions.
+2. The "finish" phase is the last phase containing all finishing actions.
+
+During each phase a conversion plan consisting of stages, which in turn consist of actions, is executed. You can see the general stages in the `--help` output and the detailed plan in the `--show-plan` output.
+
+## Using the utility
+To retrieve the latest available version of the tool, please navigate to the "Releases" section. Once there, locate the most recent version of the tool and download the attached archive.
 
-To prepare the latest version of the tool for use from a command line, please run the following commands:
+To prepare the latest version of the tool for use, please run the following commands:
 ```shell
-> wget https://github.com/plesk/ubuntu18to20/releases/download/v1.0.0/ubuntu18to20-1.0.0.zip
-> unzip ubuntu18to20-1.0.0.zip
+> unzip ubuntu18to20.zip
 > chmod 755 ubuntu18to20
 ```
 
-To monitor the conversion process, we recommend using the ['screen' utility](https://www.gnu.org/software/screen/) to run the script in the background. To do so, run the following command:
+To monitor the conversion process, we recommend using the ['screen' utility](https://www.gnu.org/software/screen/) to run the utility in the background. To do so, run the following command:
 ```shell
 > screen -S ubuntu18to20
 > ./ubuntu18to20
@@ -51,88 +53,73 @@ If you lose your SSH connection to the server, you can reconnect to the screen s
 > screen -r ubuntu18to20
 ```
 
-
 You can also call ubuntu18to20 in the background:
 ```shell
 > ./ubuntu18to20 &
 ```
+
 And monitor its status with the '--status' or '--monitor' flags:
 ```shell
 > ./ubuntu18to20 --status
 > ./ubuntu18to20 --monitor
 ... live monitor session ...
 ```
 
-After reboot to the Ubuntu 20, you can check the status of the conversion process by running the following command:
+The conversion process requires 3 reboots. It will be resumed automatically after reboot by the `plesk-dist-upgrader` systemd service. In addition to `--status` and `--monitor`, you can check the status of the conversion process by running the following command:
 ```shell
-> python3.8 ./ubuntu18to20 --status
-> python3.8 ./ubuntu18to20 --monitor
+> systemctl status plesk-dist-upgrader
 ... live monitor session ...
 ```
 
-Running ubuntu18to20 without any arguments initiates the conversion process. The script performs preliminary checks, and if any issues are detected, it provides descriptions of the problems along with guidance on how to resolve them.
-Following the preliminary checks, the tool proceeds with the distupgrade process, which is divided into two stages: the distupgrade stage, lasting approximately 20 minutes, and the finishing stage. The distupgrade stage involves the actual upgrade process, after which the server reboots.
-Upon reboot, the finishing stage commences, typically taking about 5 minutes to complete. This stage triggers a second reboot at its conclusion.
-Upon your next SSH login, you will encounter the following message:
+Running dist-upgrader without any arguments initiates the conversion process. The utility performs preliminary checks, and if any issues are detected, it provides descriptions of the problems along with guidance on how to resolve them.
+Following the preliminary checks, the tool proceeds with the dist-upgrade process, which is divided into multiple stages. Some stages end with a reboot. You can check the list of stages and steps by `./ubuntu18to20 --show-plan`.
+When dist-upgrade is finished, you will see the following login message:
 ```
 ===============================================================================
-Message from the Plesk ubuntu18to20 tool:
+Message from the Plesk dist-upgrader tool:
 The server has been upgraded to Ubuntu 20.
 You can remove this message from the /etc/motd file.
 ===============================================================================
 ```
 
-### Conversion stage options
-The conversion process consists of two stage options: "start", and "finish". To run stages individually, use the "--start", and "--finish" flags, or the "-s" flag with name of the stage you want to run.
-1. The "start" stage start distupgrade process.
-2. The "finish" stage must be called on the first boot of Ubuntu 20. You can rerun this stage if something goes wrong during the first boot to ensure that the problem is fixed and Plesk is ready to use.
-
-### Other arguments
+## Logs
+If something goes wrong, read the logs to identify the problem.
+The dist-upgrader writes its log to the `/var/log/plesk/ubuntu18to20.log` file, as well as to stdout.
+After the first reboot, the process is resumed by the `plesk-dist-upgrader` service, so its output is available in system logs (see `systemctl status plesk-dist-upgrader` and `journalctl -u plesk-dist-upgrader`).
 
-### Logs
-If something goes wrong, read the logs to identify the problem. You can also read the logs to check the status of the finish stage during the first boot.
-The ubuntu18to20 writes its log to the '/var/log/plesk/ubuntu18to20.log' file, as well as to stdout.
-
-### Revert
-If the script fails during the the "start" stage before the distupgrade performs, you can use the ubuntu18to20 script with the '-r' or '--revert' flags to restore Plesk to normal operation. The ubuntu18to20 will undo some of the changes it made and restart Plesk services. Once you have resolved the root cause of the failure, you can attempt the conversion again.
+## Revert
+If the utility fails during the the "convert" stage before actual dist-upgrade of packages, you can use the dist-upgrader utility with the `-r` or `--revert` options to restore Plesk to normal operation. The dist-upgrader will undo some of the changes it made and restart Plesk services. Once you have resolved the root cause of the failure, you can attempt the conversion again.
 Note:
-- You cannot use revert to undo the changes after the distupgrade take it's place, because packages provided by Ubuntu 20 already installed.
+- You cannot use revert to undo the changes after the dist-upgrade of packages, because packages provided by the new OS version are already installed.
+- `--revert` mode is not perfect, it can fail or be unable to restore the initial state of the system. So, the importance of creating full server backup or snapshot before starting dist-upgrade can't be stressed enough.
 
-### Check the status of the conversion process and monitor its progress
-To check the status of the conversion process, use the '--status' flag. You can see the current stage of the conversion process, the elapsed time, and the estimated time until finish.
+### Checking the status of the conversion process and monitoring its progress
+To check the status of the conversion process, use the `--status` option. You can see the current stage of the conversion process, the elapsed time, and the estimated time until finish.
 ```shell
 > ./ubuntu18to20 --status
 ```
 
-To monitor the progress of the conversion process in real time, use the '--monitor' flag.
+To monitor the progress of the conversion process in real time, use the `--monitor` option.
 ```shell
 > ./ubuntu18to20 --monitor
 ( stage 3 / action re-installing plesk components  ) 02:26 / 06:18
 ```
 
-After the first reboot you should call the script directly by python3.8 interpreter:
-```shell
-> python3.8 ./ubuntu18to20 --status
-> python3.8 ./ubuntu18to20 --monitor
-... live monitor session ...
-```
-
 ## Issue handling
-### ubuntu18to20 finish fails on the first boot
-If something goes wrong during the finish stage, you will be informed on the next SSH login with this message:
+If for some reason the process has failed, inspect the log. By default, it's put to `/var/log/plesk/ubuntu18to20.log`. If the process was interrupted before the first reboot, you can restart it with the `--resume` option. If the problem has happened after the first reboot, you can restart the process by running `systemctl restart plesk-dist-upgrader`.
+
+If something goes wrong, you will be informed on the next login with this message:
 ```
 ===============================================================================
-Message from Plesk ubuntu18to20 tool:
-Something went wrong during the final stage of Ubuntu 18 to Ubuntu 20 conversion
+Message from Plesk dist-upgrader tool:
+Something went wrong during dist-upgrade by ubuntu18to20.
 See the /var/log/plesk/ubuntu18to20.log file for more information.
 You can remove this message from the /etc/motd file.
 ===============================================================================
 ```
-You can read the ubuntu18to20 log to troubleshoot the issue. If the ubuntu18to20 finish stage fails for any reason, once you have resolved the root cause of the failure, you can retry by running 'python3.8 ubuntu18to20 -s finish'.
 
 ### Send feedback
-If you got any error, please [create an issue on github](https://github.com/plesk/ubuntu18to20/issues). To do generate feedback archive by calling the tool with '-f' or '--prepare-feedback' flags.
+If you got any error, please [create an issue on github](https://github.com/plesk/ubuntu18to20/issues). Describe your problem and attach the feedback archive or at least the log to the issue. The feedback archive can be created by calling the tool with the `--prepare-feedback` option:
 ```shell
 > ./ubuntu18to20 --prepare-feedback
 ```
-Describe your problem and attach the feedback archive to the issue.