Skip to content

Commit

Permalink
docs: refresh docs to cover recent changes
Browse files Browse the repository at this point in the history
  • Loading branch information
garethgeorge committed Oct 20, 2024
1 parent 36543c6 commit e50dbfb
Show file tree
Hide file tree
Showing 3 changed files with 4,124 additions and 5,267 deletions.
8 changes: 6 additions & 2 deletions docs/content/1.introduction/1.getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Instance ID

Username and password

* Username and password is set on first launch of Backrest. If you lose your password you can reset it by deleting the `"users"` key from the `~/.config/backrest/config.json` file and restarting the Backrest service.
* Username and password is set on first launch of Backrest. If you lose your password you can reset it by deleting the `"users"` key from the `~/.config/backrest/config.json` file (or `%appdata%\backrest\config.json` on Windows) and restarting the Backrest service.
* If you don't want to use authentication (e.g. a local only installation or if you're using an authenticating reverse proxy) you can disabled authentication.


Expand Down Expand Up @@ -85,7 +85,10 @@ The primary properties of a plan are:

- **Excludes** a list of paths to exclude from backup operations. These can be fully qualified paths (e.g. /foo/bar/baz) or wildcard paths (e.g. `*node_modules*`). See the [restic docs](https://restic.readthedocs.io/en/latest/040_backup.html#excluding-files) on excluding files for more details. Files listed in excludes map to restic's `--excludes` flag.

- **Schedule** the schedule on which backups will run. This is expressed in cron format (configurable visually in the webui).
- **Schedule** the schedule on which backups will run.

- This can be specified as an interval in hours, interval in days, or as a cron providing detailed control over when backups run (e.g. `0 0 * * *` for midnight every day).
- A clock can also be configured which determines the time the schedule is checked against. UTC or Local use the current wall-clock time but will evaluate differently when using cron expressions where the timezone matters. Last Run Time evaluates the schedule relative to the last time the task ran rather than the current wall time, this can be useful for ensuring tasks aren't skipped if the schedule is missed for some reason (e.g. on a laptop that is frequently asleep).

- **Retention Policy** the duration that snapshots are retained for. This is enforced by running `restic forget` after each backup operation if a retention policy is specified. This can be set to none for append only repos or if you'd like to manage your own retention (e.g. run forget manually). Setting some retention policy is recommended to avoid the snapshot history growing without bound. Retention may be configured either based on count or bucketed by time period. See [the restic docs](https://restic.readthedocs.io/en/latest/060_forget.html#removing-snapshots-according-to-a-policy) to understand more about these policy options. Supported modes are:

Expand All @@ -97,6 +100,7 @@ The primary properties of a plan are:
- **Backup Flags** flags that are specific to the backup command (e.g. `--force` to force a full scan of all files rather than relying on metadata). These flags are passed directly to the restic command.
- `--one-file-system` will prevent restic crossing filesystem boundaries.
- `--force` will force rereading all files on each backup rather than relying on metadata (e.g. last modified time). This can be much slower.
- `--skip-if-unchanged` will skip creating a new snapshot if nothing has changed on the filesystem. Note that the backup task will still appear in the 'Full Operation History' but not in the tree view. Snapshots will still be created if any file or folder metadata changes (e.g. permissions, access times, etc).

::alert{type="success"}
Success! Now that Backrest is configured you can sit back and let it manage your backups. You can monitor the status of your backups in the UI and restore files from snapshots as needed.
Expand Down
47 changes: 29 additions & 18 deletions docs/content/2.docs/1.operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,29 @@ Backrest executes commands by forking the [restic](https://restic.net) binary. E

When running restic commands, Backrest injects the environment variables configured in the repo into the environment of the restic process and it appends the flags configured in the repo to the command line arguments of the restic process. Logs are collected for each command. In the case of an error, Backrest captures the last ~500 bytes of output and displays this directly in the error message (the first and last 250 bytes are shown if the output is longer than 500 bytes). Logs of the command are typically also available by clicking \[View Logs\] next to an operation, these logs are truncated to 32KB (with the first and last 16KB shown if the log is longer than 32KB).

## Scheduling Operations

Operations run on configurable schedules and all support the same collection of scheduling policies that provide flexible behavior.

Available scheduling **policies** are:

* **Disabled** - the operation is disabled and will not run.
* **Cron** - a cron expression specifying when to run the operation, this allows you to specify tasks that run at specific times down to the minute with detailed control e.g. `0 0 * * *` to run daily at midnight. See [Cron Syntax](https://en.wikipedia.org/wiki/Cron#Syntax) for more information.
* **Interval Days** - the interval in days at which the operation should be run. This is useful for running tasks at a regular interval when you don't care about the exact run time. Tasks will run no more frequently than the specified interval.
* **Interval Hours** - the interval in hours at which the operation should be run. This is useful for running tasks at a regular interval when you don't care about the exact run time. Tasks will run no more frequently than the specified interval.

In addition to the scheduling policy, a schedule also specifies a **clock** which the policy is evaluated relative to:

* **Local** - the schedule is evaluated against the current wall-clock time in the local timezone.
* **UTC** - the schedule is evaluated against the current wall-clock time in UTC. (Only behaves differently from Local when using cron expressions).
* **Last Run Time** - the schedule is evaluated relative to the last time the task ran. This can be useful for ensuring tasks aren't skipped if the schedule is missed for some reason (e.g. on a laptop that is frequently asleep). It also important to use this clock for tasks that run _very_ infrequently (e.g. check and prune health check operations) to ensure they aren't skipped.

::alert{type="info"}
**Backup** operations are scheduled under plan settings in the UI. Good practice is to use the "Local" clock if running hourly or more frequently. Use the "Last Run Time" clock if running daily or less frequently.
<br/> <br/>
**Prune** and **Check** operations are scheduled under repo settings in the UI. Good practice is to run these operations infrequently (e.g. every 30 days if using "Interval Days" or on the 3rd of the month, for example, if using "cron" scheduling"). Because health operations will be infrequent, it is recommended to use the "Last Run Time" clock to ensure they are not skipped.
::

## Types of Operations

#### Backup
Expand Down Expand Up @@ -46,25 +69,16 @@ Retention policies are mapped to forget arguments:
- **By Count** maps to `--keep-last {COUNT}`
- **By Time Period** maps to the `--keep-{hourly,daily,weekly,monthly,yearly} {COUNT}` flags

Note that forget will never run for a plan if the forget policy is set to `None`.

#### Prune

[Restic docs on prune](https://restic.readthedocs.io/en/latest/060_forget.html)

Prune operations are scheduled under repo settings in the UI. A prune operation removes data from storage that is no longer referenced by any snapshot. The prune operation is run using the `restic prune` command. Prune operations apply to the entire repo and will show up under the `_system_` plan in the Backrest UI.

Prunes are run in compliance with a prune policy which specifies:

- **Schedule** - the schedule on which to run prune operations. Available types are:
- **Disabled** - prune operations are disabled. This means that the repository will grow indefinitely and will never be pruned. You should periodically run a prune operation manually if you choose this option.
- **Cron** - a cron expression specifying when to run prune operations.
- **Max Frequency Days** - the minimum number of days that must pass between prune operations.
- **Max Frequency Hours** - (Advanced) the minimum number of hours that must pass between prune operations. This is useful for running prune operations more frequently than once per day, typically this is a bad idea as prune operations are expensive and should be run infrequently.
- **Max Unused Percent** - the maximum percentage of the repository that may be left unused after a prune operation runs. Prune operations will try to repack blobs in the repository if more than this percentage of their is unused (e.g. formerly held data belonging to forgotten snapshots).
Prunes are run in compliance with a prune policy which configures a *schedule* and a *max unused percent*. The max unused percent is the percentage of data that may remain unreferenced after a prune operation. The prune operation will repack or delete unreferenced data until the repo falls under this limit, if it already is it's possible that a prune will complete immediately.

::alert{type="info"}
Prune operations are costly and may read a significant portion of your repo. It is recommended to run them infrequently (e.g. monthly or every 30 days).
Prune operations are costly and may read a significant portion of your repo. Prune costs are mitigated by running them infrequently (e.g. monthly or every 30 days), and by using a higher *max unused percent* value (e.g. 5% or 10%). A higher *max unused percent* value will result in more data being retained in the repo, but will reduce the need to repack partially unreferenced data.
::

#### Check
Expand All @@ -73,11 +87,8 @@ Prune operations are costly and may read a significant portion of your repo. It

Check operations are scheduled under repo settings in the UI. A check operation verifies the integrity of the repository. The check operation is run using the `restic check` command. Check operations apply to the entire repo and will show up under the `_system_` plan in the Backrest UI.

Checks are run in compliance with a check policy which specifies:
Checks are configured by a *schdule* determining when they run, and a single argument *read data %* which determines the percentage of the repository that should be read during a check operation. Irrespective of *read data%*, the structure of the repo will always be verified in entirety. Reading data back verifies the hashes of pack files on disk and may detect unreliable storage (e.g. an HDD running without parity). It typically does not provide much value for a reliable cloud storage provider and can be set to a low percentage or disabled.

- **Schedule** - the schedule on which to run check operations. Available types are:
- **Disabled** - check operations are disabled. You should periodically run a check operation manually if you choose this option.
- **Cron** - a cron expression specifying when to run check operations.
- **Max Frequency Days** - the minimum number of days that must pass between check operations.
- **Max Frequency Hours** - (Advanced) the minimum number of hours that must pass between check operations. This is useful for running check operations more frequently than once per day, typically this is a bad idea as check operations are expensive and should be run infrequently.
- **Read Data %** - the percentage of the repository that should be read during a check operation. A value of 100% will check the entire repository which can be very slow and can be expensive if your provider bills for egress bandwidth. Reading back data is intended to verify the integrity of pack files on-disk for potentially unreliable storage (e.g. an HDD running without parity). It typically does not provide much value for a reliable storage provider and can be set to a low percentage or disabled.
::alert{type="warning"}
A value of 100% for *read data%* will read/download every pack file in your repository. This can be very slow and, if your provider bills for egress bandwidth, can be expensive. It is recommended to set this to 0% or a low value (e.g. 10%) for most use cases.
::
Loading

0 comments on commit e50dbfb

Please sign in to comment.