Add .gitconfig parser

This commit adds yet another config handler for gopass. It is based on the format used by git itself. This has the potential to address a lot of long standing issues, but it also causes a lot of changes to how we handle configuration, so bugs are inevitable. Fixes gopasspw#1567 Fixes gopasspw#1764 Fixes gopasspw#1819 Fixes gopasspw#1878 Fixes gopasspw#2387 RELEASE_NOTES=[BREAKING] New config format based on git config. Signed-off-by: Dominik Schulz <dominik.schulz@gauner.org> Co-authored-by: Yolan Romailler <AnomalRoil@users.noreply.github.com>
dominikschulz · Nov 18, 2022 · 0c5a9ec · 0c5a9ec
1 parent 7d98071
commit 0c5a9ec
Show file tree

Hide file tree

Showing 114 changed files with 2,705 additions and 1,277 deletions.
diff --git a/.github/workflows/autorelease.yml b/.github/workflows/autorelease.yml
@@ -86,6 +86,7 @@ jobs:
         run: |
           for D in dist/*.deb; do
             curl -H"X-Filename: ${D}" -H"X-Apikey: ${APIKEY}" -XPOST --data-binary @$D https://packages.gopass.pw/repos/gopass/upload
+            curl -H"X-Filename: ${D}" -H"X-Apikey: ${APIKEY}" -XPOST --data-binary @$D https://packages.gopass.pw/repos/gopass-unstable/upload
           done
         env:
           APIKEY: ${{ secrets.APT_APIKEY }}

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,3 +1,5 @@
+# Changelog
+
 ## 1.14.10 / 2022-11-09
 
 * [BUGFIX] Correctly handle key removal on Windows (#2372, #2371)

diff --git a/Makefile b/Makefile
@@ -110,7 +110,7 @@ test-win: $(GOPASS_OUTPUT)
 		$(GO) test -test.short -run '(Test|Example)' $(pkg) || exit 1;)
 
 test-integration: $(GOPASS_OUTPUT)
-	cd tests && GOPASS_BINARY=$(PWD)/$(GOPASS_OUTPUT) GOPASS_TEST_DIR=$(PWD)/tests $(GO) test -v
+	cd tests && GOPASS_BINARY=$(PWD)/$(GOPASS_OUTPUT) GOPASS_TEST_DIR=$(PWD)/tests $(GO) test -v $(TESTFLAGS)
 
 crosscompile:
 	@echo -n ">> CROSSCOMPILE linux/amd64"

diff --git a/docs/config.md b/docs/config.md
@@ -26,7 +26,7 @@ Some configuration options are only available through setting environment variab
 | `GOPASS_GPG_BINARY` | `string` | Set this to the absolute path to the GPG binary if you need to override the value returned by `gpgconf`, e.g. [QubesOS](https://www.qubes-os.org/doc/split-gpg/). |
 | `GOPASS_PW_DEFAULT_LENGTH`   | `int`    | Set to any integer value larger than zero to define a different default length in the `generate` command. By default the length is 24 characters. |
 | `GOPASS_AUTOSYNC_INTERVAL` | `int` | Set this to the number of days between autosync runs. |
-| `GOPASS_NO_AUTOSYNC` | `bool` | Set this to `true` to disable autosync. |
+| `GOPASS_NO_AUTOSYNC` | `bool` | Set this to `true` to disable autosync. Deprecated. Please use `core.autosync` |
 
 Variables not exclusively used by gopass
 
@@ -42,31 +42,46 @@ Variables not exclusively used by gopass
 
 ## Configuration Options
 
-During start up, gopass will look for a configuration file at `$HOME/.config/gopass/config.yml`. If one is not present, it will create one. If the config file already exists, it will attempt to parse it and load the settings. If this fails, the program will abort. Thus, if gopass is giving you trouble with a broken or incompatible configuration file, simply rename it or delete it.
+During start up, gopass will look for a configuration file at `$HOME/.config/gopass/config`. If one is not present, it will create one. If the config file already exists, it will attempt to parse it and load the settings. If this fails, the program will abort. Thus, if gopass is giving you trouble with a broken or incompatible configuration file, simply rename it or delete it.
 
 All configuration options are also available for reading and writing through the sub-command `gopass config`.
 
 * To display all values: `gopass config`
 * To display a single value: `gopass config autoclip`
 * To update a single value: `gopass config autoclip false`
-* As many other sub-commands this command accepts a `--store` flag to operate on a given sub-store, provided the sub-store is a remote one. Support for different local configurations per mount was dropped in v1.9.3.
+* As many other sub-commands this command accepts a `--store` flag to operate on a given sub-store, provided the sub-store is a remote one.
+
+### Configuration format
+
+`gopass` uses a configuration format inspired by and mostly compatible with the configuration format used by git. We support
+different configuration sources that take precedence over each other, just like [git](https://mirrors.edge.kernel.org/pub/software/scm/git/docs/git-config.html).
+
+#### Configuration precendence
+
+* Hard-coded presets apply if nothing else if set
+* System-wide configuration file allows operators or package maintainers to supply system-wide defaults in /etc/gopass/config
+* User-wide (aka. global) configuration allows to set per-user settings. This is the closest equivalent to the old gopass configs. Located in `$HOME/.config/gopass/config`
+* Per-store (aka. local) configuration allow to set per-store settings, e.g. read-only. Located in `<STORE_DIR>/config`.
+* Per-store unversioned (aka `config.worktree`) configuration allows to override versioned per-store settings, e.g. disabling read-only. Located in `<STORE_DIR>/config.worktree`
+* Environment variables (or command line flags) override all other values. Read from `GOPASS_CONFIG_KEY_n` and `GOPASS_CONFIG_VALUE_n` up to `GOPASS_CONFIG_COUNT`. Command line flags take precedence over environment variables.
+
+### Configuration options
 
 This is a list of available options:
 
-| **Option**       | **Type** | Description                                                                                                                                                                                    |
-| ---------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `askformore`     | `bool`   | If enabled - it will ask to add more data after use of `generate` command.  DEPRECATED in v1.10.0                                                                                              |
-| `autoclip`       | `bool`   | Always copy the password created by `gopass generate`. Only applies to generate.                                                                                                               |
-| `autoimport`     | `bool`   | Import missing keys stored in the pass repository without asking.                                                                                                                              |
-| `autosync`       | `bool`   | Always do a `git push` after a commit to the store. Makes sure your local changes are always available on your git remote. DEPRECATED in v1.10.0                                               |
-| `concurrency`    | `int`    | Number of threads to use for batch operations (such as reencrypting).  DEPRECATED in v1.9.3                                                                                                    |
-| `cliptimeout`    | `int`    | How many seconds the secret is stored when using `-c`.                                                                                                                                         |
-| `exportkeys`     | `bool`   | Export public keys of all recipients to the store.                                                                                                                                             |
-| `recipient_hash` | `map`    | Map of recipient ids to their hashes.  DEPRECATED in v1.10.0                                                                                                                                   |
-| `usesymbols`     | `bool`   | If enabled - it will use symbols when generating passwords.  DEPRECATED in v1.9.3                                                                                                              |
-| `nocolor`        | `bool`   | Do not use color.                                                                                                                                                                              |
-| `nopager`        | `bool`   | Do not invoke a pager to display long lists.                                                                                                                                                   |
-| `notifications`  | `bool`   | Enable desktop notifications.                                                                                                                                                                  |
-| `parsing`        | `bool`   | Enable parsing of output to have key-value and yaml secrets.                                                                                                                                   |
-| `path`           | `string` | Path to the root store.                                                                                                                                                                        |
-| `safecontent`    | `bool`   | Only output _safe content_ (i.e. everything but the first line of a secret) to the terminal. Use _copy_ (`-c`) to retrieve the password in the clipboard, or _force_ (`-f`) to still print it. |
+| **Option**       | **Type** | Description | *Default* |
+| ---------------- | -------- | ----------- | --------- |
+| `core.autoclip`        | `bool`   | Always copy the password created by `gopass generate`. Only applies to generate. | `false`
+| `core.autoimport`      | `bool`   | Import missing keys stored in the pass repository without asking. | `false` |
+| `core.autosync`        | `bool`   | Always do a `git push` after a commit to the store. Makes sure your local changes are always available on your git remote. | `true` |
+| `core.cliptimeout`     | `int`    | How many seconds the secret is stored when using `-c`. | `45` |
+| `core.exportkeys`      | `bool`   | Export public keys of all recipients to the store. | `true` |
+| `core.nocolor`         | `bool`   | Do not use color. | `false`
+| `core.nopager`         | `bool`   | Do not invoke a pager to display long lists. | `false`
+| `core.notifications`   | `bool`   | Enable desktop notifications. | `true` |
+| `core.parsing`         | `bool`   | Enable parsing of output to have key-value and yaml secrets. | `true` |
+| `core.readonly`        | `bool`   | Disable writing to a store. Note: This is just a convenience option to prevent accidential writes. Enforcement can only happen on a central server (if repos are set up around a central one). | `false` |
+| `mounts.path`          | `string` | Path to the root store. | `$XDG_DATA_HOME/gopass/stores/root` |
+| `core.showsafecontent` | `bool`   | Only output *safe content* (i.e. everything but the first line of a secret) to the terminal. Use *copy* (`-c`) to retrieve the password in the clipboard, or *force* (`-f`) to still print it. | | `false` |
+| `age.usekeychain`      | `bool`   | Use the OS keychain to cache age passphrases. | `false` |
+| `domain-alias.<from>   | `string` | Alias from domain to the string value of this entry. | `` |
diff --git a/docs/setup.md b/docs/setup.md
@@ -134,7 +134,7 @@ If this fails with an error: "Inappropriate ioctl for device" run the following
 
 If you are using CSH or TCSH:
 
-```
+```bash
 setenv GPG_TTY `tty`
 ```
 
@@ -198,6 +198,8 @@ $ sudo apt update
 $ sudo apt install gopass gopass-archive-keyring
 ```
 
+Note: We also have an unstable track that sometimes contains pre-release versions. Use `https://packages.gopass.pw/repos/gopass-unstable` if you want to help with early testing.
+
 #### Manual download
 
 First, find the latest .deb release from the repository [releases page](https://github.com/gopasspw/gopass/releases). Then, download and install it:

diff --git a/fish.completion b/fish.completion
@@ -51,42 +51,6 @@ complete -c $PROG -f -n '__fish_gopass_uses_command age identities -l chars -d "
 complete -c $PROG -f -n '__fish_gopass_uses_command age identities -l help -d "show help"'
 complete -c $PROG -f -n '__fish_gopass_uses_command age identities -l version -d "print the version"'
 complete -c $PROG -f -n '__fish_gopass_needs_command' -a alias -d 'Command: Manage domain aliases'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias' -a add -d 'Subcommand: Add a new alias'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias add -l yes -d "Always answer yes to yes/no questions"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias add -l clip -d "Copy the password value into the clipboard"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias add -l alsoclip -d "Copy the password and show everything"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias add -l qr -d "Print the password as a QR Code"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias add -l unsafe -d "Display unsafe content (e.g. the password) even if safecontent is enabled"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias add -l password -d "Display only the password. Takes precedence over all other flags."'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias add -l revision -d "Show a past revision. Does NOT support RCS specific shortcuts. Use exact revision or -&lt;N&gt; to select the Nth oldest revision of this entry."'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias add -l noparsing -d "Do not parse the output."'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias add -l chars -d "Print specific characters from the secret"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias add -l help -d "show help"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias add -l version -d "print the version"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias' -a remove -d 'Subcommand: Remove an alias from a domain'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias remove -l yes -d "Always answer yes to yes/no questions"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias remove -l clip -d "Copy the password value into the clipboard"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias remove -l alsoclip -d "Copy the password and show everything"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias remove -l qr -d "Print the password as a QR Code"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias remove -l unsafe -d "Display unsafe content (e.g. the password) even if safecontent is enabled"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias remove -l password -d "Display only the password. Takes precedence over all other flags."'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias remove -l revision -d "Show a past revision. Does NOT support RCS specific shortcuts. Use exact revision or -&lt;N&gt; to select the Nth oldest revision of this entry."'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias remove -l noparsing -d "Do not parse the output."'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias remove -l chars -d "Print specific characters from the secret"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias remove -l help -d "show help"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias remove -l version -d "print the version"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias' -a delete -d 'Subcommand: Delete an entire domain'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias delete -l yes -d "Always answer yes to yes/no questions"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias delete -l clip -d "Copy the password value into the clipboard"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias delete -l alsoclip -d "Copy the password and show everything"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias delete -l qr -d "Print the password as a QR Code"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias delete -l unsafe -d "Display unsafe content (e.g. the password) even if safecontent is enabled"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias delete -l password -d "Display only the password. Takes precedence over all other flags."'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias delete -l revision -d "Show a past revision. Does NOT support RCS specific shortcuts. Use exact revision or -&lt;N&gt; to select the Nth oldest revision of this entry."'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias delete -l noparsing -d "Do not parse the output."'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias delete -l chars -d "Print specific characters from the secret"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias delete -l help -d "show help"'
-complete -c $PROG -f -n '__fish_gopass_uses_command alias delete -l version -d "print the version"'
 complete -c $PROG -f -n '__fish_gopass_needs_command' -a audit -d 'Command: Decrypt all secrets and scan for weak or leaked passwords'
 complete -c $PROG -f -n '__fish_gopass_needs_command' -a cat -d 'Command: Decode and print content of a binary secret to stdout, or encode and insert from stdin'
 complete -c $PROG -f -n '__fish_gopass_needs_command' -a clone -d 'Command: Clone a password store from a git repository'

diff --git a/helpers/changelog/main.go b/helpers/changelog/main.go
@@ -5,6 +5,10 @@
 // Changelog implements the changelog extractor that is called by the autorelease GitHub action
 // and used to extract the changelog from the CHANGELOG.md file. It's content is then used to
 // populate the release description on GitHub.
+//
+// This tool will extract every line between the first and the second subheading (##).
+// This way the changelog can have a common header under the top most heading (#) and we
+// still only get the content of the latest release in the GitHub release notes.
 package main
 
 import (
@@ -32,6 +36,10 @@ func main() {
 			in = true
 		}
 
+		if !in {
+			continue
+		}
+
 		fmt.Println(line)
 	}
 }
diff --git a/helpers/man/main.go b/helpers/man/main.go
@@ -45,6 +45,7 @@ func main() {
 		if err := cmd.Run(); err != nil {
 			os.Exit(cmd.ProcessState.ExitCode())
 		}
+
 		return
 	}
 
@@ -54,7 +55,7 @@ func main() {
 	}
 	version := semver.MustParse(strings.TrimSpace(string(vs)))
 
-	action, err := ap.New(&config.Config{}, version)
+	action, err := ap.New(config.New(), version)
 	if err != nil {
 		panic(err)
 	}

diff --git a/helpers/postrel/main.go b/helpers/postrel/main.go
@@ -152,13 +152,22 @@ func (g *ghClient) createMilestones(ctx context.Context, v semver.Version) error
 		return err
 	}
 
+	// create a milestone for the next patch version
 	if err := g.createMilestone(ctx, v.String(), 1, ms); err != nil {
 		return err
 	}
 
+	// create a milestone for the next+1 patch version
 	v.IncrementPatch()
+	if err := g.createMilestone(ctx, v.String(), 2, ms); err != nil {
+		return err
+	}
 
-	return g.createMilestone(ctx, v.String(), 2, ms)
+	// create a milestone for the next minor version
+	v.IncrementMinor()
+	v.Patch = 0
+
+	return g.createMilestone(ctx, v.String(), 90, ms)
 }
 
 func (g *ghClient) createMilestone(ctx context.Context, title string, offset int, ms []*github.Milestone) error {
@@ -607,9 +616,11 @@ func (u *repoUpdater) createPR(ctx context.Context, title, from, toOrg, toRepo s
 		fmt.Printf("❌ Creating GitHub PR failed: %s", err)
 		fmt.Printf("Request: %+v\n", newPR)
 		fmt.Printf("Response: %+v\n", resp)
+
 		return err
 	}
 	fmt.Printf("✅ GitHub PR created: %s\n", pr.GetHTMLURL())
+
 	return err
 }
 
@@ -655,6 +666,7 @@ SCAN:
 				if repl != nil {
 					fmt.Fprintln(fout, *repl)
 				}
+
 				continue SCAN
 			}
 		}
@@ -680,6 +692,7 @@ func (r *repo) commitMsg() string {
 	if r.msg != "" {
 		return r.msg
 	}
+
 	return "gopass: update to " + r.ver.String() + "\nNote: This is an auto-generated change as part of the gopass release process.\n"
 }
 
@@ -706,6 +719,7 @@ func (r *repo) updatePrepare() error {
 	if err := r.gitBranchDel(); err != nil {
 		return fmt.Errorf("git branch -d failed: %w", err)
 	}
+
 	return r.gitBranch()
 }
 
@@ -726,6 +740,7 @@ func (r *repo) gitCoMaster() error {
 	cmd.Stderr = os.Stderr
 	cmd.Dir = r.dir
 	fmt.Printf("Running command: %s\n", cmd)
+
 	return cmd.Run()
 }
 
@@ -735,6 +750,7 @@ func (r *repo) gitBranch() error {
 	cmd.Stderr = os.Stderr
 	cmd.Dir = r.dir
 	fmt.Printf("Running command: %s\n", cmd)
+
 	return cmd.Run()
 }
 
@@ -744,6 +760,7 @@ func (r *repo) gitBranchDel() error {
 	cmd.Stderr = os.Stderr
 	cmd.Dir = r.dir
 	fmt.Printf("Running command: %s\n", cmd)
+
 	return cmd.Run()
 }
 
@@ -756,8 +773,10 @@ func (r *repo) gitPom() error {
 	cmd.Dir = r.dir
 	if err := cmd.Run(); err != nil {
 		fmt.Println(buf.String())
+
 		return err
 	}
+
 	return nil
 }
 
@@ -767,6 +786,7 @@ func (r *repo) gitPush(remote string) error {
 	cmd.Stderr = os.Stderr
 	cmd.Dir = r.dir
 	fmt.Printf("Running command: %s\n", cmd)
+
 	return cmd.Run()
 }
 
@@ -788,6 +808,7 @@ func (r *repo) gitCommit(files ...string) error {
 	cmd.Stderr = os.Stderr
 	cmd.Dir = r.dir
 	fmt.Printf("Running command: %s\n", cmd)
+
 	return cmd.Run()
 }
 
@@ -799,6 +820,7 @@ func (r *repo) isGitClean() bool {
 	if err != nil {
 		panic(err)
 	}
+
 	return strings.TrimSpace(string(buf)) == ""
 }