Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --config-override feature #1169

Merged
merged 131 commits into from
Mar 3, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
131 commits
Select commit Hold shift + click to select a range
110e2b9
add test and argument handler for runtime override of configurations.
sriniv27 Jan 19, 2021
744824d
identify location to apply override in "main"
sriniv27 Jan 20, 2021
a64fa82
update gitignore
sriniv27 Jan 20, 2021
ee7cd44
remove unneeded import
sriniv27 Jan 20, 2021
53d7d8c
add jrnl interface test for overriden configurations
sriniv27 Jan 20, 2021
554d25a
trivial whitespace change
sriniv27 Jan 20, 2021
5fe6caf
implement runtime override
sriniv27 Jan 20, 2021
c4b3d51
make format
sriniv27 Jan 21, 2021
9e2b7dd
refactor override unittest
sriniv27 Jan 22, 2021
692a979
clean up unused import
sriniv27 Jan 22, 2021
2fc19c4
start writing integration test
sriniv27 Jan 22, 2021
12d2f48
add linewrap override scenario
sriniv27 Jan 23, 2021
43b8407
implement editor override step
sriniv27 Jan 23, 2021
263c79c
add dev dependencies on pytest -mock and -cov
sriniv27 Jan 23, 2021
77646ea
make format
sriniv27 Jan 23, 2021
b82391d
remove unused imports
sriniv27 Jan 23, 2021
1bd275b
make format
sriniv27 Jan 23, 2021
f978cee
Merge branch 'develop' of https://github.com/jrnl-org/jrnl into develop
sriniv27 Jan 24, 2021
1a8bcfc
rename --override to --config-override
sriniv27 Jan 24, 2021
9676290
move override implementation into own module
sriniv27 Jan 24, 2021
80d2c60
begin TDD of dot notated overrides
sriniv27 Jan 24, 2021
9bab1ae
rewrite behavior scenario
sriniv27 Jan 24, 2021
f3d8ed2
implement recursive config overrides
sriniv27 Jan 24, 2021
2d7ff73
clean up unittests
sriniv27 Jan 24, 2021
99030c4
iterate on behave step
sriniv27 Jan 24, 2021
b92dd7e
make format
sriniv27 Jan 24, 2021
ededf23
cleanup
sriniv27 Jan 24, 2021
d348dbd
move override behave tests out of core
sriniv27 Jan 25, 2021
9540d34
refactor recursive code
sriniv27 Jan 25, 2021
c31fa08
make format
sriniv27 Jan 25, 2021
c8b737e
code cleanup
sriniv27 Jan 25, 2021
e173e12
remove unused import
sriniv27 Jan 25, 2021
5da5027
update test config
sriniv27 Jan 25, 2021
9f8be33
rewrite test for better mock call expect
sriniv27 Jan 25, 2021
27a370c
make format
sriniv27 Jan 25, 2021
fe4f189
binary search misbehaving windows test
sriniv27 Jan 25, 2021
36623c1
Merge pull request #1 from sriniv27/separate-override-behave-test-fro…
sriniv27 Jan 25, 2021
bd61a78
unittest multiple overrides
sriniv27 Jan 25, 2021
ba8db5d
uncomment dot notation unittest
sriniv27 Jan 25, 2021
7a54e6f
add multiple override scenario spec
sriniv27 Jan 25, 2021
f1b2606
make format
sriniv27 Jan 25, 2021
3ea0438
make format
sriniv27 Jan 25, 2021
432e766
update unittests for new syntax
sriniv27 Jan 26, 2021
0b61c6d
update integ tests for new syntax
sriniv27 Jan 26, 2021
1578fef
update gitignore
sriniv27 Jan 26, 2021
9ac21b8
guard override application
sriniv27 Jan 26, 2021
bf89109
deserialize function as return type
sriniv27 Jan 26, 2021
3b74c2d
make format
sriniv27 Jan 26, 2021
abbb2c5
organize deserialization unittests
sriniv27 Jan 27, 2021
0f5dfad
better, more specific behave tests
sriniv27 Jan 27, 2021
b2a60fd
test different editor launch commands
sriniv27 Jan 27, 2021
e85300e
formatting
sriniv27 Jan 27, 2021
6daf4fb
handle datatypes in deserialization and update helptext
sriniv27 Jan 27, 2021
b8b5612
stick to config convention in testbed
sriniv27 Jan 28, 2021
983c9c8
update tests ith better verifications
sriniv27 Jan 28, 2021
528ce1e
make format
sriniv27 Jan 28, 2021
52e146d
space
sriniv27 Jan 28, 2021
31577ad
review feedbac
sriniv27 Jan 28, 2021
18458b7
Merge pull request #2 from sriniv27/double-delimited-text-input-for-c…
sriniv27 Jan 28, 2021
e95ab58
Bump pytest from 6.2.1 to 6.2.2
dependabot[bot] Jan 28, 2021
5e4eb66
Bump keyring from 21.8.0 to 22.0.1
dependabot[bot] Jan 28, 2021
6e0a47c
make format
sriniv27 Jan 28, 2021
c4c4155
skip on win
sriniv27 Jan 28, 2021
2063366
update deps
sriniv27 Jan 28, 2021
0bb4c80
Merge pull request #3 from sriniv27/dependabot/pip/pytest-6.2.2
sriniv27 Jan 28, 2021
63a5519
Merge pull request #4 from sriniv27/dependabot/pip/keyring-22.0.1
sriniv27 Jan 28, 2021
6e09f93
update tests with better verifications
sriniv27 Jan 28, 2021
8624fd3
skip on win
sriniv27 Jan 28, 2021
e74420b
update deps
sriniv27 Jan 28, 2021
54a5a68
Bump pytest from 6.2.1 to 6.2.2
dependabot[bot] Jan 28, 2021
1a99d65
refactor deserialization
sriniv27 Jan 29, 2021
3731ccb
Bump keyring from 21.8.0 to 22.0.1
dependabot[bot] Jan 28, 2021
2a61876
Merge branch 'develop' of https://github.com/sriniv27/jrnl into develop
sriniv27 Jan 29, 2021
9a19c37
skip on win
sriniv27 Jan 28, 2021
8ea258c
refactor deserialization
sriniv27 Jan 29, 2021
3b7e05c
update tests ith better verifications
sriniv27 Jan 28, 2021
adb7932
make format
sriniv27 Jan 28, 2021
56255cd
space
sriniv27 Jan 28, 2021
4205543
Merge branch 'develop' of https://github.com/sriniv27/jrnl into develop
sriniv27 Jan 29, 2021
ac85511
make format
sriniv27 Jan 29, 2021
dbf4650
document apply_overrides
sriniv27 Jan 29, 2021
7df83aa
update gitignore
sriniv27 Jan 30, 2021
40f93a9
document config-override enhancement
sriniv27 Jan 30, 2021
d9f441f
Merge branch 'develop' into lockfile-merge
sriniv27 Jan 31, 2021
6e658c3
Simplify config override syntax (#5)
sriniv27 Feb 1, 2021
e26fd07
update documentation to sphinx style
sriniv27 Feb 1, 2021
318e45e
variable renames
sriniv27 Feb 1, 2021
a51db6c
Bump pytz from 2020.5 to 2021.1 (#6)
dependabot[bot] Feb 1, 2021
d55fa41
Lockfile merge (#7)
sriniv27 Feb 1, 2021
2be0d37
Simplify config override syntax (#8)
sriniv27 Feb 1, 2021
f250406
Merge branch 'develop' into lockfile-merge
sriniv27 Feb 1, 2021
44310cb
formatting
sriniv27 Feb 1, 2021
f6e6436
Update pyproject.toml
sriniv27 Feb 1, 2021
ac819eb
update lockfile to remove pytest-cov and pytest-mock deps
sriniv27 Feb 1, 2021
9b80d56
update docs
sriniv27 Feb 9, 2021
c8e09b0
reuse existing mock; delete unneeded code
sriniv27 Feb 9, 2021
1df4b74
move overrides earlier in the execution
sriniv27 Feb 9, 2021
93c6221
update for passworded access
sriniv27 Feb 10, 2021
11f4a87
test that no editor is launched
sriniv27 Feb 10, 2021
336df64
remove unnecessary mocks
sriniv27 Feb 10, 2021
113d5e7
rename variable for intent
sriniv27 Feb 10, 2021
426deee
reinstate getpass deletion
sriniv27 Feb 10, 2021
a110f44
update gitignore
sriniv27 Feb 10, 2021
ec9b078
capture failure mode
sriniv27 Feb 10, 2021
e889585
remove unneeded imports
sriniv27 Feb 12, 2021
0bf4e52
renamed variable
sriniv27 Feb 12, 2021
4f93318
delete redundant step
sriniv27 Feb 12, 2021
ccba680
comment on step
sriniv27 Feb 12, 2021
0f80201
clean up step behavior description
sriniv27 Feb 12, 2021
b6fa684
[WIP] lock down journal access behavior
sriniv27 Feb 12, 2021
be2844f
skip -> wip
sriniv27 Feb 12, 2021
304b39c
correct command for overriding journal via dot keys
sriniv27 Feb 13, 2021
8d065a4
update wip test for updating a "temp" journal and then reading baack …
sriniv27 Feb 13, 2021
8d952ba
remove "mock" from poetry file
sriniv27 Feb 13, 2021
20450a9
make CI happy
sriniv27 Feb 13, 2021
531fba3
complex behavior sequence for default journal override
sriniv27 Feb 14, 2021
f36c5c6
separate out smaller pieces of logic
sriniv27 Feb 14, 2021
4ef277e
defer modification of loaded configuration to update_config
sriniv27 Feb 15, 2021
dd0c5a6
Update .gitignore
sriniv27 Feb 16, 2021
87a1912
remove skip_win
sriniv27 Feb 16, 2021
bd0394c
Merge branch 'lockfile-merge' of https://github.com/sriniv27/jrnl int…
sriniv27 Feb 16, 2021
ca8e9f8
forward override unpacking to yaml library
sriniv27 Feb 17, 2021
0b79128
merge config override step with existing config_var step in core
sriniv27 Feb 18, 2021
289a307
delete unused and redundant code
sriniv27 Feb 18, 2021
7564cfb
rebases are hard
sriniv27 Feb 18, 2021
85ae0f4
remove wip tag from test
sriniv27 Feb 20, 2021
0a2305f
remove skipped tests for windows
sriniv27 Feb 28, 2021
21c244e
Address code review
sriniv27 Feb 28, 2021
1af8c79
consolidate imports
sriniv27 Feb 28, 2021
9901ccd
Defer config_override unpacking to dict *after* base config is loaded
sriniv27 Feb 28, 2021
1327985
rename deserialize_config_args to better express intent
sriniv27 Mar 1, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,10 @@ exp/
_extras/
*.sublime-*
site/

.vscode/settings.json
coverage.xml
.vscode/launch.json
.coverage
.vscode/tasks.json
todo.txt
23 changes: 23 additions & 0 deletions docs/advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,29 @@ and can be edited with a plain text editor.
Or use the built-in prompt or an external editor to compose your
entries.

### Modifying Configurations from the Command line

You can override a configuration field for the current instance of `jrnl` using `--config-override CONFIG_KEY CONFIG_VALUE` where `CONFIG_KEY` is a valid configuration field, specified in dot-notation and `CONFIG_VALUE` is the (valid) desired override value.

You can specify multiple overrides as multiple calls to `--config-override`.
!!! note
These overrides allow you to modify ***any*** field of your jrnl configuration. We trust that you know what you are doing.

#### Examples:

``` sh
#Create an entry using the `stdin` prompt, for rapid logging
jrnl --config-override editor ""

#Populate a project's log
jrnl --config-override journals.todo "$(git rev-parse --show-toplevel)/todo.txt" todo find my towel

#Pass multiple overrides
jrnl --config-override display_format fancy --config-override linewrap 20 \
--config-override colors.title green

```

## Multiple journal files

You can configure `jrnl`to use with multiple journals (eg.
Expand Down
27 changes: 27 additions & 0 deletions docs/recipes.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,33 @@ only field 1.
jrnl -on "$(jrnl --short | shuf -n 1 | cut -d' ' -f1,2)"
```


### Launch a terminal for rapid logging
You can use this to launch a terminal that is the `jrnl` stdin prompt so you can start typing away immediately.

```bash
jrnl now --config-override editor:""
```

Bind this to a keyboard shortcut.

Map `Super+Alt+J` to launch the terminal with jrnl prompt

- **xbindkeys**
In your `.xbindkeysrc`

```ini
Mod4+Mod1+j
alacritty -t floating-jrnl -e jrnl now --config-override editor:"",
```

- **I3 WM** Launch a floating terminal with the `jrnl` prompt

```ini
bindsym Mod4+Mod1+j exec --no-startup-id alacritty -t floating-jrnl -e jrnl --config-override editor:""
for_window[title="floating *"] floating enable
```

## External editors

Configure your preferred external editor by updating the `editor` option
Expand Down
98 changes: 98 additions & 0 deletions features/overrides.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
Feature: Implementing Runtime Overrides for Select Configuration Keys

Scenario: Override configured editor with built-in input === editor:''
Given we use the config "basic_encrypted.yaml"
And we use the password "test" if prompted
When we run "jrnl --config-override editor ''"
Then the stdin prompt should have been called

Scenario: Postconfig commands with overrides
Given We use the config "basic_encrypted.yaml"
And we use the password "test" if prompted
When we run "jrnl --decrypt --config-override highlight false --config-override editor nano"
Then the config should have "highlight" set to "bool:false"
And no editor should have been called

Scenario: Override configured linewrap with a value of 23
Given we use the config "simple.yaml"
And we use the password "test" if prompted
When we run "jrnl -2 --config-override linewrap 23 --format fancy"
Then the output should be

"""
┎─────╮2013-06-09 15:39
┃ My ╘═══════════════╕
┃ fir st ent ry. │
sriniv27 marked this conversation as resolved.
Show resolved Hide resolved
┠╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
┃ Everything is │
┃ alright │
┖─────────────────────┘
┎─────╮2013-06-10 15:40
┃ Lif ╘═══════════════╕
┃ e is goo d. │
┠╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
┃ But I'm better. │
┖─────────────────────┘
"""

Scenario: Override color selections with runtime overrides
Given we use the config "basic_encrypted.yaml"
And we use the password "test" if prompted
When we run "jrnl -1 --config-override colors.body blue"
Then the config should have "colors.body" set to "blue"

Scenario: Apply multiple config overrides
Given we use the config "basic_encrypted.yaml"
And we use the password "test" if prompted
When we run "jrnl -1 --config-override colors.body green --config-override editor 'nano'"
Then the config should have "colors.body" set to "green"
And the config should have "editor" set to "nano"


Scenario Outline: Override configured editor
Given we use the config "basic_encrypted.yaml"
And we use the password "test" if prompted
When we run "jrnl --config-override editor '<editor>'"
Then the editor <editor> should have been called
Examples: Editor Commands
| editor |
| nano |
| vi -c startinsert |
| code -w |

Scenario: Override default journal
Given we use the config "basic_dayone.yaml"
And we use the password "test" if prompted
When we run "jrnl --debug --config-override journals.default features/journals/simple.journal 20 Mar 2000: The rain in Spain comes from clouds"
Then we should get no error
And we should see the message "Entry added"
When we run "jrnl -3 --debug --config-override journals.default features/journals/simple.journal"
Then the output should be
"""
2000-03-20 09:00 The rain in Spain comes from clouds

2013-06-09 15:39 My first entry.
| Everything is alright

2013-06-10 15:40 Life is good.
| But I'm better.
"""


Scenario: Make an entry into an overridden journal
Given we use the config "basic_dayone.yaml"
And we use the password "test" if prompted
When we run "jrnl --config-override journals.temp features/journals/simple.journal temp Sep 06 1969: @say Ni"
Then we should get no error
And we should see the message "Entry added"
When we run "jrnl --config-override journals.temp features/journals/simple.journal temp -3"
Then the output should be
"""
1969-09-06 09:00 @say Ni

2013-06-09 15:39 My first entry.
| Everything is alright

2013-06-10 15:40 Life is good.
| But I'm better.
"""
41 changes: 37 additions & 4 deletions features/steps/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

import ast
from collections import defaultdict
from jrnl.args import parse_args
import os
from pathlib import Path
import re
Expand All @@ -13,8 +14,11 @@
from behave import then
from behave import when
import keyring

import toml
import yaml
from yaml.loader import FullLoader


import jrnl.time
from jrnl import Journal
Expand All @@ -23,6 +27,7 @@
from jrnl.cli import cli
from jrnl.config import load_config
from jrnl.os_compat import split_args
from jrnl.override import apply_overrides, _recursively_apply

try:
import parsedatetime.parsedatetime_consts as pdt
Expand Down Expand Up @@ -114,8 +119,15 @@ def read_value_from_string(string):
return ast.literal_eval(string)

# Takes strings like "bool:true" or "int:32" and coerces them into proper type
t, value = string.split(":")
value = {"bool": lambda v: v.lower() == "true", "int": int, "str": str}[t](value)
string_parts = string.split(":")
if len(string_parts) > 1:
type = string_parts[0]
value = string_parts[1:][0] # rest of the text
value = {"bool": lambda v: v.lower() == "true", "int": int, "str": str}[type](
value
)
else:
value = string_parts[0]
return value


Expand Down Expand Up @@ -315,6 +327,7 @@ def run_with_input(context, command, inputs=""):
text = iter([inputs])

args = split_args(command)[1:]
context.args = args

def _mock_editor(command):
context.editor_command = command
Expand Down Expand Up @@ -397,8 +410,13 @@ def run(context, command, text=""):
if "cache_dir" in context and context.cache_dir is not None:
cache_dir = os.path.join("features", "cache", context.cache_dir)
command = command.format(cache_dir=cache_dir)
if "config_path" in context and context.config_path is not None:
with open(context.config_path, "r") as f:
cfg = yaml.load(f, Loader=FullLoader)
context.jrnl_config = cfg

args = split_args(command)
context.args = args[1:]

def _mock_editor(command):
context.editor_command = command
Expand Down Expand Up @@ -604,14 +622,29 @@ def journal_exists(context, journal_name="default"):
@then('the config should have "{key}" set to "{value}"')
@then('the config for journal "{journal}" should have "{key}" set to "{value}"')
def config_var(context, key, value="", journal=None):
key_as_vec = key.split(".")

if "args" in context:
parsed = parse_args(context.args)
overrides = parsed.config_override
value = read_value_from_string(value or context.text or "")
configuration = load_config(context.config_path)

if journal:
configuration = configuration["journals"][journal]

assert key in configuration
assert configuration[key] == value
if overrides:
with patch.object(
jrnl.override, "_recursively_apply", wraps=_recursively_apply
) as spy_recurse:
configuration = apply_overrides(overrides, configuration)
runtime_cfg = spy_recurse.call_args_list[0][0][0]
else:
runtime_cfg = configuration
# extract the value of the desired key from the configuration after overrides have been applied
for k in key_as_vec:
runtime_cfg = runtime_cfg["%s" % k]
assert runtime_cfg == value


@then('the config for journal "{journal}" should not have "{key}" set')
Expand Down
77 changes: 77 additions & 0 deletions features/steps/override.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
from jrnl.jrnl import run
from unittest import mock

# from __future__ import with_statement
from jrnl.args import parse_args
from behave import then

from features.steps.core import _mock_getpass, _mock_time_parse


@then("the editor {editor} should have been called")
@then("No editor should have been called")
def editor_override(context, editor=None):
def _mock_write_in_editor(config):
editor = config["editor"]
journal = "features/journals/journal.jrnl"
context.tmpfile = journal
print("%s has been launched" % editor)
return journal

if "password" in context:
password = context.password
else:
password = ""
# fmt: off
# see: https://github.com/psf/black/issues/664
with \
mock.patch("jrnl.jrnl._write_in_editor", side_effect=_mock_write_in_editor(context.jrnl_config)) as mock_write_in_editor, \
mock.patch("sys.stdin.isatty", return_value=True), \
mock.patch('getpass.getpass',side_effect=_mock_getpass(password)), \
mock.patch("jrnl.time.parse", side_effect = _mock_time_parse(context)), \
mock.patch("jrnl.config.get_config_path", side_effect=lambda: context.config_path), \
mock.patch("jrnl.install.get_config_path", side_effect=lambda: context.config_path) \
:
try :
parsed_args = parse_args(context.args)
run(parsed_args)
context.exit_status = 0
context.editor = mock_write_in_editor
expected_config = context.jrnl_config
expected_config['editor'] = '%s'%editor
expected_config['journal'] ='features/journals/journal.jrnl'

if editor is not None:
assert mock_write_in_editor.call_count == 1
assert mock_write_in_editor.call_args[0][0]['editor']==editor
else:
# Expect that editor is *never* called
mock_write_in_editor.assert_not_called()
except SystemExit as e:
context.exit_status = e.code
# fmt: on


@then("the stdin prompt should have been called")
def override_editor_to_use_stdin(context):

try:
with mock.patch(
"sys.stdin.read",
return_value="Zwei peanuts walk into a bar und one of zem was a-salted",
) as mock_stdin_read, mock.patch(
"jrnl.install.load_or_install_jrnl", return_value=context.jrnl_config
), mock.patch(
"jrnl.Journal.open_journal",
spec=False,
return_value="features/journals/journal.jrnl",
), mock.patch(
"getpass.getpass", side_effect=_mock_getpass("test")
):
parsed_args = parse_args(context.args)
run(parsed_args)
context.exit_status = 0
mock_stdin_read.assert_called_once()

except SystemExit as e:
context.exit_status = e.code
23 changes: 23 additions & 0 deletions jrnl/args.py
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,29 @@ def parse_args(args=[]):
help=argparse.SUPPRESS,
)

config_overrides = parser.add_argument_group(
"Config file override",
textwrap.dedent("Apply a one-off override of the config file option"),
)
config_overrides.add_argument(
"--config-override",
dest="config_override",
action="append",
type=str,
nargs=2,
default=[],
metavar="CONFIG_KV_PAIR",
help="""
Override configured key-value pair with CONFIG_KV_PAIR for this command invocation only.

Examples: \n
\t - Use a different editor for this jrnl entry, call: \n
\t jrnl --config-override editor: "nano" \n
\t - Override color selections\n
\t jrnl --config-override colors.body blue --config-override colors.title green
""",
)

# Handle '-123' as a shortcut for '-n 123'
num = re.compile(r"^-(\d+)$")
args = [num.sub(r"-n \1", arg) for arg in args]
Expand Down
Loading