MAINT: IPython history file per host + configurable

[ZLLentz]

Description

Main changes:

• Add a --hist-file cli argument
• Default the history file to /u1/${USER}/hutch-python/history.sqlite (local disk)
  • Previously: normal IPython history per user home directory on NFS
• Add a unit test

Minor changes:

• Make the cli args type-hinted
• Fix some minor styling inconsistencies

Motivation and Context

https://jira.slac.stanford.edu/browse/ECS-5096

• On NFS for some of our endstations, there are some file locking issues related to the ipython history file.
• Bringing it off of NFS should theoretically alleviate the issues, at the cost of fracturing history per-host.
• Use CLI over cfg file because this would hypothetically need to be configured per-session, whereas the cfg file is shared between all sessions for one hutch.

How Has This Been Tested?

• Interactively as me
• Unit tests
• Tested as xcsopr and it works as expected: each of xcs-daq and xcs-control has its own separate history file

Where Has This Been Documented?

Only here, I need to write pre-release docs

Pre-merge checklist

• Code works interactively (a real hutch config file can be loaded)
• Code contains descriptive docstrings, including context and API
• New/changed functions and methods are covered in the test suite where possible
• Test suite passes locally
• Test suite passes on GitHub Actions
• Ran docs/pre-release-notes.sh and created a pre-release documentation page
• Pre-release docs include context, functional descriptions, and contributors as appropriate

[tangkong]

I guess I should start doing this dataclass-namespace-for-cli-args thing too.

If I understand this right, the way to return to the previous behavior (storing in the ~/.ipython directory) is to invoke:

hutch-python --hist-file=:memory"

?

[tangkong]

So the default is now to have each user write to their own history file? What exactly are the conditions for having a /u1/<user> directory on a given host? I don't have one on psbuild so this defaults to the standard location (~/.ipython/profile_default/...)

[ZLLentz]

As written this only affects e.g. xppopr running on xpp-daq and similar operator accounts.
If you run as zlentz or on psbuild for example, you'll use the normal per-user file.

[ZLLentz]

Would it be prudent for normal users to have a fallback to some per-host NFS location in the ipython config directory?

[tangkong]

I think so, in the interest of keeping original ipython functionality available. I'd personally have left that as the default, and made the flag generate the file on a local disk.

Is it possible to do:

• no-flag: normal NFS location
• flag, no filename: default hist_file name in /u1/...
• flag, filename: history at the provided filename?

[ZLLentz]

I'll look into the flexibility of argparse tomorrow, but I agree that I should have made the default behavior no change from normal.

I'd like it to "just work" without too much bash-fu in e.g. the xcs3 launcher script (e.g. if xppopr do this, if not do that... is very annoying).

[tangkong]

I imagine adding just switching the xpppython script to read:

hutch-python --cfg "${HERE}/conf.yml" --hist-file

isn't too invasive, and we can switch it on as we get complaints about it.

[tangkong]

I think nargs='?' might work here:

parser = argparse.ArgumentParser()
parser.add_argument('--hist-file', dest='hist_file', nargs='?', const='/u1/...')
parser.parse_args([])  # Namespace(hist_file=None)
parser.parse_args(['--hist-file'])  # Namespace(hist_file='/u1/...')
parser.parse_args(['--hist-file', 'my_file/path'])  # Namespace(hist_file='my_file/path')

[ZLLentz]

I'll implement it exactly as you suggest, it's the cleanest overall and is opt-in

[ZLLentz]

Implemented as suggested, edited a bit so I could include another test, expanded to allow any environment vars, not just user, because why not

[ZLLentz]

If I understand this right, the way to return to the previous behavior

You've made me realize that the opr actually has no simple way to return the old previous behavior, as written. The only supported solution is to not make the special folder.

:memory: is actually an ipython built-in magic value for not using a file at all and storing the session history in the process memory.

[ZLLentz]

dataclass-namespace-for-cli-args thing too

I'm a fan of this because it helps vscode keep track of which arg is supposed to be which type

[tangkong]

LGTM, nice little extra mile with the env vars but I'd be immensely surprised if anyone used it.

[ZLLentz]

tbh it just makes it so I can add $HOSTNAME to the default later if we decide to do something like that