Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update transient.el. Don't intern transient symbols #273

Closed
wants to merge 1 commit into from

Conversation

joaotavora
Copy link

See Emacs bug#68568

Please use a dedicated feature branch for your feature request, instead of asking us to merge "your-fork/master" into the
"origin/master". The use of dedicated branches makes it much more convenient to deal with pull-requests, especially when > using Magit to do so.

I don't use Magit. This is a just a fix for a very annoying bug where completion gets completely broken just because I use a package that uses transient.

If you were about to open a pull-request asking us to merge your version of "master", then see [1] for instructions on how to quickly fix that and some information on why we ask you to do so.

YOu can push to my master of magit any time. The patch is trivial.

@psionic-k
Copy link
Contributor

I played with a backport to 0.5.2 and on master. I still see all of the commands in my completions. If that's the motivation for change, what are we doing?

(intern-soft "transient:tsc-cowsay:-b") ; transient:tsc-cowsay:-b

So the interning is happening anyway.

I'm not sure that an uninterned symbol can be used in a keymap, and if not, then the only implementations use private command names like these or re-implement everything on top repeated calls to read-key.

@joaotavora
Copy link
Author

I played with a backport to 0.5.2 and on master. I still see all of the commands in my completions. If

Likely the commands were already there from previous internations.

Try the patch from scratch or use this to unintern the existing ones:

(mapatoms (lambda (s) (when (string-match "^transient--:" (symbol-name s)) (unintern s))))

@psionic-k
Copy link
Contributor

I definitely did not just check for symbols on dirty Emacsen. You want to look at your obarray after loading GPTel etc, which just defines the prefixes.

@tarsius
Copy link
Member

tarsius commented Jan 21, 2024

I saw the bug and was gonna reply soon. Because I am on a productive bug squashing spree in another package and wanted to maintain the flow, I didn't reply immediately.

But I should sooner have helped you make your Emacs usable again:

(setq read-extended-command-predicate 'command-completion-default-include-p)

Transient does not do that itself, because "thou shalt not change options behind the users back". But I am now considering adding this:

(defun transient-command-completion-non-suffix-only-p (symbol _buffer)
  "Say whether SYMBOL should be offered as a completion.
If SYMBOL is only intended to be called as suffix command from
a transient prefix commands, return nil, t otherwise.  Consider
using `command-completion-default-include-p' instead."
  (if (get symbol 'completion-predicate)
      ;; An explicit completion predicate takes precedence.
      (funcall (get symbol 'completion-predicate) symbol buffer)
    (not (eq (car (command-modes symbol)) 'not-a-mode))))

(when (and (boundp 'read-extended-command-predicate)
           (not read-extended-command-predicate))
  (setq read-extended-command-predicate
        'transient-command-completion-non-suffix-only-p))

@joaotavora
Copy link
Author

You're right, this is not enough for all symbols. It is somehow enough for the really long ones

@joaotavora
Copy link
Author

Definitely don't set any options in user Emacs. I don't want to set that option: and I think it's perfectly legitimate to not want to and still be able to use completion elsewhere without monstrous useless transient commands following me around.

So could you explain why those symbols need to be interned in the obarray: (1) at compile-time (2) at load time (3) at run-time.

Also could you explain why some symbols need to have the actual description in their symbol names and why something like gensym couldn't be used instead?

@tarsius
Copy link
Member

tarsius commented Jan 21, 2024

Definitely don't set any options in user Emacs. I don't want to set that option:

I think you misunderstand what my predicate does. It excludes only those symbols that you don't want included. (Okay okay, it might also exclude symbols with non-nil completion-predicate, but that was a misguided last minute addition in my initial draft, just before posting it here, and I have already removed it again.)

Right now you only have a choice between setting that option to

(defun transient-command-completion-non-suffix-only-p (symbol _buffer)
  "Say whether SYMBOL should be offered as a completion.
If SYMBOL is only intended to be called as suffix command from
a transient prefix commands, return nil, t otherwise.  Consider
using `command-completion-default-include-p' instead."
  (not (eq (car (command-modes symbol)) 'not-a-mode)))

, live with the noise, or stop using transient.

(Choosing the first option will achieve your goal (and nothing else), so I hope you will reconsider.)

So could you explain why those symbols need to be interned in the obarray

Yes of course, but it will take some time because I will have to correct your misconceptions and because it is non-trivial in the first place. I hope that the presented solution is acceptable to you, at least as a workaround for another two days, while I finish the work I am currently immersed in.

@joaotavora
Copy link
Author

joaotavora commented Jan 21, 2024

Alright, i understand why interned symbols are needed finally. There are two keymaps with an indirection which is the symbol. So, here's a better patch, not much more complicated:

diff --git a/lisp/transient.el b/lisp/transient.el
index f9060f5ba85..1ee19d9944f 100644
--- a/lisp/transient.el
+++ b/lisp/transient.el
@@ -1098,6 +1098,10 @@ transient--parse-group
                   (cl-mapcan (lambda (s) (transient--parse-child prefix s))
                              spec))))))

+(defvar transient-ginormous-interned-symbols nil)
+(defun transient--intern-maybe (name)
+  (if transient-ginormous-interned-symbols (intern name) (gensym "transient:")))
+
 (defun transient--parse-suffix (prefix spec)
   (let (level class args)
     (cl-symbol-macrolet
@@ -1127,7 +1131,7 @@ transient--parse-suffix
        ((and (commandp car)
              (not (stringp car)))
         (let ((cmd pop)
-              (sym (intern
+              (sym (transient--intern-maybe
                     (format "transient:%s:%s"
                             prefix
                             (let ((desc (plist-get args :description)))
@@ -1156,7 +1160,7 @@ transient--parse-suffix
              (when-let ((shortarg (transient--derive-shortarg arg)))
                (setq args (plist-put args :shortarg shortarg)))
              (setq args (plist-put args :argument arg))))
-          (setq sym (intern (format "transient:%s:%s" prefix arg)))
+          (setq sym (transient--intern-maybe (format "transient:%s:%s" prefix arg)))
           (setq args (plist-put
                       args :command
                       `(prog1 ',sym

Before the patch:

src/emacs -Q -f fido-mode -L ~/Source/Emacs/gptel -l gptel-transient -f gptel-menu
h
C-g
M-x vc SPC

Will bring up gibberish.

After the patch it won't. You can customize transient-ginormous-interned-symbols to t to recover ginormous symbols.

I think you misunderstand what my predicate does.

No, I'm pretty sure I understand it fixes the problem. For me (so thanks!).

But this isn't about me. This is about preventing a library -- that isn't exactly an opt-out -- from polluting the obarray with commands that by definition don't need to be there except perhaps (but that depends on the implementation) for the time the transient menus are available.

in other words, this is a plain bug in my sight. We mustn't tell users of the flex completion style to either customize a bug away or stay out of a given package.

Are you sure there's no way to do exactly what transient does today while keeping the API and avoiding this pollution?

because it is non-trivial in the first place. I hope that the presented solution is acceptable to you, at least as a workaround for another two days, while I finish the work I am currently immersed in.

Sure, take your time, and don't hold back on the technical arguments.

tarsius added a commit that referenced this pull request Jan 22, 2024
By using a dedicated symbol, which uses our package prefix, we ensure
that we can set `read-extended-command-predicate', without affecting
any unrelated package.

`transient-command-completion-not-suffix-only-p' only prevents
"anonymously defined" transient suffix commands from being offered as
`read-extended-command' completion candidates.  Furthermore, because
we only set `read-extended-command-predicate' iff it is non-nil, this
does not mess with any user customization of this option.  Also note
that if the user uses `command-completion-default-include-p' instead,
then the suffix-only commands are also filtered out.

IMO it is reasonable for a package to use this feature to declare
that certain commands should absolutely never be offered as completion
candidates.  The interface of this feature is not optimized for this
use-case, but I predict that `read-extended-command-predicate' will
one day be a hook, at which point I would not have to write this
explanation in the hope that it convincingly makes the case for why
daring to us it to achieve the goal that it was designed to achieve
is not Wrong™, not even if it is a library that does it, as opposed
to a user.

It has been suggested that Transient should instead avoid `intern'ing
any symbols for these (admittedly not fully) anonymous suffix
commands.  The proposed patch does not actually work, because these
symbols *have to be* interned, and for backward compatibility reasons
`transient--init-suffix' ensures that they are.  So initially (at
load-time) these suffix command symbols would not be interned but as
the user invokes transient commands, more and more of them would
become interned.

We could avoid interning anonymously defined suffix commands
altogether, but that would have unfortunate consequences.  In fact
Transient used to do that because `read-extended-command-predicate'
did not exist yet.  But I sure wished it did, and I was considering
implementing something like that myself.  So of course once that
feature appear, I was happy to take advantage, in [1: 226db67].

Going back to not binding every suffix command to an interned symbol,
would complicate matters considerably.  It would lead to bugs, and
would slow down development, because I would constantly have to double
check changes, to reduce the risk of introducing such bugs.  The code
would be more difficult to understand.  Frankly, doing so would mean
intentionally adding accidental complexity.  I very much want to avoid
that oxymoron.

Furthermore, not interning anonymously defined suffixes, would do
nothing to hide infix arguments that are explicitly defined using
`transient-define-infix', but those should also not be offered as
completion candidates.  The approach I have chosen does.

Instead of #273.

1: 2023-08-12 226db67
   All suffix commands now must be accessed through fbound symbols
@tarsius
Copy link
Member

tarsius commented Jan 23, 2024

Unlike @psionic-k does at 68568, I agree with @joaotavora, that Transient should actively prevent the commands, that are only intended to be used from a transient prefix, from being presented as completion choices in execute-extended-command.

But I still think that the way to do that is by setting read-extended-command-predicate. I have further refined that in 522b625. That completely removes the possibility that any other commands are accidentally affected by Transient's predicate. The commit message explains why I think that is the right way to do it, answering some of the above questions in the process.

@joaotavora
Copy link
Author

@tarsius I didn't understand you meant to set this user-visible variable from transient.el's .

I'm fairly sure there are other ways to go about this, but fair enough. Anyway, if you want to do this, shouldn't you be using add-function instead of setq? Should be the go-to choice for this kinds of function-holding variables, and there's no if required.

(add-function :around read-extended-command-predicate #'funcall '((name . joaot/thingy)))   ;; dummy pass-through

@tarsius
Copy link
Member

tarsius commented Jan 23, 2024

shouldn't you be using add-function instead of setq? Should be the go-to choice for this kinds of function-holding variables, and there's no if required.

I was vaguely aware of this being a thing, but have never done it before, and am unclear on when one should and should not do that. Sounds like a good idea. (But it is more expensive.)

So I would do this, right?:

(defun transient--command-completion-not-suffix-only-p (fn symbol buffer)
  "Return nil if SYMBOL is only useful as a transient suffix command.
Otherwise, if that is non-nil, call FN and return its value.
FN is the value of `read-extended-command-predicate'.
If that is nil, return t."
  (cond ((eq (get symbol 'completion-predicate) 'transient--suffix-only)
         nil)
        (fn (funcall fn symbol buffer))
        (t)))

(static-if (boundp 'read-extended-command-predicate)
    (add-function :around read-extended-command-predicate
                  #'transient--command-completion-not-suffix-only-p))

Hm, the Custom interface (and describe-variable) don't deal with this in a useful manner:

Hide Read Extended Command Predicate: Choice: Value Menu Other predicate function: 
#[128 "\304\300\301�#\207"
  [transient--command-completion-not-suffix-only-p nil :around nil apply]
  5 advice]
    State : changed

I would have hoped that the user would be able to change the "core" value of the option, without discarding the advise. I.e., it probably would be better if read-extended-command-predicate were a hook.

@joaotavora
Copy link
Author

I would have hoped that the user would be able to change the "core" value of the option, without discarding the advise

I had exactly the same expectation. Maybe @monnier can shine some light here. Maybe it's not hard to fix...

. I.e., it probably would be better if read-extended-command-predicate were a hook.

Yup. IIUC this is one of the things add-function is supposed to allow: make a mode-tweakable hook out of any old function-holding variable. But I might be mistaken

@monnier
Copy link
Contributor

monnier commented Jan 23, 2024 via email

@tarsius
Copy link
Member

tarsius commented Jan 23, 2024

BTW, I still don't see why those symbols need to be interned, so at the very least, the code deserves a comment explaining why we have to do that (and then have to do the read-extended-command-predicate dance)

I am glad you used the term "dance" here, because that's exactly how I feel about this request.

I have already said that it would be possible to avoid interning these symbols. That is not actually true though -- but it would be possible to avoid making them fbound.

Every "suffix" consists of two parts: a command and an object. These two sides of the suffix have to somehow be linked to each other, and I decided that I would use a symbol to achieve that.

(transient-suffix-object this-command) and (oref obj command) are used to look up the other side of the suffix. The object is stored in a property I made up for that purpose, transient--suffix. For the command I chose to use the symbol's function slot (as is done nearly everywhere a command is identified by a symbol).

However, because I wanted to avoid polluting M-x, I did initially use another slot for "anonymous" commands. I was not happy with that from the start, and only saw it as a temporary work-around -- a "dance" -- only to be used until a proper solution came along, which eventually did: read-extended-command-predicate.

No longer did Transient have to bend over backward to avoid polluting M-x. As just mentioned, a suffix has two sides the command and the object, and they are linked via a symbol. While the old M-x-avoidance hack was in place, a second dichotomy was necessary; the command existed in a "call-interactive-able" form and in a "symbol-for-identification" form.

To get a symbol that can be used to locate the corresponding object, this-command had to be passed though this:

(defun transient--suffix-symbol (arg)
  "Return a symbol representing ARG.

ARG must be a command and/or a symbol.  If it is a symbol,
then just return it.  Otherwise return the symbol whose
`transient--infix-command' property's value is ARG."
  (or (cl-typep arg 'command)
      (cl-typep arg 'symbol)
      (signal 'wrong-type-argument `((command symbol) ,arg)))
  (if (symbolp arg)
      arg
    (let* ((obj (transient-suffix-object))
           (sym (oref obj command)))
      (if (eq (get sym 'transient--infix-command) arg)
          sym
        (catch 'found
          (mapatoms (lambda (sym)
                      (when (eq (get sym 'transient--infix-command) arg)
                        (throw 'found sym)))))))))

Nearly all commands that change the value of some argument can be implemented exactly the same:

(lambda ()
  (interactive)
   (let ((obj (transient-suffix-object)))
     (transient-infix-set obj (transient-infix-read obj)))
   (transient--show))

But for that to work transient-suffix-object has to be able to determine the corresponding object. With the current implementation that is done with (get this-command 'transient--object). Previously it had to pass this-command through the transient--suffix-symbol function shown above, to map through all symbols to find the one whose transient--infix-command property is eq to this-command. This also meant that even-though all those commands were equal, they could not be eq, leading to a lot of wasted memory.

I think adding a little function to read-extended-command-predicate is a better approach. It's easier to reason about, doesn't have to loop over all symbols to find the correct one, and avoids wasting memory.

But let's assume for a moment that we go back to the old kludge anyway. Some infix argument commands are not defined anonymously within the definition of a prefix command, because they are shared between multiple prefixes. For example:

(transient-define-argument magit:--gpg-sign ()
  :description "Sign using gpg"
  :class 'transient-option
  :shortarg "-S"
  :argument "--gpg-sign="
  :allow-empty t
  :reader #'magit-read-gpg-signing-key)

This is used by magit-commit, magit-merge, magit-rebase and a few other prefixes, but when called outside the context of a prefix, it is useless, just like the infixes defined anonymously for a single prefix. So this too should be excluded from M-x -- and how do we do that? We use an appropriate read-extended-command-predicate. And if we have to do that anyway to hide a bunch of named commands, can't we take advantage of that to hide the anonymously defined commands as well?

Finally, I don't think that Transient defining commands that are only useful in the context of a certain transient prefix command (and nevertheless putting these commands in the global namespace), is all that different, from a package defining commands that are only useful in the context a certain major-mode (and nevertheless putting these commands in the global namespace). If it is acceptable for these major-mode specific commands to "pollute M-x", then shouldn't it be acceptable for the transient-prefix specific commands to do the same? The pollution is a problem -- in both cases -- and read-extended-command-predicate helps addressing it -- again in both cases.

@monnier
Copy link
Contributor

monnier commented Jan 24, 2024 via email

@tarsius
Copy link
Member

tarsius commented Jan 24, 2024

Okay. I'll experiment.

@tarsius tarsius closed this Jan 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants