Forge asks for user name and password for Github requests

[DSMasterson]

Am I missing something? Magit and Forge (and MagitHub before) worked fine for me for everything I've done with my repository on GitHub except that Emacs (I suspect GHub) always asks me for the user name and password for each operation.

• I've globally set github.user in git config
• I didn't set github.HOST.user as I am not an Enterprise user.
• I've created my Personal Token
• I've added a 'USER^forge' line to ~/.authinfo
• I've added "~/.authinfo" to auth-sources
• forge-database.sqlite was created, but I can't tell what's in it.

What else am I supposed to do?

Emacs 26.3
forge-20200725.1419
ghub-20200801.808

[DSMasterson]

Oh, and I use my Personal Token as the password when Emacs asks. I don't get any errors in this process.

[tarsius]

All of that sounds about right. I cannot tell what you might have done wrong. Sorry.

[DSMasterson]

Is there any debugging steps?

[tarsius]

asks me for the user name and password for each operation.

Please post the prompts used to do so.

[DSMasterson]

It first asks for the username in the Emacs minibuffer with this prompt:

Username for 'https://github.com':

It then asks for the password in the minibuffer with:

Password for 'https://DSMasterson@github.com':

Filling in the password causes the operation (usually a Push) to complete properly.

[tarsius]

Username for 'https://github.com':

This prompt comes from url. It is possible that ghub and forge are not even involved in whatever is trying to make a connection here. Do M-x toggle-debug-on-error then do what ever gets you to that prompt, abort and post the backtrace.

Also evaluate:

(list
 (ignore-errors (forge-get-repository t))
 (ignore-errors (ghub--host nil))
 (ignore-errors (ghub--username (ghub--host nil) nil)))

If ghub--host and/or ghub--username return nil, then look at their definitions and reason about what might be wrong. Maybe use debug statements to investigate.

[DSMasterson]

Emacs does not report an error, so no backtrace.

Your list expression reports the proper host and username.

However, I'm finding that ghub.el is not loaded initially, so your list expression only works after loading the library. Even so, after loading the library, Magit still behaves the same way and works fine.

When is ghub.el loaded normally? Is it not supposed to be loaded because I'm somehow working with HTTP and, thus, the url library? Is there something I'm supposed to change to move from url to forge?

[DSMasterson]

Further updates:

1. I load magit and forge with use-package in my .emacs.
2. After starting emacs, featurep show that magit and forge are not loaded.
3. Running magit in a git directory, featurep shows magit, forge, ghub, and url loaded.
4. The forge-dispatch menu of all the Issue related commands comes up properly.
5. Just to try it, 'f f' started to fetch all issues, but asked for Username/Password.
6. Debug-on-quit, shows that this is being asked for out of magit-process-username-promp in magit-process.el
7. I thought forge & ghub were supposed to takeover querying Github?

What am I missing?

[tarsius]

I thought forge & ghub were supposed to takeover querying Github?

No. ghub.el just talks to the API of Github/Gitlab/.... To push and pull using git you still need to configure git itself.

[Boruch-Baum]

I'm having this trouble as a reproducible bug whenever attempting a commit to a "second" repository after launching emacs.
In order to reproduce:

1. exit and restart emacs
2. use magit to perform a commit for repository foo and use forge to push the commit
3. use magit to perform a commit for repository bar
4. The prompt appears, using my OS account name (ie. $USER) instead of the global git setting.
5. Upon aborting, using C-g, magit/forge fails to kill the commit buffer which had been silently and prematurely created.

[tarsius]

I'm having this trouble as a reproducible bug whenever attempting a commit to a "second" repository after launching emacs.

That sounds like a plausible cause. We had bugs that caused default-directory to be wrong, which causes things to happen in the wrong repository...

In order to reproduce:

... but I am having issues with that. It's not precise enough.

use forge to push the commit

How exactly do you do that? I don't think forge can push git data, or are you talking about something else?

The prompt appears

Please post that prompt here so I know which one you are talking about.

[Boruch-Baum]

use forge to push the commit

How exactly do you do that? I don't think forge can push git data, or are you talking about something else?

You're correct. I meant to write that I used the standard magit key sequence c c to perform the commit.

The prompt appears

Please post that prompt here so I know which one you are talking about.

Username [for https://api.github.com/repos/Boruch-Baum/emacs-key-assist]: boruch

Upon escaping the prompt, using C-g, I get the following (copied from the *Messages* buffer):

error in process filter: Quit [2 times]
Quit emacsclient request
There was a problem with the editor '/usr/bin/emacsclient.emacs --socket-name=/tmp/user/1001/emacs1001/server'. ... [Hit $ to see buffer magit-process: key-assist for details]
Quit

With an almost identical corresponding message on the first line of the magit status buffer:

GitError! There was a problem with the editor '/usr/bin/emacsclient.emacs --socket-name=/tmp/user/1001/emacs1001/server'.  [Type `$' for details]

And a commit buffer that had been silently (and prematurely) created remains. It can't be deleted from the iBuffer buffer. One needs to select it and then kill it. Here's its line in the iBuffer buffer.

Does this issue get re-opened? Doing so might attract more input from others.

 *  COMMIT_EDITMSG           263 Text             ~/Projects/emacs-related/key-assist/.git/COMMIT_EDITMSG

The error message for this additional undesirable anomaly is:

Don’t kill this buffer #<buffer COMMIT_EDITMSG>.  Instead cancel using C-c C-k

Which is misleading since the C-c C-k keybinding isn't active in the iBuffer. The final part of the message should say something like "Instead, select the buffer and then cancel the operation using C-c C-k."

[tarsius]

@Boruch-Baum Please provide the same output that I requested from @DSMasterson.

[Boruch-Baum]

Do you really want all that? Just the output from the list evaluation you asked for is 308 lines (evaluated from the minibuffer, while being prompted for the github username). Also, it includes at least two hashed strings that for personal security I wouldn't want to share unless I knew to what they decode. If you're sure you want all 308 lines, let me know and fine, I'll just redact the two hashes.

[tarsius]

Leave out the forge-get-repository part.

[Boruch-Baum]

Big difference:

("api.github.com" "Boruch-Baum")

[tarsius]

That's how it is supposed to be.

Do M-x toggle-debug-on-quit and initiate a commit. Now quitting the prompt should give you a meaningful backtrace.

[Boruch-Baum]

OK. Here goes.

Debugger entered--Lisp error: (quit)
  read-string("Username [for https://api.github.com/repos/Boruch-Baum/emacs-diredc]: " "boruch")
  url-basic-auth(#s(url :type "https" :user nil :password nil :host "api.github.com" :portspec nil :filename "/repos/Boruch-Baum/emacs-diredc" :target nil :attributes nil :fullness t :silent nil :use-cookies t :asynchronous t) t nil nil (("basic")))
  url-get-authentication("https://api.github.com/repos/Boruch-Baum/emacs-diredc" nil "basic" t (("basic")))
  url-http-handle-authentication(nil)
  url-http-parse-headers()
  url-http-content-length-after-change-function(984 1065 81)
  url-http-wait-for-headers-change-function(1 1087 1086)
  url-http-generic-filter(#<process api.github.com> "HTTP/1.1 401 Unauthorized\015\nServer: GitHub.com\015\nDate: Sun, 15 Nov 2020 23:42:34 GMT\015\nContent-Type: application/json; charset=utf-8\015\nContent-Length: 80\015\nStatus: 401 Unauthorized\015\nX-GitHub-Media-Type: github.v3; format=json\015\nX-RateLimit-Limit: 60\015\nX-RateLimit-Remaining: 55\015\nX-RateLimit-Reset: 1605484993\015\nX-RateLimit-Used: 5\015\nAccess-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset\015\nAccess-Control-Allow-Origin: *\015\nStrict-Transport-Security: max-age=31536000; includeSubdomains; preload\015\nX-Frame-Options: deny\015\nX-Content-Type-Options: nosniff\015\nX-XSS-Protection: 1; mode=block\015\nReferrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin\015\nContent-Security-Policy: default-src 'none'\015\nVary: Accept-Encoding, Accept, X-Requested-With\015\nX-GitHub-Request-Id: --redacted by boruch (sorry, Tarsius)--\015\n\015\n{\"message\":\"Bad credentials\",\"documentation_url\":\"https://docs.github.com/rest\"}")
  accept-process-output(#<process api.github.com> 1)
  url-retrieve-synchronously(#s(url :type "https" :user nil :password nil :host "api.github.com" :portspec nil :filename "/repos/Boruch-Baum/emacs-diredc" :target nil :attributes nil :fullness t :silent nil :use-cookies t :asynchronous nil) nil)
  ghub--retrieve(nil #s(ghub--req :url #s(url :type "https" :user nil :password nil :host "api.github.com" :portspec nil :filename "/repos/Boruch-Baum/emacs-diredc" :target nil :attributes nil :fullness t :silent nil :use-cookies t :asynchronous nil) :forge nil :silent nil :method "GET" :headers #f(compiled-function () #<bytecode 0x230dc49>) :handler ghub--handle-response :unpaginate nil :noerror nil :reader nil :callback nil :errorback nil :value nil :extra nil))
  ghub-request("GET" "/repos/Boruch-Baum/emacs-diredc" nil :query nil :payload nil :unpaginate nil :headers nil :username nil :auth magithub :host nil)
  ghubp-request(get "/repos/Boruch-Baum/emacs-diredc" nil nil)
  apply(ghubp-request get "/repos/Boruch-Baum/emacs-diredc" (nil nil))
  ghubp-get-repos-owner-repo(((owner (login . "Boruch-Baum")) (name . "emacs-diredc")))
  (let ((G186 (ghubp-get-context)) ghubp-contextualize-function) (push (cons 'auth 'magithub) G186) (setq ghubp-contextualize-function (lambda nil G186)) (ghubp-get-repos-owner-repo '((owner (login . "Boruch-Baum")) (name . "emacs-diredc"))))
  (ghubp-override-context auth 'magithub (ghubp-get-repos-owner-repo '((owner (login . "Boruch-Baum")) (name . "emacs-diredc"))))
  (magithub-request (ghubp-get-repos-owner-repo '((owner (login . "Boruch-Baum")) (name . "emacs-diredc"))))
  (or (magithub-request (ghubp-get-repos-owner-repo '((owner (login . "Boruch-Baum")) (name . "emacs-diredc")))) (and (not (magithub--api-available-p)) sparse-repo))
  eval((or (magithub-request (ghubp-get-repos-owner-repo '((owner (login . "Boruch-Baum")) (name . "emacs-diredc")))) (and (not (magithub--api-available-p)) sparse-repo)))
  magithub-cache(:repo-demographics (or (magithub-request (ghubp-get-repos-owner-repo '((owner (login . "Boruch-Baum")) (name . "emacs-diredc")))) (and (not (magithub--api-available-p)) sparse-repo)))
  magithub-repo()
  magithub-bug-reference-mode-on()
  run-hooks(git-commit-setup-hook)
  git-commit-setup()
  git-commit-setup-check-buffer()
  run-hooks(find-file-hook)
  after-find-file(nil t)
  find-file-noselect-1(#<buffer COMMIT_EDITMSG> "~/Projects/emacs-related/diredc/.git/COMMIT_EDITMSG" nil nil "~/Projects/emacs-related/diredc/.git/COMMIT_EDITMSG" (1843114 2051))
  find-file-noselect("/home/boruch/Projects/emacs-related/diredc/.git/COMMIT_EDITMSG")
  #f(compiled-function (files proc &optional nowait) "Find FILES and return a list of buffers created.\nFILES is an alist whose elements are (FILENAME . FILEPOS)\nwhere FILEPOS can be nil or a pair (LINENUMBER . COLUMNNUMBER).\nPROC is the client that requested this operation.\nNOWAIT non-nil means this client is not waiting for the results,\nso don't mark these buffers specially, just visit them normally." #<bytecode 0x145d5e1>)((("/home/boruch/Projects/emacs-related/diredc/.git/COMMIT_EDITMSG")) #<process server <542>> nil)
  apply(#f(compiled-function (files proc &optional nowait) "Find FILES and return a list of buffers created.\nFILES is an alist whose elements are (FILENAME . FILEPOS)\nwhere FILEPOS can be nil or a pair (LINENUMBER . COLUMNNUMBER).\nPROC is the client that requested this operation.\nNOWAIT non-nil means this client is not waiting for the results,\nso don't mark these buffers specially, just visit them normally." #<bytecode 0x145d5e1>) ((("/home/boruch/Projects/emacs-related/diredc/.git/COMMIT_EDITMSG")) #<process server <542>> nil))
  server-visit-files((("/home/boruch/Projects/emacs-related/diredc/.git/COMMIT_EDITMSG")) #<process server <542>> nil)
  server-execute(#<process server <542>> (("/home/boruch/Projects/emacs-related/diredc/.git/COMMIT_EDITMSG")) nil nil nil nil nil)
  #f(compiled-function () #<bytecode 0x270c3ad>)()
  server-execute-continuation(#<process server <542>>)
  server-process-filter(#<process server <542>> "-dir /home/boruch/Projects/emacs-related/diredc/ -current-frame -tty /dev/pts/13 dumb -file /home/boruch/Projects/emacs-related/diredc/.git/COMMIT_EDITMSG \n")

BTW, I may as well plug the repository. My package diredc is currently a candidate for MELPA, and simultaneously under consideration for ELPA or emacs-core. If you kind-of like midnight commander, the package brings many of its features to dired; it also completes the previously partial emacs support for Xfreedesktop trashing. See the README for a list of its features.

[tarsius]

So there are two issues.

1. Github rejects the request that ghub made on behalf of magithub (not forge) using a 401 (Unauthorized) status and providing this additional data {"message":"Bad credentials","documentation_url":"https://docs.github.com/rest"}.

   I got Github to react like that myself by feeding it an invalid token along my valid username. I am guessing that this happens for you because you deleted your magithub token on the webpage but did not remove id locally so magithub continues to use it.

2. The url package (not ghub) responds incorrectly, attempting to make another request, this time around using basic authentication. What it should do instead is to report the above error like so: ghub--signal-error: HTTP Error: 401, "Unauthorized", "/repos/tarsius/git-handler", ((message . "Bad credentials") (documentation_url . "https://docs.github.com/rest")).

I have added an advice to inject the correct behavior into url. The commit message has some additional details:

Advice url-http-handle-authentication with a bugfix

Upon receiving a 401 or 409 url-http-parse-headers calls
url-http-handle-authentication, which then makes another
request, which now provides the authentication information.

Unfortunately that function does not react correctly when the previous
request provided authentication information but the server rejected it.
In that case it would be appropriate to signal a http error.

On Emacs releases before 27.1 it instead makes another request, which
can result in either an infinite loop if all the information required
for a request using basic authentication is already available, until
the server refuses to comply; or, as in the case of ghub it results
in the user being asked for their username and/or password, which is
very confusing.

On Emacs 27.1 it does signal an error but without using the mechanism
intended to do so, resulting in an error that does not contain all the
information needed to properly deal with it.

---

By the way, I recommend you stop using magithub. forge still lacks a few of its features but using both packages together is asking for troubles and bad performance. One major disadvantage of magithub is that it makes network request at essentially random times (such as we have seen here, when creating a commit).

[Boruch-Baum]

I've taken your advice about removing magithub and can confirm that after unloading features magithub* I am no longer prompted for my github account name etc. I haven't yet needed to actually make and push a commit to github to further test but will report. Since you've closed the issue, I'll only report back if I encounter a problem. BTW, magithub was architectured as a set of 15+ feautures (!!), so for the benefit of potentially other readers ... (dolist (f features) (when (string-match "magithub" (symbol-name f)) (unload-feature f))) If magithub is such trouble, why not remove it from melpa already? An alternative might be to push a commit to the package that warns users when using its interactive functions that it has become a source of trouble etc. Thanks for tracking all this down, through the different packages and emacs versions.

…

-- hkp://keys.gnupg.net CA45 09B5 5351 7C11 A9D1 7286 0036 9E45 1595 8BC0

[tarsius]

If magithub is such trouble, why not remove it from melpa already?

@vermiculus It might be time to think about that.

[tarsius]

Reported in https://debbugs.gnu.org/cgi/bugreport.cgi?bug=50511.
Fixed in emacs commit 0b98ea5fbe276c67206896dca111c000f984ee0f.