Unable to create GitRepository from private repos - authentication required.

[ldvy]

Describe the bug

Hello team,

I installed Flux and pointed it toward a private repository inside my organization. It worked perfectly - Flux created an SSH key in this repository using the supplied PAT and could track all the changes.
Git credentials were stored in the environment variables, and the full command looked like the following:
flux bootstrap github --owner=<organization>--repository=flux-config --branch=main --path=<k8s cluster name> --namespace=demo

The problem later arose when I tried to add another private Github repository containing my application's source code.
I ran the following command:
flux create source git <name> --url=https://github.com/<org>/<repo> --username=<my own username> --password=<password for my account> --branch=main

This exact command is mentioned in the following document for basic auth:
https://fluxcd.io/docs/cmd/flux_create_source_git/#examples

I also tried to use the PAT supplied upon installing Flux instead of the password, but it returned the same error.

Then I created a personal Github repository (also private) and executed the above commands while replacing the repo URL, but I got the same error message.

I also tried creating a k8s secret with the basic auth credentials, generating a YAML file for the git repository, and specifying the secret name inside the secretRef block under spec, but the error remained the same.
The entire YAML file is listed below:

---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
  name: demo-repo
  namespace: demo
spec:
  interval: 1m0s
  ref:
    branch: main
  secretRef:
    name: <secret name>
  url: https://github.com/<org or my user>/<repo>

As a last resort, I tried to change the git implementation to felibgit2 but got the below error:
failed to checkout and determine revision: unable to fetch-connect to remote '<my repo>': early EOF

Cloning the repository manually using basic auth works.

Please help me to identify the issue and add a new GitRepository.
I appreciate any help you can provide.

Steps to reproduce

1. Install Flux using basic auth.
2. Try to add a new GitRepository by running the following command and specifying basic auth credentials:
   flux create source git <name> --url=https://github.com/<org>/<repo> --username=<my own username> --password=<password for my account> --branch=main
3. Try to add a new git repo using YAML files and basic auth

Expected behavior

Flux to successfully authenticate to Github and be able to pull the repository.

Screenshots and recordings

No response

OS / Distro

Windows 11, Ubuntu 20.04

Flux version

v0.31.4

Flux check

✚ generating GitRepository source
► applying secret with repository credentials
✔ authentication configured
► applying GitRepository source
✔ GitRepository source updated
◎ waiting for GitRepository source reconciliation
✗ failed to checkout and determine revision: unable to list remote for 'https://github.com//': authentication required

Git provider

Github

Container Registry provider

No response

Additional context

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[kingdonb]

What version of Flux are you using exactly?

• Enrich early EOF error message image-automation-controller#410

This PR landed 6 days ago in Image Automation Controller (not source controller, where your error is coming from) and it looks related.

Early EOF, it is stated there, is a red-herring signal that masks a different error, likely indicates a failure to authenticate.

The PR in Image Automation Controller is supposed to make the error message a bit clearer, we may need a similar one for source controller. (cc: @pjbgf WDYT?)

It would be helpful to know exactly which version of Flux you were using when you reported this, @ldvy sorry I see I missed this was in the original post, v0.31.4

[pjbgf]

For the repository that is using the PAT, can you please check whether the Secret which it refers to contains the correct token in the password field and the correct user account in the username field?

Alternatively, you can have all the sources being fetched via SSH. For that you just need to ensure you have deploy keys defined as a secret (pretty much as the bootstrap does for your flux-system GitRepository). Depending on your security requirements, you could have a single secret shared across all sources or you can split them based on your trust boundaries.

More information can be found here: https://fluxcd.io/docs/components/source/gitrepositories/#ssh-authentication

[kingdonb]

If this information helped, or if you have not been able to resolve this issue, we hope your Flux work is going well!

The link shared above shows this:

flux create secret git podinfo-auth \
    --url=ssh://git@github.com/stefanprodan/podinfo \
    --private-key-file=./identity

Convenience wrapper for Flux to create a secret from an existing key. It also shows the URL format that is expected if you want to use SSH authentication. There are more examples there, including ones where Flux generates a key and only stores it in the cluster, as well as how to export to YAML manifests for encryption which you can stock in the cluster using SOPS.

It has been a while and no response, I'm going to go ahead and close this, but if you need more support please let us know.