Page not reacting while long-running migration/mirror initialization

[kb-1000]

• Gitea version (or commit ref): 1.4.0
• Git version: not relevant, 2.16.3.windows.1
• Operating system: not relevant, Windows
• Database (use [x]):
  • PostgreSQL
  • MySQL
  • MSSQL
  • SQLite
• Can you reproduce the bug at https://try.gitea.io:
  • Yes (provide example URL) https://try.gitea.io/kaeptmblaubaer1000/cpython.git
  • No
  • Not relevant
• Log gist: not relevant

Description

The response is delayed as long as the download takes.
It should rather either show that the download is in progress, or the progress (like GitHub Desktop does).

Screenshots

[tankerkiller125]

This really needs to be resolved, on large repos the page will timeout and break the repo causing a 500 error.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 weeks. Thank you for your contributions.

[techknowlogick]

Please review https://docs.gitea.io/en-us/config-cheat-sheet/ for git.timeout settings and increase the timeouts.

[kb-1000]

This is not the cause of the issue... the issue is that the server waits until completion before sending the response page.

[kb-1000]

Please reopen this issue!

[zeripath]

Thank you @kaeptmblaubaer1000 I think I understand the issue.

https://github.com/go-gitea/gitea/blob/master/routers/repo/repo.go#L211

func MigratePost(ctx *context.Context, form auth.MigrateRepoForm) {
	...
	repo, err := models.MigrateRepository(ctx.User, ctxUser, models.MigrateRepoOptions{
		Name:        form.RepoName,
		Description: form.Description,
		IsPrivate:   form.Private || setting.Repository.ForcePrivate,
		IsMirror:    form.Mirror,
		RemoteAddr:  remoteAddr,
	})
	...
}

https://github.com/go-gitea/gitea/blob/master/models/repo.go#L899

func MigrateRepository(doer, u *User, opts MigrateRepoOptions) (*Repository, error) {
	...
	if err = git.Clone(opts.RemoteAddr, repoPath, git.CloneRepoOptions{
		Mirror:  true,
		Quiet:   true,
		Timeout: migrateTimeout,
	}); err != nil {
		return repo, fmt.Errorf("Clone: %v", err)
	}
	...
}

In the same goroutine as the POST! For big repositories this clone could take a very long time indeed. Meanwhile the user is sat on the other end of that POST waiting.

[zeripath]

The problem is how to fix this. Clearly we should put this clone in a separate goroutine and return. But we probably need to monitor the migration somehow.

I guess a quick fix is to see if just sticking the clone and it's following parts in a goroutine is enough.

[stale]

This issue has been automatically closed because of inactivity. You can re-open it if needed.

[zeripath]

Not stale - sorry that was my fault for not removing the stale tag. It shouldn't happen anymore.

[zeripath]

@kaeptmblaubaer1000 is there any chance you could test my pr to see if it solves your issue?

[kb-1000]

If it doesn't wait for completion of mirror / import clone before returning the response page, then it solved the issue... but unfortunately I can't test it, as I don't have Go on my computer (only on my phone) and I don't currently use Gitea anyway, because there is currently no need for it for me.

[tankerkiller125]

@zeripath If you are currently running Gitea someplace try cloning the emoji-data repo (https://github.com/iamcal/emoji-data) if it continues and doesn't freeze AND you don't get a 500 error when you go to the repo in gitea then you've resolved the issue. If it freezes then it's not fixed and if you get a 500 error then that means that some sort of "Please Wait while we clone the Repo" page needs to be created and displayed.

Note: the Emoji-Data repo is about 2GB (raw git) make sure you have enough space before cloning.

[mrsdizzie]

Should be fixed as these have been moved into queues and now show "loading" icons for migration instead of just waiting until finished