Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Too many files in collection dropbox blocks creating new items #5392

Closed
cjcolvar opened this issue Sep 27, 2023 · 1 comment
Closed
Assignees
Labels

Comments

@cjcolvar
Copy link
Member

cjcolvar commented Sep 27, 2023

Describe the bug
Avalon passes all of the filenames of files in the dropbox into a single lsof command which fails with Errno::E2BIG if there are too many files in the dropbox.
Report in slack by Sahab Bolhari.

Expected behavior
Avalon can create new items in a collection no matter how many files are in the dropbox.

Screenshots/Screen Recording
Truncated error message:

avalon_1 | F, [2023-09-27T20:02:04.115601 #1] FATAL -- : [458a3c42-248e-4b9c-9d00-f86b187772fc]
avalon_1 | [458a3c42-248e-4b9c-9d00-f86b187772fc] Errno::E2BIG (Argument list too long - /usr/sbin/lsof -Fcpan0 "2021-053-05.flac" "2021-070-01.flac" "1998-169-17.flac" "1989-021-01.flac" "2023-039 (4).flac" "2016-014-04.flac" "2019-100-11.flac" "2021-041-21.flac" "1998-067-01.flac" "2019-037-01.flac" "2017-017-01.flac" "2016-076-07.flac" "2018-088-01.flac" "2022-053a-08.flac" "2022-080-11.flac" "2016-105-03.flac" "2022-107.flac" "2022-030-08.flac" "1994-020-04.flac" "2023-115 (4).flac" "2018-035.flac" "2018-011-02.flac" "1986-010-02.flac" "2017-080-03.flac" "1989-112-03.flac" "1992-041-01.flac" "2015-090-05.flac" "1985-100-25.flac" "1993_103.flac" "1996-010-01.flac" "1997-158-07.flac" "2023-078.flac" "2018-095-11.flac" "2019-005-03.flac" "1985-100-04.flac" "2015-019-02.flac" "2022-110-02.flac" "1997-133.flac"

Environment (please complete the following information):
Docker

Release:
Is this issue associated with a specific release? Unknown

Additional notes
See https://github.com/avalonmediasystem/avalon/blob/main/lib/avalon/batch.rb#L36-L42

@cjcolvar cjcolvar added the bug label Sep 27, 2023
@joncameron
Copy link
Contributor

Could implement a paging of the files or some other way to rework this (a splat, xargs style thing) to avoid an obscenely long line for getting the files list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants