-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can not import dataset or create task -> cvat_worker_import: ./backend_entrypoint.sh: run: expected 1 argument #6999
Comments
There is a weird warning in traefik DEBUG logs for
There is a
Both
|
More weird actions: I was getting the tus PATCH error when creating a task. All of a sudden, It allows me to create a task, but it sits at 0% indefinately. docker compose logscvat_server | 2023-10-14 11:52:34,548 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.2:0 - "GET /api/auth/rules HTTP/1.0" 304 Not Modified
cvat_server |
cvat_server | 2023-10-14 11:52:35,261 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/server/health/?format=json&org= HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:35,394 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "OPTIONS /api/auth/password/change?org= HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:35,395 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "OPTIONS /api/auth/password/reset?org= HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:35,536 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/users/self?org= HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:35,554 DEBG 'uvicorn-0' stderr output:
cvat_server | [2023-10-14 11:52:35,554] WARNING django.request: Not Found: /api/user-agreements
cvat_server |
cvat_server | 2023-10-14 11:52:35,555 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/user-agreements?org= HTTP/1.0" 404 Not Found
cvat_server |
cvat_server | 2023-10-14 11:52:35,772 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/users/self?org= HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:35,889 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/server/annotation/formats?org= HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:35,890 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/server/about?org= HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:35,917 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/organizations?org=&page_size=500&page=1 HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:35,918 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/server/plugins?org= HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:36,232 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/tasks?org=zm&page=1&page_size=10 HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:36,401 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/lambda/requests?org=zm HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:36,439 DEBG 'uvicorn-1' stderr output:
cvat_server | [2023-10-14 11:52:36,439] INFO cvat.apps.engine.cache: Starting to get chunk from cache: key data_10_0_preview
cvat_server |
cvat_server | 2023-10-14 11:52:36,440 DEBG 'uvicorn-1' stderr output:
cvat_server | [2023-10-14 11:52:36,440] INFO cvat.apps.engine.cache: Ending to get chunk from cache: key data_10_0_preview, is_cached False
cvat_server |
cvat_server | 2023-10-14 11:52:36,441 DEBG 'uvicorn-1' stderr output:
cvat_server | [2023-10-14 11:52:36,440] INFO cvat.apps.engine.cache: Starting to prepare chunk: key data_10_0_preview
cvat_server |
cvat_server | 2023-10-14 11:52:36,445 DEBG 'uvicorn-1' stderr output:
cvat_server | [2023-10-14 11:52:36,444] WARNING django.request: Bad Request: /api/tasks/10/preview
cvat_server |
cvat_server | 2023-10-14 11:52:36,446 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/tasks/10/preview?org=zm HTTP/1.0" 400 Bad Request
cvat_server |
cvat_server | 2023-10-14 11:52:36,520 DEBG 'uvicorn-1' stderr output:
cvat_server | [2023-10-14 11:52:36,520] WARNING django.request: Not Found: /api/functions/requests/
cvat_server |
cvat_server | 2023-10-14 11:52:36,520 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/functions/requests/?org=zm HTTP/1.0" 404 Not Found
cvat_server |
cvat_server | 2023-10-14 11:52:36,532 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/tasks/10/status?org=zm HTTP/1.0" 200 OK
cvat_server |
cvat_worker_import | ./backend_entrypoint.sh: run: expected 1 argument
cvat_worker_import exited with code 1
cvat_server | 2023-10-14 11:52:43,189 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "DELETE /api/tasks/10 HTTP/1.0" 204 No Content
cvat_server |
cvat_server | 2023-10-14 11:52:43,570 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.2:0 - "GET /api/auth/rules HTTP/1.0" 304 Not Modified
cvat_server |
cvat_server | 2023-10-14 11:52:54,044 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.2:0 - "GET /api/auth/rules HTTP/1.0" 304 Not Modified
cvat_server |
cvat_ui | 172.23.0.5 - - [14/Oct/2023:17:52:56 +0000] "GET /tasks?page=1 HTTP/1.1" 200 691 "https://cvat.<REDACTED>.com/tasks/create?projectId=3" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36" "172.23.0.1"
cvat_server | 2023-10-14 11:52:57,384 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/server/health/?format=json&org= HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:57,500 DEBG 'uvicorn-1' stderr output:
cvat_server | [2023-10-14 11:52:57,499] WARNING django.request: Not Found: /api/user-agreements
cvat_server |
cvat_server | 2023-10-14 11:52:57,502 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/user-agreements?org= HTTP/1.0" 404 Not Found
cvat_server |
cvat_server | 2023-10-14 11:52:57,519 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "OPTIONS /api/auth/password/change?org= HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:57,522 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "OPTIONS /api/auth/password/reset?org= HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:57,526 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/users/self?org= HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:57,761 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/users/self?org= HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:57,882 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/server/about?org= HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:57,914 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/server/plugins?org= HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:57,915 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/server/annotation/formats?org= HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:57,917 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/organizations?org=&page_size=500&page=1 HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:58,186 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/tasks?org=zm&page=1&page_size=10 HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:58,345 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/lambda/requests?org=zm HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:52:58,422 DEBG 'uvicorn-1' stderr output:
cvat_server | [2023-10-14 11:52:58,422] WARNING django.request: Not Found: /api/functions/requests/
cvat_server |
cvat_server | 2023-10-14 11:52:58,423 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/functions/requests/?org=zm HTTP/1.0" 404 Not Found
cvat_server |
cvat_server | 2023-10-14 11:53:00,643 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.2:0 - "GET /api/auth/rules HTTP/1.0" 304 Not Modified
cvat_server |
cvat_server | 2023-10-14 11:53:06,156 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/projects?org=zm&names_only=true&page=1&page_size=10&search= HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:53:07,097 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.2:0 - "GET /api/auth/rules HTTP/1.0" 304 Not Modified
cvat_server |
cvat_server | 2023-10-14 11:53:07,356 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/projects/3 HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:53:07,525 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/labels?project_id=3&org=zm&page_size=500&page=1 HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:53:14,658 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/projects/3 HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:53:14,837 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/labels?project_id=3&org=zm&page_size=500&page=1 HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:53:15,258 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "POST /api/tasks?org=zm HTTP/1.0" 201 Created
cvat_server |
cvat_server | 2023-10-14 11:53:15,564 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "POST /api/tasks/11/data?org=zm HTTP/1.0" 202 Accepted
cvat_server |
cvat_server | 2023-10-14 11:53:17,356 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "POST /api/tasks/11/data?org=zm HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:53:17,591 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "POST /api/tasks/11/data?org=zm HTTP/1.0" 202 Accepted
cvat_server |
cvat_server | 2023-10-14 11:53:17,845 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/tasks/11/status?org=zm HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:53:19,223 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.2:0 - "GET /api/auth/rules HTTP/1.0" 304 Not Modified
cvat_server |
cvat_server | 2023-10-14 11:53:22,084 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/tasks?org=zm&page=1&page_size=10 HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:53:22,252 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/lambda/requests?org=zm HTTP/1.0" 200 OK
cvat_server |
cvat_server | 2023-10-14 11:53:22,282 DEBG 'uvicorn-0' stderr output:
cvat_server | [2023-10-14 11:53:22,282] INFO cvat.apps.engine.cache: Starting to get chunk from cache: key data_11_0_preview
cvat_server |
cvat_server | 2023-10-14 11:53:22,284 DEBG 'uvicorn-0' stderr output:
cvat_server | [2023-10-14 11:53:22,283] INFO cvat.apps.engine.cache: Ending to get chunk from cache: key data_11_0_preview, is_cached False
cvat_server |
cvat_server | 2023-10-14 11:53:22,284 DEBG 'uvicorn-0' stderr output:
cvat_server | [2023-10-14 11:53:22,284] INFO cvat.apps.engine.cache: Starting to prepare chunk: key data_11_0_preview
cvat_server |
cvat_server | 2023-10-14 11:53:22,286 DEBG 'uvicorn-0' stderr output:
cvat_server | [2023-10-14 11:53:22,286] WARNING django.request: Bad Request: /api/tasks/11/preview
cvat_server |
cvat_server | 2023-10-14 11:53:22,286 DEBG 'uvicorn-0' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/tasks/11/preview?org=zm HTTP/1.0" 400 Bad Request
cvat_server |
cvat_server | 2023-10-14 11:53:22,336 DEBG 'uvicorn-1' stderr output:
cvat_server | [2023-10-14 11:53:22,335] WARNING django.request: Not Found: /api/functions/requests/
cvat_server |
cvat_server | 2023-10-14 11:53:22,337 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.1:0 - "GET /api/functions/requests/?org=zm HTTP/1.0" 404 Not Found
cvat_server |
cvat_server | 2023-10-14 11:53:28,677 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.23.0.2:0 - "GET /api/auth/rules HTTP/1.0" 304 Not Modified
cvat_server |
cvat_worker_import exited with code 1 Files in volume╰─❯ tree /cvat-vol
/cvat-vol
├── assets
├── cache
│ └── events
│ ├── events_000.db
│ ├── events_001.db
│ └── events.db
├── data
│ └── 11
│ ├── compressed
│ ├── original
│ └── raw
│ ├── bf0aac0878b0a3d2_jpg.rf.43ce523ba85e649db07055557d72cc9e.jpg
│ ├── bf2f25f3ed9ff4d5_jpg.rf.e9d07ec8b85372d40366bc2770b7d537.jpg
│ ├── c1d8b110186e095a_jpg.rf.1515701021c709417f9e92537a1d3714.jpg
│ ├── c274e026ac907eb9_jpg.rf.2388de7229ba0dd3051326f64c1a770b.jpg
│ ├── c650ff8d3e8e75b3_jpg.rf.a469a8f0edb8fddf8e2b70b102f5a5eb.jpg
│ ├── c7df5232291486a3_jpg.rf.d18b52a383576cfe6c4ec343fca07a80.jpg
│ ├── c91ee912164d8ecb_jpg.rf.5a70d8b40bde25a3a4cb09337bf0c6f9.jpg
│ ├── cb4b754537798d23_jpg.rf.310e3c4544bf89d2f3109956b301db20.jpg
│ ├── cb8c75fc1c7ccf73_jpg.rf.a6d315cf4a3ef0824075df8c8561838b.jpg
│ ├── ccc1a2d44a290368_jpg.rf.8e8d6f87f1d1327caf5f80e0c96cdad9.jpg
│ ├── cdb1e8621d1c624e_jpg.rf.b744b93a7af0771f1e5cb955155c8451.jpg
│ ├── ce97f7bc90e97109_jpg.rf.0ee971ca2acc7c8826290a5a80153777.jpg
│ ├── d027c6e32db60e3c_jpg.rf.15a865d02f390ef84fca8e053946db31.jpg
│ ├── d27e094e98374ff8_jpg.rf.a0e0eba539117b97eb6433295c4cfb44.jpg
│ ├── d28d71c2690c16ed_jpg.rf.b7c4b2a5e5870d9fcf8644c109c5457f.jpg
│ └── d2fe2b47668e9d8e_jpg.rf.c3c019d146ec1fc0bcf7fbeb12c7bc7f.jpg
├── docker-dummy
├── jobs
├── models
├── projects
│ ├── 2
│ └── 3
├── storages
├── tasks
│ └── 11
│ └── artifacts
└── tmp
18 directories, 20 files In the logs I see this: cvat_worker_import | ./backend_entrypoint.sh: run: expected 1 argument
cvat_worker_import exited with code 1 Possibly this is the issue? |
Apparently I had set cvat/ui dev c0b16e084d1f 45 hours ago 145MB
cvat/server dev d2b9351cedf0 46 hours ago 2.77GB
traefik latest cc365cbb0397 3 days ago 151MB
cvat/ui <none> d99892aa9005 4 days ago 145MB
cvat/server <none> 8c2b8b050d3a 4 days ago 2.77GB
postgres 15-alpine 9f959158ce82 10 days ago 237MB
cvat/server latest ed304b59905c 11 days ago 3.14GB
cvat/ui <none> 5358101b1b13 3 weeks ago 144MB
cvat/server <none> e685cf17c422 3 weeks ago 2.78GB
postgres <none> ab8fb914369e 2 months ago 237MB
traefik v2.9 d1e26b5f8193 6 months ago 138MB
clickhouse/clickhouse-server 22.3-alpine d4088d71d1dc 6 months ago 773MB
eqalpha/keydb x86_64_v6.3.2 b9c88d8f9d06 8 months ago 129MB
grafana/grafana-oss 9.3.6 f925787d0f79 8 months ago 317MB
timberio/vector 0.26.0-alpine d8ecc9831523 10 months ago 122MB
openpolicyagent/opa 0.45.0-rootless 8723f2dc306a 12 months ago 84.3MB |
Hi, sorry to bug again but can anyone possibly take a look at this? I can't create any tasks or import any datasets so, I can't use cvat. I really don't want to use roboflow. I would prefer to run cvat but, it has been unbelievably unreliable. I've spent more time debugging and fixing cvat than I have working on my projects. Any advice or pointers would be appreciated. |
I nuked the install and managed to get things working again. |
CVAT is proxied by Cloudflare tunnel
Actions before raising this issue
CSRF_TRUSTED_ORIGINS
fix (for POST on django admin page) - Add CVAT_HOST to CSRF_TRUSTED_ORIGINS #6322CVAT_HOST=cvat.<REDACTED>.com
CVAT_BASE_URL=https://cvat.<REDACTED>.com
'--entryPoints.web.forwardedHeaders.trustedIPs'
to all Cloudflare IP ranges and my local host's IP that is running cloudflared (fixes not being able to POST aka create tasks) - Cant create task - CVAT served via cloudflare tunnels - Content Mismatch blocks uploads. #6917 (comment)Steps to Reproduce
Expected Behaviour
I expect to be able to import datasets into the project
Possible Solution
I see an error in the logs:
I am assuming this error is in regard to importing the new task. Possible fix is to debug why it is not receiving its 1 argument.
Context
My CVAT instance is proxied by Cloudflare tunnel.
This error looks exactly like the error I was having when trying to create a task, but there is not a
Content-Mismatch
warning -> #6917 (comment)Error
My site is HTTPS only and tus seems to be requesting HTTP, this may be a red herring though. Also,
cvat_worker_import
container is throwing errors:How can I fix this?
Environment
The text was updated successfully, but these errors were encountered: