Data pre-processing Soccernet, missing Cache files? #7
Comments
|
Hi, If possible could you provide the commands you ran, the one that fails and the error please ? It's been a while now I've worked on this code so I need some help to refresh my memory ^^' |
|
Hi, Thank you for the prompt response! I've attached two scripts that I've run. The first was completed successfully and was for extracting the frames. The second is where I'm getting stuck, which is with the preprocessing labels. Extract Frames: #!/bin/bash
fps=2
input_folder="/SATA/SoccerNet/"
output_folder="/SATA/soccernet_as_extracted_${fps}fps/"
split=train
export PYTHONPATH=$PYTHONPATH:/home/liam/soccer/action_spotting/eztorch
python run/datasets/extract_soccernet.py \
--input-folder $input_folder \
--output-folder $output_folder \
--fps $fps \
--split $split
split=test
python run/datasets/extract_soccernet.py \
--input-folder $input_folder \
--output-folder $output_folder \
--fps $fps \
--split $splitPrecompute Labels #!/bin/bash
radius_label=0.5
dataset_json=/SATA/soccernet_as_extracted_2fps/test.json # Path to the JSON.
frame_dir=/SATA/soccernet_as_extracted_2fps/test/ # Path to the decoded videos.
fps=2
# not working, where do I get this file and what is it for?
# Is this something I should download from SoccerNet? If so, where do I get it?
cache_dir=/SATA/soccernet_as_extracted_2fps/cache/
export PYTHONPATH=$PYTHONPATH:/home/liam/soccer/action_spotting/eztorch
python run/datasets/precompute_soccernet_labels.py \
--radius-label $radius_label \
--data-path $dataset_json \
--path-prefix $frame_dir \
--fps $fps
#--cache-dir $cache_dirAnd here is the output I get from running the precompute labels: Let me know if there is anything else I can do to help, and thank you so much for the help :) |
|
Hi there, Looking through the code, I found that if left empty, the Cache directory is created. The check for if the cache argument is None fails always because it defaults to an empty string, so it's never a None type. I added a check for an empty string. and now the code continues, but I've run into a new error with regards to how labels are dealt with: I've made another change moving the Thanks for the help! |
|
From my code which is fairly poorly written, I understand that cache_dir should be provided AND should not exist, otherwise it will try to load data. So it should be a path to a directory not existing not to a file. When you provided the cache_dir argument did you create previously the cache_dir ? if not, then there is an issue that I don't grasp. Maybe there is not a write access, but it seems unlikely if you put the cache close to the data. The precomputing of labels consist on creating a tensor for each frame to know if one or multiple action occurs so the tensor for each video is of shape: |
|
I made a PR #10 could you try to go onto this branch ? I cannot test right now because I don't have the data on my hands and I hope I won't need to. I've overriden some changes from your PR #9. I was quite concerned on why you changed the np types from string to bytes for the paths and the .gitignore local changes should be kept local I think 😄 |
|
Hi there, Ya, sorry about that; the changes going from bytes to strings were because I was getting internal Python errors. The error was with regards to np.string being deprecated by numpy >= 2.0. Downgrading numpy however, resulted in more cascading errors. Looking at the documentation, I believe the functionality remained unchanged, so I included it. Thank you for making the PR #10, I'll give it a try and get back to you. |
|
Hi there, I'm running into issues with the inference now. Mainly, not being sure which config files to use for the script: The default files are not provided in the repository. I've tried a few different config entry point files. The current one I'm trying to work with is the This makes me think this isn't the correct file, either. Sorry to keep bugging you about this project, I really appreciate all the help though. I'm new to the Hyrda library, so I might just be using it completely wrong with this setup. |
|
Using hydra, the config path is relative to the script launched using Python I gave this exemple for vivit: config_path="../eztorch/configs/run/finetuning/vivit"
config_name="vivit_tiny_soccernet_uniform"I guess the config name you use is correct and path too otherwise it wouldnt launch the code Did you install eztorch in your environement ? If yes, could you verify timm is installed ? |
|
Hi, Thanks for the help; I got it working by downgrading the timm version. All of my libraries are of different versions than what the code was built upon. I'm still determining how that happened since I downloaded them using the requirement.txt file. I think I ran into an issue when downloading dependencies and manually downloaded some, which resulted in the mismatches. Again thank you so much for the help, I greatly appreciated it. |
I have pre-processed the SoccerNetV2 dataset using the extract frames pre-processing script.
Now I'm trying to pre-compute the labels but I'm missing the cache files and the num_timestamps_per_half.pt file.
Is there a certain place I can find this file on the Soccernet dataset website or is this something I should generate myself?
Thank you for the help.
The text was updated successfully, but these errors were encountered: