TextVQA accuracy

[crx15215682925]

Hello, I have got the json file of TextVQA, can you tell me how to get the accuracy? Only the eval code for Textcaps is currently provided.

[soonchangAI]

For TextVQA validation accuracy:

2 GPU:

python -m torch.distributed.launch --nproc_per_node 2 TAP/tools/run.py --tasks vqa --datasets m4c_textvqa --model m4c_split \ --config $config \ --run_type val \ --save_dir $save_directory \ --resume_file $model.ckpt \ training_parameters.distributed True
single GPU:

python TAP/tools/run.py --tasks vqa --datasets m4c_textvqa --model m4c_split \ --config $config \ --run_type val \ --save_dir $save_directory \ --resume_file $model.ckp
For TextVQA test accuracy, the test set label is not publicly release, So:

1. Predict and save as JSON

python TAP/tools/run.py --tasks vqa --datasets m4c_stvqa --model m4c_split \ --config $config \ --run_type inference --evalai_inference 1 \ --save_dir $model.ckpt

2. Upload to https://eval.ai/web/challenges/challenge-page/874/ for accuracy calculation

[seuppp]

Hi @soonchangAI, thanks for your reply.

Now I find the address provided in data/README.md is invalid. Could you please provide me with a new dataset download address? Thanks.

[seuppp]

Hi @crx15215682925

I find the address provided in data/README.md is invalid. Could you please provide me with a new dataset download address? Thanks.