Add inference API of AMR

[h-munakata]

Summary

1. Add inference API of AMR

• Add msclap in dependencies
• Add configuration for AMR in lighthouse/models.py
  I implemented encode_audio() separately from encode_video().

model.encode_audio("api_example/1a-ODBWMUAE.wav")

2. Modularize PANNs and CLAP

• Class AudioEncoder in lighthouse/feature_extractor/audio_encoder.py only have model selector and lighthouse/feature_extractor/audio_encoders/{clap_a | pann}.py has individual model.

Future work

• Add Gradio demo

[h-munakata]

The test has failed... I'm checking.

[awkrail]

Could you add msclap before 'clip@git+https://github.com/openai/CLIP.git' ?

[awkrail]

Could you remove magic number of False? Instead, please insert variable X = False and then use it here for readability?

[awkrail]

@h-munakata In addition, no tests are newly provided. Hence, could you add some tests for CLAP feature extractor and AMR inference API?

[h-munakata]

@h-munakata In addition, no tests are newly provided. Hence, could you add some tests for CLAP feature extractor and AMR inference API?

I see. I will add some tests in tests/test_models.py later.

[h-munakata]

For the test and the Gradio demo, I think {moment | qd | cg | ...}-detr for CLAP features is useful.
Should I prepare the pre-trained weights of all seven models trained with Clotho-Moment?

[awkrail]

For the test and the Gradio demo, I think {moment | qd | cg | ...}-detr for CLAP features is useful.
Should I prepare the pre-trained weights of all seven models trained with Clotho-Moment?

Yes, I agree. Could you prepare the pre-trained weights on your Zenodo?

[h-munakata]

Yes, I agree. Could you prepare the pre-trained weights on your Zenodo?

Sure. I will upload after the training.

[awkrail]

@h-munakata Sorry, I misunderstood your question. I think that in your paper, CG-DETR (or QD-DETR) achieved the highest performance. So, no need for training. All you need to do is upload the current trained models on Zenodo and accessible from Lighthouse.

[h-munakata]

My intention in training all models is to add a CLAP feature to the double for loop of FEATURE and MODEL used in the demo and test to make it easier to handle.
As you said, I will stop training all models and define AUDIO_FEATURE variable in the demo and test.

All you need to do is upload the current trained models on Zenodo and accessible from Lighthouse.

I see. I'll upload the model in the next commit for the test.

[awkrail]

@h-munakata BTW, could you finish implementing web demo tomorrow? I will tag the current version as v1.0, and wondering whether you can finish this implementation by tomorrow.

[h-munakata]

BTW, could you finish implementing web demo tomorrow?

Yes, I want to make it in time for the DCASE workshop the day after tomorrow.