How to get the videoclip features?

[ziweiji]

Could you please guide me on how to extract text and video features from VideoClip? I'm using the checkpoint from fairseq/examples/MMPT/, but I'm unable to obtain the same features as you. Could you provide the code for this process?

[FOXamber]

#old_feat is MIL-NCE feature
inter_old_feat = images[start+i]
inter_old_feat = inter_old_feat.unsqueeze(0).unsqueeze(0)
bsz = inter_old_feat.size(0)
seq_len = inter_old_feat.size(1)
max_video_len = config.dataset.max_video_len
padding = torch.zeros(
bsz, max_video_len - seq_len, inter_old_feat.size(-1))
vfeats = torch.cat([inter_old_feat, padding], dim=1)
vmasks = torch.cat([
torch.ones((bsz, seq_len), dtype=torch.bool),
torch.zeros((bsz, max_video_len - seq_len), dtype=torch.bool)
],
dim=1
)
output = model(caps, cmasks, vfeats, vmasks)
inter_new_feat = output["pooled_video"].squeeze()
new_feat[start+i] = inter_new_feat