The code for our INTERSPEECH 2020 paper - Jointly Fine-Tuning "BERT-like'" Self Supervised Models to Improve Multimodal Speech Emotion Recognition

This repository contains the implementation of the paper -- Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis

Deep Multiset Canonical Correlation Analysis - An extension of CCA to multiple datasets

Real-world photo sequence question answering system (MemexQA). CVPR'18 and TPAMI'19

IISAN: Efficiently Adapting Multimodal Representation for Sequential Recommendation with Decoupled PEFT

Yet another Ph.D. adventure.

Collects a multimodal dataset of Wikipedia articles and their images

Companion Repo for the Vision Language Modelling YouTube series - https://bit.ly/3PsbsC2 - by Prithivi Da. Open to PRs and collaborations

Code for COLING2020 paper: Probing Multimodal Embeddings for Linguistic Properties.

Segment-level autoencoders for multimodal representation

My master thesis: Siamese multi-hop attention for cross-modal retrieval.

Multimodal Bi-Transformers (MMBT) in Biomedical Text/Image Classification

A detailed description on how to extract and align text, audio, and video features at word-level.

Together Yet Apart: Multimodal Representation Learning for Personalised Visual Art Recommendation

All experiments were done to classify multimodal data.

FuseMap: Integrate spatial transcripomics with universal gene, cell, and tissue embeddings.

PyTorch Implementation of HUSE: Hierarchical Universal Semantic Embeddings ( https://arxiv.org/pdf/1911.05978.pdf )

Python implementation of the Multimodal Eigenwords (MM-Eigenwords) 🐍

User modelling using Multi-modal fusion

Gowers Method for finding latent networks of multi-modal data