Reduce model size when using Bert Embedder

[faizan30]

System (please complete the following information):

• OS: Linux
• Python version: 3.7.1
• AllenNLP version: 0.8.3

Question

• I'm working on a text classification task and I'm using PretrainedBertEmbedder. The size of classification model is over 400Mb. How can I reduce the model size while still using Bert?

Token embedder:

"token_embedders": {
"bert": {
"type": "bert-pretrained",
"pretrained_model": "bert-base-uncased",
"top_layer_only": true
}
}

Token Indexer:
"token_indexers": {
"bert": {
"type": "bert-pretrained",
"pretrained_model": ".pretrained/bert/bert-base-uncased.tar.gz",
"do_lowercase": true,
"max_pieces": 100
}
}

[kernelmachine]

I don't think you can reduce the model size beyond your classifier's parameters, unless you retrain a smaller BERT. If you need more efficient training, some tricks like mixed precision training (#2149) and gradient accumulation are in the works (#2721)

[faizan30]

Are BERT embeddings being saved as part of weights file? If yes is there a way to seperate my model weights and bert weights?

[faizan30]

Thanks for the response @kernelmachine .