RuntimeError: Expected tensor for argument #1 'indices' to have one of the following scalar types: Long, Int; but got torch.cuda.FloatTensor instead (while checking arguments for embedding)

[neeraj1909]

Environment info

• transformers version: 4.12.5
• Platform: Linux-5.10.90+-x86_64-with-debian-bullseye-sid
• Python version: 3.7.12
• PyTorch version (GPU?): 1.9.1 (True)
• Tensorflow version (GPU?): 2.6.2 (True)
• Using GPU in script?: Yes
• Using distributed or parallel set-up in script?: No

Who can help

@LysandreJik @vanpelt @arfon

Information

I am using Captum for interpreting the attributions of the tokens in each layer using Layer-Conductance
lc = LayerConductance(predict, model.bert.encoder.layer[i])

Now, in the line
layer_attributions = lc.attribute(inputs=input_ids, baselines=ref_input_ids, additional_forward_args=(attention_mask,))
RuntimeError is generating.

A helper function to perform forward pass of the model and make predictions.
def predict(input_ids, attention_mask=None): outputs, attention_weights = model(input_ids=input_ids, attention_mask=attention_mask) preds = torch.softmax(outputs, dim = 1)[0][1].unsqueeze(0) return preds

Model I am using: "google/muril-base-cased"

#2952

Expected behavior

Code is working fine during training and prediction but raising errors while interpreting the layers with captum.

Any help would be greatly appreciated.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[mangalm96]

@neeraj1909 I am facing the same issue, did you manage to figure this out by any chance?