From b0b44b35fd7622ff9d50a25086840b47ccd2df48 Mon Sep 17 00:00:00 2001
From: Lunwen He <lwhecser@gmail.com>
Date: Mon, 18 Nov 2024 12:40:21 -0800
Subject: [PATCH] Update base for Update on "allow customized head_dim"

This is for resolving the ask in this [post](https://fb.workplace.com/groups/pytorch.edge.users/permalink/1574875706716050/).

Similar change in HF: https://github.com/huggingface/transformers/pull/32502

Differential Revision: [D65974454](https://our.internmc.facebook.com/intern/diff/D65974454/)

[ghstack-poisoned]