Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PR 636 miss fix for FusedAddRMSNorm function #645

Closed
Atream opened this issue Dec 4, 2024 · 2 comments
Closed

PR 636 miss fix for FusedAddRMSNorm function #645

Atream opened this issue Dec 4, 2024 · 2 comments

Comments

@Atream
Copy link
Contributor

Atream commented Dec 4, 2024

In #636, you write const uint32_t smem_size = (ceil_div(num_warps, 4) * 4 + d) * sizeof(float); in GemmaFusedAddRMSNorm but not in FusedAddRMSNorm function.

@yzh119
Copy link
Collaborator

yzh119 commented Dec 4, 2024

Hi @Atream , yes I think you are correct, would you mind creating a pull request to fix it?

@Atream
Copy link
Contributor Author

Atream commented Dec 4, 2024

Yes, I've created a pull request to address the issue. You can find it here #646 . Please review and let me know if there's anything else that needs to be adjusted. Thanks for your response!

yzh119 pushed a commit that referenced this issue Dec 4, 2024
…646)

Fix smem_size in FusedAddRMSNorm which is missed in #636 
Fix issue #645
@Atream Atream closed this as completed Dec 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants