This repository contains various experiments in extending the effective context length of models trained with Rotary Positional Embeddings.
Formalized results will be provided in an upcoming paper. In the meantime, have fun with it 🙂
forked from jquesnelle/yarn
-
Notifications
You must be signed in to change notification settings - Fork 0
License
jon-tow/scaled-rope
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
No description, website, or topics provided.
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published
Languages
- Python 100.0%