GitHub - mijing/FastBPE: 效率优化的bpe算法

Name		Name	Last commit message	Last commit date
Latest commit History 2 Commits
README		README
bpe.py		bpe.py
bpe_fast.py		bpe_fast.py
sample.txt		sample.txt

Repository files navigation

bpe.py 是原始算法
执行可以直接: python bpe.py sample.txt


bpe_fast.py 是效率优化的算法, 提升1000x
执行可以直接: python bpe_fast.py sample.txt

效率优化的bpe算法

No releases published

No packages published