Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add BROS #23190

Merged
merged 120 commits into from
Sep 14, 2023
Merged

Add BROS #23190

Show file tree
Hide file tree
Changes from 112 commits
Commits
Show all changes
120 commits
Select commit Hold shift + click to select a range
3db764f
add Bros boilerplate
jinhopark8345 May 10, 2023
2dc368a
copy and pasted modeling_bros.py from official Bros repo
jinhopark8345 May 10, 2023
5603061
update copyright of bros files
jinhopark8345 May 10, 2023
dbc56b8
copy tokenization_bros.py from official repo and update import path
jinhopark8345 May 10, 2023
e2a2d9d
copy tokenization_bros_fast.py from official repo and update import path
jinhopark8345 May 10, 2023
90ce711
copy configuration_bros.py from official repo and update import path
jinhopark8345 May 10, 2023
6126da1
remove trailing period in copyright line
jinhopark8345 May 10, 2023
63139eb
copy and paste bros/__init__.py from official repo
jinhopark8345 May 10, 2023
596d1a7
save formatting
jinhopark8345 May 14, 2023
764e8df
remove unused unnecessary pe_type argument - using only crel type
jinhopark8345 May 14, 2023
f35348f
resolve import issue
jinhopark8345 May 14, 2023
892dd2d
remove unused model classes
jinhopark8345 May 14, 2023
37c7d9f
remove unnecessary tests
jinhopark8345 May 18, 2023
d878de0
remove unused classes
jinhopark8345 May 18, 2023
772d20e
fix original code's bug - layer_module's argument order
jinhopark8345 May 18, 2023
6ef6ca7
clean up modeling auto
jinhopark8345 May 18, 2023
c338261
add bbox to prepare_config_and_inputs
jinhopark8345 May 18, 2023
7379457
set temporary value to hidden_size (32 is too low because of the of the
jinhopark8345 May 18, 2023
602e2d9
remove decoder test, update create_and_check* input arguemnts
jinhopark8345 May 18, 2023
79b886c
add missing variable to model tests
jinhopark8345 May 18, 2023
5f35f68
do make fixup
jinhopark8345 May 20, 2023
3eace5d
update bros.mdx
jinhopark8345 May 21, 2023
9f0e8ca
add boilerate plate for no_head inference test
jinhopark8345 May 21, 2023
66ff6ce
update BROS_PRETRAINED_MODEL_ARCHIVE_LIST (add naver-clova-ocr prefix)
jinhopark8345 May 21, 2023
f3e9dab
add prepare_bros_batch_inputs function
jinhopark8345 May 21, 2023
7022d4c
update modeling_common to add bbox inputs in Bros Model Test
jinhopark8345 May 21, 2023
f9aab55
remove unnecessary model inference
jinhopark8345 May 22, 2023
41e1ad9
add test case
jinhopark8345 May 22, 2023
94cf5fc
add model_doc
jinhopark8345 May 23, 2023
d10e166
add test case for token_classification
jinhopark8345 May 24, 2023
2845c23
apply fixup
jinhopark8345 May 24, 2023
55d5d7b
update modeling code
jinhopark8345 Jul 23, 2023
e41ab5d
update BrosForTokenClassification loss calculation logic
jinhopark8345 Aug 1, 2023
4ef71fd
revert logits preprocessing logic to make sure logits have original s…
jinhopark8345 Aug 1, 2023
d735bd5
- update class name
jinhopark8345 Aug 8, 2023
5ce570e
- add BrosSpadeOutput
jinhopark8345 Aug 8, 2023
4933093
add boilerate plate for no_head inference test
jinhopark8345 Aug 8, 2023
0d53a2d
add prepare_bros_batch_inputs function
jinhopark8345 May 21, 2023
7228d98
add test case
jinhopark8345 May 22, 2023
9e758d7
add test case for token_classification
jinhopark8345 May 24, 2023
7be8d1d
update modeling code
jinhopark8345 Jul 23, 2023
ca9f5e8
update BrosForTokenClassification loss calculation logic
jinhopark8345 Aug 1, 2023
a6e77d7
revert logits preprocessing logic to make sure logits have original s…
jinhopark8345 Aug 1, 2023
13639d7
Merge remote-tracking branch 'upstream/main' into add-bros
jinhopark8345 Aug 9, 2023
725e145
apply masking on the fly
jinhopark8345 Aug 9, 2023
f5113b3
add BrosSpadeForTokenLinking
jinhopark8345 Aug 10, 2023
a955d3c
update class name
jinhopark8345 Aug 13, 2023
0cb524f
separate the logits calculation logic and loss calculation logic
jinhopark8345 Aug 13, 2023
5939860
update logic for loss calculation so that logits shape doesn't change
jinhopark8345 Aug 14, 2023
179c4f9
update typo
jinhopark8345 Aug 14, 2023
24d55f9
update prepare_config_and_inputs
jinhopark8345 Aug 14, 2023
aa28567
update dummy node initialization
jinhopark8345 Aug 15, 2023
d1a120f
update last_hidden_states getting logic to consider when return_dict …
jinhopark8345 Aug 15, 2023
ed5efb3
update box first token mask param
jinhopark8345 Aug 15, 2023
2d7bcc7
bugfix: remove random attention mask generation
jinhopark8345 Aug 15, 2023
8379565
update keys to ignore on load missing
jinhopark8345 Aug 15, 2023
632fde5
run make style and quality
jinhopark8345 Aug 15, 2023
1f2a956
apply make style and quality of other codes
jinhopark8345 Aug 15, 2023
d83b042
update box_first_token_mask to bool type
jinhopark8345 Aug 16, 2023
794dbba
update index.md
jinhopark8345 Aug 16, 2023
863155f
apply make style and quality
jinhopark8345 Aug 16, 2023
d18a5e6
Merge remote-tracking branch 'upstream/main' into add-bros
jinhopark8345 Aug 16, 2023
6f7c3d3
apply make fix-copies
jinhopark8345 Aug 16, 2023
eb7ba73
pass check_repo
jinhopark8345 Aug 17, 2023
16f4830
update bros model doc
jinhopark8345 Aug 18, 2023
ecce552
docstring bugfix fix
jinhopark8345 Aug 18, 2023
d927015
add checkpoint for doc, tokenizer for doc
jinhopark8345 Aug 18, 2023
46ec931
Update README.md
jinhopark8345 Aug 18, 2023
4433162
Update docs/source/en/model_doc/bros.md
jinhopark8345 Aug 18, 2023
828c9b0
Update bros.md
jinhopark8345 Aug 18, 2023
41de331
Update src/transformers/__init__.py
jinhopark8345 Aug 18, 2023
fa52d90
Update docs/source/en/model_doc/bros.md
jinhopark8345 Aug 18, 2023
d219760
Apply suggestions from code review
jinhopark8345 Aug 18, 2023
3b64b10
apply suggestions from code review
jinhopark8345 Aug 19, 2023
6811e44
apply suggestions from code review
jinhopark8345 Aug 19, 2023
8922ffa
revert test_processor_markuplm.py
jinhopark8345 Aug 19, 2023
09c3f82
Update test_processor_markuplm.py
jinhopark8345 Aug 19, 2023
a4d2e91
apply suggestions from code review
jinhopark8345 Aug 19, 2023
6bce6e1
apply suggestions from code review
jinhopark8345 Aug 21, 2023
0b3e750
apply suggestions from code review
jinhopark8345 Aug 21, 2023
a10fbac
update BrosSpadeELForTokenClassification head name to entity linker
jinhopark8345 Aug 21, 2023
336a94c
add doc string for config params
jinhopark8345 Aug 21, 2023
9da2fa4
update class, var names to more explicit and apply suggestions from c…
jinhopark8345 Aug 21, 2023
e2e304f
remove unnecessary keys to ignore
jinhopark8345 Aug 21, 2023
f621427
update relation extractor to be initialized with config
jinhopark8345 Aug 21, 2023
8a7d54c
add bros processor
jinhopark8345 Aug 21, 2023
fb7a991
apply make style and quality
jinhopark8345 Aug 21, 2023
9a47510
update bros.md
jinhopark8345 Aug 21, 2023
ab706c0
remove bros tokenizer, add bros processor that wraps bert tokenizer
jinhopark8345 Aug 21, 2023
5222230
revert change
jinhopark8345 Aug 21, 2023
3ef8bd5
apply make fix-copies
jinhopark8345 Aug 21, 2023
2a5a010
update processor code, update itc -> initial token, stc -> subsequent…
jinhopark8345 Aug 21, 2023
7761029
add type hint
jinhopark8345 Aug 21, 2023
e9449d1
remove unnecessary condition branches in embedding forward
jinhopark8345 Aug 21, 2023
b001e88
Merge remote-tracking branch 'upstream/main' into add-bros
jinhopark8345 Aug 21, 2023
6a22091
fix auto tokenizer fail
jinhopark8345 Aug 21, 2023
c16e4d8
update docstring for each classes
jinhopark8345 Aug 23, 2023
66f1446
Merge remote-tracking branch 'upstream/main' into add-bros
jinhopark8345 Aug 23, 2023
3f07cb4
update bbox input dimension as standard 2 points and convert them to 4
jinhopark8345 Aug 24, 2023
20a2bee
update bros docs
jinhopark8345 Aug 24, 2023
14e5591
Merge remote-tracking branch 'upstream/main' into add-bros
jinhopark8345 Aug 30, 2023
52dcb38
apply suggestions from code review : update Bros -> BROS in bros.md
jinhopark8345 Sep 2, 2023
6cdcaf2
1. box prefix var -> bbox
jinhopark8345 Sep 2, 2023
983ac62
replace einsum with torch matmul
jinhopark8345 Sep 4, 2023
007333a
apply style and quality
jinhopark8345 Sep 4, 2023
a51a66d
remove unused argument
jinhopark8345 Sep 4, 2023
0403675
remove unused arguments
jinhopark8345 Sep 4, 2023
e15b019
update docstrings
jinhopark8345 Sep 4, 2023
2b6a8f4
apply suggestions from code review: add BrosBboxEmbeddings, replace
jinhopark8345 Sep 5, 2023
039afcb
Merge remote-tracking branch 'upstream/main' into add-bros
jinhopark8345 Sep 5, 2023
0fb70f1
Merge remote-tracking branch 'upstream/main' into add-bros
jinhopark8345 Sep 8, 2023
1a8558b
revert einsum update
jinhopark8345 Sep 10, 2023
8eb78e1
update bros processor
jinhopark8345 Sep 10, 2023
44a0fc9
apply suggestions from code review
jinhopark8345 Sep 14, 2023
19993a7
add conversion script for bros
jinhopark8345 Sep 14, 2023
8fe9f5a
Apply suggestions from code review
jinhopark8345 Sep 14, 2023
e1d0c73
Merge remote-tracking branch 'upstream/main' into add-bros
jinhopark8345 Sep 14, 2023
9e883fb
fix readme
jinhopark8345 Sep 14, 2023
8223fed
apply fix-copies
jinhopark8345 Sep 14, 2023
187c411
Merge remote-tracking branch 'upstream/main' into add-bros
jinhopark8345 Sep 14, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -310,6 +310,7 @@ Current number of checkpoints: ![](https://img.shields.io/endpoint?url=https://h
1. **[BLOOM](https://huggingface.co/docs/transformers/model_doc/bloom)** (from BigScience workshop) released by the [BigScience Workshop](https://bigscience.huggingface.co/).
1. **[BORT](https://huggingface.co/docs/transformers/model_doc/bort)** (from Alexa) released with the paper [Optimal Subarchitecture Extraction For BERT](https://arxiv.org/abs/2010.10499) by Adrian de Wynter and Daniel J. Perry.
1. **[BridgeTower](https://huggingface.co/docs/transformers/model_doc/bridgetower)** (from Harbin Institute of Technology/Microsoft Research Asia/Intel Labs) released with the paper [BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learning](https://arxiv.org/abs/2206.08657) by Xiao Xu, Chenfei Wu, Shachar Rosenman, Vasudev Lal, Wanxiang Che, Nan Duan.
1. **[Bros](https://huggingface.co/docs/transformers/model_doc/bros)** (from NAVER CLOVA) released with the paper [BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents](https://arxiv.org/abs/2108.04539) by Teakgyu Hong, Donghyun Kim, Mingi Ji, Wonseok Hwang, Daehyun Nam, Sungrae Park.
1. **[ByT5](https://huggingface.co/docs/transformers/model_doc/byt5)** (from Google Research) released with the paper [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://arxiv.org/abs/2105.13626) by Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel.
1. **[CamemBERT](https://huggingface.co/docs/transformers/model_doc/camembert)** (from Inria/Facebook/Sorbonne) released with the paper [CamemBERT: a Tasty French Language Model](https://arxiv.org/abs/1911.03894) by Louis Martin*, Benjamin Muller*, Pedro Javier Ortiz Suárez*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot.
1. **[CANINE](https://huggingface.co/docs/transformers/model_doc/canine)** (from Google Research) released with the paper [CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation](https://arxiv.org/abs/2103.06874) by Jonathan H. Clark, Dan Garrette, Iulia Turc, John Wieting.
Expand Down
1 change: 1 addition & 0 deletions README_es.md
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,7 @@ Número actual de puntos de control: ![](https://img.shields.io/endpoint?url=htt
1. **[BLOOM](https://huggingface.co/docs/transformers/model_doc/bloom)** (from BigScience workshop) released by the [BigScience Workshop](https://bigscience.huggingface.co/).
1. **[BORT](https://huggingface.co/docs/transformers/model_doc/bort)** (from Alexa) released with the paper [Optimal Subarchitecture Extraction For BERT](https://arxiv.org/abs/2010.10499) by Adrian de Wynter and Daniel J. Perry.
1. **[BridgeTower](https://huggingface.co/docs/transformers/model_doc/bridgetower)** (from Harbin Institute of Technology/Microsoft Research Asia/Intel Labs) released with the paper [BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learning](https://arxiv.org/abs/2206.08657) by Xiao Xu, Chenfei Wu, Shachar Rosenman, Vasudev Lal, Wanxiang Che, Nan Duan.
1. **[Bros](https://huggingface.co/docs/transformers/model_doc/bros)** (from NAVER) released with the paper [BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents](https://arxiv.org/abs/2108.04539) by Teakgyu Hong, Donghyun Kim, Mingi Ji, Wonseok Hwang, Daehyun Nam, Sungrae Park.
1. **[ByT5](https://huggingface.co/docs/transformers/model_doc/byt5)** (from Google Research) released with the paper [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://arxiv.org/abs/2105.13626) by Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel.
1. **[CamemBERT](https://huggingface.co/docs/transformers/model_doc/camembert)** (from Inria/Facebook/Sorbonne) released with the paper [CamemBERT: a Tasty French Language Model](https://arxiv.org/abs/1911.03894) by Louis Martin*, Benjamin Muller*, Pedro Javier Ortiz Suárez*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot.
1. **[CANINE](https://huggingface.co/docs/transformers/model_doc/canine)** (from Google Research) released with the paper [CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation](https://arxiv.org/abs/2103.06874) by Jonathan H. Clark, Dan Garrette, Iulia Turc, John Wieting.
Expand Down
1 change: 1 addition & 0 deletions README_hd.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,7 @@ conda install -c huggingface transformers
1. **[BLOOM](https://huggingface.co/docs/transformers/model_doc/bloom)** (from BigScience workshop) released by the [BigSicence Workshop](https://bigscience.huggingface.co/).
1. **[BORT](https://huggingface.co/docs/transformers/model_doc/bort)** (एलेक्सा से) कागज के साथ [बीईआरटी के लिए ऑप्टिमल सबआर्किटेक्चर एक्सट्रैक्शन](https://arxiv.org/abs/ 2010.10499) एड्रियन डी विंटर और डैनियल जे पेरी द्वारा।
1. **[BridgeTower](https://huggingface.co/docs/transformers/model_doc/bridgetower)** (हरबिन इंस्टिट्यूट ऑफ़ टेक्नोलॉजी/माइक्रोसॉफ्ट रिसर्च एशिया/इंटेल लैब्स से) कागज के साथ [ब्रिजटॉवर: विजन-लैंग्वेज रिप्रेजेंटेशन लर्निंग में एनकोडर्स के बीच ब्रिज बनाना](<https://arxiv.org/abs/2206.08657>) by Xiao Xu, Chenfei Wu, Shachar Rosenman, Vasudev Lal, Wanxiang Che, Nan Duan.
1. **[Bros](https://huggingface.co/docs/transformers/model_doc/bros)** (NAVER से) Teakgyu Hong, Donghyun Kim, Mingi Ji, Wonseok Hwang, Daehyun Nam, Sungrae Park. द्वाराअनुसंधान पत्र [BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents](https://arxiv.org/abs/2108.04539) के साथ जारी किया गया
1. **[ByT5](https://huggingface.co/docs/transformers/model_doc/byt5)** (Google अनुसंधान से) साथ में कागज [ByT5: पूर्व-प्रशिक्षित बाइट-टू-बाइट मॉडल के साथ एक टोकन-मुक्त भविष्य की ओर] (https://arxiv.org/abs/2105.13626) Linting Xue, Aditya Barua, Noah Constant, रामी अल-रफू, शरण नारंग, मिहिर काले, एडम रॉबर्ट्स, कॉलिन रैफेल द्वारा पोस्ट किया गया।
1. **[CamemBERT](https://huggingface.co/docs/transformers/model_doc/camembert)** (इनरिया/फेसबुक/सोरबोन से) साथ में कागज [CamemBERT: एक टेस्टी फ्रेंच लैंग्वेज मॉडल](https:// arxiv.org/abs/1911.03894) लुई मार्टिन*, बेंजामिन मुलर*, पेड्रो जेवियर ऑर्टिज़ सुआरेज़*, योआन ड्यूपॉन्ट, लॉरेंट रोमरी, एरिक विलेमोन्टे डे ला क्लर्जरी, जैमे सेडाह और बेनोइट सगोट द्वारा।
1. **[CANINE](https://huggingface.co/docs/transformers/model_doc/canine)** (Google रिसर्च से) साथ में दिया गया पेपर [कैनाइन: प्री-ट्रेनिंग ए एफिशिएंट टोकनाइजेशन-फ्री एनकोडर फॉर लैंग्वेज रिप्रेजेंटेशन]( https://arxiv.org/abs/2103.06874) जोनाथन एच क्लार्क, डैन गैरेट, यूलिया टर्क, जॉन विएटिंग द्वारा।
Expand Down
1 change: 1 addition & 0 deletions README_ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -321,6 +321,7 @@ Flax、PyTorch、TensorFlowをcondaでインストールする方法は、それ
1. **[BLOOM](https://huggingface.co/docs/transformers/model_doc/bloom)** (BigScience workshop から) [BigScience Workshop](https://bigscience.huggingface.co/) から公開されました.
1. **[BORT](https://huggingface.co/docs/transformers/model_doc/bort)** (Alexa から) Adrian de Wynter and Daniel J. Perry から公開された研究論文: [Optimal Subarchitecture Extraction For BERT](https://arxiv.org/abs/2010.10499)
1. **[BridgeTower](https://huggingface.co/docs/transformers/model_doc/bridgetower)** (Harbin Institute of Technology/Microsoft Research Asia/Intel Labs から) released with the paper [BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learning](https://arxiv.org/abs/2206.08657) by Xiao Xu, Chenfei Wu, Shachar Rosenman, Vasudev Lal, Wanxiang Che, Nan Duan.
1. **[Bros](https://huggingface.co/docs/transformers/model_doc/bros)** (NAVER から) Teakgyu Hong, Donghyun Kim, Mingi Ji, Wonseok Hwang, Daehyun Nam, Sungrae Park. から公開された研究論文 [BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents](https://arxiv.org/abs/2108.04539)
1. **[ByT5](https://huggingface.co/docs/transformers/model_doc/byt5)** (Google Research から) Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel から公開された研究論文: [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://arxiv.org/abs/2105.13626)
1. **[CamemBERT](https://huggingface.co/docs/transformers/model_doc/camembert)** (Inria/Facebook/Sorbonne から) Louis Martin*, Benjamin Muller*, Pedro Javier Ortiz Suárez*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot から公開された研究論文: [CamemBERT: a Tasty French Language Model](https://arxiv.org/abs/1911.03894)
1. **[CANINE](https://huggingface.co/docs/transformers/model_doc/canine)** (Google Research から) Jonathan H. Clark, Dan Garrette, Iulia Turc, John Wieting から公開された研究論文: [CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation](https://arxiv.org/abs/2103.06874)
Expand Down
1 change: 1 addition & 0 deletions README_ko.md
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,7 @@ Flax, PyTorch, TensorFlow 설치 페이지에서 이들을 conda로 설치하는
1. **[BLOOM](https://huggingface.co/docs/transformers/model_doc/bloom)** (from BigScience workshop) released by the [BigScience Workshop](https://bigscience.huggingface.co/).
1. **[BORT](https://huggingface.co/docs/transformers/model_doc/bort)** (Alexa 에서) Adrian de Wynter and Daniel J. Perry 의 [Optimal Subarchitecture Extraction For BERT](https://arxiv.org/abs/2010.10499) 논문과 함께 발표했습니다.
1. **[BridgeTower](https://huggingface.co/docs/transformers/model_doc/bridgetower)** (from Harbin Institute of Technology/Microsoft Research Asia/Intel Labs) released with the paper [BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learning](https://arxiv.org/abs/2206.08657) by Xiao Xu, Chenfei Wu, Shachar Rosenman, Vasudev Lal, Wanxiang Che, Nan Duan.
1. **[Bros](https://huggingface.co/docs/transformers/model_doc/bros)** (NAVER 에서 제공)은 Teakgyu Hong, Donghyun Kim, Mingi Ji, Wonseok Hwang, Daehyun Nam, Sungrae Park.의 [BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents](https://arxiv.org/abs/2108.04539)논문과 함께 발표했습니다.
1. **[ByT5](https://huggingface.co/docs/transformers/model_doc/byt5)** (Google Research 에서) Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel 의 [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://arxiv.org/abs/2105.13626) 논문과 함께 발표했습니다.
1. **[CamemBERT](https://huggingface.co/docs/transformers/model_doc/camembert)** (Inria/Facebook/Sorbonne 에서) Louis Martin*, Benjamin Muller*, Pedro Javier Ortiz Suárez*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot 의 [CamemBERT: a Tasty French Language Model](https://arxiv.org/abs/1911.03894) 논문과 함께 발표했습니다.
1. **[CANINE](https://huggingface.co/docs/transformers/model_doc/canine)** (Google Research 에서) Jonathan H. Clark, Dan Garrette, Iulia Turc, John Wieting 의 [CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation](https://arxiv.org/abs/2103.06874) 논문과 함께 발표했습니다.
Expand Down
1 change: 1 addition & 0 deletions README_zh-hans.md
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,7 @@ conda install -c huggingface transformers
1. **[BLOOM](https://huggingface.co/docs/transformers/model_doc/bloom)** (from BigScience workshop) released by the [BigScience Workshop](https://bigscience.huggingface.co/).
1. **[BORT](https://huggingface.co/docs/transformers/model_doc/bort)** (来自 Alexa) 伴随论文 [Optimal Subarchitecture Extraction For BERT](https://arxiv.org/abs/2010.10499) 由 Adrian de Wynter and Daniel J. Perry 发布。
1. **[BridgeTower](https://huggingface.co/docs/transformers/model_doc/bridgetower)** (from Harbin Institute of Technology/Microsoft Research Asia/Intel Labs) released with the paper [BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learning](https://arxiv.org/abs/2206.08657) by Xiao Xu, Chenfei Wu, Shachar Rosenman, Vasudev Lal, Wanxiang Che, Nan Duan.
1. **[Bros](https://huggingface.co/docs/transformers/model_doc/bros)** (来自 NAVER) 伴随论文 [BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents](https://arxiv.org/abs/2108.04539) 由 Teakgyu Hong, Donghyun Kim, Mingi Ji, Wonseok Hwang, Daehyun Nam, Sungrae Park 发布。
1. **[ByT5](https://huggingface.co/docs/transformers/model_doc/byt5)** (来自 Google Research) 伴随论文 [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://arxiv.org/abs/2105.13626) 由 Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel 发布。
1. **[CamemBERT](https://huggingface.co/docs/transformers/model_doc/camembert)** (来自 Inria/Facebook/Sorbonne) 伴随论文 [CamemBERT: a Tasty French Language Model](https://arxiv.org/abs/1911.03894) 由 Louis Martin*, Benjamin Muller*, Pedro Javier Ortiz Suárez*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot 发布。
1. **[CANINE](https://huggingface.co/docs/transformers/model_doc/canine)** (来自 Google Research) 伴随论文 [CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation](https://arxiv.org/abs/2103.06874) 由 Jonathan H. Clark, Dan Garrette, Iulia Turc, John Wieting 发布。
Expand Down
1 change: 1 addition & 0 deletions README_zh-hant.md
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,7 @@ conda install -c huggingface transformers
1. **[BLOOM](https://huggingface.co/docs/transformers/model_doc/bloom)** (from BigScience workshop) released by the [BigScience Workshop](https://bigscience.huggingface.co/).
1. **[BORT](https://huggingface.co/docs/transformers/model_doc/bort)** (from Alexa) released with the paper [Optimal Subarchitecture Extraction For BERT](https://arxiv.org/abs/2010.10499) by Adrian de Wynter and Daniel J. Perry.
1. **[BridgeTower](https://huggingface.co/docs/transformers/model_doc/bridgetower)** (from Harbin Institute of Technology/Microsoft Research Asia/Intel Labs) released with the paper [BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learning](https://arxiv.org/abs/2206.08657) by Xiao Xu, Chenfei Wu, Shachar Rosenman, Vasudev Lal, Wanxiang Che, Nan Duan.
1. **[Bros](https://huggingface.co/docs/transformers/model_doc/bros)** (from NAVER) released with the paper [BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents](https://arxiv.org/abs/2108.04539) by Teakgyu Hong, Donghyun Kim, Mingi Ji, Wonseok Hwang, Daehyun Nam, Sungrae Park.
1. **[ByT5](https://huggingface.co/docs/transformers/model_doc/byt5)** (from Google Research) released with the paper [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://arxiv.org/abs/2105.13626) by Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel.
1. **[CamemBERT](https://huggingface.co/docs/transformers/model_doc/camembert)** (from Inria/Facebook/Sorbonne) released with the paper [CamemBERT: a Tasty French Language Model](https://arxiv.org/abs/1911.03894) by Louis Martin*, Benjamin Muller*, Pedro Javier Ortiz Suárez*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot.
1. **[CANINE](https://huggingface.co/docs/transformers/model_doc/canine)** (from Google Research) released with the paper [CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation](https://arxiv.org/abs/2103.06874) by Jonathan H. Clark, Dan Garrette, Iulia Turc, John Wieting.
Expand Down
2 changes: 2 additions & 0 deletions docs/source/en/_toctree.yml
Original file line number Diff line number Diff line change
Expand Up @@ -634,6 +634,8 @@
title: BLIP-2
- local: model_doc/bridgetower
title: BridgeTower
- local: model_doc/bros
title: Bros
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
title: Bros
title: BROS

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied suggestions from code review!

- local: model_doc/chinese_clip
title: Chinese-CLIP
- local: model_doc/clip
Expand Down
2 changes: 2 additions & 0 deletions docs/source/en/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ The documentation is organized into five sections:
1. **[BLOOM](model_doc/bloom)** (from BigScience workshop) released by the [BigScience Workshop](https://bigscience.huggingface.co/).
1. **[BORT](model_doc/bort)** (from Alexa) released with the paper [Optimal Subarchitecture Extraction For BERT](https://arxiv.org/abs/2010.10499) by Adrian de Wynter and Daniel J. Perry.
1. **[BridgeTower](model_doc/bridgetower)** (from Harbin Institute of Technology/Microsoft Research Asia/Intel Labs) released with the paper [BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learning](https://arxiv.org/abs/2206.08657) by Xiao Xu, Chenfei Wu, Shachar Rosenman, Vasudev Lal, Wanxiang Che, Nan Duan.
1. **[Bros](model_doc/bros)** (from NAVER CLOVA) released with the paper [BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents](https://arxiv.org/abs/2108.04539) by Teakgyu Hong, Donghyun Kim, Mingi Ji, Wonseok Hwang, Daehyun Nam, Sungrae Park.
1. **[ByT5](model_doc/byt5)** (from Google Research) released with the paper [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://arxiv.org/abs/2105.13626) by Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel.
1. **[CamemBERT](model_doc/camembert)** (from Inria/Facebook/Sorbonne) released with the paper [CamemBERT: a Tasty French Language Model](https://arxiv.org/abs/1911.03894) by Louis Martin*, Benjamin Muller*, Pedro Javier Ortiz Suárez*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot.
1. **[CANINE](model_doc/canine)** (from Google Research) released with the paper [CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation](https://arxiv.org/abs/2103.06874) by Jonathan H. Clark, Dan Garrette, Iulia Turc, John Wieting.
Expand Down Expand Up @@ -308,6 +309,7 @@ Flax), PyTorch, and/or TensorFlow.
| BLIP-2 | ✅ | ❌ | ❌ |
| BLOOM | ✅ | ❌ | ✅ |
| BridgeTower | ✅ | ❌ | ❌ |
| Bros | ✅ | ❌ | ❌ |
| CamemBERT | ✅ | ✅ | ❌ |
| CANINE | ✅ | ❌ | ❌ |
| Chinese-CLIP | ✅ | ❌ | ❌ |
Expand Down
Loading