Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mes/longformer on beaker copy all #250

Open
wants to merge 67 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
95296ad
pretraining script
ibeltagy Jul 16, 2020
325693e
wip
ibeltagy Jul 16, 2020
985acc9
wip
ibeltagy Jul 17, 2020
023dd78
wip
ibeltagy Jul 17, 2020
08230ac
wip
ibeltagy Jul 17, 2020
fb65d57
.
ibeltagy Jul 17, 2020
0e80cde
pad chunks or start next doc
ibeltagy Jul 17, 2020
6ca7d1b
todo
ibeltagy Jul 17, 2020
a2aa4f7
wip
ibeltagy Jul 17, 2020
62a69d5
wip
ibeltagy Jul 17, 2020
3e3a478
wip
ibeltagy Jul 18, 2020
3bc5354
wip
ibeltagy Jul 18, 2020
1a91024
wip
ibeltagy Jul 18, 2020
5fa21f2
wip
ibeltagy Jul 18, 2020
18eb003
wip
ibeltagy Jul 18, 2020
607e446
wip
ibeltagy Jul 18, 2020
d4659de
wip
ibeltagy Jul 18, 2020
c7c53cb
wip
ibeltagy Jul 19, 2020
0a07daf
wip
ibeltagy Jul 22, 2020
827576c
wip
ibeltagy Jul 22, 2020
1a6498c
tpu
ibeltagy Jul 22, 2020
3e82548
wip
ibeltagy Jul 22, 2020
adadd42
wip
ibeltagy Jul 23, 2020
9e191a0
pretraining script
ibeltagy Jul 16, 2020
9d18808
wip
ibeltagy Jul 16, 2020
6e24cee
wip
ibeltagy Jul 17, 2020
a2ab9b3
wip
ibeltagy Jul 17, 2020
e3f4ba9
wip
ibeltagy Jul 17, 2020
f9e654b
.
ibeltagy Jul 17, 2020
9c2646d
pad chunks or start next doc
ibeltagy Jul 17, 2020
433a2e2
todo
ibeltagy Jul 17, 2020
ec47270
wip
ibeltagy Jul 17, 2020
77e105d
wip
ibeltagy Jul 17, 2020
af08b5a
wip
ibeltagy Jul 18, 2020
d105023
wip
ibeltagy Jul 18, 2020
1183999
wip
ibeltagy Jul 18, 2020
20e8208
wip
ibeltagy Jul 18, 2020
224824d
wip
ibeltagy Jul 18, 2020
4a12730
wip
ibeltagy Jul 18, 2020
c936d24
wip
ibeltagy Jul 18, 2020
510801b
wip
ibeltagy Jul 19, 2020
9184b71
wip
ibeltagy Jul 22, 2020
4ae991a
wip
ibeltagy Jul 22, 2020
aea2a98
tpu
ibeltagy Jul 22, 2020
69b717a
wip
ibeltagy Jul 22, 2020
5f641c0
wip
ibeltagy Jul 23, 2020
e3ddeca
wip
ibeltagy Jul 23, 2020
21c9e57
Merge branch 'trainer' of github.com:allenai/longformer into trainer
ibeltagy Jul 23, 2020
00ce1e9
wip
ibeltagy Jul 23, 2020
56b9c6a
wip
ibeltagy Jul 23, 2020
8fca187
wip
ibeltagy Jul 25, 2020
9dd76b7
wip
ibeltagy Jul 25, 2020
d40983a
wip
ibeltagy Jul 25, 2020
f0f6a30
wip
ibeltagy Jul 25, 2020
a6e37df
Merge branch 'trainer' of github.com:allenai/longformer into trainer
ibeltagy Jul 25, 2020
9eb6fdf
wip
ibeltagy Jul 25, 2020
14b6074
wip
ibeltagy Jul 25, 2020
5b97bd6
wip
ibeltagy Jul 25, 2020
71d7a9d
wip
ibeltagy Jul 25, 2020
97a126d
wip
ibeltagy Jul 25, 2020
c873da2
wip
ibeltagy Jul 25, 2020
d602869
faster gradnorm
ibeltagy Jul 28, 2020
ffd06dd
allow changing seqlen at runtime
ibeltagy Jul 28, 2020
129a3f9
log and resume data preprocessing
ibeltagy Jul 30, 2020
1c42f96
multiprocessed preprocessing
ibeltagy Jul 30, 2020
c20264e
wip
ibeltagy Aug 3, 2020
ff96351
Save this directory as a dataset and use it directly on a plain base …
meslater1030 Aug 3, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
wip
  • Loading branch information
ibeltagy committed Jul 23, 2020
commit 20e8208af7617c132bf1d7e2008a4569c97f7beb
2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
pytorch-lightning @ git+http://github.com/ibeltagy/pytorch-lightning.git@v0.8.5_fixes#egg=pytorch-lightning

torch==1.3.0
torch==1.3.1
transformers==3.0.2
tensorboardX
test-tube==0.7.5