Skip to content

Commit

Permalink
updating test since space before is arbitrarily removed
Browse files Browse the repository at this point in the history
  • Loading branch information
itazap committed Sep 6, 2024
1 parent 1511100 commit b7b2b09
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ def test_tokenizer_decode_added_tokens(self):
# fmt: on

batch_tokens = tokenizer.batch_decode(sample_ids)
self.assertEqual(batch_tokens, ["k s ɾ ɾ l ɭʲ!?!? $$$", "j ð s j ð s oːɹ $$$"])
self.assertEqual(batch_tokens, ["k s ɾ ɾ l ɭʲ ! ? ! ? $$$", "j ð s j ð s oːɹ $$$"])

@staticmethod
def get_from_offsets(offsets, key):
Expand Down

0 comments on commit b7b2b09

Please sign in to comment.