Fixing tokenizer when transformers is installed without tokenizers#26236
Merged
ArthurZucker merged 3 commits intohuggingface:mainfrom urialon:fixSep 27, 2023
+3
transformers is installed without tokenizers#26236