Skip to content

Port tokenization for the multilingual model - #2

Merged
thomwolf merged 2 commits into
huggingface:masterfrom
elyase:patch-1
Nov 10, 2018
Merged

Port tokenization for the multilingual model#2
thomwolf merged 2 commits into
huggingface:masterfrom
elyase:patch-1

Add test for Chinese tokenization

4d124ba
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs