Bibliography:

1. ​ https://research.google/blog/a-fast-wordpiece-tokenization-system/

2. ​ BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

3. ​ https://github.com/huggingface/tokenizers

4. ​ https://www.tensorflow.org/text/guide/subwords_tokenizer

5. Wikipedia Bibliography:

   1. ​ Lexical Analysis § Tokenization

   2. ​ https://en.wikipedia.org/wiki/Aho%E2%80%93Corasick_algorithm

   3. ​ Trie