Very interesting to see an LLM with weights and the code base.
They also talk about tokenizer fertility in the HF model card [1]<p>[1]: <a href="https://huggingface.co/Aleph-Alpha/Pharia-1-LLM-7B-control" rel="nofollow">https://huggingface.co/Aleph-Alpha/Pharia-1-LLM-7B-control</a>