There are some issue using T5 for regular people since the tokenizer has some issues. mT5 though, that one is pretty solid to start with.

Example: https://stackoverflow.com/questions/75851029/t5-fine-tuned-m...