I'm worried about the version hell of relying on HuggingFace's transformers.
In the model card : config.json [1] and generation_config.json there are strings "transformers_version": "4.28.1" and it refers to some common architecture "GPTNeoXForCausalLM" .
Which if I follow the string soft link correctly means, it refers to this file [2] with a long history of commits including some recents [3].
I don't understand how to get out of the version hell. I can install and pin a version of the transformer library, but sooner or later if I chain different foundation models I'll have to do this for each different model card that has its own version.
And then those transformer library version are probably dependent on some pytorch versions.
I don't understand how you can't reliably track a bug, or a change in behavior, or guarantee that the model that you spend millions of dollar training doesn't rust, is reproducible or become unusable due to this mess.
(And also the weights are de-serialized pickled python dictionary which is from a safety point of view equivalent to running untrusted binaries and very worrying on its own.)
I've been diving in lately and while it's not efficient, the only way to do manage is to create a new conda/mamba environment, or a custom Docker image for all the conflicting packages.
In terms of pickled files, we may ask them to release the models in safetensors. That would be a big improvement.
There's not much we can do about dependencies on pytorch or other python libraries. Perhaps people can make more independent implementations. The redundancy in implementations would help.
Tensorflow saved models are a great way to solve the problem... Save the computation graph and weights, and drop all the crusty code dependencies. I think ONNX models are similar. I expect there should be a Jax equivalent at some point, as Jax is basically perfectly designed for this (everything is expressed in lax operations, which allows changing implementations for cpu/gpu/tpu freely... So just save the list of lax ops).
> And then those transformer library version are probably dependent on some pytorch versions.
They are not, and I dont think the model even cares about the transformers version. I run git transformers/diffusers and PyTorch 2.1 in all sorts of old repos, and if it doesnt immediately work, usually theres just small changes to APIs here and there that make scripts unhappy, and that you can manually fix.
In the model card : config.json [1] and generation_config.json there are strings "transformers_version": "4.28.1" and it refers to some common architecture "GPTNeoXForCausalLM" .
Which if I follow the string soft link correctly means, it refers to this file [2] with a long history of commits including some recents [3].
I don't understand how to get out of the version hell. I can install and pin a version of the transformer library, but sooner or later if I chain different foundation models I'll have to do this for each different model card that has its own version.
And then those transformer library version are probably dependent on some pytorch versions.
I don't understand how you can't reliably track a bug, or a change in behavior, or guarantee that the model that you spend millions of dollar training doesn't rust, is reproducible or become unusable due to this mess.
(And also the weights are de-serialized pickled python dictionary which is from a safety point of view equivalent to running untrusted binaries and very worrying on its own.)
[1]https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b/b... [2]https://github.com/huggingface/transformers/blob/v4.28.1/src... [3]https://github.com/huggingface/transformers/commits/v4.28.1/...