====== Software =====
See also this [[https://github.com/josephmisiti/awesome-machine-learning|list of ML related software]].

===== Deep Learning =====

==== Current, Recommended Frameworks ====
  * **Pytorch**
    * Huggingface
    * LLMs
      * [[https://unsloth.ai/|Unsloth]] [[https://github.com/unslothai/unsloth|github]] Incredibly fast. As of March 2025, this is the fastest single GPU framework for LLMs
      * [[https://github.com/ggml-org/llama.cpp|llama.cpp]]
      * LLVM
    * **Multi-GPU training**
      * Note: recommendations can change frequently (every few months it seems)
      * As of March 2025: Nilay recommends DeepSpeed, which is integrated with HuggingFace Accelerate.  See here: [[https://huggingface.co/docs/accelerate/en/usage_guides/deepspeed|HuggingFace Accelerate - DeepSpeed]]

==== Miscellaneous, Perhaps Less Used, or Older Ones ====
  * Fairseq (pytorch)
    * Tutorial on how to add a new model to fairseq: [[https://fairseq.readthedocs.io/en/latest/tutorial_simple_lstm.html|Tutorial: Simple LSTM]] Nilay used this tutorial when adding his model to fairseq
    * [[https://github.com/pytorch/fairseq/blob/main/examples/translation/README.md|Translation example]]
    * [[https://fairseq.readthedocs.io/en/latest/command_line_tools.html|CLI Documentation]]
  * OpenNMT (pytorch)
  * [[https://github.com/IntelLabs/nlp-architect|NLP Architect]] (Intel's NLP software)
  * [[https://minitorch.github.io/|MiniTorch]] (educational)
  * [[https://jax.readthedocs.io/en/latest/notebooks/quickstart.html|JAX]] + [[https://github.com/google/flax|Flax]]
  * Tensorflow
    * [[https://www.tensorflow.org/tfmodels/nlp/fine_tune_bert|Tensorflow - Fine-tuning a BERT model]]
  * Keras
    * [[https://huggingface.co/docs/transformers/training|Fine-tune a pretrained Huggingface model in Keras]] and [[https://huggingface.co/course/chapter3/3?fw=tf|Huggingface: Fine-tuning a model with Keras]]
  * Export formats
    * [[https://en.wikipedia.org/wiki/Open_Neural_Network_Exchange|ONNX]] The industry de-facto standard.  See this [[https://www.youtube.com/watch?v=pkBNjAYZKWA|video]] for how to import into Mathematica.  Videos: [[https://www.youtube.com/watch?v=dJUzO0bivaA|PyTorch to Tensorflow using ONNX]] [[https://www.youtube.com/watch?v=bE1N7sq3xIA|PyTorch, TensorFlow, Keras Model File Conversion]]

==== MT and Seq2seq ====
See [[nlp:Machine Translation#Software|MT Software]].
  * [[https://github.com/EdinburghNLP/nematus|Nematus]] (Tensorflow) Rico Sennrich's NMT system
  * [[https://github.com/marian-nmt/marian-dev|Marian]] [[https://arxiv.org/pdf/1804.00344.pdf|paper]] (C++) Optimized implementations of RNNs and Transformer
  * [[https://github.com/awslabs/sockeye|Sockeye]] (MXNet) Amazon's NMT system

===== Experimental Pipeline ======
  * [[https://wandb.ai/site|Weights and biases]]
  * [[https://www.tensorflow.org/tensorboard|Tensorboard]] [[https://pytorch.org/tutorials/intermediate/tensorboard_tutorial.html|PyTorch tutorial]] [[https://pytorch.org/tutorials/recipes/recipes/tensorboard_with_pytorch.html|PyTorch Recipe]]
  * [[https://dvc.org/|Data Version Control (DVC)]]


===== Resources =====
  * [[https://github.com/josephmisiti/awesome-machine-learning|Awesome Machine Learning]] A curated list of awesome machine learning frameworks, libraries and software.

===== Related Pages =====
  * [[Hugging Face]]
  * [[nlp:Transformers#Software|Transformers - Software]]
  * **[[nlp:Software|NLP - Software]]**
  * [[PyTorch]]
  * [[Tensorflow]]