Differences

This shows you the differences between two versions of the page.

--- ml:software [2022/04/22 08:21] – [Deep Learning] jmflanig
+++ ml:software [2025/10/09 17:40] (current) – [Current, Recommended Frameworks] jmflanig
@@ Line 3: / Line 3: @@
 ===== Deep Learning =====
-  * Huggingface (pytorch)
+==== Current, Recommended Frameworks ====
+  * **Pytorch**
+    * Huggingface
+    * LLMs
+      * [[https://unsloth.ai/|Unsloth]] [[https://github.com/unslothai/unsloth|github]] Incredibly fast. As of March 2025, this is the fastest single GPU framework for LLMs
+      * [[https://github.com/ggml-org/llama.cpp|llama.cpp]]
+      * LLVM
+    * **Multi-GPU training**
+      * Note: recommendations can change frequently (every few months it seems)
+      * As of March 2025: Nilay recommends DeepSpeed, which is integrated with HuggingFace Accelerate.  See here: [[https://huggingface.co/docs/accelerate/en/usage_guides/deepspeed|HuggingFace Accelerate - DeepSpeed]]
+==== Miscellaneous, Perhaps Less Used, or Older Ones ====
   * Fairseq (pytorch)
     * Tutorial on how to add a new model to fairseq: [[https://fairseq.readthedocs.io/en/latest/tutorial_simple_lstm.html|Tutorial: Simple LSTM]] Nilay used this tutorial when adding his model to fairseq
@@ Line 12: / Line 24: @@
   * [[https://minitorch.github.io/|MiniTorch]] (educational)
   * [[https://jax.readthedocs.io/en/latest/notebooks/quickstart.html|JAX]] + [[https://github.com/google/flax|Flax]]
+  * Tensorflow
+    * [[https://www.tensorflow.org/tfmodels/nlp/fine_tune_bert|Tensorflow - Fine-tuning a BERT model]]
+  * Keras
+    * [[https://huggingface.co/docs/transformers/training|Fine-tune a pretrained Huggingface model in Keras]] and [[https://huggingface.co/course/chapter3/3?fw=tf|Huggingface: Fine-tuning a model with Keras]]
   * Export formats
-    * [[https://en.wikipedia.org/wiki/Open_Neural_Network_Exchange|ONNX]] The industry de-facto standard.  See this [[https://www.youtube.com/watch?v=pkBNjAYZKWA|video]] for how to import into Mathematica.  Videos: [[https://www.youtube.com/watch?v=dJUzO0bivaA|PyTorch to Tensorflow using ONNX]]
+    * [[https://en.wikipedia.org/wiki/Open_Neural_Network_Exchange|ONNX]] The industry de-facto standard.  See this [[https://www.youtube.com/watch?v=pkBNjAYZKWA|video]] for how to import into Mathematica.  Videos: [[https://www.youtube.com/watch?v=dJUzO0bivaA|PyTorch to Tensorflow using ONNX]] [[https://www.youtube.com/watch?v=bE1N7sq3xIA|PyTorch, TensorFlow, Keras Model File Conversion]]
 ==== MT and Seq2seq ====
@@ Line 20: / Line 36: @@
   * [[https://github.com/marian-nmt/marian-dev|Marian]] [[https://arxiv.org/pdf/1804.00344.pdf|paper]] (C++) Optimized implementations of RNNs and Transformer
   * [[https://github.com/awslabs/sockeye|Sockeye]] (MXNet) Amazon's NMT system
+===== Experimental Pipeline ======
+  * [[https://wandb.ai/site|Weights and biases]]
+  * [[https://www.tensorflow.org/tensorboard|Tensorboard]] [[https://pytorch.org/tutorials/intermediate/tensorboard_tutorial.html|PyTorch tutorial]] [[https://pytorch.org/tutorials/recipes/recipes/tensorboard_with_pytorch.html|PyTorch Recipe]]
+  * [[https://dvc.org/|Data Version Control (DVC)]]
 ===== Resources =====
@@ Line 25: / Line 47: @@
 ===== Related Pages =====
+  * [[Hugging Face]]
   * [[nlp:Transformers#Software|Transformers - Software]]
-  * [[nlp:Software|NLP - Software]]
+  * **[[nlp:Software|NLP - Software]]**
   * [[PyTorch]]
+  * [[Tensorflow]]