ml:pytorch

This is an old revision of the document!

PyTorch

Tutorials
- https://pytorch.org/tutorials/
- https://pytorch.org/tutorials/beginner/nn_tutorial.html
- https://pytorch.org/tutorials/recipes/recipes/loading_data_recipe.html (DataLoader)
- https://pytorch.org/tutorials/beginner/data_loading_tutorial.html
- https://pytorch.org/tutorials/recipes/recipes/defining_a_neural_network.html
- https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html Good seq2seq tutorial and starter code (doesn't have batching)
- pytorch-seq2seq example code Good seq2seq code with batching
- How to code The Transformer in Pytorch
Frameworks
- PyTorch Lightning A higher-level API for PyTorch that removes a bunch of boilerplate code
Models
- Transformer
  - https://github.com/SamLynnEvans/Transformer Nice implementation (Rongwen used it) with a tutorial: How to code The Transformer in Pytorch
Books
- Deep Learning with PyTorch Excerpts Online
Performance
- Profiling
  - Can et al 2020 - NVidia - Profiling and Optimizing Deep Neural Networks with DLProf and PyProf
  - Also Scalene - High-performance CPU and memory profiler for Python. Great for profiling PyTorch. video
- Issues
  - Advanced indexing gradient is extremely slow when there are many duplicate indices (Has a workaround using Tensor.index_select)

ml/pytorch.1633215726.txt.gz · Last modified: 2023/06/15 07:36 (external edit)