A curated list of the most cited deep learning papers implementation.
- Fundamentals of NLP
- Foundation of Deep Learning for NLP
- Language Models and Pretraining
- Advanced Language Models
- Multimodal Learning
- Fine-Tuning and Prompting
- Efficient NLP Models
- RLHF and Fine-Tuning
- Recent State-of-the-Art Models
- Specialized Topics and Applications
- Efficient Estimation of Word Representations in Vector Space, Mikolov et al.
pdf - Distributed Representations of Words and Phrases and Their Compositionality, Mikolov et al.
pdf - Linguistic Regularities in Continuous Space Word Representations, Mikolov et al.
pdf - GloVe: Global Vectors for Word Representation, Pennington et al.
pdf - Sequence to Sequence Learning with Neural Networks, Sutskever et al.
pdf - Neural Machine Translation by Jointly Learning to Align and Translate, Bahdanau et al.
pdf
- Attention Is All You Need, Vaswani et al.
pdf - Deep Contextualized Word Representations, Peters et al.
pdf - Universal Language Model Fine-tuning for Text Classification, Howard and Ruder
pdf
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, Devlin et al.
pdf - Improving Language Understanding by Generative Pre-training (GPT), Radford et al.
pdf - Language Models Are Few-Shot Learners (GPT-2), Radford et al.
pdf
- Language Models Are Few-Shot Learners (GPT-3), Brown et al.
pdf - Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (T5), Raffel et al.
pdf - XLNet: Generalized Autoregressive Pretraining for Language Understanding, Yang et al.
pdf - RoBERTa: A Robustly Optimized BERT Pretraining Approach, Liu et al.
pdf
- Learning Transferable Visual Models From Natural Language Supervision (CLIP), Radford et al.
pdf - Zero-Shot Text-to-Image Generation (DALL-E), Ramesh et al.
pdf - Few-Shot Learning with Multimodal Models (Flamingo), Alayrac et al.
pdf
- Making Pre-trained Language Models Better Few-shot Learners, Gao et al.
pdf - Low-Rank Adaptation of Large Language Models (LoRA), Hu et al.
pdf
- DistilBERT: A Distilled Version of BERT, Sanh et al.
pdf - Generating Long Sequences with Sparse Transformers, Child et al.
pdf
- Training Language Models to Follow Instructions with Human Feedback (InstructGPT), OpenAI
pdf - Proximal Policy Optimization, Schulman et al.
pdf
- LLaMA: Open and Efficient Foundation Language Models, Meta AI
pdf - Gemini AI (2024): Monitor papers or blogs by OpenAI and Google DeepMind.
- Understanding / Generalization / Transfer
- Optimization / Training Techniques
- Unsupervised / Generative Models
- Convolutional Network Models
- Image Segmentation / Object Detection
- Image / Video / Etc
- Distilling the knowledge in a neural network (2015), G. Hinton et al. [pdf]
- Deep neural networks are easily fooled: High confidence predictions for unrecognizable images (2015), A. Nguyen et al. [pdf]
- How transferable are features in deep neural networks? (2014), J. Yosinski et al. [pdf]
- CNN features off-the-Shelf: An astounding baseline for recognition (2014), A. Razavian et al. [pdf]
- Learning and transferring mid-Level image representations using convolutional neural networks (2014), M. Oquab et al. [pdf]
- Visualizing and understanding convolutional networks (2014), M. Zeiler and R. Fergus [pdf]
- Decaf: A deep convolutional activation feature for generic visual recognition (2014), J. Donahue et al. [pdf]
- Training very deep networks (2015), R. Srivastava et al. [pdf]
- Batch normalization: Accelerating deep network training by reducing internal covariate shift (2015), S. Loffe and C. Szegedy [pdf]
- Delving deep into rectifiers: Surpassing human-level performance on imagenet classification (2015), K. He et al. [pdf]
- Dropout: A simple way to prevent neural networks from overfitting (2014), N. Srivastava et al. [pdf]
- Adam: A method for stochastic optimization (2014), D. Kingma and J. Ba [pdf]
- Improving neural networks by preventing co-adaptation of feature detectors (2012), G. Hinton et al. [pdf]
- Random search for hyper-parameter optimization (2012) J. Bergstra and Y. Bengio [pdf]
- Pixel recurrent neural networks (2016), A. Oord et al. [pdf]
- Improved techniques for training GANs (2016), T. Salimans et al. [pdf]
- Unsupervised representation learning with deep convolutional generative adversarial networks (2015), A. Radford et al. [pdf]
- DRAW: A recurrent neural network for image generation (2015), K. Gregor et al. [pdf]
- Generative adversarial nets (2014), I. Goodfellow et al. [pdf]
- Auto-encoding variational Bayes (2013), D. Kingma and M. Welling [pdf]
- Building high-level features using large scale unsupervised learning (2013), Q. Le et al. [pdf]
- EffNet: An Efficient Structure for Convolutional Neural Networks, Freeman et al.
pdf - Squeeze-and-Excitation Networks, Hu et al.
pdf - MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications, Howard et al.
pdf - ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices, Zhang et al.
pdf - Residual Attention Network for Image Classification, Wang et al.
pdf - Densely Connected Convolutional Networks, Huang et al.
pdf - Xception: Deep Learning with Depthwise Separable Convolutions, Chollet.
pdf - DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations, Liu et al.
pdf - Practical Riemannian Neural Networks, Marceau-Caron and Ollivier.
pdf - Rethinking the Inception Architecture for Computer Vision, Szegedy et al.
pdf - Deep Residual Learning for Image Recognition, He et al.
pdf - Very Deep Convolutional Networks for Large-Scale Image Recognition, Simonyan et al.
pdf - Going Deeper with Convolutions, Szegedy et al.
pdf⭐ - ImageNet Classification with Deep Convolutional Neural Networks(AlexNet), Krizhevsky et al.
pdf⭐
- You only look once: Unified, real-time object detection (2016), J. Redmon et al. [pdf]
- Fully convolutional networks for semantic segmentation (2015), J. Long et al. [pdf]
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2015), S. Ren et al. [pdf]
- Fast R-CNN (2015), R. Girshick [pdf]
- Rich feature hierarchies for accurate object detection and semantic segmentation (2014), R. Girshick et al. [pdf]
- Spatial pyramid pooling in deep convolutional networks for visual recognition (2014), K. He et al. [pdf]
- Semantic image segmentation with deep convolutional nets and fully connected CRFs, L. Chen et al. [pdf]
- Learning hierarchical features for scene labeling (2013), C. Farabet et al. [pdf]
- Image Super-Resolution Using Deep Convolutional Networks (2016), C. Dong et al. [pdf]
- A neural algorithm of artistic style (2015), L. Gatys et al. [pdf]
- Deep visual-semantic alignments for generating image descriptions (2015), A. Karpathy and L. Fei-Fei [pdf]
- Show, attend and tell: Neural image caption generation with visual attention (2015), K. Xu et al. [pdf]
- Show and tell: A neural image caption generator (2015), O. Vinyals et al. [pdf]
- Long-term recurrent convolutional networks for visual recognition and description (2015), J. Donahue et al. [pdf]
- VQA: Visual question answering (2015), S. Antol et al. [pdf]
- DeepFace: Closing the gap to human-level performance in face verification (2014), Y. Taigman et al. [pdf]:
- Large-scale video classification with convolutional neural networks (2014), A. Karpathy et al. [pdf]
- Two-stream convolutional networks for action recognition in videos (2014), K. Simonyan et al. [pdf]
- 3D convolutional neural networks for human action recognition (2013), S. Ji et al. [pdf]