Deep Learning:
Category Topics Mathematics Techniques/Algorithms
1. Foundations Artificial Neural Linear Algebra Backpropagation, Gradient
Networks (ANNs), (Vectors, Matrices), Descent
Feedforward Calculus
Networks, Activation (Derivatives),
Functions Probability (Sigmoid,
Softmax)
2. Activation Sigmoid, Tanh, Calculus Function Approximation
Functions ReLU, Leaky ReLU, (Differentiable
ELU, Softmax Functions)
3. Loss Functions Calculus (Gradients, SGD, Momentum,
Optimization (MSE, Hessians), RMSProp, Adam, AdamW
Cross-Entropy, Hinge Numerical
Loss), Optimization
Gradient-Based
Optimization
4. Dropout, Batch Probability L1/L2 Regularization
Regularization Normalization, (Regularization),
Weight Decay, Early Statistics (Variance
Stopping Reduction)
5. Deep Fully Connected Linear Algebra CNN Architectures (VGG,
Architectures Networks, (Convolutions, ResNet), RNN Architectures
Convolutional Neural Tensors), Calculus, (LSTMs, GRUs), Attention
Networks (CNNs), Probability Mechanisms (Transformers)
Recurrent Neural
Networks (RNNs),
Transformers
6. Transfer Pretrained Models, Linear Algebra ResNet, EfficientNet, BERT
Learning Fine-Tuning, Feature (Vector Spaces),
Extraction Optimization
7. Generative Variational Probability (Latent GAN Variants (DCGAN,
Models Autoencoders Variables), CycleGAN), Variational
(VAEs), Generative Information Theory Inference
Adversarial Networks
(GANs), Diffusion
Models
8. Sequence RNNs, LSTMs, Linear Algebra, Encoder-Decoder Models,
Models GRUs, Transformers, Probability (Markov Self-Attention, BERT, GPT
Attention Chains), Statistics
Mechanisms
9. Deep Q-Learning, Probability (Markov DQN, PPO, A3C
Reinforcement Policy Gradient Decision Processes),
Learning Methods, Actor-Critic Linear Algebra
Models
10. SHAP, LIME, Probability (Feature Saliency Maps, Grad-CAM
Explainability Attention Attribution), Linear
Visualization, Algebra
Gradient-Based
Interpretability
11. Computer Image Classification, Linear Algebra Faster R-CNN, YOLO, Mask
Vision Object Detection, (Convolutions), R-CNN, U-Net
Semantic Geometry (Affine
Segmentation Transformations),
Probability
12. Natural Text Embeddings Linear Algebra Seq2Seq, BERT, GPT, T5,
Language (Word2Vec, GloVe, (Embeddings), Transformers
Processing Transformers), Probability
Language Models, (n-Grams), Statistics
Sentiment Analysis,
Text Summarization
13. Contrastive Learning, Probability (Mutual SimCLR, BYOL, MAE,
Self-Supervise Masked Modeling, Information), Linear DINO
d Learning Representation Algebra
Learning
14. Zero-Shot/ Generalization from Probability (Bayesian GPT (Zero-Shot), Few-Shot
Few-Shot Limited Data Inference), Statistics Transformers
Learning
15. Neural Hyperparameter Optimization (Search Grid Search, Random
Architecture Tuning, Automated Algorithms), Linear Search, Bayesian
Search Model Design Algebra Optimization, AutoML
16. Adversarial Adversarial Probability Fast Gradient Sign Method
ML Examples, Robust (Perturbation (FGSM), Projected Gradient
Models Analysis), Linear Descent (PGD)
Algebra
17. Training Weight Initialization Probability Cyclical Learning Rates,
Tricks (He, Xavier), (Distribution of Mixup, Cutout
Learning Rate Weights), Linear
Schedulers, Data Algebra
Augmentation
18. Graph Node Classification, Graph Theory, Graph Convolutional
Neural Graph Embeddings, Linear Algebra Networks (GCNs), Graph
Networks Graph Convolutions Attention Networks (GATs)
19. Time Sequence Linear Algebra, LSTMs, Temporal
Series Forecasting, Statistics Convolutional Networks
Temporal (Autocorrelation)
Convolutions
20. Large Pretraining, Probability BERT, GPT-4, LLaMA,
Language Fine-Tuning, (Next-Token Bloom
Models Foundation Models Prediction), Linear
Algebra
