Table of Contents

  1. The Deep Learning Revolution
    1. The Impact of Deep Learning
    2. A Tutorial Example
    3. A Brief History of Machine Learning
  2. Probabilities
    1. The Rules of Probability
    2. Probability Densities
    3. The Gaussian Distribution
    4. Transformation Of Densities
    5. Information Theory
    6. Bayesian Probabilities
  3. Standard Distributions
    1. Discrete Variables
    2. The Multivariate Gaussian
    3. Periodic Variables
    4. The Exponential Family
    5. Nonparametric Methods
    6. Histograms
    7. Kernel densities
    8. Nearest-neighbours
  4. Single Layer Networks: Regression
    1. Linear Regression
    2. Decision Theory
    3. The Bias-Variance Trade-off
  5. Single-layer Networks: Classification
    1. Discrimination Functions
    2. Decision Theory
    3. Generative Classifiers
    4. Discriminative Classifiers
  6. Deep Neural Networks
    1. Limitations of Fixed Basis Functions
    2. Mulilayer Networks
    3. Deep Networks
    4. Error Functions
    5. Mixture Density Networks
  7. Gradient Descent
    1. Error Surfaces
    2. Gradient Descent Optimization
    3. Convergence
    4. Normalization
    5. Batch Normalization
    6. Layer Normalization
  8. Backpropegations
    1. Evaluation of Gradients
    2. Automatic Differntiation
  9. NineRegularization
    1. Induction Bias
    2. Weight Decay
    3. Learning Curves
    4. Parameter Sharing
    5. Residual Connections
    6. Model Averaging
  10. Convolutional Networks
    1. Computer Vision
    2. Convulutional Filters
    3. General Graph Networks
  11. Structured Distributions
    1. Graphical Models
    2. Conditional Independence
    3. Sequence Models
  12. Transformers
    1. Attention
    2. Natural Language
    3. Transformer Language Models
    4. Multimodal Transformers]]
  13. Graph Neural Networks
    1. Machine Learning on Graphs
    2. Neural Message-Passing
    3. General Graph Networks
  14. Sampling
    1. Basic Sampling Algorithms
    2. Markov Chgain Monte Carlo
    3. Langevin Sampling
  15. FifteenLatentVariables
    1. k-means Clustering
    2. Mixture of Gaussians
    3. Expectation-Maximizastion Algorithm
    4. Evidence Lower Bound
  16. Continuous Latent Variables
    1. Principal Component Analysis
    2. Probabilistic Latent Variables
    3. Evidence Lower Bound
    4. Nonlinear Latent Variable Models
  17. Generative Adversarial Networks
    1. Adversarial Training
    2. Image GANs
  18. Normalizing Flows
    1. Coupling Flows
    2. Autoregressive Flows
    3. Continuous Flows
  19. Autoencoders
    1. Deterministic Autoencoders
    2. Variational Autoencoders
  20. Diffusion Models
    1. Forward Encoder
    2. Reverse Encoder
    3. Score Matching
    4. Guided Diffusion