A subset of Artificial Intelligence (AI) that focuses on the use of statistical algorithms that can learn from data and generalize onto unseen data.
Core Foundations:
- Probability & Statistics
- Linear Algebra
- Optimization
Core Concepts:
- Representation: How models structure knowledge
- Evaluation: How to measure model performance
- Optimization: Best model parameters
Map of Content
Paradigms
Deep Learning · Supervised ML · Unsupervised ML · Self-supervised learning (SSL) · Reinforcement Learning · Representation Learning
Training mechanics
Neural Network · Backpropogation · Gradient Descent · Optimizer · Loss Function · Learning Rate · Weight Decay · Training Loop · Activation Function · Softmax · Logits · Normalization Layers · Universal Approximation Theorem (UAT)
Architectures
Transformer · Attention · Autoencoders · Autoregressive Models · Graph Neural Network (GNN) · E(n) Equivariant Graph Neural Networks (EGNN) · e3nn (Euclidean 3D Neural Networks) · GotenNet · Joint Embedding Predictive Architectures (JEPAs)
Representation & latent space
Latent Space · Contrastive learning · Bootstrap Your Own Latent (BYOL) · Kernel method · PCA (Principal Component Analysis) · Hierarchical Tensor Refinement (HTR)
Classical ML
Support vector machine (SVM) · Kernel method · PCA (Principal Component Analysis)
Reinforcement learning
Reinforcement Learning · Monte Carlo Tree Search (MCTS)
Molecular ML & dynamics
Markov State Model (MSM) · VAMP · VAMP-1 · VAMP-2 · VAMP-E · VAMPnets · tICA (Time-lagged Independent Component Analysis) · TS-DAR · Collective Variable (CV) · Nudged Elastic Band (NEB)
Interpretability & alignment
Mechanistic Interpretability · AI Ethics · Instrumental convergence · Computational Functionalism
AI (broad)
Artificial Intelligence (AI) · Artificial General Intelligence (AGI) · Artificial Superintelligence (ASI)