Machine Learning Knowledge Base

These are study notes on building machine learning systems end to end. They compress the process into three stages, design, development, and deployment, and follow one idea through all of them: most of the work in production ML is engineering and data, not algorithms.

Scope: classic (predictive) ML systems. LLMs and foundation models are not covered.

        flowchart LR
  subgraph design[2 · Design]
    direction TB
    A[Scope the<br>project] --> B[Define data and<br>a baseline] --> C[Label and<br>organize data]
  end
  subgraph development[3 · Development]
    direction TB
    D[Train a<br>model] --> E[Analyze<br>errors] --> F[Audit before<br>launch]
  end
  subgraph deployment[4 · Deployment]
    direction TB
    G[Deploy in<br>production] --> H[Monitor and<br>maintain]
  end
  design --> development --> deployment
  design <-. iterate .-> development
  development <-. iterate .-> deployment
    

From a business problem to a model serving users. Each stage is a set of steps, and work moves back and forth between stages: error analysis can call for more data, and monitoring triggers retraining. Adapted from DeepLearning.AI, MLOps Specialization.

1 · Overview

New here? Start with the lifecycle, which walks all three stages once, then use the project checklist as a step-by-step guide. The speech recognition case study shows the stages applied to one system.

• 1.1 Introduction: why ML, types of ML systems, and the main challenges

• 1.4 MLOps: maturity levels and what to automate at each stage

2 · Design

Design decides what to build before anything is trained: which business problem is worth solving, whether ML is the right tool, which single objective the model should optimize, and what data it needs, defined consistently enough that a model can learn from it.

• 2.1 Scoping

• 2.2 Data

3 · Development

Development turns scoped data into a working model: set up validation, get a baseline fast, then iterate with error analysis, more often by improving the data than the model. It ends with an audit before anything reaches users.

• 3.1 Modeling Overview

• 3.2 Validation and Hyperparameter Tuning

• 3.3 Model Baseline

• 3.4 Error Analysis

• 3.5 Prioritizing Improvements

• 3.6 Skewed Datasets

• 3.7 Performance Auditing

• 3.8 Data-centric AI Development

• 3.9 Experiment Tracking

4 · Deployment

Deployment puts the model in front of real data: choosing an architecture, rolling out gradually with a way back, making builds reproducible, and monitoring the system once it is live. A first deployment is only about halfway through the project, since live traffic reveals what development could not.

• 4.1 Key Challenges in Deployment

• 4.2 Deployment Architecture

• 4.3 Common Deployment Patterns

• 4.4 Reproducibility and CI/CD

• 4.5 Monitoring

• 4.6 Case Study: Defect Inspection in Manufacturing

Appendix

• Templates: the three documents worth writing along the way: one-pagers, design docs, and after-action reviews

• Sources: the courses, books, and articles these notes draw on