Skip to content
anythingai.space
MLOpsintermediate13 min · 8 steps

ML Infrastructure and Orchestration

Pipelines as DAGs, the orchestrators that run them — Airflow, Kubeflow, TFX — and the CPUs, GPUs and TPUs underneath.

ONE NIGHT OF CRON JOBSextractextract (ran long)train03:00 train on stale dataevaluate01:00 → 08:00No dependencies, no retries, no record of what ran onwhich data
requestsignaldataheavy workresponse
1 / 8
ProblemStep 1 of 8

Cron jobs that don't know about each other

The first ML “platform” is often a few cron jobs on one GPU box: extract at 01:00, train at 03:00, evaluate at 05:00. Each job assumes the previous one finished.

One night the extract runs long. Training starts anyway on yesterday's data, evaluation passes, and nobody notices — there's no dependency tracking, no retry and no record of what ran.

Watch outTime-based scheduling is not dependency management.

Lesson 5 of 7 in MLOps