Data Science Wire

Where should train-only preprocessing state live in an MLOps stack?

Reddit r/MLOps22h4 min read

Disclosure: I maintain DATADOC, an open-source project focused on one part of the MLOps handoff that keeps breaking in practice: how to move preprocessing from an experiment into a repeatable inference contract. The workflow I am testing is: Profile the raw tabular data. Generate a deterministic plan. Fit medians, vocabularies, scaling values, and outlier bounds on the training split only. Save that learned state as an inspectable pipeline.json. Validate schema and apply the frozen state to validation, test, or production batches. The reason I chose JSON instead of pickle is operational: revie

Read the full story at Reddit r/MLOps

More in MLOps / LLMOps