House Price MLOps Pipeline
Validated data pipeline and model API with Airflow, Great Expectations and alerting.
Architecture
How it works
- New data files
- Airflow ingestion DAG
- Great Expectations checkpoint
- PostgreSQL + run stats
- FastAPI model service
- Streamlit app
Feature contract
A shared feature module defines every input field and its valid range, so training, the serving API and the Airflow DAGs build features the same way. That keeps training and serving in sync.
Ingest and validate
An Airflow DAG picks up new data and runs it through a Great Expectations checkpoint. Passing and failing rows are split, and run statistics are written to PostgreSQL.
Alert
When a checkpoint fails, the pipeline posts an alert to Microsoft Teams, so bad data is caught before it reaches the model.
Serve
A FastAPI service loads the model and answers predictions; a scheduled prediction DAG scores new data. The Streamlit app only talks to the API, never to the model or database directly.
Run anywhere
Every service runs from one Docker Compose file, so the whole stack starts with a single command.
Information
A model is only as reliable as the data reaching it. This project wraps a house-price regression model in the production pieces around it: data validation, alerting, a serving API and an app.
EPITA · Data Science in Production
- Airflow
- Great Expectations
- FastAPI
- PostgreSQL
- Streamlit
- Docker Compose