CasaMotion real-time mobility data platform
Personal project, data engineering
Streaming platform for Casablanca mobility data: Kafka ingestion, PyFlink and Spark processing, a Bronze-Silver-Gold warehouse on PostgreSQL/PostGIS and live dashboards.
CasaMotion is a real-time data platform for Casablanca mobility data. Events are ingested through Kafka, processed with PyFlink for streaming and Spark for batch, and land in a Bronze-Silver-Gold warehouse on PostgreSQL with PostGIS for the geospatial layer. Airflow schedules the batch jobs and Grafana serves the live dashboards.
The project exists to practise the full data engineering chain on a realistic, messy dataset: late events, duplicates, schema drift and geospatial joins. Source code is public on GitHub.