Work / Platforms & apps
∑
Learning analytics pipeline.
Self-hosted event ingestion and metrics for learning records, plus OAuth 2.0 single sign-on so client organizations authenticate straight into the platform.
- Stack
- Kafka, ClickHouse, OAuth 2.0
- Scale
- Per-learner event streams
- Role
- Designed and operated
Flow
- Learning events
- Kafka
- Stream processing
- ClickHouse
- Metrics
- Dashboards
How it works
Every learner interaction is published as an event to Kafka. Processors aggregate progress, time on task, and completion into metrics stored in ClickHouse, which handles the column-oriented queries reporting needs.
The stack is self-hosted, which kept per-event cost near zero and learner data inside the platform's own boundary.
Hard parts
- Backfilling years of historical records into a new schema without stalling live ingestion.
- Right-sizing ClickHouse for bursty, term-start traffic.
Outcome
Client-facing reporting that used to be manual became a live dashboard.