Six sources, one warehouse
iCal feeds, Google Calendar, Zotero, BibTeX/RIS files, local markdown notes, and OpenAlex enrichment - all ingested idempotently into a bronze/silver/gold Postgres layout managed by dbt.
Portfolio capstone - Data Engineering
Treat student life as a data product.
StudentOps ingests calendar feeds, Zotero, markdown notes, and academic references through a layered Postgres warehouse, then surfaces what is due, what is planned, and what is worth reading - with an AI assistant that refuses to invent citations.
Built with Python, dbt, Prefect, Streamlit, Astro. Deployed on Fly.io and Cloudflare Pages. Costs about AUD 12 a month to run live.
What it does
Honest AI
Reading summariser, Anki flashcard generator, citation suggester. Every DOI the suggester proposes is validated against OpenAlex and CrossRef; unresolved DOIs are dropped silently. No hallucinated references reach the user.
Observable by default
Every flow opens a row in meta.flow_runs. The Pipelines page shows freshness per source, success rate over the last fifty runs, and today's AI spend.
How it works
Stack
- Python 3.11+ ingestion, transforms, AI
- PostgreSQL 16 warehouse (Supabase)
- dbt-core SQL transforms + SCD2 snapshots
- Prefect 3 orchestration
- Streamlit dashboard
- Astro 5 marketing + docs site
- Tailwind CSS styling
- Anthropic Claude AI features
- Resend transactional email
- Cloudflare Pages marketing host
- Fly.io app + worker host
- GitHub Actions CI/CD