At Servicepipe, I designed and built a lakehouse data platform from scratch for 130 TB of web traffic events per month. It brings ingestion, transformation, and serving together for analytics, BI, and machine learning workloads.
I redesigned the ingestion pipeline as Kafka and Flink streaming jobs writing to Iceberg, cutting end-to-end data latency from 5 minutes to 15 seconds. I also migrated Iceberg object storage from MinIO to SeaweedFS.
Earlier at Servicepipe, I designed a ClickHouse service that lets security analysts query and replay historical attack patterns. I built dbt models and Airflow DAGs for analytics-ready datasets, along with traffic dashboards used by security and product teams.
I also developed Python tooling and human-in-the-loop data aggregation services for analysts. In a separate project, I built Vault-MoreTech, a secrets management service that won the More.Tech hackathon.
