Looking for a job

Surajkumar Sikchi

@surajkumarsikchi

Message

Data engineering and backend engineer building scalable AWS metadata and search platforms.

India

Message

What I'm looking for

I’m looking for a role where I can build scalable data platforms on AWS—owning Spark ETL, search indexing, and observability—while tackling distributed systems problems and delivering reliable, high-performance backend engineering.

I’m a Software Engineer focused on Data Engineering, building large-scale metadata enrichment, search, and observability platforms on AWS. I designed Apache Spark pipelines processing 500K+ media records, improved metadata match rates from 32% to 73%, and reduced OpenSearch indexing time from ∼83 hours to 35 minutes.

In my internship, I co-designed a high-throughput Logstash seeding architecture and architected a Data Quality Framework using broadcast joins, anti-joins, and fuzzy matching. I engineered OpenSearch ingest pipelines, enrich policies, and automated lookup workflows, and built Grafana dashboards and alerting to improve production visibility, reliability, and scalability. I also led Spark ETL pipeline development and production support, including Delta processing architectures and an upstream data source migration from Trino to MySQL.

Experience

Work history, roles, and key accomplishments

Software Engineer Intern

Nielsen

Jan 2026 - Jun 2026 (5 months)

Co-designed a high-throughput Logstash seeding architecture that reduced OpenSearch indexing time from ~83 hours to 35 minutes and improved metadata match accuracy from 32% to 73%. Built Apache Spark metadata enrichment workflows on AWS (EMR on EKS), implemented OpenSearch ingest/enrichment automation, and added Grafana-based data quality and system observability.