ryan willson
@ryanwillson1
Senior Data Engineer and architect delivering scalable lakehouse platforms and real-time pipelines.
What I'm looking for
I am a Senior Data Engineer and Big Data Architect with 10+ years building real-time pipelines and lakehouse platforms across healthcare, finance, and retail. I specialize in reducing latency and cloud costs while ensuring compliance with HIPAA, GDPR, and SOX.
I've designed Kafka and Spark streaming at scale, built multi-zone lakehouses on AWS S3, and standardized onboarding to cut time-to-data by ~50%. I've led Fortune-level deployments, authored Python and SQL frameworks for cleansing and identity graphs, and tuned EMR/Glue/Redshift to lower ETL runtimes and cloud spend.
I deliver measurable business outcomes—cutting inventory discrepancies, shrinking batch windows, and improving match rates—while mentoring teams and implementing infrastructure-as-code and observability best practices.
Experience
Work history, roles, and key accomplishments
Software Engineer
SecureHavenCo
Feb 2025 - Aug 2025 (6 months)
Built a unified retail data model and automated 15 near-real-time pipelines (under 15-minute SLAs), cutting inventory discrepancies by 35% and reducing cloud costs by 20%. Delivered self-service dashboards that reduced prep time from two hours to ten minutes and shrank batch windows by 40%.
Software Engineer
Secure HavenCo
Feb 2025 - Aug 2025 (6 months)
Designed and maintained scalable ETL/ELT pipelines integrating inventory, orders, customers, and shipping; migrated legacy RDBMS and flat-file workflows into AWS analytics stacks to accelerate financial and billing reports and enable real-time sales insights.
Designed Kafka and Spark streaming processing 500M+ events/day, reducing latency ~45% and built a multi-zone lakehouse on S3 that halved time-to-data and improved match rates by 20%. Tuned EMR/Glue/Redshift to cut ETL runtimes and cloud spend by ~30% while ensuring compliance for Fortune-level customers.
Led enterprise RDBMS-to-PostgreSQL migrations and architected petabyte-scale real-time and batch data pipelines using Spark and AWS (Glue/EMR), reducing downstream data errors by 40% and cutting infrastructure costs and latency.
Implemented end-to-end tagging achieving >98% event capture and reduced defects 40%; built A/B testing pipelines that shortened analysis from days to hours and increased conversion by 7%. Modeled unified profiles in AEP to boost activation reach 20% while operationalizing GDPR consent and deletion workflows.
Lead Software Developer
Blue Medora
Feb 2017 - Apr 2019 (2 years 2 months)
Led two agile squads to deliver 10 cloud monitoring integrations in Kotlin/Java, doubling coverage and cutting time-to-market per integration by 50%. Delivered a serverless public API ingesting millions of metrics/day and drove CI/CD improvements that reduced incidents 30% and increased deployment cadence to weekly.
Delivered governance roadmaps and implemented lineage for 90% of critical datasets to meet HIPAA/GDPR/SOX; built InfoSphere/DataStage pipelines that shrank nightly batches 25% and led MDM consolidation of 20 sources to improve report accuracy and reduce duplicates.
Delivered enterprise data integration and governance solutions for healthcare and finance clients, designing ETL and metadata workflows to improve data reliability, lineage, and MDM practices.
Education
Degrees, certifications, and relevant coursework
University of Denver
Master of Science, Computer Science
Completed a Master of Science in Computer Science focusing on advanced topics in software and data engineering.
University of California, Berkeley
Bachelor of Science, Electrical Engineering & Computer Science
Earned a Bachelor of Science in Electrical Engineering & Computer Science with foundational training in software and systems engineering.
Tech stack
Software and tools used professionally
Availability
Location
Authorized to work in
Job categories
Skills
Interested in hiring ryan?
You can contact ryan and 90k+ other talented remote workers on Himalayas.
Message ryanFind your dream job
Sign up now and join over 100,000 remote workers who receive personalized job alerts, curated job matches, and more for free!
