ARAbdul Rafey
@abdulrafey
Senior Data Engineer specializing in Python and data integration.
United StatesWhat I'm looking for
I am a Senior Data Engineer with extensive experience in transforming legacy data pipelines and enhancing data processing efficiency. Currently, I lead the migration of data pipelines from R to Python at edX, utilizing Amazon ECS and Apache Airflow for orchestration. My work has significantly improved the reliability of our data processes and ensured seamless integration with various analytics tools.
Previously, I served as a Senior Software Engineer at Arbisoft Ltd, where I orchestrated data extraction from numerous e-commerce websites using advanced web scraping techniques. I also led a development team on full-stack projects, implementing innovative solutions that saved costs and improved operational efficiency. My passion for mentoring and training new team members has fostered a collaborative and productive work environment.
Experience
Work history, roles, and key accomplishments
Senior Data Engineer
edx | 2U
Jan 2023 - Present (2 years 4 months)
Led the migration of legacy data pipelines from R to Python, utilizing Amazon ECS for containerized deployment and Apache Airflow for orchestration. Developed ETL pipelines, upgraded dbt versions, and ensured seamless data integration from Salesforce and Braze to the data warehouse.
Senior Data Engineer
edX | 2U
Jan 2023 - Present (2 years 4 months)
Led the migration of legacy data pipelines from R to Python, utilizing Amazon ECS for containerized deployment and Apache Airflow for orchestration. Developed ETL pipelines, upgraded dbt versions, and migrated data from Salesforce and Braze to the data warehouse, ensuring seamless integration and reliability.
Senior Software Engineer
Arbisoft Ltd
Jan 2021 - Present (4 years 4 months)
Orchestrated data extraction from e-commerce websites using frameworks like Scrapy and Selenium. Led a development team on full-stack projects and created a Chrome extension that streamlined processes, saving significant time and costs.
Senior Software Engineer
Arbisoft Ltd
Sep 2021 - Present (3 years 8 months)
Orchestrated data extraction from e-commerce websites using frameworks like Scrapy and Selenium. Led a development team on full-stack projects, developed a custom middleware for user agent rotation, and created a Chrome extension to streamline processes.
Associate Software Engineer
Nextbridge Ltd (vteams)
May 2019 - Sep 2021 (2 years 4 months)
Led a team in creating full-stack web applications using PHP and Laravel. Developed a custom e-commerce store and an in-house recruitment solution, significantly reducing costs and generating revenue.
Associate Software Engineer
Nextbridge Ltd (vteams)
May 2019 - Jan 2021 (1 year 8 months)
Led a team in creating full-stack web applications using PHP and Laravel. Developed a custom e-commerce store and an in-house recruitment solution, significantly reducing costs and generating substantial revenue.
Education
Degrees, certifications, and relevant coursework
CICOMSATS University Islamabad
Bachelor of Computer Science (Hons), Computer Science
2016 - 2020
Completed a Bachelor of Computer Science (Hons) program, focusing on various aspects of computer science including programming, data structures, and software development methodologies.
Availability
Location
United StatesAuthorized to work in
Job categories
Skills
Interested in hiring Abdul?
You can contact Abdul and 50k+ other talented remote workers on Himalayas.
Message AbdulFind your dream job
Sign up now and join over 85,000 remote workers who receive personalized job alerts, curated job matches, and more for free!
