NHNiyamul Hoque
@niyamulhoque
Big Data Engineer with expertise in data processing and analytics.
IndiaWhat I'm looking for
I am a Big Data Engineer with nearly three years of experience specializing in batch and streaming data processing using technologies like Spark, Python, Kafka, and AWS. My journey has been marked by a commitment to excellence, earning multiple Impact awards for my contributions to high-impact projects across various industries.
At Emids Technologies, I was promoted from Junior Associate Developer to Software Engineer within three years, where I played a pivotal role in developing the CoreLake product. My achievements include consolidating data from over ten sources, resulting in a 25% increase in retrieval speed and a 15% reduction in storage costs. I have a proven track record of enhancing workflow performance by up to 50%, demonstrating my ability to deliver results that matter.
My technical expertise spans various big data frameworks and cloud services, including AWS, Azure, and Google Cloud. I am passionate about leveraging data to drive decision-making and improve operational efficiency, and I continuously seek opportunities to innovate and optimize data processing workflows.
Experience
Work history, roles, and key accomplishments
Software Engineer
Emids Technologies
Mar 2022 - Present (3 years 4 months)
Accelerated team learning of real-time data processing by architecting a Kafka-based proof-of-concept, which informed key architectural decisions. Developed a key POC feature for an internal proprietary tool that ingests mock health data from an API endpoint source, using MSK (Kafka) coupled with PySpark and Python, and then lands streaming data into target systems.
Software Engineer
Emids Technologies
Mar 2022 - Present (3 years 4 months)
Piloted the end-to-end integration of Snowflake as both a source and target within a proprietary ETL tool, specifically tailored for the Elevance Health project, completed within 6 months. Spearheaded the development of Snowflake connectors as both source and target in CoreLake, extending the platform
’s integration flexibility.
Software Engineer
Emids Technologies
Mar 2022 - Present (3 years 4 months)
Implemented end-to-end data pipelines with Airflow and Spark Scala, ensuring seamless data flow from source systems to target databases, which reduced data latency by 6 hours using optimized methods. Engineered and implemented new features in our Corelake platform to further enhance ingestion capability from multiple sources, processing data through efficient data quality checks and validation, an
Software Engineer
Emids Technologies
Mar 2022 - Present (3 years 4 months)
Revamped proprietary ETL tool 'CoreLake' utilizing Spark, Python, and AWS, accelerating data ingestion speeds by 60% and reducing data processing time by 45% across all pipelines. Consolidated data from multiple sources and stored it efficiently in Data lakes, resulting in a 25% increase in data retrieval speed and a 15% reduction in storage costs.
Education
Degrees, certifications, and relevant coursework
STSRM Institute of Science and Technology
Bachelor of Technology, Electronics and Communication Engineering
Grade: 8.1 CGPA
Completed a Bachelor of Technology in Electronics and Communication Engineering. Focused on core concepts and practical applications within the field.
Tech stack
Software and tools used professionally
Availability
Location
IndiaAuthorized to work in
Salary expectations
Job categories
Interested in hiring Niyamul?
You can contact Niyamul and 90k+ other talented remote workers on Himalayas.
Message NiyamulFind your dream job
Sign up now and join over 100,000 remote workers who receive personalized job alerts, curated job matches, and more for free!
