I am a highly skilled and results-oriented Azure Data Engineer with 3.1 years of experience in designing, implementing, and optimizing data solutions on the Microsoft Azure platform. Throughout my career, I have been adept at developing scalable data pipelines, integrating data from various sources, and ensuring data quality and security. I am passionate about leveraging data engineering and analytics to drive business insights and data-driven decision-making.
In my most recent role at Sublime Information Technologies, I applied a variety of transformations, including aggregation and analytical transformations, to process and analyze data effectively. I worked with various Azure services such as Azure Data Lake, Azure Data Factory, Azure Functions, Azure Key-Vault, Azure Databricks, and Azure SQL DB. I successfully built data pipelines in Azure Data Factory to load data into ADLS and utilized Azure Synapse for Data Warehousing. Additionally, I have experience with Snowflake for Data Warehousing, integrating it with PySpark.
Prior to that, at DMI India, I leveraged Azure Data Bricks and Spark for data extraction, transformation, and aggregation from multiple file formats. I implemented scalable and efficient data pipelines using Azure Data Factory and Azure Databricks to seamlessly ingest and integrate diverse data sources into Azure SQL Database and Azure Data Lake Storage. I also optimized SQL queries and data transformation tasks, resulting in a significant reduction in overall data processing time and enhancing data pipeline efficiency.
