Results-driven Data Engineer with 5+ years of experience in designing,
developing, and optimizing large-scale data pipelines and cloud data platforms on
Microsoft Azure. Proven expertise in building scalable ETL/ELT workflows using
Azure Data Factory, transforming big data with Databricks (PySpark), and
delivering insights through Power BI dashboards and Azure Synapse Analytics.
Strong communicator with hands-on experience in automation, data modeling,
and implementing best practices in cloud data engineering.
Professional Summary:
Designed and deployed 10+ scalable ETL pipelines using Azure Data Factory,
integrating data from both cloud and on-premise sources into Azure Data Lake,
improving data availability by 40%.
Proficient in SQL, PySpark, and cloud-native tools such as Azure Data
Factory, Azure SQL Database, Azure Synapse Analytics, and Azure
Databricks.
Hands on experience on Creating ADF Pipeline for Full Load and Incremental Load.
Developed generalized and parameterized Azure Data Factory Pipelines,
Activities, Data Sets, and Linked Services.
Developed Azure Data Factory Pipelines for executing the Data bricks
Notebooks scripts to perform Extract, Transform (Apply business logic) and Load
on files in Data Lake, creating views and custom transformations using python
(PySpark, Spark Sql) languages.
Extensively used Joins and sub-Queries to simplify complex queries involving
multiple tables. Involved in extracting the data from different data sources like
SQL Server, SAP and CSV files, etc.
Experienced in working in Agile and DevOps environments, utilizing Azure
DevOps for CI/CD pipelines to automate deployments and improve software
delivery speed and reliability.
Expertise in developing end-to-end ETL workflows and automating data
processing, enabling real-time and batch processing for business-critical
applications.
Proven track record of collaborating with cross-functional teams, including
data scientists, business analysts, and stakeholders, to deliver data-driven
solutions that drive decision- making.
Skill Summary:
Category Skills
Cloud Platform Azure (ADF, Databricks, Synapse Analytics, Azure SQL DB, ADLS
Gen2, Logic Apps)
Languages Python ,PySpark, SQL
&
Framework
s
Tools & BI Power BI, Git, Azure DevOps, JIRA, Confluence
Methodologies Agile/Scrum, CI/CD, DevOps
Experience:
Capgemini, Hyderabad, India(July 2021 to present)
Developed and executed comprehensive ETL pipelines utilizing Azure Data
Factory to seamlessly integrated both structured and unstructured data from on-
premises and cloud- based sources into Azure Data Lake.
Automated CI/CD pipelines for Azure Databricks deployment using GitHub
Actions, improving release cycles and reducing deployment time by 40%.
Operated within an Aglie framework utilizing Jira to manage tasks, facilitate
sprint planning, and conduct performance retrospective, there by enhancing team
productivity.
Leveraged ServiceNow for incidents and change management, ensuing
compliance with SLAs ad streamlining documentation process.
Administrated metadata and data lineage management through Unity Catalog,
facilitating enhanced data discovery, traceability and governance.
Working as a team lead with 15+ resources tagged under me. Guiding and
training my team technically.
Maintained close collaboration with business stakeholders, enhancing
communication and delivering solutions aligned with business KPIs.
Implemented data refresh and job scheduling through ADF triggers, significantly
improving pipeline reliability and cutting down manual effort by 70%.
Managed large data sets using Azure Synapse Analytics and SQL Server,
optimizing queries for multi-terabyte environments.
Wipro, Hyderabad, India(Feb 2018 to July 2019)
Joined as a L2 support resource and helped address user concerns on priority.
Developed monitored and alerting system utilizing Azure Monitor and Log
Analytics to ensure optimal ETL performance and uptime.
Designed and managed ETL processes using Azure Data Factory, automating
the extraction, transformation, and loading of data from legacy systems to cloud-
based data platforms.
Established data quality checks and validation processes with Azure Data
Factory to ensure data integrity and accuracy.
Spearheaded internal training sessions on Azure Data Services, Spark, and
Power BI, accelerating team upskilling and improving productivity .
Developed and deployed a sample ETL pipeline using Azure Data Factory,
loading JSON and CSV files from Blob Storage into Azure SQL Database for
querying.
