Executive Summary
Result-oriented Senior AI Cloud Data Engineer with over 10 years of software industry experience, including 9+ years architecting and scaling enterprise Big Data & Cloud pipelines across Azure and GCP platforms. Specialized in PySpark, Azure Databricks, GCP BigQuery, Cloud Composer (Airflow), and Delta Lake. Demonstrated track record in Aviation, Retail, Mortgage Insurance, and Healthcare domains delivering high-throughput batch/real-time ingestion frameworks, data warehouse optimization, and AI workflow automations.
Core Competencies
Languages & Frameworks
Python, PySpark, SQL, Scala, Spark SQL
Cloud Platforms & Services
Azure (ADF, Databricks), GCP (BigQuery, Dataproc, Composer)
Storage & Data Lakehouse
Delta Lake, HDFS, Google Cloud Storage, Azure Data Lake (ADLS Gen2)
Orchestration & AI
Airflow (Cloud Composer), Azure Data Factory, n8n AI Automations
Professional Experience
EMAKINA (EPAM Systems), Doha, Qatar
Project: Duty Free Business Intelligence | Client: Leading Aviation in Qatar
- Architected scalable ingestion framework transferring high-volume transaction data from on-premise sources to GCP BigQuery.
- Developed PySpark transformations producing curated datasets for predicting duty-free retail sales trends and flight catering food waste reduction.
- Orchestrated end-to-end data workflows using Cloud Composer (Airflow) DAGs with automated error retries and SLA alerts.
- Optimized BigQuery query execution using custom partitioning and clustering strategies, reducing query cost by 40% and pipeline latency by 50%.
EPAM Systems, Chennai, India
Project: Aviation Inventory Analytics | Client: Leading Aviation in Qatar
- Designed Azure Databricks ingestion pipelines for flight booking predictions and flight operations (Flight 360) analytics.
- Applied Delta Lake table optimizations (compaction, Z-Ordering) to eliminate small file bottlenecking and improve query speed by 45%.
- Integrated multi-source datasets using optimized PySpark joins, windowing functions, and caching strategies.
EPAM Systems, Chennai, India
Project: Data Factory for CTC | Client: Retail & Distribution
- Engineered ADF pipelines extracting data from RDBMS/file systems into Azure Databricks Delta Lake.
- Implemented Spark optimizations including broadcast joins, salting for skewed keys, and partition coalesce.
Accenture, Chennai, India
Project: Home Buyer Transformation | Client: NA Mortgage Insurance
- Constructed PySpark pipelines in Databricks for mortgage risk assessment analytics and automated decision-making.
- Utilized Azure Data Factory (ADF) to ingest data from heterogeneous sources into Azure Data Lake.
Tata Consultancy Services (TCS), Chennai, India
Project: Healthcare Data Modernization | Client: US Healthcare
- Re-engineered legacy COBOL SQL queries into optimized PySpark & Hive transformations on HDFS.
- Designed partitioned & bucketed Hive tables for historical claims analysis and AWS Redshift export.
Certifications
Databricks Certified Associate Developer for Apache Spark
Issued: April 2025 • Active till March 2027
Microsoft Certified: Azure Data Fundamentals
Microsoft Certified Professional
Google Cloud Skill Badges
Cloud Security, Foundational Data & AI, Looker Models
Education
Bachelor of Engineering (B.E.)
Electrical and Electronics Engineering
National Engineering College, Anna University (2012 – 2016)
CGPA: 9.13 / 10.0