Hello, Welcome !!

I am a Data Engineer with experience in designing and implementing end-to-end data pipelines for both real-time streaming and batch processing systems, leveraging scalable cloud-based data platforms. I specialize in using Python, SQL, GCP, and AWS to deliver efficient and impactful data solutions that drive informed business decisions.

Projects

Streaming Data Pipeline: A project focused on real-time data processing using Kafka and Databricks.

Dataflow Data Pipeline: Capture, process, and visualize COVID-19 data in both real-time and batch modes, offering insights into the evolving pandemic landscape.

Streaming Data Pipeline with AWS Glue: The pipeline efficiently captures, processes, and visualizes streaming data for robust business analytics.

Work Experience

Data Engineer | Nov-23 - Present
Implemented end-to-end streaming pipelines between client APIs and Databricks.
Utilized AWS EC2 instances to extract real-time data using Python.
Leveraged Kafka streams and Apache Spark for ETL processes.
Transformed data using DBT for integration with Salesforce.

Data Engineer | Aug-21 - Nov-23
Implemented ETL processes for British Telecom using PySpark.
Utilized Big Data ETL tools including DataProc, DataFlow, Spark, Airflow, and Pub/Sub.
Developed BigQuery transformations using DBT.
Led migration of data from Oracle to GCP.