Farman Ahmed

Senior Data Platform Engineer

PayPal

Improved payment-domain data availability by 40% by engineering Bronze–Silver–Gold lakehouse pipelines using Azure Databricks, Delta Lake, Auto Loader, DLT, Kafka, and Spark Structured Streaming. Reduced Spark processing costs by 22% and critical-job runtime by 35% through execution-plan optimization, partitioning, caching, and Delta Lake file optimization.

Azure DatabricksDelta LakeAuto LoaderDLTKafkaSpark Structured StreamingPySparkCI CDData QualityData LineageRBACObservability

Senior Data Platform Engineer

WPP

Reduced new-source onboarding time by 35% by designing parameterized ingestion pipelines using Azure Data Factory, Azure Databricks, and PySpark for 15 enterprise systems. Improved cross-system data reconciliation accuracy to 99% through automated validation, exception handling, and consistency checks across ADLS Gen2, Azure SQL, and reporting layers.

Azure Data FactoryAzure DatabricksPySparkADLS Gen2Azure SQLPower BIData ReconciliationData ValidationSpark Optimization

Senior Data Engineer

Amazon

Automated ingestion of 60 enterprise datasets using AWS S3, Glue, EMR, Redshift, Athena, and Step Functions, reducing manual processing effort by 45%. Built governed and versioned datasets supporting 12 ML feature pipelines using SageMaker Feature Store and AWS data lake services.

S3AWS GlueAmazon EMRRedshiftAthenaStep FunctionsSagemakerGitHub ActionsAWS CodePipelineTerraformAWS CDKCloudwatchIAMKMSMLFlow

Data Engineer

Accelerated analytics dataset delivery by 35% and reduced manual pipeline interventions by 40% by developing Snowflake, dbt, Matillion, and automated orchestration workflows. Improved reporting query performance by 30% through SQL tuning, Databricks optimization, and dimensional modeling; expanded governed catalog coverage to 100 datasets.

SnowflakeDBTMatillionSQLDatabricksGCPDataflowDataprocApache SparkKafkaData QualityCloud Monitoring

Associate Big Data Engineer

Accelerated legacy ETL execution by 40% by migrating transformations to Apache Spark on Cloudera/Hadoop and optimizing Apache Hive workloads. Reduced data freshness lag by 50% through CDC and incremental ingestion pipelines using Apache NiFi, Kafka, and reusable processing logic.

Apache SparkClouderaHadoopApache HiveApache NiFiKafkaCDCData LineageRBACGovernance