Anirudh Varma Penumetcha

Sr. Data Engineer

Cognizant Technology Solutions

Led modernization of enterprise batch applications and Python/shell scripts to containerized workloads on AWS EKS and Rancher-managed Kubernetes. Re-engineered Oracle-to-AWS Glue/S3-to-Snowflake pipelines processing approximately 80 billion rows, reducing batch window from 6 hours to 3 hours.

AWSAmazon EKSKubernetesRancherPythonPySparkAWS GlueSnowflakeOracleStonebranch UACGitHub ActionsHarborHarnessHelmDockerEFSFluent BitSplunkInformatica PowerCenterSSHSFTP

Sr. Data Engineer

Designed serverless NLP pipelines processing approximately 200,000 documents per run, reducing processing time from 4 hours to 1 hour. Optimized distributed PySpark and Spark DataFrame workloads by 85% and accelerated EMR/HDFS-to-Snowflake processing by 35%.

NLPPySparkSparkSnowflakeAirflowEMRHDFSData QualityAnomaly DetectionGovernance

Sr. Data Engineer

Built end-to-end ETL/ELT pipelines using AWS Glue, PySpark, Azure Data Factory, S3, and Snowflake. Developed Airflow DAGs and Spark Streaming modules with Kafka to automate dependencies and process high-volume event and batch data.

AWS GluePySparkAzure Data FactoryS3SnowflakeAirflowKafkaFlaskAPI GatewayLambdaEC2EMR

Data Engineer

Designed BigQuery pipelines, geospatial datasets, and reusable spatial functions that improved data retrieval, analysis, and query performance. Implemented data-quality and governance practices and resolved geospatial processing issues.

BigQueryGeospatialData QualityGovernancePerformance Optimization

Data Engineer

Developed Databricks, PySpark, Spark SQL, Hive, Azure Data Factory, SSIS, and Snowflake pipelines to migrate and transform data from legacy and on-premises platforms. Built Azure and Power BI reporting workflows with incremental refresh and automated CI/CD.

DatabricksPySparkSpark SQLHiveAzure Data FactorySSISSnowflakePower BIGitJenkinsPythonBASHMySQL