Senior Data Engineer / Databricks Expert

Stermedia20K PLN – 30K PLN per monthWorldwide

Apply now

Opens the original posting on Himalayas.

Apply now Source: Himalayas

Apply on the original site

Companies in this catalog do not charge applicants. Never send money to apply.

Stermedia .ai is looking for an experienced Senior Data Engineer with strong Databricks expertise to join a data platform team building a global analytics environment for a leading pharmaceutical organization.

The platform integrates data from more than 80 facilities worldwide , including laboratories, manufacturing systems, logistics platforms, and operational environments. Its goal is to transform distributed datasets into reliable, analytics-ready data that supports reporting, operational monitoring, and data-driven decision-making.

You’ll contribute to the full delivery lifecycle — from direct client discussions and business requirements analysis to solution design, implementation, and production deployment. Working alongside international data engineering, analytics, and machine learning teams, you’ll focus on building scalable data pipelines and transformation layers using Databricks, Apache Spark, Python, SQL, and Delta Lake .

Loading job…

Senior Data Engineer / Databricks Expert at Stermedia · ApplyPick

Design, develop, and maintain data pipelines in Databricks using Python, PySpark, and SQL
Build and maintain Delta Lake tables and transformation layers following bronze, silver, and gold architecture
Integrate data from multiple source systems into consistent, analytics-ready datasets
Implement incremental processing, change data capture, and schema evolution where required
Develop and orchestrate production workflows, including dependencies, scheduling, retries, and monitoring
Optimize Spark workloads, SQL queries, and compute usage to improve performance and manage costs
Implement automated data quality checks, validation, and pipeline tests
Support data access management, governance, and lineage using Unity Catalog
Maintain reusable code, technical documentation, and Git-based development workflows
Collaborate with business stakeholders, data engineers, analysts, and machine learning specialists to translate requirements into reliable data solutions
Required skills
Strong commercial experience with Databricks , including developing and operating production data pipelines
Advanced knowledge of Python, PySpark, and SQL
Hands-on experience with Apache Spark , distributed data processing, and performance troubleshooting
Practical experience with Delta Lake , including incremental loads, merge operations, and schema management
Strong understanding of lakehouse architecture , ETL/ELT patterns , and data modeling
Experience orchestrating and monitoring workflows in Databricks
Practical knowledge of Unity Catalog , including permissions, data organization, and lineage
Experience optimizing pipeline performance and compute resource usage
Familiarity with cloud storage and services in at least one major cloud environment: Azure, AWS, or GCP
Experience with Git, code reviews, automated testing, and CI/CD workflows
Strong analytical and communication skills, with the ability to work directly with international stakeholders
Good command of English
Nice to have
Experience with DBT , including models, tests, macros, and integration with Databricks
Experience with Structured Streaming and Databricks Auto Loader

Senior Data Engineer / Databricks Expert

Apply now

Senior Data Engineer / Databricks Expert

Apply now

About the project

Requirements

Familiarity with Terraform and infrastructure as code

The possibility of remote work

Apply now