Back to all jobs
U

Senior Databricks Consultant / Architect

unisongroup

Singapore · sg 3d ago

Job description

# Senior Databricks Consultant / Architect > Unison Group · Singapore, Singapore · — · Posted 2026-11-13 **Workplace:** on_site **Department:** Bharat ## Description ### Role Overview We are looking for an experienced Senior Databricks Consultant / Architect to lead the design, architecture, development, and implementation of enterprise-scale data engineering and analytics solutions using Azure Databricks. The ideal candidate should have strong expertise in Lakehouse architecture, PySpark, Delta Lake, cloud data platforms, and data governance. ### Key Responsibilities - Design and architect scalable, secure, and high-performance Lakehouse solutions using Databricks. - Lead end-to-end data engineering and data platform implementation. - Define enterprise data architecture, standards, best practices, and technical roadmaps. - Design and implement data pipelines using PySpark, Spark SQL, Delta Lake, and Databricks Workflows. - Develop and optimize ETL/ELT pipelines for batch and streaming data processing. - Implement Medallion Architecture (Bronze, Silver, Gold) and Delta Live Tables (DLT). - Work extensively with Unity Catalog for data governance, security, access control, and lineage. - Integrate Databricks with Azure services such as ADLS Gen2, Azure Data Factory, Event Hubs, and Azure Key Vault. - Optimize Spark workloads, cluster configurations, SQL queries, and overall platform performance. - Implement CI/CD pipelines, automated deployments, and DevOps practices. - Provide technical leadership, conduct design reviews, and mentor data engineering teams. - Collaborate with business stakeholders, architects, and engineering teams to translate business requirements into technical solutions. ### Mandatory Technical Skills - Strong hands-on experience with **Azure Databricks and Apache Spark**. - Expertise in PySpark, Spark SQL, Python, and SQL. - Strong knowledge of Delta Lake and Lakehouse architecture. - Experience implementing Medallion Architecture. - Expertise in Unity Catalog, Delta Live Tables (DLT), and Auto Loader. - Strong understanding of ETL/ELT design and data pipeline development. - Experience with Azure Data Lake Storage (ADLS Gen2) and Azure Data Factory. - Knowledge of Databricks Workflows, cluster management, and performance tuning. - Experience with data security, governance, and access management. - Hands-on experience with Git, Azure DevOps, and CI/CD. ### Good to Have - Experience with AWS Databricks or GCP Databricks. - Knowledge of Kafka, Event Hubs, and real-time data streaming. - Experience with Databricks SQL and Lakehouse Monitoring. - Knowledge of Terraform and Infrastructure as Code. - Experience with MLflow and the Databricks Machine Learning ecosystem. - Databricks Professional-level certifications. ### Candidate Profile - Strong experience in enterprise data architecture and large-scale data engineering implementations. - Ability to independently drive architecture decisions and technical delivery. - Excellent communication, stakeholder management, and problem-solving skills. - Ability to work with cross-functional teams and provide technical direction. ## Apply [Apply at Unison Group](https://apply.workable.com/unisongroup/j/7B51F64936/apply) --- Powered by [Workable](https://www.workable.com)

Similar open jobs