Hiring the leaders and builders who turn Databricks into a real enterprise data platform

A global manufacturer chose Azure Databricks to pull together scattered data and support finance, supply chain, sales and manufacturing analytics. Its systems included SAP, other business apps, plant systems and data from outside the company.

The plan was to hire a few data engineers. The real gap was bigger. Nobody owned the platform architecture, the data products, governance, how data came in, or how any of it ran in production.

"Data Engineer" was too broad to be useful here. Some candidates had strong SQL and pipeline skills but little Spark or Databricks depth. Others knew Databricks well but had never touched SAP, plant data or anything near real time. A third group could build pipelines but had never shaped data products the business would adopt.

The company needed a balanced team that could build the platform and tie it to results people cared about.

We worked backward from the first priorities: ERP reporting, supply chain visibility and plant performance. Those outcomes told us the order to hire in, from leadership to architecture to builders.

We also separated the must have Databricks skills from the ones that could transfer. Direct Unity Catalog, Spark, Delta Lake and production pipeline experience mattered for the platform architect and senior engineers. Strong SQL, Python, modeling and CI/CD could carry over into some of the builder roles.

Leadership:

• Head of Data Platform
• Owns platform strategy, the roadmap, team design, data product delivery, governance and partner management

Architects:

• Databricks Lakehouse Architect
• Enterprise Data Architect
• Data Security and Governance Architect

Builders:

• Databricks Data Engineers
• Spark and Streaming Engineers
• Platform Engineers
• Analytics Engineers
• BI and Semantic Model Developers
• Data Quality and Observability Engineers

Domain and adoption:

• Data Product Owners
• Data Governance Lead
• Finance, supply chain and manufacturing data stewards

Evidence that they could show:

• Production Databricks or a comparable lakehouse they had shipped
• SQL, Python, PySpark, Delta Lake and orchestration
• ERP, API, batch, CDC or streaming integration
• Data modeling across Bronze, Silver and Gold layers
• Unity Catalog, lineage, access control and data quality
• CI/CD, automated testing, observability and cost control
• One clear case of a pipeline turning into a business product people used

We did not treat every Databricks resume as the same. We sorted candidates into platform builders, pipeline engineers, analytics engineers and data product leaders. In interviews we asked what each person had personally designed, built and run.

We also checked whether they could work across SAP teams, cloud infrastructure, security and the business, not just inside a central data group.

Our advisors vetted both halves of the team. Senior data platform leaders checked the leadership candidates on roadmap, governance and data product thinking. Working Databricks architects and engineers tested the builders on Spark, Delta Lake, Unity Catalog and real production pipelines. We found much of the talent through our data engineering and Databricks community, where people are known for what they have actually shipped.

A hiring plan in sequence, tied to the data products that mattered most. Roles were clearer, architects and engineers stopped overlapping, and the company had a better base for judging systems integrator help.

Instead of a project team that would scatter after the migration, the manufacturer owned its platform and had repeatable ways of working. It could add plants, domains and AI use cases without rebuilding the whole setup each time.