Data Platform Engineer (CPT) - Datafin IT Recruitment

ENVIRONMENT:

A renowned Independent Asset Management Firm seeks a Data Platform Engineer in a hands-on Engineering role with a blend of architectural design, DevOps and governance responsibilities across Azure, Databricks, and on-premises environments. You will play a pivotal role in building, maintaining, and evolving the firm’s Enterprise Data Platform from existing hybrid on-prem SQL and Azure Cloud technologies, into a fully Azure Cloud and Databricks architecture. You will ensure a secure, scalable, and performant data platform infrastructure — enabling Analytics, Data Science, and Enterprise Data Integration across the business. The ideal candidate will require a BSc Computer Science or equivalent, Microsoft AZ and/or DP Certifications, Databricks Data Platform, Data Engineer, Platform Architect Certifications and 7-10 years technology experience with proficiency in Azure Cloud Services, Bicep, Terraform, SQL Server, Python & FinOps awareness and cloud cost optimization experience.

DUTIES:

Platform Engineering & Management -

• Build, configure, and manage Azure infrastructure components supporting the Core Data Platform.
• Develop and maintain Infrastructure-as-Code using Bicep and Terraform.
• Manage and maintain the Azure Databricks environments, including workspace creation, access management, and environment configuration.
• Oversee Azure DevOps and GitHub pipelines for CI/CD of data infrastructure and integration components.
• Implement FinOps practices to optimize cloud cost management and forecasting.

Database & Integration Management -

• Support on-premises SQL Server environments (3 Line), including performance troubleshooting, tuning, and high-availability configurations.
• Collaborate with Data Integration teams to ensure smooth data movement between on-premises and cloud environments.
• Oversee and maintain secure, high-performance data pipelines and connectivity between systems.

Security, Governance & Compliance -

• Design and enforce role-based access controls (RBAC) across the Databricks platform and Core Data subscriptions.
• Ensure compliance with internal security policies, regulatory standards, and data governance frameworks.
• Implement and maintain platform monitoring, logging, and alerting for proactive issue resolution.

Architecture & Standards -

• Contribute to architectural design and technical standards for Azure and Databricks solutions.
• Evaluate and recommend improvements to infrastructure patterns and automation processes.
• Ensure platform architecture aligns with best practices, scalability, and resilience requirements.

Collaboration & Continuous Improvement -

• Partner with Data Engineering, Architecture, and Security teams to deliver end-to-end solutions.
• Mentor junior engineers and promote knowledge sharing within the Core Data team
• Identify automation opportunities to improve efficiency and platform reliability.

REQUIREMENTS:

Qualifications –

• BSc Computer Science or equivalent.
• Microsoft AZ and/or DP Certifications.
• Databricks Data Platform, Data Engineer, Platform Architect Certifications.

Experience/Skills -

• 7-10 Years technology experience, with at least 3 years in the cloud platform/data platform roles.
• Azure Cloud Services (compute, networking, storage, security, cost management).
• Infrastructure as Code: Bicep and Terraform.
• Azure Databricks management (workspace setup, security, clusters, policies).
• Azure DevOps and/or GitHub Actions pipeline management.
• Python scripting for automation and tooling.
• SQL Server administration and performance tuning (3rd-line level).
• FinOps awareness and cloud cost optimization experience.

Advantageous –

• Experience with Azure Monitor, Log Analytics, and Application Insights.
• Knowledge of data governance tools.
• Familiarity with CI/CD principles for data pipelines.
• Understanding of data integration frameworks and patterns (ETL/ELT).
• Exposure to enterprise identity management (Entra ID).

ATTRIBUTES:

• The ability to ‘approach and own’ and continuously looks for opportunities to develop.
• High conviction and be comfortable sharing opinions.
• The ability to build and maintain meaningful relationships.
• A client-focused and collaborative approach.
• A curiosity about technology and its potential to drive innovation.