Azure DevOps Engineer (m/f/d)
Halian View all jobs
- Abu Dhabi
- Contract
- Full-time
- Databricks Asset Bundles (DABs): Enhance our DABs processes to enhance developer experience and velocity.
- CI/CD Pipeline Development: Build end-to-end automation in
- Governance: Mature and manage security policies and access controls
- ADF Management: Manage our ADF resources, optimizing for reliability and cost.
- Monitoring & Cost Optimization: Enhance observability across storage and compute to ensure cost effective operations
- Bachelor's degree in a relevant field (e.g., Data Engineering, Software Engineering, Computer Science)
- Extensive hands-on experience with Azure Cloud services; strong understanding of resource provisioning, networking, and security in Azure.
- Deep experience with Azure Databricks administration (clusters, scaling, job scheduling, monitoring).
- Strong experience with IaC (Terraform or Bicep), managing changes via code, and infrastructure versioning.
- Competence with version control (Git) and implementing CI/CD pipelines for data workloads/infra.
- Familiarity with Spark internals (how compute works, resource allocation, partitioning, memory), to aid in optimizing infrastructure.
- Knowledge of cost management on cloud (spot/preemptible VMs, auto-scaling, rightsizing).
- Strong scripting/automation skills (Python / PowerShell / Bash).
- Experience working with data engineering, analytics and BI teams; understanding of pipeline flows, data dependencies, and common data bottlenecks.
- Good to have Databricks or Azure certifications