Data Engineer
Employer not yet confirmed · UAE · onsite
Data Engineer
Data Product Design & Delivery
- Design and deliver end-to-end data products for assigned business domains across the bronze, silver, and gold layers, including the dimensional and semantic models that serve reporting and AI.
- Build robust ingestion from Amana's enterprise systems — Oracle Fusion (Finance, HCM, P2P, CRM), Hyperion, CMiC, project and operations systems, SharePoint, and Power Platform — using change data capture, incremental, and API-based patterns.
- Lead workstreams within the Synapse Analytics to Databricks migration: re-engineering pipelines, running parallel loads, reconciling results, and preparing for cutover.
- Contribute reusable frameworks, templates, and patterns, and propose improvements to the standards owned by the Data Engineering Lead (Data Architect).
Data Quality, Governance & Security
- Implement data quality frameworks, monitoring, and reconciliation across your domains, and drive issues to resolution at source with data owners.
- Implement access control, lineage, classification, and data-handling rules in Unity Catalog and Microsoft Purview, including PDPL controls for personal and HR data.
- Partner with the Data Governance Office to turn domain policies into working controls and documented, certified datasets.
Performance, Reliability & Cost
- Tune Spark and SQL workloads for performance and cost within the platform's cluster and workload policies.
- Own the operational health of your domains: monitoring, alerting, incident response, and root-cause fixes.
- Apply CI/CD, automated testing, and DataOps practices using Git and Azure DevOps.
Analytics & AI Enablement
- Build AI-ready data products — feature tables and document and vector pipelines — for machine learning, RAG, and agent use cases, in partnership with the Data Scientist and AI Engineer.
- Deliver Power BI semantic models and certified datasets with the Data Analyst, supporting self-service adoption through the Amana Digital Academy.
- Apply AI-assisted engineering — code generation, testing, documentation, and lineage discovery — with appropriate human review, and help the team adopt it safely.
Collaboration & Mentoring
- Translate requirements from finance, project, and operations stakeholders into technical designs with clear delivery outcomes.
- Mentor Data Engineers through pairing, design reviews, and code review.
- Explain design choices, trade-offs, and risks clearly to technical and business audiences.
Required Qualifications
- 5+ years of hands-on data engineering experience, including at least 2 years with Databricks, Apache Spark, and Delta Lake in production.
- Designed and delivered end-to-end data products — from ingestion to modelled, consumable data — for one or more business domains.
- Experience integrating complex enterprise sources such as Oracle Fusion, SAP, Microsoft Dynamics, or CMiC, including incremental and API-based extraction.
- Data quality and access controls implemented in practice using Unity Catalog, Microsoft Purview, or equivalent tooling.
- Strong SQL, Python, and PySpark, with Git-based CI/CD delivery (Azure DevOps or equivalent).
- Excellent written and verbal communication skills across technical and business audiences.
Required (held, or achieved after joining within the period shown):
- Databricks Certified Data Engineer Associate or Microsoft Certified: Azure Databricks Data Engineer Associate (DP-750) — within 6 months
- Databricks Certified Data Engineer Professional — within 12 months
- Preferred (one or more):
- Databricks Certified Generative AI Engineer Associate
- Microsoft Certified: Fabric Data Engineer Associate (DP-700)
- Microsoft Certified: Fabric Analytics Engineer Associate (DP-600)
- Microsoft Certified: Azure Data Engineer Associate (DP-203 — retired 2025; accepted if held)