Role Summary:
We are seeking a senior data engineer with strong Microsoft Azure experience who is ready to grow into a data architect role. You will design and build reliable data pipelines and lakehouse/warehouse solutions while contributing to data architecture decisions (patterns, standards, governance, and non-functional requirements). This is a hands-on role with increasing ownership of solution design and technical leadership.
Key Responsibilities:
- Design, build, and optimize end-to-end data ingestion, transformation, and serving pipelines on Azure (batch and streaming).
- Contribute to solution architecture: define target-state data platform, reference patterns, and architecture artifacts (HLD/LLD, diagrams, and NFRs).
- Implement Lakehouse/warehouse solutions using proven patterns (e.g., medallion architecture, dimensional modeling, and data vault where applicable).
- Partner with stakeholders to translate business requirements into scalable data models, data products, and SLAs.
- Drive data quality, observability, and reliability (validation rules, reconciliation, monitoring, alerting, and runbooks).
- Implement security and governance: RBAC, managed identities, Key Vault, encryption, data access policies, and lineage.
- Establish CI/CD and infrastructure-as-code practices for data workloads; enforce coding and review standards.
- Optimize performance and cost across storage, compute, and orchestration.
- Mentor junior engineers and lead technical discussions across engineering, DevOps, and analytics teams.
Required Skills & Qualifications :
Azure Data Engineering
- Strong hands-on experience with Azure data services such as Azure Data Factory (ADF), Azure Synapse
- Analytics, Azure Databricks, and/or Microsoft Fabric (Data Engineering).
- Experience with storage and lakehouse components: ADLS Gen2, Delta Lake / Parquet, and data partitioning strategies.
- Experience with relational and NoSQL stores (e.g., Azure SQL, SQL Server, Cosmos DB) and data integration patterns.
- Streaming and event-driven exposure: Event Hubs / Kafka, Stream Analytics, and/or Spark Structured Streaming.
Programming & Data
- Strong SQL skills; ability to write efficient, maintainable transformations and tune queries.
- Proficiency in Python and/or Scala (preferred) for data processing and automation.
- Solid understanding of data modeling (dimensional/star schema), metadata management, and data quality concepts.
Platform Engineering
- Experience with Git-based development and CI/CD using Azure DevOps or GitHub Actions.
- Infrastructure-as-code exposure (Terraform, Bicep/ARM) and environment promotion strategies (dev/test/prod).
- Observability practices: logs, metrics, lineage, and pipeline monitoring (e.g., Azure Monitor, Log Analytics).
Architecture Mindset
- Ability to reason about non-functional requirements: performance, scalability, resiliency, security, and cost.
- Strong communication skills to explain trade-offs and guide stakeholders toward practical solutions.
Nice-to-Have:
- Experience with Microsoft Purview (catalog, lineage, governance) and enterprise data security policies.
- Exposure to API-based integration patterns and services such as Azure Functions, Logic Apps, and API
- Management.
- Experience with containerization and orchestration (Docker, AKS) for data workloads.
- Familiarity with analytics/BI layers (Power BI) and semantic modeling concepts.
- Cloud certifications (preferred): DP-203, AZ-305 (or equivalent).
What Success Looks Like (First 90 Days)
- Understand the current data landscape and propose improvements aligned to business outcomes and
- platform standards.
- Deliver at least one production-grade pipeline/data product with monitoring, quality checks, and
- documentation.
- Create or refine architecture artifacts (data flows, target architecture, integration patterns, and standards).
- Establish repeatable CI/CD practices and improve reliability/performance for key workloads.
Education:
Bachelor’s degree in Computer Science/Engineering (or equivalent practical experience).
Experience:
8–10 years
