Senior Data Engineer, AI & Data, Azure Databricks

About the role

EY GDS AI & Data practice builds trusted, modern data platforms that power analytics and responsible AI. As a Senior Data Engineer you will design and deliver lakehouse solutions, secure batch and streaming pipelines, and AI‑ready data products on Databricks, collaborating with data scientists and security teams. Trivandrum office, on-site work model, competitive salary and relocation assistance available.

What you’ll do

• Own end‑to‑end data product delivery from design to production
• Build lakehouse architectures using Delta Lake, Unity Catalog, and medallion patterns
• Develop secure batch and streaming pipelines with Spark and Delta Live Tables
• Integrate native AI capabilities such as Mosaic AI, MLflow, and vector search
• Apply governance and security controls via Unity Catalog and CI/CD pipelines
• Mentor engineers and lead technical delivery within Agile teams
• Collaborate with data scientists, analytics, and security stakeholders
• Troubleshoot performance, reliability, and cost issues

What you’ll bring

• 5-8 years data engineering or analytics engineering experience
• Bachelor's or Master's in Computer Science, Engineering, or related field
• Hands‑on Python, PySpark, SQL, Delta Lake
• Experience with Databricks platform and Azure/GCP/AWS integration
• Knowledge of cloud data security, CI/CD, and infrastructure as code
• Databricks certification preferred
• Experience with AI/ML pipelines, RAG or semantic search a plus
• Consulting mindset with strong communication skills

Skills