Senior Lead Data Engineer, Data & AI

About the role

Data & AI team at Srijan builds modern, scalable data platforms and AI services for enterprise customers. The Senior Lead Data Engineer designs and implements Databricks‑based pipelines, optimizes Spark workloads, and supports LLMOps and vector search workflows. Works on hybrid model from Gurugram with occasional onsite collaboration, driving CI/CD automation and data governance.

What you’ll do

• Design and build scalable data platforms on Databricks and Delta Lake
• Develop and optimize large‑scale PySpark pipelines and Spark SQL queries
• Implement performance tuning and troubleshooting for Spark workloads
• Create and maintain CI/CD pipelines with GitLab automation
• Integrate REST APIs and support AI/ML workflows including LLMOps and RAG pipelines
• Enforce data governance, metadata management, and quality standards
• Conduct code reviews and mentor junior engineers
• Collaborate with DevOps, ML, and business teams to deliver AI analytics solutions

What you’ll bring

• 5+ years data engineering experience
• Expert in Python and PySpark
• Deep knowledge of Apache Spark internals and performance tuning
• Hands‑on with Databricks, Delta Lake, and Spark SQL
• Experience building CI/CD pipelines using GitLab
• Familiar with LLM evaluation, fine‑tuning, and vector‑search architectures
• Retail domain exposure preferred
• Strong stakeholder communication and technical leadership

Skills