Senior Data Engineer

About the role

Iris Software seeks a Senior Data Engineer in Noida, UP, India, to design scalable data engineering solutions and modern Lakehouse architectures. The role involves building high-performance batch and streaming pipelines using PySpark, Databricks, and Kafka while mentoring team members. This is an on-site position focused on delivering trusted data products and ensuring platform reliability.

What you’ll do

• Design scalable data engineering solutions and define ingestion/transformation architectures.
• Optimize Snowflake or Delta Lake solutions to support enterprise-scale data platforms.
• Implement high-performance batch and streaming data pipelines.
• Architect workflow orchestration solutions using Apache Airflow or Databricks Workflows.
• Drive data quality, validation, reconciliation, and governance practices.
• Mentor team members on PySpark, Snowflake, Delta Lake, Kafka, and best practices.
• Troubleshoot complex data processing and workflow issues through root cause analysis.
• Promote responsible use of AI-assisted engineering capabilities to improve productivity.

What you’ll bring

• Expertise in PySpark and modern distributed data processing frameworks.
• Experience with Snowflake or Delta Lake on Databricks for enterprise-scale platforms.
• Proficiency in designing event-driven architectures using Apache Kafka or Amazon Kinesis.
• Ability to architect workflow orchestration using Apache Airflow or Databricks Workflows.

Nice to have

• Strong background in Python, SQL, and AWS cloud services (S3, Dynamo DB, RDS).
• Demonstrated experience in data quality, validation, governance, and observability practices.
• Proven ability to troubleshoot complex data processing and streaming platform issues.
• Experience mentoring team members and collaborating with cross-functional stakeholders.

Skills

Benefits

• World-class benefits supporting financial, health, and well-being needs.
• Personalized employee experience for holistic professional and personal growth.
• Continuous learning and mentorship opportunities.