Data Engineer, Compute Operations

About the role

News Corp India seeks a Data Engineer to build and operate its primary Data Warehouse within the GCP ecosystem. You will architect high-performance, automated data environments, designing efficient pipelines and applying a 'Thinking Data' mindset where AI and advanced analytics are standard tools. Based in Bangalore, this full-time role focuses on solving critical business problems through scalable ETL/ELT development, performance optimization, and operational maintenance of data pipelines.

What you’ll do

• Develop, deploy, and maintain scalable ETL/ELT pipelines in GCP using Cloud Composer and PySpark on Dataproc
• Implement BigQuery features like Partitioning, Clustering, and Materialized Views to improve query performance and support cost management
• Assist in implementing BigLake components for unified governance of structured and unstructured data
• Use Terraform and CircleCI to deploy and manage pipeline code and infrastructure for modular, reproducible environments
• Implement logging, alerting, and data quality checks to monitor data flow and ensure reliability
• Collaborate with senior engineers and cross-functional teams to translate business requirements into efficient data solutions
• Proactively learn new GCP services and tools to enhance data processing capabilities

What you’ll bring

• Mid-level experience with 2-4 years of hands-on Data Engineering experience
• Expert-level BigQuery skills including Partitioning, Clustering, and Slot Management
• Proficiency in PySpark on Dataproc for large-scale dataset processing
• Strong Python and SQL coding skills with modular, testable code
• Experience with Cloud Composer (Airflow) for orchestration
• Proficiency in Terraform for Infrastructure-as-Code and CI/CD tools
• Solid knowledge of GCP services including BigQuery, Dataform, Cloud Composer, Cloud Storage, Cloud Functions, and Dataproc
• Understanding of Data Modeling (Star Schema, Snowflake) and Data Operations

Skills