Data Engineer-Data Platforms-Google
IBM · Posted Oct 23
Hybrid cloud, artificial intelligence, enterprise software, quantum computing, and IT consulting services
Get a personal compatibility score
Add a resume for personal matches
About the role
IBM Consulting works with clients across industries to shape their hybrid cloud and AI journeys, supported by IBM technology, strategic partners, and Red Hat. The role sits in data engineering on Google's Cloud ecosystem, covering batch and real-time pipelines, data migration, and data layer design. The engineer will build and maintain solutions using BigQuery, DataFlow, DataProc, PubSub, Cloud Composer, and open-source tools such as Spark, Airflow, and dbt.
What you will do
- Design Data Pipelines: Design and develop batch and real-time data pipelines for Data Warehouse and Datalake using Google services such as DataProc, DataFlow, PubSub, BigQuery, and Big Table.
- Develop Data Engineering Solutions: Utilize Google Cloud Storage, BigTable, BigQuery DataProc with Spark and Hadoop, and Google DataFlow with Apache Beam or Python to build and maintain data engineering solutions.
- Manage Data Platforms: Schedule and manage the data platform using Google Cloud Scheduler and Cloud Composer (Airflow), ensuring efficient data pipeline operations.
- Implement Data Migration: Develop and implement data migration solutions using Google services, ensuring seamless data transfer between systems.
- Optimize Data Layer: Design and optimize the data layer using Google services such as BigQuery, Big Table, and Cloud Spanner, ensuring efficient data storage and retrieval.
- Data Migration Solutions: Experience working with Google services to develop and implement data migration solutions, ensuring seamless data transfer between systems.
Skills used in this role
What the employer is looking for
- Google Cloud Ecosystem Expertise: Exposure to designing, building, and maintaining data engineering solutions on Google's Cloud ecosystem, including services such as Google DataProc, DataFlow, PubSub, BigQuery, Big Table, Cloud Spanner, CloudSQL, and AlloyDB.
- Data Pipeline Development Experience: Exposure to developing and managing batch and real-time data pipelines for Data Warehouse and Datalake using Google services and open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala.
- Google Cloud Services Proficiency: Experience working with Google Cloud Storage, BigTable, BigQuery DataProc with Spark and Hadoop, and Google DataFlow with Apache Beam or Python to build and maintain data engineering solutions.
- Data Platform Management Knowledge: Exposure to scheduling and managing the data platform using Google Cloud Scheduler and Cloud Composer (Airflow) for efficient data pipeline operations.
- Data Layer Design Understanding: Experience working with data layer design using Google services such as BigQuery, Big Table, and Cloud Spanner for efficient data storage and retrieval.
- Open Source Technologies: Exposure to utilizing open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala for developing and managing batch and real-time data pipelines.
- Cloud Composer Expertise: Exposure to using Cloud Composer (Airflow) for scheduling and managing the data platform, ensuring efficient data pipeline operations.
Benefits and support
- Compensation and benefits are detailed in the job posting
About IBM
International Business Machines Corporation (IBM) is a global technology and consulting company that provides hybrid cloud, artificial intelligence, and enterprise infrastructure solutions. Operating in over 175 countries, IBM helps businesses across industries modernize operations, secure data, and drive digital transformation through platforms like watsonx and Red Hat OpenShift. Founded over a century ago, the company remains a pioneer in high-performance computing, enterprise automation, and quantum technology.
- Industry
- Information Technology
- Company size
- 260,000+ employees
- Founded
- June 16, 1911
- Location
- Armonk, New York, USA
- Funding stage
- Public Company
Funding
Public Company · $3.5B raised
- ·Post Ipo Debt$150M
Leadership
Chairman, President and Chief Executive Officer
Senior Vice President and Chief Financial Officer
Senior Vice President, Software and Chief Commercial Officer
Senior Vice President, IBM Consulting
Senior Vice President, Marketing and Communications
Recent coverage
IBM Newsroom
Expanding enterprise data engineering with Apache Flink® stream processing2026-10-20
IBM Newsroom
IBM named a Leader in the 2026 IDC MarketScape for Worldwide Agentic AI Platforms2026-10-14
IBM Newsroom
Cleveland Clinic, RIKEN and IBM Finalists for 2026 ACM Gordon Bell Prize in Quantum-Centric Supercomputing2026-09-09
IBM Newsroom
Wimbledon and IBM Introduce New AI-Powered Fan Experiences and Modernized Digital Platforms for The Championships 20262026-07-08