Data Engineer-Data Platforms-AWS
IBM · Posted Oct 23
Hybrid cloud, artificial intelligence, enterprise software, quantum computing, and IT consulting services
Get a personal compatibility score
Add a resume for personal matches
About the role
IBM Consulting helps clients shape hybrid cloud and AI journeys, supported by strategic partners, IBM technology, and Red Hat. This Data Engineer role focuses on AWS data platforms, designing, building, and operating batch and real-time data pipelines and data layers using services such as AWS Glue, EMR, Kinesis, Redshift, and DynamoDB.
What you will do
- Design and Develop Data Pipelines: Design, build, and operate batch and real-time data pipelines using AWS services such as AWS EMR, AWS Glue, Glue Catalog, and Kinesis, ensuring seamless integration and operation of data engineering solutions.
- Create Data Layers: Create data layers on AWS RedShift, Aurora, and DynamoDB, and migrate data using AWS DMS.
- Manage Data Services: Schedule and manage data services on the AWS Platform, ensuring efficient operation of data engineering solutions.
- Develop Batch and Real-time Pipelines: Develop batch and real-time data pipelines for Data Warehouse and Datalake, utilizing AWS Kinesis and Managed Streaming for Apache Kafka.
- Utilize Open Source Technologies: Utilize open source technologies like Apache Airflow and dbt, Spark / Python or Spark / Scala on AWS Platform to support data engineering solutions.
- Data Layer Creation: Experience working with AWS RedShift, Aurora, and DynamoDB to create data layers and migrate data using AWS DMS.
- Data Service Management: Experience scheduling and managing data services on the AWS Platform, ensuring efficient operation of data engineering solutions.
- Proficiency with AWS Databrew: Experience working with AWS Glue Databrew to support data engineering solutions, including data preparation and data quality tasks.
Skills used in this role
What the employer is looking for
- Exposure to AWS Toolset: Experience working with AWS services such as AWS EMR, AWS Glue, Glue Catalog, and Kinesis to design, build, and operate batch and real-time data pipelines.
- Data Pipeline Development: Exposure to developing batch and real-time data pipelines for Data Warehouse and Datalake, utilizing AWS Kinesis and Managed Streaming for Apache Kafka.
- Open Source Technologies: Exposure to utilizing open source technologies like Apache Airflow and dbt, Spark / Python or Spark / Scala on AWS Platform to support data engineering solutions.
- Knowledge of Lambda Functions: Exposure to using Lambda functions with Python to support data engineering solutions, including data processing and data transformation tasks.
- Familiarity with RedShift Spectrum: Experience working with RedShift Spectrum to support data engineering solutions, including data warehousing and data analytics tasks.
Benefits and support
- Compensation and benefits are detailed in the job posting
About IBM
International Business Machines Corporation (IBM) is a global technology and consulting company that provides hybrid cloud, artificial intelligence, and enterprise infrastructure solutions. Operating in over 175 countries, IBM helps businesses across industries modernize operations, secure data, and drive digital transformation through platforms like watsonx and Red Hat OpenShift. Founded over a century ago, the company remains a pioneer in high-performance computing, enterprise automation, and quantum technology.
- Industry
- Information Technology
- Company size
- 260,000+ employees
- Founded
- June 16, 1911
- Location
- Armonk, New York, USA
- Funding stage
- Public Company
Funding
Public Company · $3.5B raised
- ·Post Ipo Debt$150M
Leadership
Chairman, President and Chief Executive Officer
Senior Vice President and Chief Financial Officer
Senior Vice President, Software and Chief Commercial Officer
Senior Vice President, IBM Consulting
Senior Vice President, Marketing and Communications
Recent coverage
IBM Newsroom
Expanding enterprise data engineering with Apache Flink® stream processing2026-10-20
IBM Newsroom
IBM named a Leader in the 2026 IDC MarketScape for Worldwide Agentic AI Platforms2026-10-14
IBM Newsroom
Cleveland Clinic, RIKEN and IBM Finalists for 2026 ACM Gordon Bell Prize in Quantum-Centric Supercomputing2026-09-09
IBM Newsroom
Wimbledon and IBM Introduce New AI-Powered Fan Experiences and Modernized Digital Platforms for The Championships 20262026-07-08