Data Engineer-Data Platforms-AWS
IBM · Posted Oct 5
Hybrid cloud, artificial intelligence, enterprise software, quantum computing, and IT consulting services
Get a personal compatibility score
Add a resume for personal matches
About the role
IBM Consulting works with leading companies across industries to shape their hybrid cloud and AI journeys, supported by IBM technology, strategic partners, and Red Hat. As a Data Engineer specializing in Data Platforms on AWS, the role focuses on designing, building, and operating batch and real-time data pipelines and data layers on the AWS Cloud ecosystem. The engineer will work with AWS EMR, Glue, Kinesis, Redshift, Aurora, and DynamoDB, along with open source technologies such as Apache Airflow, dbt, and Spark with Python or Scala.
What you will do
- Design and Develop Data Pipelines: Design, build, and operate batch and real-time data pipelines using AWS services such as AWS EMR, AWS Glue, Glue Catalog, and Kinesis, ensuring seamless integration and operation of data engineering solutions.
- Create Data Layers: Create data layers on AWS RedShift, Aurora, and DynamoDB, and migrate data using AWS DMS.
- Manage Data Services: Schedule and manage data services on the AWS Platform, ensuring efficient operation of data engineering solutions.
- Develop Batch and Real-time Pipelines: Develop batch and real-time data pipelines for Data Warehouse and Datalake, utilizing AWS Kinesis and Managed Streaming for Apache Kafka.
- Utilize Open Source Technologies: Utilize open source technologies like Apache Airflow and dbt, Spark / Python or Spark / Scala on AWS Platform to support data engineering solutions.
Skills used in this role
What the employer is looking for
- Exposure to AWS Toolset: Experience working with AWS services such as AWS EMR, AWS Glue, Glue Catalog, and Kinesis to design, build, and operate batch and real-time data pipelines.
- Data Pipeline Development: Exposure to developing batch and real-time data pipelines for Data Warehouse and Datalake, utilizing AWS Kinesis and Managed Streaming for Apache Kafka.
- Data Layer Creation: Experience working with AWS RedShift, Aurora, and DynamoDB to create data layers and migrate data using AWS DMS.
- Open Source Technologies: Exposure to utilizing open source technologies like Apache Airflow and dbt, Spark / Python or Spark / Scala on AWS Platform to support data engineering solutions.
- Data Service Management: Experience scheduling and managing data services on the AWS Platform, ensuring efficient operation of data engineering solutions.
- Proficiency with AWS Databrew: Experience working with AWS Glue Databrew to support data engineering solutions, including data preparation and data quality tasks.
- Knowledge of Lambda Functions: Exposure to using Lambda functions with Python to support data engineering solutions, including data processing and data transformation tasks.
- Familiarity with RedShift Spectrum: Experience working with RedShift Spectrum to support data engineering solutions, including data warehousing and data analytics tasks.
Benefits and support
- Compensation and benefits are detailed in the job posting
About IBM
International Business Machines Corporation (IBM) is a global technology and consulting company that provides hybrid cloud, artificial intelligence, and enterprise infrastructure solutions. Operating in over 175 countries, IBM helps businesses across industries modernize operations, secure data, and drive digital transformation through platforms like watsonx and Red Hat OpenShift. Founded over a century ago, the company remains a pioneer in high-performance computing, enterprise automation, and quantum technology.
- Industry
- Information Technology
- Company size
- 260,000+ employees
- Founded
- June 16, 1911
- Location
- Armonk, New York, USA
- Funding stage
- Public Company
Funding
Public Company · $3.5B raised
- ·Post Ipo Debt$150M
Leadership
Chairman, President and Chief Executive Officer
Senior Vice President and Chief Financial Officer
Senior Vice President, Software and Chief Commercial Officer
Senior Vice President, IBM Consulting
Senior Vice President, Marketing and Communications
Recent coverage
IBM Newsroom
IBM Introduces Self-Hosted Deployment for IBM Bob to Help Enterprises Advance AI Sovereignty and Governance2026-10-01
Cybersecurity Dive
IBM and Red Hat Collaborate with Lightwell to Enhance Java Security, Launch Clearinghouse2026-10-06
IBM Newsroom
IBM announcements at Think 2026 to advance the agentic era2026-05-05
IBM Newsroom
IBM and Aramco Explore Collaboration to Accelerate AI and Innovation Across Saudi Arabia2026-05-05