Sr. Data Engineer-Hadoop,Hive,Spark,Trino, Star Rocks, Clickhouse,Data engineering-3+yrs AI exposure
Visa · Posted Oct 29
Digital payment technology platform and global payment card services corporation
Get a personal compatibility score
Add a resume for personal matches
About the role
Visa is a global payments technology company facilitating transactions between consumers, merchants, financial institutions and government entities across more than 200 countries and territories. The data engineering team builds and operates data products and large-scale pipelines supporting business and product features, with a focus on distributed data systems, governance, and quality. Sr. Data Engineers collaborate with product, business, and cybersecurity stakeholders to design architectures, build batch and real-time data pipelines, and apply automation and AI/ML services to improve development and operations.
What you will do
- Collaborate proactively with stakeholders to clarify requirements and identify key product stakeholders.
- Translate functional requirements into architecture designs for one or more components, leveraging existing architecture design patterns, and communicate these to Data Engineers.
- Implement extensible, maintainable, and reusable code using appropriate coding patterns, guidelines, and best practices; participate in code reviews to ensure standards are followed.
- Provide end-to-end ownership of data products or data pipelines, including mitigation of technical debt and documentation of errors or unexpected bugs.
- Identify and conduct required testing for moderately complex use cases, including unit testing, coding standards, and security scans.
- Build tools and standard automation processes to assist in transforming, managing, accessing, deploying, and monitoring data processes in batches and in real time.
- Write queries to extract and compile raw data across end-to-end pipelines, and implement orchestration techniques to automate data extraction logic.
- Ensure adherence to data management principles, governance, and tools to maintain data quality across multiple features.
- Develop and update troubleshooting guidance and procedures for reviewing, addressing, and fixing advanced problems flagged by testing.
- Manage software upgrades and server patches for security remediation where applicable.
Skills used in this role
What the employer is looking for
- 3+ years of relevant work experience and a Bachelor's degree, OR 5+ years of relevant work experience.
- Experience in solving data problems using data technologies
- Experience with Hadoop, Hive, HBase, Spark, Trino, or comparable distributed data systems.
- Experience in designing and implementing data pipelines and large-scale data processing jobs.
- Experience in writing, modifying, and reviewing high-quality, testable, and efficient code.
- Data pipelines, ETL/ELT, data-quality frameworks, Java or Python, SQL, Git, and production troubleshooting.
- Experience in developing unit, integration, and end-to-end tests with a focus on automation.
- Experience in building and maintaining CI/CD pipelines and deploying applications using containerization and orchestration tools.
- Experience in monitoring applications in production and troubleshooting issues using observability tools.
- Experience in integrating AI or ML services into applications, where applicable.
- Experience in collecting and analyzing metrics to guide optimizations and improvements.
- Experience in partnering with stakeholders to clarify requirements and ensure deliverables meet business needs.
- Experience in applying secure coding practices and complying with regulatory standards.
Preferred qualifications
- 3 or more years of work experience with a bachelor’s degree or more than 2 years of work experience with an Advanced Degree (e.g. Masters, MBA, JD, MD).
- Experience in contributing in Open-source components.
- Use/Build AI agents, and agentic patterns to improve development, testing, operations, troubleshooting, automation, onboarding, and documentation.
- Experience in enterprise-class systems engineered for high availability, low latency, scalability, and supportability. Architecture depth across storage/compute separation, query and table formats, streaming, metadata, governance, and interoperability.
- Experience in building and pushing code into production.
- Experience in implementing and supporting real-time systems.
Benefits and support
- Compensation and benefits are detailed in the job posting
About Visa
Visa Inc. is a multinational payment card services corporation that facilitates electronic funds transfers worldwide. The company empowers global commerce through its secure digital payment network, connecting consumers, merchants, financial institutions, and government entities across more than 200 countries and territories.
- Industry
- Banking
- Company size
- 34100 employees
- Founded
- 1958-09-18
- Location
- San Francisco, California, USA
- Funding stage
- Public Company
Funding
Public Company · $17.9B raised
- 2008-03-19IPO$17.9B
Leadership
Chief Executive Officer
Chief Financial Officer
Chief Product and Strategy Officer
Group President, Global Markets
Vice Chair, Chief People and Corporate Affairs Officer
Recent coverage
GlobalData
Visa Expands Strategic Partnerships and AI Travel Assistant Offerings2025-10-01
Visa Newsroom
Visa Reports Strong Fiscal Year Financial Results and Processing Volumes2025-10-25
Visa Newsroom
Visa Appoints New Leaders across Global Markets and Technology Divisions2025-08-11