Kafka Senior Engineer - Officer
State Street · Posted Oct 13
Investment servicing, custody, asset management, and trading solutions for institutional investors
Get a personal compatibility score
Add a resume for personal matches
About the role
State Street's Platform Engineering Middleware team runs the bank's vital messaging plumbing, executing over 2 million job runs per day in Autosys, supporting 3,500 web sites and handling 5 billion messages per day across Kafka and MQ. The team is responsible for platform engineering, design, and implementation of next generation messaging infrastructure, including the first Kafka cluster on public cloud. This senior engineer role leads Kafka messaging design and engineering across the enterprise, writes automation and Kafka producer/consumer/connector code, tunes and monitors the platform, and acts as a final level of production support.
What you will do
- Design, Engineer, and Evangelize Kafka Messaging across the enterprise
- Lead in the design and engineering of our first Kafka Cluster on a Public Cloud (Azure, AWS, other) using Ansible, Terraform, Python
- Lead and assist in providing direction for Junior Engineers
- Work with Enterprise Architecture team in identifying and implementing use cases best suited for Kafka
- Write code to automate tasks ( request processing, product installations, health checks, proactive monitoring, alerting, reporting, predictive analytics )
- Write Kafka producer code and Kafka consumer code within consumer groups to test the Kafka client APIs, develop Kafka application best practices, and help safely onboard Kafka applications to our Kafka platforms (across programming languages/platforms)
- Configure and test pre-built Kafka Connectors for integrating Kakfa with other systems
- Design, code, and test new Kafka Connectors where needed for integrating Kakfa with other systems
- Perform optimization and tuning of Kafka
- Design monitoring solutions for the platform and baseline statistics reporting
- Work on creating best practices, standards, Integration patterns and on-boarding process automation
- Ability to work with application development teams to understand usage pattern and provide solutions
- Lead in collaborating with other infrastructure teams to achieve high resiliency and scalability of our Kafka Messaging platforms
- Provide technical expertise in scoping end to end business requirements and assist in reflecting them in implementation for different Kafka connectors
- Conduct technical reviews of integration implementation for all SDLC phases to assess the quality of integration solutions built on Kafka
- Strive to provide highly resilient Kafka Messaging systems using the latest high availability (HA) and disaster recovery (DR) technology solution
Skills used in this role
What the employer is looking for
- Love of writing code/programming
- Solid coding ability to write robust, fault tolerant, well tested code. Any scripting/programming language is fine but with willingness to learn Python, Ansible, Terraform, Puppet, Shell. Experience with source code control system
- Experience working on Red Hat Linux
- Deep level of understanding of Kafka Messaging or another Messaging technology
- Experience with finding root cause for problems with Kafka or Applications using Kafka (or another Messaging technology)
- Experience with tuning Kafka systems (or other Messaging systems)
- Experience with supporting Application architectures SEDA, Kappa, and Lambda or similar
- This is a “hands on keyboard” engineering position
- L4 ability to drive a project and collaborate within our team and with other application and infrastructure teams
- L4 ability to troubleshoot complex issues
- Occasional weekend work may be required for Prod deployments/changes
- Our engineers will need to create new docs and SOPs for use by our support team and perform some cross training of the support team members
Preferred qualifications
- Programming and automation experience.
- 8+ years of experience preferred but not required depending on candidate
- Computer science degree preferred but not required depending on candidate
- Experience with high availability (HA) and disaster recovery (DR) solutions for middleware or other platforms
Benefits and support
- Competitive and comprehensive benefits packages with generous medical care, insurance and savings plans
- Flexible Work Program
- Development programs and educational support
- Paid volunteer days
- Matching gift program
- Employee networks
About State Street
State Street Corporation is a leading global financial services holding company providing institutional investors with investment servicing, investment management, and investment research and trading. Operating in more than 100 geographic markets worldwide, the company manages and administers trillions of dollars in assets through its core divisions like State Street Global Advisors and State Street Alpha. Headquartered in Boston, Massachusetts, it traces its origins back to 1792.
- Industry
- Financial Services
- Company size
- 51000+ employees
- Founded
- 1792-06-25
- Location
- Boston, Massachusetts, USA
- Funding stage
- Public Company
Leadership
Chairman and Chief Executive Officer
John Woods
Chief Financial Officer
Mostapha Tahiri
President of State Street Alpha & Chairman of Asia Pacific
Ann Fogarty
Enterprise Chief Operating Officer
President and Chief Executive Officer of State Street Investment Management
Recent coverage
Business Wire
State Street Corporation Releases 2026 Digital Assets Study2026-10-06
State Street
State Street Appointed as Transfer Agent by Wellington Management on HKEX Integrated Fund Platform2026-09-29
State Street
State Street Corporation to Report Third-Quarter 2026 Financial Results and Host Conference Call Webcast on October 142026-09-23
State Street
Yuanta Funds Selects State Street for First Overseas Securities Lending Program2026-09-15