Senior Data Engineer II - Real Time Data Engineering
Senior Data Engineer II - Real Time Data Engineering
Requirements
Proven experience building, deploying, and operating production-grade, low-latency stream processing pipelines using Apache Kafka or related technologies (Kafka Streams, Flink, Spark Streaming).
Deep hands-on expertise in handling idempotency, duplicate events, out-of-order logs, late-arriving data, and exactly-once processing semantics in financial or mission-critical domains.
Strong knowledge of windowing techniques such as tumbling, sliding, and session windows, along with stateful stream processing and high-throughput real-time aggregation.
Advanced experience in streaming data modeling for real-time consumer patterns and downstream analytical stores, including event schemas, state-store schemas, and schema evolution with Protobuf or Avro.
Skill in defensive engineering practices, designing systems with dead-letter queues, backpressure management, circuit breakers, and state-store recovery.
What you'll be doing
Designing and building foundations of a real-time liquidity observability stack and core streaming data pipelines powering global financial operations.
Architecting high-throughput Kafka and Flink streaming pipelines to process global bank transactions and balance demand versus supply intraday.
Building low-latency streaming pipelines to deliver instant, global liquidity visibility across currencies, jurisdictions, and nostro accounts.
Fueling machine learning forecasting models and routing high-value transactions with fine-grained event controls.
Bulletproofing event architecture against out-of-order data, network partitions, and schema shifts.
Leveraging a Medallion architecture with sub-second OLAP querying to provide Treasury Ops with precise liquidity visibility.
Collaborating closely with Product Managers, Data Scientists, Quantitative Analysts, and Treasury Operations in autonomous, cross-functional teams.
Driving clarity and quality via clear communication, design reviews, and writing well-tested, maintainable code.