Senior Data Engineer II - Real Time Data Engineering- Java

Requirements

  • Proficient with Java or Scala
  • Proven experience building, deploying, and operating production-grade, low-latency stream processing pipelines using Apache Kafka (or Kafka Streams, Flink, Spark Streaming)
  • Deep hands-on experience handling duplicate events, out-of-order log processing, late-arriving data, and exactly-once processing semantics in financial or mission-critical domains
  • Strong expertise in windowing techniques (tumbling, sliding, session windows), stateful stream processing, and high-throughput real-time aggregation across streaming data pipelines
  • Advanced experience in streaming data modeling for both real-time stream consumer patterns and downstream analytical stores (e.g., event schemas, state store schemas, schema evolution with Protobuf/Avro)
  • Instinctive defensive engineering skills including design for dead-letter queues, backpressure management, circuit breakers, and state-store recovery

What you'll be doing

  • Design and build the real-time liquidity observability stack and core streaming data pipelines that power global financial operations
  • Architect high-throughput Kafka and Flink streaming pipelines processing global bank transactions
  • Balance demand against supply intraday with low-latency streaming pipelines delivering instant, global liquidity visibility
  • Route high-value transactions instantaneously with fine-grained event controls
  • Fuel ML forecasting models to maximize yield on surplus capital
  • Bulletproof event architecture against out-of-order spikes, network partitions, and schema shifts
  • Master a Medallion architecture pairing streaming pipelines with sub-second OLAP querying
  • Collaborate closely with Product Managers, Data Scientists, Quantitative Analysts, and Treasury Ops in a highly autonomous, cross-functional team
  • Engage in clear, actionable communication and thorough design reviews to raise engineering standards
  • Navigate trade-offs across latency, cost, regulatory boundaries, and customer experience using RFCs, whiteboard sessions, and stakeholder feedback
AI Summary ✨

Similar jobs

  • Python Software Engineer - Data Infrastructure

    Man GroupLondon, UK3 days ago
    Registered sponsor
  • Senior Software Engineer II - Data Onboarding & Reporting

    WiseGreater London, UK3 days ago
    Registered sponsor
    ≈ €133k – €173k / year base
  • Data Engineer

    SmarketsLondon, UK3 days ago
    Registered sponsor
  • See all jobs in UK →