Lead Software Engineer – AIML Data Platform (Data, Python, Containers/Kubernetes)

Requirements

  • Formal training or certification on software engineering concepts and advanced applied experience.
  • Hands-on practical experience delivering system design, application development, testing, and operational stability.
  • Demonstrable strength in a cloud-based, cloud-native engineering environment.
  • Advanced Python skills; additional languages such as Java, Kotlin, or Scala are a plus.
  • Deep knowledge of distributed-systems fundamentals including partitioning, parallelism, state, fault tolerance, and performance tuning.
  • Production experience with data engineering and streaming platforms at scale.
  • Practical machine learning engineering experience covering inference, evaluation, monitoring, and deployment.
  • Real-world experience delivering LLM systems beyond prototypes with attention to quality, cost, safety, and governance.
  • Experience leading effective use of AI-assisted software development tools and setting team standards for validating AI outputs.
  • Strong understanding of responsible AI use, including data sensitivity, secure input/output handling, and resiliency and security expectations.
  • Proficient in all aspects of the Software Development Life Cycle, including databases and data structures.
  • Advanced knowledge of agile methodologies such as CI/CD, application resiliency, and security.
  • Proven leadership in technical delivery across teams with clear, pragmatic, and evidence-driven communication.

Nice to Haves

  • In-depth knowledge of the financial services industry and IT systems.
  • Experience with regulated or high-governance environments, agent frameworks, MCP, RAG, and vector search technologies.
  • Familiarity with Kubernetes, modern observability tools, real-time analytics, and search-centric architectures.

What You'll Be Doing

  • Executing creative software solutions across batch, streaming, ML, and LLM workloads while balancing latency, throughput, resilience, cost, and governance.
  • Developing secure, high-quality production code and leading code reviews and incident responses.
  • Engineering high-volume data platforms using technologies like Spark, Ray, Kafka, Iceberg, and OpenSearch.
  • Productionizing ML workflows including training, evaluation, deployment, monitoring, and lifecycle management.
  • Building and maintaining robust LLM systems with focus on accuracy, latency, cost, safety, and auditability.
  • Driving adoption of AI-assisted engineering practices to improve code quality and delivery speed.
  • Applying software development life cycle tools and automation to enhance value and reliability.
  • Championing responsible AI use and coaching teams on safe and compliant AI adoption.
  • Leading evaluation of external vendors, startups, and internal teams to assess technology fit and design.
  • Identifying opportunities for automation to enhance operational stability and providing technical leadership and mentorship.
AI Summary ✨

Similar jobs

  • Data Platform Engineer

    ClickHouse4 locations2 days ago
    NewRemoteRegistered sponsor
  • Senior Data Engineer I - Global KYC & Onboarding

    WiseGreater London, UK2 days ago
    Registered sponsor
    ≈ €105k – €133k / year base
  • Research Engineer, Gemini Omni Data Infrastructure, DeepMind

    GoogleLondon, UK3 days ago
    Registered sponsor
  • Senior Data Engineer II - Real Time Data Engineering- Java

    WiseGreater London, UK5 days ago
    Registered sponsor
    ≈ €126k – €156k / year base
  • See all jobs in UK →