Senior Software Engineer, Data Platforms

What you'll be doing

  • Design and enhance scalable, fault-tolerant data architectures supporting real-time and batch processing using Apache Kafka, Apache Flink, and lakehouse technologies.
  • Improve data semantics, classification, and discovery to enhance data accessibility and usability across eBay.
  • Develop and maintain automated data pipelines for efficient and reliable data processing from multiple sources into lakehouse architecture.
  • Optimize data consumption through materialized views and data subsumption techniques to reduce redundancy and improve performance.
  • Collaborate with data scientists, business analysts, and engineering teams to convert requirements into effective data solutions.
  • Explore and integrate emerging data engineering technologies to improve scalability, efficiency, and reliability of data infrastructure.

Requirements

  • Experience with Apache Kafka, Apache Flink, and other streaming technologies for large-scale data processing.
  • Strong understanding of lakehouse architectures and hands-on experience in large-scale data solution implementation.
  • Knowledge of data semantics, data discovery, and governance for effective data management.
  • Experience designing, developing, and automating data pipelines, including materialized views.
  • Familiarity with data subsumption techniques to reduce data redundancy and improve system performance.
  • Proficiency in Java, Scala, Python, or similar programming languages.
  • 5+ years of relevant software engineering or data engineering experience.
AI Summary ✨

Similar jobs

  • Data Platform Engineer - Cybersecurity

    WorkdayDublin, Irelanda month ago
    RemoteRegistered sponsor
    €70.2k – €105.4k / year base
  • Staff Software Engineer - Data Platform

    ToastDublin, Irelanda month ago
    Registered sponsor
  • Quantitative Data Engineer - EOT QEDD - Experienced Hire

    Susquehanna (SIG)Dublin, Ireland2 months ago
    Registered sponsor
  • See all jobs in Ireland →