Senior Software Engineer, Data Platforms

Requirements

• Extensive experience with Apache Kafka, Apache Flink, and other relevant streaming technologies.
• Deep understanding of lakehouse architecture and its implementation in large-scale environments.
• Strong knowledge of data semantics, discovery processes, and data governance.
• Proven ability to design and implement automated data pipelines and materialized views.
• Experience with data subsumption techniques and tools.
• Proficiency in Java, Scala, Python, or similar languages.
• 5+ years of directly applicable experience.

What You'll Be Doing

• Architectural Leadership: Design and enhance data architecture for real-time and batch processing using Kafka, Flink, and lakehouse principles.
• Data Semantics and Discovery: Implement systems for effective data semantics management.
• Pipeline Automation: Develop and maintain automated data pipelines for efficient data flow.
• Data Consumption Optimization: Create strategies for optimal materialized views and subsumption.
• Cross-functional Collaboration: Work with data scientists and business analysts to drive data solutions.
• Innovation and Research: Stay updated on industry developments and propose new technologies.

Nice to Haves

Perks and Benefits