Staff Product Analyst - Agentic AI Tracking

Requirements:

• Collaborate with Product and Engineering to define, implement, and document tracking requirements for agent actions, state transitions, and user-agent interactions.
• Develop and maintain monitoring systems, dashboards, and alerts for agentic interaction data correctness.
• Investigate anomalies in agent event logs, trace data discrepancies, and drive timely resolutions.
• Partner with AI Research and Data Science teams to align tracking with evaluation metrics.
• Validate interaction data for user behavior analysis and feature assessment.
• Create and maintain documentation for agent tracking specifications and QA frameworks.
• Identify and implement process improvements for agent data collection pipelines.

Nice to haves:

• Experience with Python for data analysis and log parsing.
• Background in data pipelines, ETL processes, or real-time event streaming.
• Prior experience working closely with AI or ML teams for data validation.

What you'll be doing:

• Analyze agent behavior, measure the impact of new agentic features, and develop evaluation pipelines.
• Collaborate with various teams to ensure tracking accuracy and data quality.
• Validate interaction data for user behavior analysis and agent success metrics.

Perks and benefits:

• Hybrid working model with face-to-face collaboration in Berlin campus
• 27 days holiday with additional days based on service years
• Development opportunities with educational budget and access to courses
• Health perks like gym subsidy, meditation, and health checkups
• Financial benefits including Employee Share Purchase Plan and insurance
• Dining perks with meal vouchers and corporate discounts