Requirements

Bachelor's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 2+ years related experience OR Master's Degree in related field AND 1+ year(s) related experience OR Doctorate in related field OR equivalent experience
1+ years experience in applied science, machine learning, deep learning, natural language processing or large-scale data mining
Strong programming skills in languages such as Python, and experience with relevant libraries and frameworks

Nice to Haves

Ph.D in computer science, machine learning, or a related field
Experience working with LLM and OpenAI models
Understanding of Responsible AI principles and their application in AI product development
Experience with large-scale data analysis and experimentation
Expertise in developing tools for automation and familiarity with quality analysis frameworks
Excellent analytical and problem-solving skills with a keen attention to detail
Excellent written and verbal communication skills

Drive research initiatives for online metrics development to continuously assess and improve the real-time performance of Copilot products
Develop and implement methodologies for offline quality measurement
Lead the development of tools for automating quality analysis
Design and execute experiments to evaluate the impact of model changes
Hands-on engagement with model training activities