Member of Technical Staff – AI Evaluations, Health
- Location
- Job Number
- City
- Team
- Country
- Discipline
MAI Health is bringing world-class AI to healthcare, and this role puts you at the front line of that effort: embedded in our partnership with Mayo Clinic, one of the world’s leading medical institutions. As an Evals Engineer, you will work with Mayo Clinic to ensure the agents we deploy are safe and effective. These agents help patients with everything from figuring out whether Mayo is the right fit for their care, to navigating their visit, understanding their conditions, and answering billing questions. This role is critical to defining what “good” looks like for a Mayo Clinic agent starting from an explicit specification of intended behavior, grounded in good medical practice and building the evaluation system that ensures quality improves over time.
Being passionate and opinionated about human-computer interaction, you will work at the nexus of product, research, and clinical practice. Your job is to evaluate the configured system patients actually encounter: the model together with its orchestration harness, tools, and connected data. You will be responsible for ensuring outputs are high quality, factual, and safe in a domain where the bar for all three is the highest anywhere. We’re looking for someone with an abundance of positive energy, empathy, and kindness, in addition to being highly effective. The right candidate takes initiative, thrives with ambiguity, and is comfortable being the face of MAI inside a partner organization.
This is a hybrid role based in NYC, with regular travel to Mayo Clinic (Rochester, MN) for on-site working sessions with clinical and technical stakeholders, and to London to work with the larger MAI Health Evals team.
Responsibilities
- Own the end-to-end evaluation strategy for MAI Health’s Mayo Clinic project: defining metrics, building datasets, and translating results into actionable steps for response-quality improvement, with clinical accuracy and patient safety as the top priorities.
- Build layered evaluations that span the full interaction spectrum from static single-turn benchmarks, adaptive multi-turn simulations with user simulators (including adversarial cases with hidden goals), and grounded, agentic evaluations that exercise the agent’s tools, retrieval, and connected data, scoring trajectories as well as final responses.
- Be a strong member of the MAI Health Evaluations team: learn from its practices and frameworks, translate them to the Mayo context, and feed what you learn back to improve the overall team’s capabilities.
- Work embedded with Mayo clinicians and subject-matter experts to design high-quality tooling and run evaluations for LLM-based tools. This includes writing clinically-grounded evaluation guidelines and exemplar-tied, case-specific rubrics, recruiting and calibrating physician raters, and maintaining held-out clinician-authored test sets to detect overfitting.
- Design and write LLM-as-judge prompts for medical content; build and run automated evaluations that scale clinical review, calibrate autograders against expert clinician annotation, and report judge–human agreement alongside inter-rater agreement so the measurement itself is trustworthy.
- Evaluate patient-facing AI experiences for factuality, groundedness, safety, appropriate escalation, and health literacy. This includes safeguards for psychologically vulnerable users and safety-netting behaviors and ensuring outputs are trustworthy whether the customer is a consumer or a clinician.
- Serve as the technical partner ensuring high-quality evaluations are created at Mayo: gathering requirements, demonstrating capabilities, triaging quality issues, and representing partner needs back to MAI engineering and research teams.
- Partner with data teams to develop scalable pipelines and dashboards tracking pre-launch and in-deployment eval performance.
- Contribute to privacy-preserving monitoring of production conversations, pairing it with clinical review of concerning interactions, so evaluation evolves with real-world use.
- Translate evaluation insights into actionable product improvements and inform launch/no-launch decisions for clinical features, maintaining configuration-specific records (model version, harness, evaluation date) that support lifecycle oversight and change control.
- Monitor trends in model performance, clinician and patient feedback, and the clinical-AI benchmark landscape (e.g., medical QA benchmarks, safety frameworks) to continuously refine evaluation approaches.
- Own key projects end-to-end, proactively identifying risks and proposing solutions to ensure timely delivery.
- Coordinate cross-team and cross-organization collaboration: align MAI engineering, research, data science, and Mayo clinical/IT stakeholders on goals, deliverables, and status.
- Handle sensitive health data responsibly, working within HIPAA and partner data-governance requirements.
- Embody our Culture and Values.
Qualifications
Required Qualifications
- Bachelor’s Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
- 3+ years leading multi-disciplinary projects: defining requirements, developing project plans, and executing with cross-functional teams.
Preferred Qualifications: - Experience in healthcare, health tech, or other regulated, safety-critical domains; familiarity with clinical workflows, medical terminology, or working with clinician stakeholders.
- Comfort operating inside a partner organization and translating between their world and yours.
- Computer science background; experience in training/evaluation of LLMs, including LLM-as-judge design, user simulation, and agentic/trajectory evaluation.
- Technical depth in software development, data science, and machine learning. While you are not expected to write code on the critical path, you’re able to define and execute zero-to-one prototypes, build tools to support the larger eval pipeline, and speak the language of the engineers, researchers, and clinicians you work with.
- Familiarity with HIPAA, PHI handling, and/or lifecycle regulatory approaches to clinical AI (e.g., FDA guidance on AI-enabled devices and predetermined change-control plans, EU AI Act).
- Experience building and evaluating ML-powered or LLM-powered products.
Software Engineering IC4 – The typical base pay range for this role across the U.S. is USD $119,800 – $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $160,200 – $261,000 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
Software Engineering IC5 – The typical base pay range for this role across the U.S. is USD $142,800 – $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 – $304,200 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
Similar jobs
Senior Applied Scientist
Principal Applied Scientist
Principal Applied Scientist
The Crawler Team, part of Microsoft Search’s Web Data organization, builds and operates Microsoft’s internet-scale web crawling platform that powers Bing Search, Microsoft Copilot, indexing, content discovery, web freshness, and AI grounding experiences. The team develops highly scalable distributed systems that discover, prioritize, retrieve, and process web content at massive scale while continuously improving content quality, freshness, efficiency, and reliability. As a Principal Software Engineer, you will drive the technical vision, architecture, and execution of critical infrastructure that enables Microsoft’s Search and AI platforms to understand and leverage the web.
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Starting January 26, 2026, Microsoft AI (MAI) employees who live within a 50- mile commute of a designated Microsoft office in the U.S. or 25-mile commute of a non-U.S., country-specific location are expected to work from the office at least four days per week. This expectation is subject to local law and may vary by jurisdiction.
Responsibilities
- Lead the architecture, design, and evolution of large-scale distributed systems responsible for web crawling, content acquisition, content processing, and retrieval at internet scale.
- Drive technical strategy and innovation in crawl scheduling, content freshness, real-time content discovery, and intelligent crawl prioritization systems.
- Build and operate highly reliable, performant, and efficient services that process billions of web crawling and content acquisition requests daily.
- Apply AI, machine learning, and data-driven approaches to improve crawl efficiency, content quality, resource utilization, and platform intelligence.
- Provide technical leadership across multiple teams and organizations, influencing architecture, engineering practices, and long-term platform direction.
- Partner closely with engineers, applied scientists, product managers, and platform teams across Search and AI to deliver shared infrastructure and end-to-end solutions.
- Lead efforts to improve platform scalability, reliability, observability, operational excellence, and developer productivity.
- Mentor senior engineers and drive engineering excellence through technical guidance, design reviews, and cross-team collaboration.
Qualifications
Required Qualifications:
- Bachelor’s Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
- OR equivalent experience.
- Experience designing, building, and operating scalable distributed services at scale.
Preferred Qualifications:
- Master’s Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
- OR Bachelor’s Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
- OR equivalent experience.
#MicrosoftAI #Search #Copilot
Software Engineering IC5 – The typical base pay range for this role across the U.S. is USD $142,800 – $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 – $304,200 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
Similar jobs
Senior Applied Scientist
Principal Applied Scientist
Principal Applied Scientist
At Microsoft AI, our Health team is on a mission to build and serve medical superintelligence at global scale. We’re responsible for ensuring that Microsoft AI’s models and services are useful, trusted and safe across diverse health journeys. For this role, the Health team at Microsoft AI is looking for a hands-on, product-focused clinician who will sit inside an engineering-heavy cross-functional team, and who is excited about contributing to the next generation of systems that will transform the field.
Across all our hires, it’s important for colleagues to share our enthusiasm about the role of technology and AI in health and healthcare, but also appreciate the challenges and risks of delivering effective solutions in a complex and safety critical space. You will need to be highly self-sufficient, and be able to span from high level strategy through to ground-level execution on a wide range of tasks — on this team, no task is beneath anyone. This also means you will play a major role in cultivating and promoting a positive team culture.
Microsoft’s wider mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Responsibilities
As a member of Microsoft AI’s Health team, you will be someone who:
- Contributes to prompting, prototyping, building agents, scripting and data analysis
- Defines evaluation strategies, and creates the tests, datasets and LLM scorers needed to know whether a system is safe and working
- Collaborates with and bridges the gap between clinicians, researchers, product, designers, engineering, and partnership teams.
- Contributes to safety, ethics, and privacy policies, and identifies gaps in our tools, technologies and methods to ensure overall rigour.
- Contributes to the clinical direction of our team’s efforts, bringing a clinician’s judgement to what we build and in what order.
Qualifications
Required/minimum qualifications:
- Medical Degree with more than 2 years of associated postgraduate clinical experience
- Experience working in the health AI space (in academic or industry settings), including developing, evaluating, and deploying health AI products
- Hands-on AI engineering experience
- A track record of owning things end-to-end — founding or leading a project, company, charity, PhD, or wearing multiple hats at an early stage startup
- Comfort with ambiguity and uncertainty, and the ability to work at high speed with autonomy in a rapidly changing environment
- Pragmatism about when something is good enough to ship, and when it isn’t
- Strong collaboration, communication and presentation skills
- Ability to produce high quality written and presentation materials with excellent attention to detail
Additional or preferred qualifications:
- Higher degree (e.g. Masters or PhD) in a related field
Applied Sciences IC5 – The typical base pay range for this role across United Kingdom is £ 93,500.00 – £ 161,800.00 per year. Certain roles may be eligible for benefits and other compensation.
Find additional benefits and pay information here:
https://careers.microsoft.com/v2/global/en/corporate-pay/united-kingdom-corporate-pay.html
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
Similar jobs
Senior Applied Scientist
Principal Applied Scientist
Principal Applied Scientist
Microsoft AI Experiences is a customer‑first organization building AI‑powered products that help people and businesses achieve more. As the world undergoes a profound AI platform shift, we are looking for product leaders who can operate several steps ahead anticipating what the future will demand and translating that vision into clarity, strategy, and execution today.
As a team, we play a central role in discovery, crawling, selection, document processing & aggregation of web graph. Work in our team is unique in that you’ll be able to work with trillions of documents, which pushes the boundaries of our infrastructure and computational resources. Moreover, it provides interesting challenges that haven’t been encountered before, thereby combining research and product applications.
We are looking for the best and brightest to join our team and help us apply the next generation of LLM, NLP, deep learning, large scale distributed systems and informational retrieval techniques. As a Senior Product Manager, you will build the next generation of Unified Index Platform (UIP) which is being used for featurizing, filtering and indexing for Search, Grounding and Training. UIP supports all data types and structured data across web, pdf, image, video, local, shopping etc. The candidate will focus on defining KPIs for success, doing deep data analysis, writing product specifications and delivering complex large projects.
This is a highly collaborative, cross‑functional role ideal for someone who thrives in operational complexity, is deeply organized, and loves working at the intersection of people, process, and product quality. You will partner closely with engineering, research, design, data science, legal, privacy, and security teams to build compliant, scalable, high‑quality AI systems that serve hundreds of millions of users.
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Starting January 26, 2026, Microsoft AI (MAI) employees who live within a 50- mile commute of a designated Microsoft office in the U.S. or 25-mile commute of a non-U.S., country-specific location are expected to work from the office at least four days per week. This expectation is subject to local law and may vary by jurisdiction.
Responsibilities
- Strategy & Vision
- Define the multi‑year product vision and strategy for Microsoft AI Experience Index Pipelines, related Success metrics, Feedback loops, and Testing mechanisms.
- Create strategies for processing all kinds of entities/data types, real time injection of critical data at scale, and on-boarding different teams that want to process and build large scale indexes.
- Execution & Delivery
- Define success criteria, OKRs, and KPIs across multiple product surfaces and organizational boundaries.
- Establish methodologies for on-boarding SLM/LLMs and platform support for large scale publish-subscribe mechanisms.
- Drive platform architecture decisions in partnership with engineering and research teams.
- Plan, size, and continuously optimize capacity (compute, storage, data throughput, and cost) to build, scale, and operate training pipelines reliably.
- Cross‑Functional Leadership
- Partner with Legal, Privacy, Security, HR, and Compliance teams to ensure platforms are compliant‑by‑design.
- Collaborate with teams across Microsoft that need to process, build or access very large indexes for Search or AI scenarios.
- Define and evangelize processes for developers and researchers to improve labeling workflows, model‑as‑a‑judge systems, and agentic evaluation pipelines.
- Innovation & Technical Depth
- Research, design, and operationalize new evaluation techniques and metrics that improve the quality and reliability of LLM and ML‑based products.
- Go deep into new technical areas quickly and provide clarity to engineering teams on requirements, architecture, and tradeoffs.
Qualifications
Required Qualifications:
- Bachelor’s Degree AND 5+ years experience in product/service/program management or software development
- OR equivalent experience.
Preferred Qualifications:
- Bachelor’s Degree AND 8+ years experience in product/service/program management or software development
- OR equivalent experience.
- 4+ years experience managing cross-functional and/or cross-team projects.
- Outstanding Analytical skills
- Experience in writing good detailed technical specifications for all new features
- Good Understanding of LLM, Big Data, Machine Learning, and Data Mining.
- Understanding of Security, Privacy and Compliance practices.
- Attention to detail, planning, organization, and project management skills
- Ability to quickly adapt to new technology and go deep in new focus areas.
- Excellent oral and written communication skills with the ability to collaborate effectively with a large number of partners and customers and translate requirements into a product vision and roadmap.
#MicrosoftAI #ProductManagement #ProductManager #SeniorProductManager #ProductLeadership #AIProductManagement #Search #InformationRetrieval #SearchEngine #WebSearch #DataEngineering #BigData #DataMining #DistributedSystems #DataInfrastructure #DataPlatform #Indexing #WebCrawling #ArtificialIntelligence #AI #GenerativeAI #LLM #MachineLearning #DeepLearning #NLP #AIProducts
Product Management IC4 – The typical base pay range for this role across the U.S. is USD $119,800 – $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $160,200 – $261,000 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
Similar jobs
Senior Applied Scientist
Principal Applied Scientist
Principal Applied Scientist
At Microsoft AI, we are inventing an AI Companion for everyone – an AI designed with real personality and emotional intelligence that’s always in your corner. Defined by effortless communication, extraordinary capabilities, and a new level of connection and support, we want Copilot to define the next wave of technology. This is a rare opportunity to be a part of a team crafting something that challenges everything we know about software and consumer products.
At Microsoft AI, our Health team is on a mission to help millions of users better understand and proactively manage their health and wellbeing. We’re responsible for ensuring that Microsoft AI’s models and services are useful, trusted and safe across diverse customer health journeys.
We’re on the lookout for a talented and passionate Principal Product Designer to deliver end-to-end experiences for our health AI product, crafting world-class products that empower users and promote positive health outcomes. You’ll own a key product area, driving end-to-end design from zero-to-one
Responsibilities
- Ship world-class AI applications that delight consumers with an emphasis on craft, quality, trust, and positive user and societal impact
- Own an entire product area and be responsible for understanding user needs and behaviors, defining product requirements, managing end-to-end product development, launches and iterations.
- Cross-functional communication and dependency management, scoping and scheduling, and seeing through implementation by collaborating with engineering through the finish line.
- Find your own path to get things done despite roadblocks to get your work into the hands of users quickly and iteratively.
- Set the bar for excellence and quality
- Enjoy working in a fast-paced, design-driven, product development cycle.
- Translate business goals into strategy, user experience and technical requirements in close collaboration with UX designers and AI model training teams.
- Define goals and performance indicators, set up and oversee experiments, measure success with data and research.
- Collaborate effectively and communicate clearly with cross-functional teams, including product managers, designers, and other engineers, to build exceptional consumer-grade applications.
Qualifications
Required Qualifications:
- Master’s Degree in Industrial Design, Product Design, Human Computer Interaction, User Experience, Interaction Design, or related field AND 6+ years experience working in product or service design OR Bachelor’s Degree in Industrial Design, Product Design, Human Computer Interaction, User Experience, Interaction Design, or related field AND 8+ years experience working in product or service design OR equivalent experience (e.g., demonstrated experience working in product or service design or using design thinking to solve problems).
Preferred Qualifications
- Master’s Degree in Industrial Design, Product Design, Human Computer Interaction, User Experience, Interaction Design, or related field AND 8+ years experience working in product or service design OR Bachelor’s Degree in Industrial Design, Product Design, Human Computer Interaction, User Experience, Interaction Design, or related field AND 12+ years experience working in product or service design OR equivalent experience (e.g., demonstrated experience working in product or service design or using design thinking to solve problems).
- A portfolio or work samples that demonstrate shipping world class consumer products from vision through launch.
- Ability to collaborate effectively with engineering to ensure details make it to the hands of the user.
- Experience that demonstrates proven ability to advocate for design thinking, influence design outcomes, discover problems and tackle them in fast-paced and ambiguous environments.
- Proven ability to collaborate and contribute to a positive, inclusive work environment, fostering knowledge sharing and growth within the team.
- Ability to own your work in a fast-paced. start-up like environment, manage multiple priorities, and adapt to ever-changing requirements and deadlines.
- Proven track record of setting design vision for innovative products that require bold thinking and risk taking.
- Ability to break large bodies of work into a phased execution plan, and lead a cross-functional team from idea to polished implementation.
- Experience with diving into research and leveraging both qualitative and quantitative signals for design impact.
- Experience prompting LLMs, working with generative AI, or a demonstrated curiosity and desire to learn
- Not required, but familiarity with the healthcare space is a plus!
Product Design IC5 – The typical base pay range for this role across United Kingdom is £ 93,500.00 – £ 161,800.00 per year. Certain roles may be eligible for benefits and other compensation.
Find additional benefits and pay information here:
https://careers.microsoft.com/v2/global/en/corporate-pay/united-kingdom-corporate-pay.html
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
Similar jobs
Senior Applied Scientist
Principal Applied Scientist
Principal Applied Scientist
At Microsoft AI, we are inventing an AI Companion for everyone – an AI designed with real personality and emotional intelligence that’s always in your corner. Defined by effortless communication, extraordinary capabilities, and a new level of connection and support, we want Copilot to define the next wave of technology. This is a rare opportunity to be a part of a team crafting something that challenges everything we know about software and consumer products.
At Microsoft AI, our Health team is on a mission to help millions of users better understand and proactively manage their health and wellbeing. We’re responsible for ensuring that Microsoft AI’s models and services are useful, trusted and safe across diverse customer health journeys.
We’re on the lookout for a talented and passionate Principal Product Designer to deliver end-to-end experiences for our health AI product, crafting world-class products that empower users and promote positive health outcomes. You’ll own a key product area, driving end-to-end design from zero-to-one
Responsibilities
- Ship world-class AI applications that delight consumers with an emphasis on craft, quality, trust, and positive user and societal impact
- Own an entire product area and be responsible for understanding user needs and behaviors, defining product requirements, managing end-to-end product development, launches and iterations.
- Cross-functional communication and dependency management, scoping and scheduling, and seeing through implementation by collaborating with engineering through the finish line.
- Find your own path to get things done despite roadblocks to get your work into the hands of users quickly and iteratively.
- Set the bar for excellence and quality
- Enjoy working in a fast-paced, design-driven, product development cycle.
- Translate business goals into strategy, user experience and technical requirements in close collaboration with UX designers and AI model training teams.
- Define goals and performance indicators, set up and oversee experiments, measure success with data and research.
- Collaborate effectively and communicate clearly with cross-functional teams, including product managers, designers, and other engineers, to build exceptional consumer-grade applications.
Qualifications
Required Qualifications:
- Master’s Degree in Industrial Design, Product Design, Human Computer Interaction, User Experience, Interaction Design, or related field AND 6+ years experience working in product or service design OR Bachelor’s Degree in Industrial Design, Product Design, Human Computer Interaction, User Experience, Interaction Design, or related field AND 8+ years experience working in product or service design OR equivalent experience (e.g., demonstrated experience working in product or service design or using design thinking to solve problems).
Preferred Qualifications
- Master’s Degree in Industrial Design, Product Design, Human Computer Interaction, User Experience, Interaction Design, or related field AND 8+ years experience working in product or service design OR Bachelor’s Degree in Industrial Design, Product Design, Human Computer Interaction, User Experience, Interaction Design, or related field AND 12+ years experience working in product or service design OR equivalent experience (e.g., demonstrated experience working in product or service design or using design thinking to solve problems).
- A portfolio or work samples that demonstrate shipping world class consumer products from vision through launch.
- Ability to collaborate effectively with engineering to ensure details make it to the hands of the user.
- Experience that demonstrates proven ability to advocate for design thinking, influence design outcomes, discover problems and tackle them in fast-paced and ambiguous environments.
- Proven ability to collaborate and contribute to a positive, inclusive work environment, fostering knowledge sharing and growth within the team.
- Ability to own your work in a fast-paced. start-up like environment, manage multiple priorities, and adapt to ever-changing requirements and deadlines.
- Proven track record of setting design vision for innovative products that require bold thinking and risk taking.
- Ability to break large bodies of work into a phased execution plan, and lead a cross-functional team from idea to polished implementation.
- Experience with diving into research and leveraging both qualitative and quantitative signals for design impact.
- Experience prompting LLMs, working with generative AI, or a demonstrated curiosity and desire to learn
- Not required, but familiarity with the healthcare space is a plus!
Product Design IC5 – The typical base pay range for this role across United Kingdom is £ 93,500.00 – £ 161,800.00 per year. Certain roles may be eligible for benefits and other compensation.
Find additional benefits and pay information here:
https://careers.microsoft.com/v2/global/en/corporate-pay/united-kingdom-corporate-pay.html
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
Similar jobs
Senior Applied Scientist
Principal Applied Scientist
Principal Applied Scientist
At Microsoft AI, our Health team is on a mission to build and serve medical superintelligence at global scale. We’re responsible for ensuring that Microsoft AI’s models, knowledge grounding systems and user applications are useful, trusted and safe across diverse health journeys. For this role, the Health team at Microsoft AI is looking for experienced hands-on product managers who will sit inside a cross-functional team, and who are excited about contributing to the next generation of AI systems that will transform the field.
Across all our hires, it’s important for colleagues to share our enthusiasm about the role of technology and AI in health and healthcare, but also appreciate the challenges and risks of delivering effective solutions in a complex and safety critical space. You will need to be highly self-sufficient, and be able to span from high level strategy through to ground-level execution on a wide range of tasks — on this team, no task is beneath anyone. This also means you will play a major role in cultivating and promoting a positive team culture.
Role
We are building multiple products, across the tech stack, that aim to enable the delivery of trusted medical knowledge at the point of need. That work spans the model layer, how our systems retrieve, rank and ground answers in authoritative medical evidence, and how we know when they are right — through to the applications that put that intelligence in front of users inside their real workflows.
These are zero-to-one roles: you’ll set direction for your area and stay close enough to the work to ship it, partnering daily with engineers, designers, researchers, and clinicians.
We are explicitly looking for product builders who have deep , first-hand experience of using coding agents and other AI systems to expand the reach and quality of their work.
Microsoft’s wider mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Responsibilities
- Own a product area end to end — define requirements; drive development from concept to shipped experience
- Translate strategy into user experience and technical requirements in close collaboration with UX and AI model development teams
- Shape how medical knowledge is sourced, grounded and surfaced — working with engineering and clinical partners on retrieval, source quality, provenance and the trade-offs behind them
- Break large, ambiguous problems into phased execution plans and lead a cross-functional team through them
- Manage scoping, scheduling, dependencies, and communication across engineering, design, clinical, privacy, and compliance partners
- Define goals and performance indicators, run experiments, and measure success through data, user research, and structured evaluation alongside clinical subject-matter experts — including defining what “good” looks like for a health answer
- Set the quality bar in a domain where accuracy, transparency, and trust are non-negotiable
Qualifications
Required Qualifications
- Bachelor’s or Master’s degree in Engineering, Computer Science, Mathematics, or a clinical or health-related discipline preferred
- 8+ years of experience in product management or equivalent
- Experience working with LLMs and/or generative AI, or demonstrated curiosity and desire to learn
- Demonstrated ability to operate independently in a fast-paced environment, manage multiple priorities, and adapt as the landscape shifts
- Experience shipping in regulated or high-consequence domains, treating privacy, compliance, and safety review as design constraints rather than afterthoughts
Preferred Qualifications
- Experience working in healthcare, ideally on software used by consumers, clinicians or within care delivery settings
- Experience with search, ranking, retrieval or recommendation systems, or with information quality and provenance problems
- Experience partnering with subject matter experts to define quality bars, ground truth, and evaluation methodology
Product Management IC5 – The typical base pay range for this role across United Kingdom is £ 93,500.00 – £ 161,800.00 per year. Certain roles may be eligible for benefits and other compensation.
Find additional benefits and pay information here:
https://careers.microsoft.com/v2/global/en/corporate-pay/united-kingdom-corporate-pay.html
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
Similar jobs
Senior Applied Scientist
Principal Applied Scientist
Principal Applied Scientist
Member of Technical Staff – Software Engineer, Health AI
- Location
- Job Number
- City
- Team
- Country
- Discipline
At Microsoft AI, our health team is on a mission to help millions of users better understand and proactively manage their health and wellbeing. We’re responsible for ensuring that Microsoft AI’s models and services are useful, trusted and safe across diverse customer health journeys.
We’re assembling a world-class team of builders with backgrounds in healthcare, big tech, and frontier AI research labs. Our goal is to translate cutting-edge research—like MAI-DxO (microsoft.ai/new/the-path-to-medical-superintelligence)—into transformative products for millions of users across Microsoft Copilot, our enterprise partners and beyond.
We’re looking for engineers who are self-starters, care deeply about user experience, and take pride in solving real customer needs. You’ll collaborate in a fast-paced, iterative environment to bring innovative technology to life, with safety, reliability, and scale front and centre.
We’re hiring across multiple domains: backend, web, mobile, and Applied AI. If you have a strong technical spike in one or more of these areas and a proven track record of crafting high-impact user experiences, we’d love to hear from you.
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Responsibilities
- Collaborate with AI researchers, clinicians, product managers, and designers to bring world-class AI health products to the world.
- Own the end-to-end development of features, from ideation and specification through to deployment and iteration.
- Design, build, and optimize production-grade code (with coding agents), delivering robust features within a much larger existing architecture.
- Collaborate with Applied AI & clinicians to evaluate AI features ensuring they remain effective and safe throughout their lifecycle
- Work independently across a wide range of our stack, shipping delightful user experiences.
- Ensure resilience, maintainability, and security above all else.
- Build the hiring pipelines, onboarding frameworks, or software development best practices as needed to scale an engineering team around you. Guide peers, contributing to a culture of technical excellence and continuous improvement.
Qualifications
Required Qualifications:
- Bachelor’s or Higher Degree in Computer Science, or related technical discipline AND strong software engineering experience with coding in languages/frameworks including, but not limited to, C#, C, C++, Java, Python, Rust, Typescript, Swift, Kotlin.
- Demonstrated expertise building products at scale, with domain expertise in one or more of distributed systems, cloud infrastructure, web, mobile, Applied AI.
- Experience using AI coding agents to build production software, while maintaining strong engineering judgement and high standards for safety, quality, and maintainability
- Experience collaborating in cross functional teams, working through ambiguity to deliver high quality products.
- Experience building products from 0 to 1 with a bias towards shipping and learning, while balancing a high-quality bar.
- Proven ability to collaborate and contribute to a positive, inclusive work environment, fostering knowledge sharing and growth within the team.
Preferred qualifications:
- Experience in healthcare technology, particularly with regulated medical devices or similar high-trust, safety-critical products.
- Demonstrated written and verbal communication skills with the ability to work closely with cross-functional teams, including product managers, designers, and other engineers.
- Passion for learning new technologies and staying up to date with industry trends, best practices, and emerging technologies and patterns in AI.
- Experience designing and operating agentic AI systems, including tool use, orchestration, evaluation, guardrails, observability, and failure handling.
Software Engineering IC5 – The typical base pay range for this role across United Kingdom is £ 93,500.00 – £ 161,800.00 per year. Certain roles may be eligible for benefits and other compensation.
Find additional benefits and pay information here:
https://careers.microsoft.com/v2/global/en/corporate-pay/united-kingdom-corporate-pay.html
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
Similar jobs
Senior Applied Scientist
Principal Applied Scientist
Principal Applied Scientist
Member of Technical Staff – Software Engineer, Health AI
- Location
- Job Number
- City
- Team
- Country
- Discipline
At Microsoft AI, our health team is on a mission to help millions of users better understand and proactively manage their health and wellbeing. We’re responsible for ensuring that Microsoft AI’s models and services are useful, trusted and safe across diverse customer health journeys.
We’re assembling a world-class team of builders with backgrounds in healthcare, big tech, and frontier AI research labs. Our goal is to translate cutting-edge research—like MAI-DxO (microsoft.ai/new/the-path-to-medical-superintelligence)—into transformative products for millions of users across Microsoft Copilot, our enterprise partners and beyond.
We’re looking for engineers who are self-starters, care deeply about user experience, and take pride in solving real customer needs. You’ll collaborate in a fast-paced, iterative environment to bring innovative technology to life, with safety, reliability, and scale front and centre.
We’re hiring across multiple domains: backend, web, mobile, and Applied AI. If you have a strong technical spike in one or more of these areas and a proven track record of crafting high-impact user experiences, we’d love to hear from you.
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Responsibilities
- Collaborate with AI researchers, clinicians, product managers, and designers to bring world-class AI health products to the world.
- Own the end-to-end development of features, from ideation and specification through to deployment and iteration.
- Design, build, and optimize production-grade code (with coding agents), delivering robust features within a much larger existing architecture.
- Collaborate with Applied AI & clinicians to evaluate AI features ensuring they remain effective and safe throughout their lifecycle
- Work independently across a wide range of our stack, shipping delightful user experiences.
- Ensure resilience, maintainability, and security above all else.
- Build the hiring pipelines, onboarding frameworks, or software development best practices as needed to scale an engineering team around you. Guide peers, contributing to a culture of technical excellence and continuous improvement.
Qualifications
Required Qualifications:
- Bachelor’s or Higher Degree in Computer Science, or related technical discipline AND strong software engineering experience with coding in languages/frameworks including, but not limited to, C#, C, C++, Java, Python, Rust, Typescript, Swift, Kotlin.
- Demonstrated expertise building products at scale, with domain expertise in one or more of distributed systems, cloud infrastructure, web, mobile, Applied AI.
- Experience using AI coding agents to build production software, while maintaining strong engineering judgement and high standards for safety, quality, and maintainability
- Experience collaborating in cross functional teams, working through ambiguity to deliver high quality products.
- Experience building products from 0 to 1 with a bias towards shipping and learning, while balancing a high-quality bar.
- Proven ability to collaborate and contribute to a positive, inclusive work environment, fostering knowledge sharing and growth within the team.
Preferred qualifications:
- Experience in healthcare technology, particularly with regulated medical devices or similar high-trust, safety-critical products.
- Demonstrated written and verbal communication skills with the ability to work closely with cross-functional teams, including product managers, designers, and other engineers.
- Passion for learning new technologies and staying up to date with industry trends, best practices, and emerging technologies and patterns in AI.
- Experience designing and operating agentic AI systems, including tool use, orchestration, evaluation, guardrails, observability, and failure handling.
Software Engineering IC5 – The typical base pay range for this role across United Kingdom is £ 93,500.00 – £ 161,800.00 per year. Certain roles may be eligible for benefits and other compensation.
Find additional benefits and pay information here:
https://careers.microsoft.com/v2/global/en/corporate-pay/united-kingdom-corporate-pay.html
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
Similar jobs
Senior Applied Scientist
Principal Applied Scientist
Principal Applied Scientist
Join our innovative Web Data Platform Team as a Senior Software Engineer and help us revolutionize the future of web document processing and rendering!
At Web Data Document Processing & Rendering team, you’ll play a key role in building cutting-edge batch and streaming systems as part of our platform that powers Bing Search and Microsoft AI training scenarios, empowers data scientists to seamlessly integrate their machine learning models, driving transformative insights at massive scale. Here you will be working with eye-popping scales, processing tens of billions of documents per day and managing hundreds of petabytes of data. The opportunity to make an impact on millions of users with each feature you ship is truly motivating.
In today’s AI-driven world, where data is more valuable than ever, we’re embarking on an exciting mission to elevate our platform, scaling it even further to unlock new possibilities and serve a growing array of use cases and customers. If you’re passionate about building high-performance systems that shape the future of data processing, this is the place for you!
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Starting January 26, 2026, Microsoft AI (MAI) employees who live within a 50- mile commute of a designated Microsoft office in the U.S. or 25-mile commute of a non-U.S., country-specific location are expected to work from the office at least four days per week. This expectation is subject to local law and may vary by jurisdiction.
Responsibilities
- Design and develop large-scale distributed software services and solutions following modern software engineering practices through design and code reviews.
- Collaborate with appropriate stakeholders to determine user requirements for a scenario, drive identification of dependencies and the development of design documents for a product, application, service, or platform.
- Perform system design with well-defined interfaces across multiple components, leveraging data/telemetry to inform decisions.
- Develop best-in-class engineering for our services, ensuring components are modular, secure, reliable, testable, diagnosable, maintainable, actively monitored, and reusable.
- Improve test coverage, organize and implement integration tests, and resolve problem areas.
- Troubleshoot and optimize automation, reliability, and monitoring for Live Site.
- Embody our culture and values
Qualifications
Required Qualifications:
- Bachelor’s Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
- OR equivalent experience.
Preferred Qualifications:
- Master’s Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
- OR Bachelor’s Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
- OR equivalent experience.
#MicrosoftAI
Software Engineering IC4 – The typical base pay range for this role across the U.S. is USD $119,800 – $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $160,200 – $261,000 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
Similar jobs
Senior Applied Scientist
Principal Applied Scientist
Principal Applied Scientist