AI Analyst
Hello Heart
About Hello Heart:
Hello Heart is on a mission to change the way people care for their hearts. The company provides the first app and connected heart monitor to help people track and manage their heart health. With Hello Heart, users take steps to control their risk of heart attacks and stroke – the leading cause of death in the United States. Peer-reviewed studies have shown that high-risk users of Hello Heart have seen meaningful drops in blood pressure, cholesterol and even weight. Recognized as the digital leader in preventive heart health, Hello Heart is trusted by more than 130 leading Fortune 500 and government employers, national health plans, and labor organizations. Founded in 2013, Hello Heart has raised more than $138 million from top venture firms and is a best-in-class solution on the American Heart Association’s Innovators’ Network and CVS Health Point Solutions Management platform. Visit www.helloheart.com for more information.
About the Role:
Hello Heart is seeking a talented and motivated AI Analyst to join our Clinical Data Science team. As an AI Analyst, you will be responsible for conducting in-depth analysis of our data science products, from research to production. Your work will directly contribute to improving healthcare practices and enhancing the well-being of individuals.
Responsibilities:
- Monitor and analyze the performance of production ML models & AI agents, with a strong focus on our LLM-based conversational agents.
- Develop and maintain comprehensive monitoring dashboards (e.g., in Tableau) to track key model performance metrics, data drift, and user interactions in real-time.
- Dive deep into large-scale conversational data, user inputs, and LLM outputs to identify key patterns, trends, and failure points.
- Generate actionable insights and research leads from your analysis to guide improvements in model accuracy, prompt engineering, and agent logic.
- Design, run, and analyze A/B tests to rigorously evaluate the impact of new models, prompts, and conversational flows on user behavior and clinical outcomes.
- Collaborate closely with data scientists, engineers, and product managers to translate analytical findings into tangible improvements for our AI-driven solutions.
Qualifications:
- 3+ years of hands-on experience
- Bachelor’s degree in statistics, computer science, engineering or another related field.
- Strong analytical and critical thinking skills, with the ability to interpret complex data and generate meaningful insights.
- Proficiency in SQL
- Proficiency in Python with Pandas
- Hands-on experience working programmatically with Large Language Models (LLMs), including running experiments, applying prompt engineering techniques, and handling structured outputs.
- Proficiency in Tableau or equivalent and data visualization
- Familiarity with research methodologies, experimental design, and statistical modeling techniques (bayesian methods, propensity score matching, etc).
Advantage for candidates with:
- Experience in healthcare data analysis, including Electronic Medical Records (EMR) or clinical data.
- Experience in conducting AB tests for ML models in production, modeling the causal effects of treatment interventions.
- Experience with dbt (data build tool) or similar data transformation tools.
Hello Heart has a positive, diverse, and supportive culture - we look for people who are collaborative, creative, and courageous. Oh, and if you want to see some recent evidence of the fun things we do at Hello Heart, check out our Instagram page.