[Image created with OpenAI’s DALL·E]
AI in Healthcare: Why We Still Need a Human at the Controls
AI is rolling full steam into healthcare. From generative AI tools being studied for mental-health treatment, to models assessing cancer risk years in advance, to machine learning accelerating drug discovery, developments that once sounded like science fiction are becoming increasingly real.
But here’s the question we should be asking: Is AI really ready to drive?
Where AI Is Already Making an Impact
One of the most interesting examples comes from Dartmouth, where researchers at the Geisel School of Medicine led the first clinical trial of a generative AI-powered therapy chatbot.
The system, called Therabot, was evaluated with participants diagnosed with major depressive disorder, generalized anxiety disorder, or an eating disorder. Researchers reported significant reductions in symptoms among participants who used it. Users also reported levels of trust and communication with the system that researchers described as comparable to working with a mental-health professional.
But the researchers were equally clear about the limitations. The results were promising, not permission to remove clinicians from the equation. The research team emphasized that generative AI was not ready to operate autonomously in mental-health treatment, where high-risk situations require careful safeguards and human oversight. Read more about Dartmouth’s Therabot research.
Other digital mental-health tools were also gaining attention. Wysa, for example, received FDA Breakthrough Device designation for an AI-led conversational agent studied in connection with chronic pain and associated depression and anxiety.
Beyond mental health, AI was already pushing boundaries in diagnostics, risk assessment, and drug discovery.
- MIT’s Jameel Clinic has supported AI-driven drug-discovery research, including work aimed at identifying new antibiotic candidates.
- Mirai, developed through MIT research, uses mammograms to assess a patient’s breast-cancer risk up to five years in advance.
- Sybil, developed by researchers at MIT and Mass General Brigham, analyzes low-dose CT scans to predict lung-cancer risk up to six years in advance.
These aren’t examples of AI replacing medicine. They’re examples of AI expanding what clinicians and researchers may be able to see, analyze, and discover. Explore the MIT Jameel Clinic’s work in AI and health.
The Catch: AI Isn’t Always Trustworthy
Here’s the part we can’t gloss over: AI systems have limitations, and those limitations become especially important when health decisions are involved.
- Generative AI can produce inaccurate information. A response can sound confident and convincing while still being wrong.
- AI doesn’t experience empathy. A system can generate language that sounds compassionate, but generating an empathetic response isn’t the same as experiencing or understanding another person’s emotions.
- Not every health-related AI tool has undergone the same level of clinical evaluation. A research system developed and tested with clinicians is very different from a general-purpose chatbot being used informally for medical or mental-health advice.
- AI can inherit limitations from its data and design. Bias, incomplete data, lack of transparency, privacy concerns, and poor implementation can all affect how safely a system performs.
That’s why the distinction between AI capability and responsible AI implementation is so important in healthcare.
Why Humans Still Need to Be in the Driver’s Seat
I like to think of AI as the train engine: powerful, fast, and capable of taking us places we couldn’t reach otherwise.
But the engine isn’t the entire system.
Healthcare involves context, accountability, judgment, ethics, trust, privacy, and human relationships. A model can identify patterns in enormous amounts of data. It can help researchers discover possibilities they might not otherwise see. It can potentially extend access to information and support.
But knowing what to do with that information is another question.
That’s why human oversight is essential. AI can support clinicians, researchers, and healthcare organizations, but people still need to evaluate its output, understand its limitations, establish safeguards, and remain accountable for how it is used.
The most promising future isn’t one where humans simply hand over the controls. It’s one where technology expands human capability while people remain responsible for the decisions surrounding it.
Innovation Requires Trust
There’s another reason human oversight is so important: healthcare technology only creates value when people can trust the systems they’re being asked to use.
That trust can’t come from innovation alone.
Patients need to understand when AI is being used. Clinicians need enough information to evaluate its recommendations. Organizations need clear processes for privacy, security, governance, accessibility, and accountability. Researchers need rigorous methods for determining whether a tool actually works for the populations it is intended to serve.
Moving quickly may be valuable in technology. In healthcare, moving responsibly is just as important.
The goal shouldn’t simply be to build AI that can do more. It should be to build and use AI in ways people can reasonably trust.
My Takeaway
AI in healthcare is full of promise. It is already changing research, risk assessment, drug discovery, and the way we think about access to certain kinds of support.
But AI isn’t magic, and innovation doesn’t eliminate the need for judgment.
The future of healthcare doesn’t need to be framed as AI versus humans.
The more useful question is how AI and humans can work together safely, responsibly, and effectively.
AI can analyze. It can predict. It can generate. It can identify patterns at a scale humans can’t.
Humans still have to decide how those capabilities should be used.