Artificial intelligence is reshaping healthcare at a pace that outstrips oversight, raising urgent concerns about how to deploy it without deepening existing risks. In medicine, AI promises to ease clinician workloads and speed drug discovery, but those benefits hinge on addressing ethical gaps before they become systemic failures.
Experts warn that the rush to integrate AI into care could worsen disparities if biases in training data go unchecked. The technology’s potential to improve outcomes depends on three core questions: accuracy across diverse populations, protection of patient privacy, and clear accountability for failures.
Accuracy and Equity: The Hidden Bias Risk
Clinicians point to a fundamental flaw in AI development: most tools are built on historical medical data that already reflects systemic biases. Sanjay Juneja, a hematologist and AI specialist at Citrus Oncology, notes that there is no reason to think AI will be spared from the biases already embedded in medicine.
Juneja cites studies showing racial and socioeconomic disparities in diagnostic accuracy, where AI models trained on skewed datasets perform poorly for underrepresented groups. Without proactive testing across demographics, similar errors could become permanent features of clinical decision-making.
The problem extends beyond race. Rural patients, those with rare conditions, and non-English speakers often lack representation in training data, creating blind spots in AI recommendations. Juneja argues that equity is not an afterthought—it must be the starting point. Every day an AI tool operates with unaddressed bias is another day of compounded injustice, he says.
Privacy and Transparency: The Unseen Threat
Beyond bias, AI in healthcare introduces new vulnerabilities for patient data. Dr. Adirim highlights a growing threat: AI systems capable of autonomous cyberattacks. This year alone, reports have documented AI agents planning and executing breaches, a risk that becomes catastrophic when applied to interconnected hospital systems holding life-critical data.
A compromised AI tool in a clinic does not just expose records—it can manipulate treatment protocols, alter prescription dosages, or even disable monitoring equipment. The stakes are not just about privacy, Adirim says. They are about patient safety. She points to the lack of standardized disclosure practices: many hospitals deploy AI without informing patients whether decisions are human-driven or algorithmic.
Juneja emphasizes that transparency is not optional—it is the foundation of trust. If you are using AI to recommend a treatment, the patient deserves to know, he says. Ethical adoption means being upfront about when and how these tools influence care. Yet current regulations offer little guidance on how to document AI use or audit its decisions after deployment.
The gap between promise and reality is widening. While AI vendors tout breakthroughs, independent evaluations remain rare. Adirim’s recent research found that federal oversight lacks clear standards for validating AI equity, monitoring post-deployment performance, or assigning responsibility when systems fail.
State-level efforts vary widely, with some requiring AI bias audits and others imposing no requirements at all.
Accountability: The Missing Command Structure
The third ethical question, who is responsible when AI fails, has no clear answer. Juneja draws a parallel to the infamous Apocalypse Now scene where Captain Willard demands to know who’s in charge, only to be met with silence. In AI governance, they are still at that bridge, he says. Every day an AI tool operates without a named accountability officer is another day of unchecked risk.
Adirim’s coauthored paper identifies structural gaps at every level: federal agencies lack unified AI oversight, state laws conflict, and local hospitals often rely on vendor assurances rather than independent testing. The result is a patchwork where accountability evaporates when failures occur. If an AI misdiagnoses a patient, who investigates, Adirim asks. Who compensates them? Right now, the answer is often no one.
The federal government has taken limited steps. Juneja acknowledges the challenge: balancing innovation with safeguards requires a careful approach that encourages progress while insulating patients from harm.
Yet the pressure to deploy AI quickly persists. Hospitals adopt untested tools to cut costs, insurers use algorithms to deny claims without human review, and pharma companies race to commercialize AI-driven therapies before safety data is complete. The ethical framework exists in theory; but in practice, it is often ignored until after the damage is done.
Given the risks, experts advocate for a more cautious approach, emphasizing the need for robust testing and validation across diverse populations. Dr. Adirim suggests that healthcare organizations should prioritize independent evaluations of AI tools, rather than relying on vendor assurances.
The importance of transparency in AI use cannot be overstated, as it directly impacts patient trust and safety. Clear disclosure practices are essential, enabling patients to understand when and how AI influences their care. This transparency also extends to cybersecurity, where the threat of autonomous cyberattacks demands clear standards and regulations.
Implementation and Governance
Effective AI governance requires a structured approach, with defined accountability and meaningful human oversight. Dr. Juneja stresses the importance of having a plan in place, with named individuals responsible for specific aspects of the workflow. This approach helps mitigate the risk of errors and ensures that issues are addressed promptly.
Dr. Adirim’s coauthored paper highlights the need for updated regulations that reflect the AI era, including standards for validation, equity, and transparency. The federal government must invest in the capacity of healthcare organizations to evaluate AI independently, rather than relying on vendor assurances. By doing so, the evaluation of AI tools can be accelerated, and the benefits of AI can be realized while minimizing the risks.
Dr. Juneja reported a consulting relationship with Incyte Corporation, while Dr. Adirim reported no conflicts of interest, according to the disclosures.
