The promise of artificial intelligence in healthcare often arrives wrapped in sleek marketing language about efficiency, precision and transformation. But for many physicians, the memory of electronic health record rollouts – and the endless clicking that came with them – still lingers like a warning.Dr. Niki Panich believes the next wave of technology cannot afford to repeat those mistakes.Panich, chief medical officer at Penguin AI and a practicing family physician, is pushing for what she calls "glass box traceability" – a system in which AI-generated recommendations expose the reasoning behind them instead of operating as mysterious black boxes.If hospitals and health systems want physicians to use AI in clinical practice, the physicians need to understand how an AI system arrived at its recommendation, Panich said.Pulling back the curtainIn Panich's view, transparency at the bedside has to go far beyond a confidence score hidden deep inside software menus."'Showing the work' at the point of care has to be more than a citation or a confidence score buried three clicks deep," she said. "For a physician moving between patients, transparency means seeing, at a glance, what the AI considered, what it weighted most heavily, and what it set aside."She said clinicians should be able to see the specific chart details, evidence and clinical guidelines that influenced a recommendation. If an AI tool flags a patient as being at high risk for readmission, for example, physicians should know exactly why."The standard to aim for is a colleague walking you through their thinking the way a trusted partner would," she said.Panich described a scenario in which an AI-generated discharge recommendation suggested placing a patient on a new anticoagulant medication. Embedded in the system's reasoning was a nursing note documenting that the patient had recently fallen at home."The physician hadn't gotten to that note yet," she said. "Without the visibility, she might have signed off and moved on. With it, she pauses, calls the family, and rethinks the plan entirely."The human factorPanich argues that medicine rarely fits neatly inside textbook answers. The right treatment can shift because of fears, family responsibilities or cultural realities that may never appear in a dataset."The injectable example is exactly right, and it is the kind of detail that rarely shows up in any guideline or training set," she said. "The patient who is terrified of needles. The grandmother who is the primary caregiver for her husband and cannot afford to be sedated for a procedure on Tuesday."That is why she believes AI systems should present options rather than dictate care paths."AI should surface options, present context and leave room for alternatives at every step," she said.Just as important, she said, healthcare organizations must make clear that physicians remain responsible for the final decision."Health systems have to make clear, in policy and in practice, that AI recommendations are inputs to a clinical decision the physician owns," Panich said.She believes that balance allows AI to handle the exhausting work of sorting through information while physicians focus on the patient sitting in front of them.Earning trust after the EHR eraFor AI to become a routine part of medicine, Panich said the industry must confront concerns about bias, oversight and safety directly."A wrong recommendation in medicine can cost a person their life," she said.She called for broader and more representative training data, ongoing bias audits and clear reporting about where models perform well – and where they do not."It means transparent reporting on where models perform well and where they fall short," she said.But technology alone will not win over skeptical physicians, especially those who still associate healthcare innovation with administrative overload."The pain of EHR rollouts came from being asked to do more documentation, more clicking, more administrative work, all under the banner of progress," Panich said.Earning credibilityInstead, she believes AI will earn credibility by tackling some of the least glamorous but most draining parts of medicine: prior authorizations, chart reviews, insurance denials and appeals."When AI proves itself first by removing administrative burden, by making the day shorter and the work lighter in ways clinicians can feel, the trust to lean on it for clinical decision support follows naturally," she said.For Panich, the future of AI in medicine depends less on replacing doctors than on building tools physicians can challenge, question and trust."You earn that trust with your sleeves rolled up, doing the unglamorous work of taking weight off the people closest to patient care," she said.Follow Bill's health IT coverage on LinkedIn: Bill SiwickiEmail him: [email protected]Healthcare IT News is a HIMSS Media publication.WATCH NOW: The cost savings of SDOH