Big Hospitals Are Vetting Their AI. Your Practice Should Too.

A physical therapy clinic called us on a Monday, a little rattled. They had turned on an AI chatbot from some cheap vendor a month earlier and left it running on their website. Nobody had looked at it since. Then a patient forwarded a screenshot. The bot had confidently told her the clinic accepted her insurance and quoted a copay. Both were wrong. She showed up, got a bill she did not expect, and left a one star review about it. The tool had been making things up to sound helpful, and no human had heard a word of it for weeks.

Nobody at that clinic was careless. They did what a lot of busy practices do. They bought an AI tool, switched it on, and trusted it to behave. The problem is that trusting AI to behave, without ever checking, is exactly the thing the biggest names in medicine have decided they will not do.

What just happened at the top of the food chain

This week, Mount Sinai Health System and Denver Health joined PACT AI, a coalition that brings large organizations together with independent assurance firms whose entire job is to verify that AI systems actually do what they claim to do. Reported by Becker's Hospital Review and Modern Healthcare, the move puts these systems alongside outside auditors who test AI for bias, accuracy, and safety rather than taking a vendor's word for it.

It is not the only sign of the shift. The Coalition for Health AI, a separate group of clinicians and health systems, has published a whole blueprint for trustworthy healthcare AI, and it has since joined forces with the Joint Commission to turn those ideas into real standards. The message from the top of the industry is clear and a little humbling: nobody who is serious about AI in healthcare is just plugging it in and hoping.

So why are so many small practices doing exactly that?

The gap between a hospital and your front desk

Mount Sinai has a governance committee, a research center, and now a coalition of auditors. Your practice has a front desk person named Dana who is already covering two roles. You are never going to match that, and you do not need to. But the instinct behind all that hospital machinery translates perfectly to a three provider clinic, and it fits on a sticky note: never let AI talk to a patient without a way to check what it said.

The good news is that the AI a small practice actually touches is far simpler than a hospital's clinical models. You are not deploying an algorithm that reads scans or recommends chemo. You are using AI to answer the phone after hours, respond to website chats, book appointments, send reminders, or draft replies to reviews. Lower stakes, yes. But these tools speak to your patients in your name, and a wrong answer here still costs you a booking, a review, or trust. Responsible AI at your level is not complicated. It is just a handful of habits most practices skip.

The five questions that replace an audit team

You cannot hire an assurance firm. You can ask the same questions they would, before you sign anything. If a vendor gets cagey on any of these, walk. We wrote a fuller version of this in our guide on what to check before you sign an AI vendor, but here is the short list.

• What data does it touch, and is it covered? If the AI handles any patient information, you need a signed business associate agreement and a straight answer on HIPAA. No BAA, no deal.
• Can I read what it said? You must be able to pull transcripts of the calls and chats it handled. A tool you cannot review is a tool you cannot trust.
• What happens when it does not know? The right answer is that it hands off to a human. The dangerous answer is that it guesses to sound helpful, which is exactly what burned that PT clinic.
• Who is accountable if it is wrong? If the AI tells a patient the wrong thing, you want a vendor who owns it and fixes it, not one who points at the fine print.
• Can I shut it off fast? You should be able to pause it in minutes, not file a support ticket and wait three days.

Set it and forget it is the actual danger

Here is the part almost nobody does, and it is the cheapest safeguard there is. Listen to your AI. In the first few weeks after you turn on any AI phone or chat tool, read or listen to a sample of what it actually handled every single week. Not all of it. Ten calls. Twenty chats. You are checking three things: did it get the facts right, did it sound like your practice, and did it know when to get out of the way and pass someone to a human.

This is the small practice version of what Mount Sinai is paying auditors to do at scale. They call it monitoring. You can call it Friday afternoon with a coffee. The clinics that do this catch the wrong copay in week one instead of week six, when it is a single fix instead of a bad review. The ones that skip it are the ones who call us rattled on a Monday.

Tell patients. It builds trust, it does not break it

A lot of practice owners worry that admitting an AI answered the phone will feel cold or cheap. The opposite is true. Patients are fine talking to a virtual assistant that says up front what it is and offers a human anytime. What they cannot stand is feeling tricked into thinking a robot was a person. A simple line does the whole job: you are chatting with our virtual assistant, just say human and I will connect you.

This is not only good manners, it is where the rules are heading. A number of states are moving toward requiring practices to disclose AI use in patient interactions, and honesty about it is becoming a genuine trust signal in its own right. We dug into this in our piece on why AI transparency builds patient trust. The practices that get ahead of it look modern and confident. The ones that hide it look like they have something to hide.

How we think about it at EtherealMinds

We build and run an AI receptionist for healthcare practices, so we have skin in this game, and it is exactly why we are strict about it. Our AI receptionist is set up the way this whole article argues it should be. It works from rules your practice approves, it identifies itself to patients, it hands off to a human the moment a call is urgent or outside what it should handle, and every conversation is logged so you and we can review it. We monitor it, not just install it. That is the difference between AI that grows your practice and AI that embarrasses it while you are not listening.

We wire the same thinking into the rest of what we build. When AI drafts a social post or answers a form on one of our websites built to convert patients, a human still owns the final word. Automation should carry the boring load. It should never carry the whole responsibility. If you want the wider view on where AI fits for a smaller clinic, we laid it out in what small practices should actually do about AI.

The real lesson from the big systems

When Mount Sinai and Denver Health join a coalition to vet their AI, it is easy to read it as a story about hospitals with money to burn. It is not. It is a story about the most sophisticated players in healthcare deciding that trusting AI blindly is the one thing they will not risk. You do not need their budget to borrow their judgment. Ask the hard questions before you buy. Listen to what your AI tells your patients. Keep a human in the loop and tell people the truth. That is responsible AI for a practice your size, and it is the difference between a tool that works for you and one that goes rogue while you are not listening.