A cross-sectional study measured AI-generated type 2 diabetes information on quality, readability, and transparency — and found variance patients can’t detect. ‘The AI told me’ can’t be corrected because there’s no source. When a clinic owns its vetted content and an assistant answers only from it, all three axes become controllable.
A general chatbot’s answer reads exactly as authoritative when it’s right and when it’s wrong.
A patient with type 2 diabetes asks an AI chatbot how to manage their diet. The answer that comes back might be excellent, or it might be subtly wrong, written at a reading level they can’t follow, and impossible to trace to any source. A new cross-sectional study set out to measure exactly that — the quality, readability, and transparency of AI-generated health information — and the results are a useful warning for any clinic thinking about how its patients get educated online.
The three things that were measured
The study assessed online and AI-generated information about type 2 diabetes on three axes, and each one maps to a real risk:
- Quality — is the information actually correct and complete? Wrong guidance about a chronic condition compounds over years.
- Readability — can a normal person understand it? Health information written above a patient’s reading level is functionally useless, however accurate.
- Transparency — can you tell where it came from and whether it’s trustworthy? An answer with no traceable source can’t be verified, corrected, or defended.
These aren’t academic niceties. For a chronic condition like diabetes — managed largely by the patient, at home, for the rest of their life — the quality of everyday information is a genuine determinant of outcomes. And the study’s framing makes clear that AI-generated material varies on all three, which is precisely the problem: variance, in a domain where consistency is the point.
The transparency gap is the quiet danger
Of the three, transparency is the one most people underestimate. A general AI chatbot produces a fluent, confident paragraph about managing blood sugar — and gives you no way to know whether it’s drawn from a clinical guideline, a decade-old forum post, or a plausible-sounding blend of both. It reads exactly as authoritative when it’s right and when it’s wrong.
For a clinician, that’s the nightmare. You can correct a patient who cites a specific bad website. You cannot correct a patient who says “the AI told me,” because there’s no source to examine, no author to weigh, nothing to point at. The information has authority without accountability. And a chronic-disease patient makes small self-management decisions daily, each one nudged by whatever they read last.
The pattern behind this and the chatbot studies
This sits alongside the broader finding that the public is already pouring health questions into general assistants. The common thread is the same: general-purpose AI produces health information of uncontrolled quality, unknown readability, and no transparency — and patients can’t tell the difference. The tool is fluent enough to be trusted and generic enough to be wrong.
The constructive question for a clinic isn’t whether patients should use AI for health information. They already do. It’s whether the information they get can be made to score well on exactly the three axes this study measured — quality, readability, and transparency — instead of being left to chance.
Where owned infrastructure changes the equation
This is where a self-hosted, content-owned approach becomes relevant — and, in a medical context, precision matters, so let’s be exact. Platforms like VBWD are infrastructure, not medicine: self-hosted, source-available software for running your own content and assistant, not a clinical or diagnostic system. But that infrastructure maps onto the study’s three axes in a way a general chatbot structurally cannot.
Quality becomes controllable. When a clinic owns its patient-education content — in its own content system, reviewed by its own clinicians — and an assistant answers only from that vetted material rather than the open internet, quality stops being a lottery and becomes an editorial responsibility the organisation actually holds.
Readability becomes a choice. You control the source text, so you write it at the reading level your patients need, in the languages they speak. A grounded assistant then draws on material you’ve already made readable, instead of generating prose at whatever level the model defaults to.
Transparency becomes possible. This is the decisive one. When answers come from a defined corpus you maintain, you can say where information came from, keep it current, and stand behind it. “The assistant told me” becomes “our clinic’s vetted guidance says,” with a source a clinician can point to, examine, and correct. Authority regains its accountability.
The boundary, stated plainly
None of this makes software into a clinician, and in a medical context that caveat is not boilerplate — it’s the whole safety case. A grounded assistant serving your own vetted diabetes-education content is a patient-information tool, and a good one. It is not a diagnostic device, it does not replace the consultation, and it must never be positioned as clinical advice for an individual. Owning the content and the infrastructure improves quality, readability, and transparency; it does not turn education into diagnosis, and the study is a reminder of why that line matters.
The takeaway
This research measured what everyone half-knew: AI health information is a mixed bag of uncontrolled quality, uneven readability, and near-zero transparency — and patients can’t tell the good from the bad. A clinic can’t fix the general internet, but it can offer patients something better on its own turf: vetted content it controls, written to be understood, served by an assistant that answers only from sources the clinic can name and stand behind. In a chronic-disease world run largely by patients at home, that’s not a small improvement. It’s the difference between information you can trust and information that merely sounds like it.
General information for healthcare and technology decision-makers, not medical, legal, or regulatory advice. AI patient-information tools are not a substitute for professional care; any clinical deployment requires validation, governance, and compliance review appropriate to the jurisdiction.
