AI changes & verification
A dated record of what changes in the Medi-Chi clinical AI: which model serves it, which deterministic rules constrain it, what goes into the libraries it reads from, and which citations have been re-checked — including how many were wrong and what was done about them.
Maintained by Derek Doran (AHPRA CMR0002211465), who is responsible for the clinical content of this platform. This page is self-published and is not an independent audit; where no external verification exists, we say so. Last entry: 3 August 2026.
- Citation re-check
Named-source registry behind every point and herb recommendation
Source strings shown with diagnoses, acupuncture points and herbal references are now resolved to a catalogued work with an author, date and public link, so each claim can be traced rather than taken on trust.
- Added a registry of 21 named works — classical texts (Sù Wèn, Líng Shū, Nàn Jīng, Shāng Hán Lùn, Jīn Guì Yào Lüè, Shén Nóng Běn Cǎo Jīng, Zhēn Jiǔ Dà Chéng and others), plus the WHO acupuncture benchmarks and WHO Standard Acupuncture Point Locations.
- Citations render as clickable links that open the work's title, author, date, a one-line description of what it is, and a link to a public copy.
- Source text that names no catalogued work stays plain and non-clickable rather than implying verification that has not happened.
- An automated test asserts that every point and herb source string in the reasoning data resolves to a real registry entry, so a new unverifiable source fails the build.
- Citation re-check
Full PubMed citation audit across articles and clinical content
Every PubMed identifier used anywhere on the site was re-checked against the live PubMed record to remove dead and mismatched IDs.
- Built a repeatable verification script that resolves each PMID against PubMed and compares the returned record with the citation as written.
- The check now runs in continuous integration, so a citation that stops resolving is caught rather than left on the page.
Citations re-checked
- Checked
- 191
- Corrected
- 47
- Scope
- PubMed identifiers across blog articles and disease reference files
Method: scripts/verify-pubmed-citations.ts, resolved against the live PubMed record
Outcome: 43 citations in the article library and 4 in disease files were corrected or replaced; none were left pointing at a dead identifier.
- Citation re-check
Automated retraction monitoring on cited literature
Cited papers are re-checked on a schedule for retraction and expression-of-concern notices, and flagged references are replaced rather than quietly left in place.
- A scheduled job audits the cited literature and reports anything carrying a retraction notice.
- Flagged citations are reviewed by the responsible clinician before a replacement is published.
- Retrieval library
Source-link integrity checking across the condition library
Every external source URL in the condition library is checked for reachability, so guideline and regulator links do not rot silently.
- Condition source links are validated in batches with a per-request timeout, and failures are surfaced for review.
- Citation numbering in condition files is validated at build time: a superscript that points at a missing source fails the build.
- Rules & logic
Classical-source reasoning made mandatory in AI output
The diagnosis prompt and post-processing rules now require pattern reasoning to name the classical text it draws on, instead of asserting a pattern without provenance.
- Pattern reasoning must cite a named classical source (for example Sù Wèn, Líng Shū or Shāng Hán Lùn).
- Pīnyīn, Chinese characters and standard point nomenclature are preserved verbatim through translation and export rather than paraphrased.
- The app deliberately does not display confidence percentages, which would imply a statistical accuracy that has not been measured.
- Safety engine
Deterministic safety engine in front of AI output
Risky output is blocked or flagged by rules that run outside the model, so safety does not depend on the model behaving well.
- A rules-based engine screens for red-flag presentations and routes users to an urgent-care screen with an explanation.
- Restricted substances (including Má Huáng, Fù Zǐ, Cǎo Wū and Chuān Wū) are blocked from herbal output for regulatory compliance.
- Safety acknowledgement is required before clinical tooling output can be exported or printed.
- Model
Current model: Gemini Flash via the Lovable AI Gateway
Clinical text generation runs on Gemini Flash through the Lovable AI Gateway. The model is used off-the-shelf — it is not fine-tuned on clinical data, and user data is not used for training.
- Retrieval over curated in-app libraries, not model fine-tuning: the model is given vetted material to work from rather than being trained on it.
- No user-submitted case data, tongue photographs or clinical notes are sent for model training.
- Model output is constrained by the safety and citation rules listed elsewhere in this log; the model is not the last word before publication.
What this log does not claim. Medi-Chi has not been independently audited, is not a registered medical device, and has not been evaluated in a clinical trial. Nothing here should be read as certification.
Spotted a citation that does not check out? Report an issue and it will be corrected and logged here. For the full method, see How our AI works and Trust & Security.