Zhejiang University has introduced OmniPT, an AI-powered universal pathology assistant combining vision and language models for seamless human-computer interaction. This advanced tool is deployed at the pathology department of the First Affiliated Hospital of Zhejiang University School of Medicine in Hangzhou, targeting prevalent cancers such as gastric, colorectal, and cervical. The hospital is the first clinical institution in China to integrate AI in pathology diagnostics.
OmniPT has demonstrated breakthroughs in cancer classification, grading, and the identification of vascular and neural invasion. It also predicts disease progression with 80 to 90 percent accuracy, according to hospital officials.
Pathology involves meticulous examination of tissue samples to diagnose diseases, but patients and families often remain unaware of this critical process. China’s pathology sector faces acute workforce shortages, with only 30,000 registered professionals against a demand of 150,000 to 200,000, said Zhang Jing, vice-president of the hospital’s Yuhang branch and chair of its pathology department.
"The shortage is especially severe in remote areas, despite being less critical in cities like Beijing, Shanghai, and Hangzhou," Zhang explained. He noted that the long training period further exacerbates the gap, with young pathologists often lacking experience.
In this context, OmniPT is transforming clinical diagnosis. Developed by Professor Song Mingli’s team from Zhejiang University’s College of Computer Science and Technology, the system reduces diagnostic time and improves accuracy by handling repetitive tasks.
"It acts as an assistant, enabling pathologists to focus on critical judgments," Zhang said, highlighting OmniPT’s efficiency in tasks like mitosis counting, vital for diagnosing gliomas. "While manual counting under a microscope can take up to an hour, OmniPT completes it in less than 10 seconds. It flags uncertain findings for the pathologist’s review."
The tool’s computational capabilities also enhance precision and detect details often overlooked by fatigued doctors. OmniPT automates over 90 percent of repetitive tasks, streamlining workflows and addressing professional shortages.
"We control the AI, not the other way around," Zhang emphasized. "It enables us to solve complex pathology challenges, especially in underserved areas or for less experienced practitioners. This significantly improves efficiency, reduces costs, and minimizes errors."