Medical Artificial Intelligence · asia
AI improves accuracy in identifying rare kidney cancer, but physicians also become more likely to accept incorrect advice
A pathology interpretation study involving 21 physicians shows AI’s potential to help identify rare kidney cancer, while revealing the cost of trust: those who benefit more from correct advice may also be more likely to follow incorrect advice.
Diagnosing rare cancers sometimes hinges on whether physicians consider them as a possibility. If tissue resembles that of a common cancer, the additional tests needed to establish the diagnosis may never be initiated. A study announced by Asan Medical Center in Seoul, South Korea, shows that AI can help physicians identify this type of easily overlooked kidney cancer. However, physicians who are more willing to accept its judgments are also more susceptible to incorrect advice.
The hospital’s English-language announcement on November 19 described a study already published in *npj Digital Medicine*, rather than a new paper published that day. The Korean-language announcement had been released on November 12, while *Digital Chosun Ilbo* reported that the paper went online on August 13. Led by Namkug Kim and Yong Mee Cho, the study aimed to help screen for fumarate hydratase-deficient renal cell carcinoma (FHdRCC), identifying cases requiring further testing.
According to the hospital, FHdRCC accounts for less than 0.4% of kidney cancers and progresses rapidly, yet its tissue morphology can resemble that of other kidney cancers. Confirming the diagnosis requires additional tests, including genetic testing. Among the 39 patients with this cancer included in the study, only 36% were initially suspected of having it. This illustrates the gap the model seeks to address: bringing a rare possibility into diagnostic consideration in the first place.
The team used 335 digital pathology images to develop and evaluate the model. The area under the curve (AUC), a measure of the ability to distinguish the disease, was 0.954 in internal validation and 0.820 and 0.867 in two external validation cohorts, respectively. Values closer to 1 indicate better discrimination. The lower external performance also highlights that the model’s capabilities vary across data sources, and these figures cannot be treated directly as clinical diagnostic accuracy rates.
Whether AI helped physicians in practice was examined in a separate interpretation experiment. According to *Digital Chosun Ilbo*, participants included seven genitourinary pathology experts, seven pathologists specializing in other fields, and seven residents. Two weeks apart, they interpreted the same set of 30 difficult cases, first without AI and then with AI assistance. Of these, 15 were FHdRCC and 15 were other kidney cancers.
With AI, physicians’ diagnostic accuracy rose from 60.0% to 73.3%, an increase of 13.3 percentage points. Agreement among physicians also improved, while interpretation time did not increase significantly. However, the overall improvement masked differences in how physicians used AI. An exploratory analysis divided physicians into 14 who more readily accepted AI advice and seven who were more cautious. The former benefited more from correct advice but were also more likely to accept incorrect advice.
The median scores on an indicator of overreliance on incorrect AI advice were 0.800 and 0.600 in the two groups, respectively. This reflects the risk of automation bias: once a system provides an answer, people may adjust their judgments to follow it. The study found no significant association between physicians’ tendency to accept AI advice and their years of experience or subspecialty. It therefore suggested that diagnostic expertise and the ability to use AI effectively may be distinct skills, and that training should also address how to recognize and question incorrect advice.
These findings still have clear limits to their applicability. The interpretation experiment involved only 21 physicians from a single institution and 30 cases, with just five cases in which AI was wrong. The behavioral grouping was also exploratory. Rare cancer accounted for half of the cases, far removed from its low prevalence in routine clinical practice. The current evidence supports AI’s potential as an aid in a controlled interpretation setting; it has yet to demonstrate increased rates of actual additional testing, earlier diagnosis, or improved survival. Whether it can balance alerting physicians with avoiding misleading them in clinical practice still requires larger-scale validation.