Legacy AI Triumphs: A 10-Year-Old CNN Outperforms Modern LLMs in Dermatology

By Dr. Magnus Lynch | October 25, 2026 | ⏱️ 5 min read

As a practicing dermatologist, I recently conducted a comparative study to evaluate the diagnostic capabilities of the latest "frontier" Large Language Models (LLMs)—specifically GPT-6 Astra and Claude Fable 5.1. To my surprise, these cutting-edge systems were significantly less accurate than a decade-old architecture.

"A ten-year-old network, fine-tuned for roughly twenty minutes, outperformed the most advanced AI models of 2026 at identifying skin cancer."

---

🔬 The Experimental Setup

The test utilized the MILK10k dataset (released via the International Skin Imaging Collaboration in 2025). Each case provided two perspectives:

1. A standard clinical close-up photograph.
2. A dermoscopic image (captured via an illuminated magnifying lens).

Test Sample Composition

I randomly selected 200 lesions, all of which had been confirmed via histopathology. The difficulty was elevated because every lesion—even the benign ones—had been biopsied, meaning they were clinically ambiguous enough to worry a human doctor.

The 200-lesion breakdown:

• 100 High-Risk (Do Not Discharge):
  • 30 Melanomas
  • 25 Basal Cell Carcinomas (BCC)
  • 25 Squamous Cell Carcinomas (SCC), including Bowen’s disease
  • 20 Actinic Keratoses
• 100 Low-Risk (Safe to Discharge):
  • 50 Benign Naevi (moles)
  • 50 Seborrhoeic Keratoses

---

🛠️ Methodology & Workflow

The LLMs were given the images and asked for the most likely of six diagnoses, a confidence level, a secondary choice, and a "discharge" recommendation. Meanwhile, I used a fine-tuned version of Inception v3 (originally released by Google in 2015).

Technical Implementation of the CNN

The Inception v3 model was pretrained on general imagery and then fine-tuned across all layers using a single H100 GPU. To ensure a fair test, the 200 test lesions (and all related images of those patients) were strictly excluded from the training and validation sets.

The model's logic can be simplified as: $\text{Final Score} = (w_1 \cdot \text{Clinical Image}) + (w_2 \cdot \text{Dermoscopic Image})$ Where $w_2$ (dermoscopic weight) is significantly higher.

Notably, this model is lightweight enough to execute locally in a mobile browser via WebGPU, ensuring total privacy as no data is uploaded.

---

📊 Results: The Performance Gap

The CNN achieved a first-choice accuracy of $\frac{125}{200} = 62\%$, while the LLMs hovered around $42\%$.

Diagnostic Accuracy Table

---

🔍 Nuance and Failures

The "Favorite Answer" Bias

The LLMs exhibited a tendency to over-rely on specific diagnoses regardless of the evidence:

• GPT-6 Astra had a "BCC bias," labeling 75 lesions as Basal Cell Carcinoma (including 14 melanomas).
• Claude Fable 5.1 had an "Actinic Keratosis bias," labeling 53 lesions as such.

Critical Errors

In some cases, the LLMs were dangerously overconfident. For one invasive melanoma (under 0.8mm), GPT-6 Astra called it a BCC with 95% confidence. In another instance, both LLMs misidentified a melanoma as a seborrhoeic keratosis and suggested the patient be discharged.

Visual Examples

Case 1: The Danger of Misdiagnosis Both LLMs incorrectly identified this as a harmless seborrhoeic keratosis.

Case 2: Successful Identification All models correctly identified this Basal Cell Carcinoma.

Case 3: High Confidence, Wrong Answer GPT-6 Astra was 95% sure this was a BCC; it was actually a melanoma.

Case 4: Over-Caution LLMs called this an actinic keratosis and refused to discharge the patient.

🏁 Final Thoughts

While the LLMs are versatile, the specialized, fine-tuned CNN remains superior for this specific medical task. When allowing for a second-choice diagnosis, the CNN's accuracy jumped to 173/200, dwarfing the LLMs' scores of 134 and 138.