Responsible AI

Model card & safety

What the model is, what it was trained on, where it fails, and the guardrails built around it. Written for the clinicians who rely on it and the reviewers who audit it.

Intended use

DermaMarker is clinical decision support for general practitioners and nurses in settings without a dermatologist on call. It prioritises referrals and drafts documentation. It is not a diagnostic device — every output is framed as a preliminary assessment for a licensed clinician to confirm or overrule.

Model & training data

ArchitectureEfficientNet-B0, transfer learning from ImageNet
RuntimeONNX, served by FastAPI
DatasetHAM10000 — 10,015 dermatoscopic images, 7 classes
ProvenanceMedical University of Vienna & Cliff Rosendahl practice (Australia); >50% histopathology-confirmed
Narrative layerClaude (claude-sonnet-5) for bilingual summaries, with a deterministic rule-based fallback

Known limitations

  • Skin-tone representation. HAM10000 is drawn overwhelmingly from lighter Fitzpatrick skin types (I–III). Accuracy on darker skin tones — common across the Gulf, South Asia, and Africa — is not established. The confidence gate below is a partial mitigation; fine-tuning on regional data (e.g. dermoscopy sets that include Fitzpatrick IV–VI) is the roadmap fix.
  • Dermatoscopy vs phone photos. Training images are dermatoscopic. Smartphone photos degrade accuracy; the UI instructs users on framing and flags low-confidence results instead of hiding the uncertainty.
  • Seven classes only. Conditions outside the HAM10000 taxonomy (e.g. cutaneous infections, drug eruptions) will be forced into the nearest class. Out-of-distribution detection is on the roadmap.
  • Class imbalance. 67% of training images are benign nevi. Training uses weighted sampling to counter this, but rare classes (dermatofibroma, vascular lesions) have thin support.

Safety guardrails

  • Asymmetric triage thresholds. Melanoma at just 30% probability or basal cell carcinoma at 40% already escalates to urgent — the system is deliberately tuned to over-refer rather than under-refer. A missed melanoma costs a life; an unnecessary referral costs a consultation.
  • Confidence gating. Any top prediction below 50% confidence is automatically bumped to monitor with a re-scan recommendation, and the uncertainty is stated in the report rather than suppressed.
  • Degradation, not failure. Each pipeline step (classify → triage → report → store) fails independently and visibly. If the AI narrative layer is unavailable, a deterministic rule-based report is generated instead — the clinician always gets the raw probabilities.
  • Human in the loop by design. The product output is a referral recommendation addressed to a clinician, never a diagnosis addressed to a patient.

Privacy & data handling

  • No patient identifiers are collected — scans store the image, the model output, and timestamps only.
  • Images are stored in Supabase object storage; deployments can be region-pinned to satisfy data-residency requirements (e.g. UAE health data regulations).
  • The referral note is generated on demand and printed or copied by the clinician — it is not transmitted anywhere by the system.

Regulatory status

DermaMarker is a research prototype. It is not registered as a medical device with any regulator (UAE MOHAP, FDA, CE). Deployment in clinical settings would require conformity assessment as clinical decision support software.