Skip to content
sibisaiPublic

About

MedLens - Medical image analysis & education platform (FastAPI/React). EfficientNet-V2 models for MRI, X-ray, OCT with Grad-CAM + AI-powered explanations. Includes Learn Mode quiz and progress dashboard.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Latest commit

Β 

History

37 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

MedLens

A medical image analysis and education platform with deep learning models, Grad-CAM visualization, and AI-powered explanations. Built with PyTorch, FastAPI, and React.

Demo

πŸ”— Live Demo: medlens-two.vercel.app

Overview

This project provides:

  • REST API for medical image classification with Grad-CAM visualization
  • AI-generated explanations using Claude vision API
  • Analyze Mode: Upload images for diagnosis with visual + written explanations
  • Learn Mode: Unlimited quiz for practicing diagnosis with instant feedback
  • Dashboard: Track learning progress with accuracy charts and weak area identification
  • Extensible architecture for adding new classification models

Models

Model Task Classes Accuracy AUC
brain_tumor MRI Classification Glioma, Meningioma, Pituitary, No Tumor 99% -
pneumonia Chest X-Ray Normal, Pneumonia 88% 0.958
bone_fracture X-Ray Fracture Detection Fractured, Not Fractured 98% 0.998
retinal_oct Retinal OCT Analysis CNV, DME, Drusen, Normal 99% -

Results

πŸ“Š Click on any model below to view training curves, confusion matrix, and Grad-CAM visualizations.

🧠 Brain Tumor MRI Classification (99% accuracy)

Brain Tumor MRI Classification

4-class classification of brain MRI scans.

Metric Value
Test Accuracy 99%
Classes Glioma, Meningioma, Pituitary, No Tumor

Training Progress

Brain Tumor Training

Confusion Matrix

Brain Tumor Confusion Matrix

Grad-CAM Visualizations

Brain Tumor Grad-CAM

🫁 Pneumonia Detection (88% accuracy, 0.958 AUC)

Pneumonia Detection

Binary classification from chest X-rays.

Metric Value
Test Accuracy 88%
AUC Score 0.958
Classes Normal, Pneumonia

Training Progress

Pneumonia Training

Confusion Matrix

Pneumonia Confusion Matrix

ROC Curve

Pneumonia ROC Curve

Grad-CAM Visualizations

Pneumonia Grad-CAM

🦴 Bone Fracture Detection (98% accuracy, 0.998 AUC)

Bone Fracture Detection

Binary classification of X-rays across all anatomical regions.

Metric Value
Test Accuracy 98.22%
AUC Score 0.998
Classes Fractured, Not Fractured

Training Progress

Bone Fracture Training

Confusion Matrix

Bone Fracture Confusion Matrix

ROC Curve

Bone Fracture ROC Curve

Grad-CAM Visualizations

Bone Fracture Grad-CAM

πŸ‘οΈ Retinal OCT Analysis (99% accuracy)

Retinal OCT Analysis

4-class classification of retinal OCT scans for eye disease detection.

Metric Value
Test Accuracy 99.38%
Classes CNV, DME, Drusen, Normal

Training Progress

Retinal OCT Training

Confusion Matrix

Retinal OCT Confusion Matrix

Grad-CAM Visualizations

Retinal OCT Grad-CAM

Project Structure

medical-image-classifier/
β”œβ”€β”€ api/
β”‚   β”œβ”€β”€ app/
β”‚   β”‚   β”œβ”€β”€ main.py             # FastAPI application
β”‚   β”‚   β”œβ”€β”€ models/
β”‚   β”‚   β”‚   β”œβ”€β”€ base.py         # Base classifier interface
β”‚   β”‚   β”‚   β”œβ”€β”€ brain_tumor.py  # Brain tumor classifier
β”‚   β”‚   β”‚   β”œβ”€β”€ pneumonia.py    # Pneumonia classifier
β”‚   β”‚   β”‚   β”œβ”€β”€ bone_fracture.py # Bone fracture classifier
β”‚   β”‚   β”‚   └── retinal_oct.py  # Retinal OCT classifier
β”‚   β”‚   └── utils/
β”‚   β”‚       β”œβ”€β”€ gradcam.py      # Grad-CAM visualization
β”‚   β”‚       └── llm.py          # Claude LLM integration
β”‚   β”œβ”€β”€ scripts/
β”‚   β”‚   β”œβ”€β”€ generate_explanations.py  # Batch generate explanations
β”‚   β”‚   └── generate_overlays.py      # Batch generate Grad-CAM overlays
β”‚   β”œβ”€β”€ weights/                # Model weights (not tracked in git)
β”‚   β”œβ”€β”€ Dockerfile
β”‚   β”œβ”€β”€ fly.toml
β”‚   └── requirements.txt
β”œβ”€β”€ frontend/
β”‚   β”œβ”€β”€ src/
β”‚   β”‚   β”œβ”€β”€ pages/              # HomePage, AnalyzePage, LearnPage, DashboardPage
β”‚   β”‚   β”œβ”€β”€ components/         # React components
β”‚   β”‚   β”‚   β”œβ”€β”€ shared/         # Navigation
β”‚   β”‚   β”‚   └── quiz/           # Quiz components
β”‚   β”‚   β”œβ”€β”€ hooks/              # Custom hooks
β”‚   β”‚   └── utils/              # API, sample data, dashboard data
β”‚   β”œβ”€β”€ public/
β”‚   β”‚   └── samples/            # Sample images and pre-cached overlays
β”‚   β”œβ”€β”€ package.json
β”‚   └── vite.config.js
β”œβ”€β”€ notebooks/
β”‚   β”œβ”€β”€ brain_tumor_classifier.ipynb
β”‚   β”œβ”€β”€ pneumonia_classifier.ipynb
β”‚   β”œβ”€β”€ bone_fracture_classifier.ipynb
β”‚   └── retinal_oct_classifier.ipynb
└── assets/                     # README images

Technical Details

Model Architecture

All models use EfficientNet-V2-S pretrained on ImageNet with custom classification heads:

  • Input: 224x224 RGB images
  • Backbone: EfficientNet-V2-S (frozen early layers)
  • Classifier: Dropout(0.3) β†’ Linear(1280, 512) β†’ ReLU β†’ Dropout(0.15) β†’ Linear(512, num_classes)

Grad-CAM

Visualizations generated by backpropagating target class scores and computing weighted activation maps from the last convolutional layer.

LLM Explanations

Image-specific explanations generated using Claude Haiku vision API, describing anatomical findings and Grad-CAM focus areas.

Frontend Features

  • Analyze Mode: Model selection, sample gallery, drag-and-drop upload, Grad-CAM visualization with opacity slider, AI explanations
  • Learn Mode: Unlimited quiz, instant feedback with Grad-CAM, pre-cached explanations, session tracking
  • Dashboard: Accuracy stats, per-model charts, weak area identification, session history
  • Mobile responsive design

Datasets

Model Dataset
Brain Tumor Brain Tumor MRI Dataset
Pneumonia Chest X-Ray Pneumonia
Bone Fracture Fracture Multi-Region X-ray
Retinal OCT Retinal OCT Images

API Reference

Full API documentation available at medical-image-api.fly.dev/docs

Endpoints

Method Endpoint Description
GET /health Health check
GET /models List available models
POST /predict/{model_name} Classification
POST /predict/{model_name}/gradcam Classification with Grad-CAM
POST /explain/{model_name} AI-generated explanation

Local Development

API

cd api
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Add weights to weights/
# Add ANTHROPIC_API_KEY to .env

python -m uvicorn app.main:app --reload --port 8000

API docs: http://localhost:8000/docs

Frontend

cd frontend
npm install

# Create .env.local for local development
echo "VITE_API_URL=http://localhost:8000" > .env.local

npm run dev

Frontend: http://localhost:5173

License

MIT

Acknowledgments

About

MedLens - Medical image analysis & education platform (FastAPI/React). EfficientNet-V2 models for MRI, X-ray, OCT with Grad-CAM + AI-powered explanations. Includes Learn Mode quiz and progress dashboard.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages