Plant Disease Detection Using Mobile Images and Video

Case Study: Plant Doctor

Research Goal: Plant Doctor was developed to monitor plant health automatically using video footage, without needing to pick leaves or take them to a lab. The researchers tested the system on urban plants and trees in Tokyo. The main goal was to create a non-invasive way to track plant health repeatedly over time using accessible video [5].

System Architecture

Plant Doctor combines three computer vision models into one pipeline:

1. YOLOv8 — Leaf Detection

YOLOv8 detects leaves in each video frame and draws bounding boxes around them. The authors chose YOLOv8 because it provides a good balance between accuracy and detection speed.

2. DeepSORT — Leaf Tracking

After leaves are detected, DeepSORT tracks each leaf across consecutive frames. It assigns a unique ID to each leaf so the system knows when the same leaf appears in multiple frames, avoiding duplicate counts.

3. Best-ROI Selection

Because camera movement can cause blur or bad angles, the software evaluates all views of a tracked leaf and picks the single clearest region of interest (ROI) for segmentation.

4. DeepLabV3+ — Damage Segmentation

The best view of the leaf is passed to DeepLabV3+, which segments the damaged spots (caused by pests, fungi, or bacteria) and calculates the percentage of the leaf affected.

Dataset and Training

The researchers trained the YOLOv8-PD leaf detection model on 4,595 total images from three sources:

• Smartphone images: 56.7% of the dataset (≈ 2,608 images), taken in outdoor conditions in Shinjuku, Tokyo.
• DSLR images: 22.3% of the dataset (≈ 1,025 images), taken with a Nikon D600 and a 300 mm telephoto lens.
• PlantVillage images: 21.0% of the dataset (≈ 962 images), laboratory benchmark photos.

For the damage segmentation model, the authors created a dataset of 124 manually annotated sick-leaf images to train DeepLabV3+ [5].

Reported Results

Here are the main results reported in the paper:

Important Findings and Limitations

This study highlights two key lessons for computer vision projects:

1. Why Damage Segmentation Is Harder Than Leaf Detection

The model achieved high accuracy for detecting leaves (mAP@0.5 ≈ 0.99) and segmenting whole leaves (IoU = 0.96), but damage segmentation was lower (IoU = 0.76).

• Whole leaves have clear shapes and good contrast against backgrounds.
• Damaged areas have fuzzy borders, variable colors, and small spots that are harder to separate.
• Sunlight reflections on leaves can look like damaged spots.
• The damage segmentation model was trained on only 124 annotated images, because pixel-level labeling takes a lot of manual work.

2. The 50-Minute Video Processing Limitation

A 1-minute video taking 50 minutes means the current implementation is not real-time. This shows an important point: a detection model can be very accurate while the full video pipeline remains too slow for real-time mobile use.

3. Where the Pipeline Can Fail

• Tracking errors: DeepSORT can assign new IDs to the same leaf if the camera moves back and forth quickly.
• Overlapping leaves: Clusters of leaves make it harder to calculate single-leaf areas accurately.
• Blurred background leaves: Defocused leaves in the background are sometimes mistakenly included in the foreground leaf mask.

Lessons for Mobile Deployment

To make a system like Plant Doctor practical on a phone or drone:

• Separate the tasks: Run lightweight leaf detection on the mobile device and send selected frames to a server for segmentation.
• Downsample frames: Instead of processing every frame, sampling a few frames per second can significantly reduce DeepSORT's computational load while still tracking leaves reliably.
• Model optimization: Quantizing models to INT8 can speed up inference on mobile chips.

Comparison with Other Research

The single-image CNN from Iftikhar et al. [2] is fast and very accurate on clean lab images, but it only gives one label per photo. Plant Doctor [5] works on real-world video and measures how much of each leaf is damaged. However, it is much slower and is not yet fast enough for real-time use.

Why This Case Study Matters

Plant Doctor is a great example for this tutorial because it moves beyond single still images to multi-view video. It shows how detection, tracking, and segmentation can work together, while also giving honest data about processing time and real-world challenges [5].