Bloom or Bluff? Benchmarking Vision–Language Models Against Classical Machine Learning for Harmful Algal Bloom Detection from Satellite Imagery.
Saved in:
| Title: | Bloom or Bluff? Benchmarking Vision–Language Models Against Classical Machine Learning for Harmful Algal Bloom Detection from Satellite Imagery. |
|---|---|
| Authors: | Narula, Harsh Deep Singh1 (AUTHOR) |
| Source: | Remote Sensing. Jul2026, Vol. 18 Issue 13, p2147. 21p. |
| Subjects: | Multispectral imaging, Support vector machines, United States. National Oceanic & Atmospheric Administration, Classification algorithms, Chlorophyll spectra, Algal blooms, Machine learning, Remote-sensing images |
| Geographic Terms: | Lake Erie |
| Abstract: | Highlights: What are the main findings? Commercial vision-language models (VLMs; GPT-4o, GPT-5.5, Claude Sonnet 4.6) flagged harmful algal blooms with 73–93% false positive rates on bloom-absent satellite images, below the best RGB classifier and a trivial always-present baseline. The VLMs collapsed 60–70% of severity predictions into "moderate" regardless of actual conditions and identified at most one of the two severe blooms. A multi-spectral SVM (10 Sentinel-2 bands) achieved the strongest overall detection (F1 = 0.833, 79.5% accuracy, 27% false positive rate), outperforming every other method tested. Multi-spectral feature importances ranked the chlorophyll indices NDCI and Floating Algae Index among the top predictors, the same spectral signatures used in NOAA's operational algorithms. What are the implications of the main findings? The 12-percentage-point accuracy gain from RGB to multi-spectral features within the same SVM quantifies the spectral information gap, carried by the red-edge and shortwave infrared bands, that constrains RGB-only models and VLM vision encoders. For aquatic monitoring that requires spectral discrimination, domain-specific classifiers on multi-spectral data remain the right tool, and VLMs cannot substitute for them in spectral analysis. VLMs may serve a complementary role as narrative generators for non-specialist audiences, but their inability to access spectral bands is a hard boundary for current architectures. In recent years, there has been growing interest in applying vision–language models (VLMs) to quantitative remote sensing. This study evaluates whether three commercial VLMs (GPT-4o, GPT-5.5, and Claude Sonnet 4.6) can detect and classify the severity of harmful algal blooms (HABs) from Sentinel-2 satellite imagery of western Lake Erie and compares them against classical machine learning classifiers (Random Forest (RF), Support Vector Machine (SVM), and eXtreme Gradient Boosting (XGBoost)) trained on both a three-band red, green, blue (RGB) composite representation of the imagery and a 10-band multi-spectral reflectance representation. Forty bloom events identified from the National Oceanic and Atmospheric Administration (NOAA) Harmful Algal Bloom Operational Forecast System (HAB-OFS) severity assessments were assembled into the evaluation dataset, spanning seven bloom seasons (2019–2025). For binary bloom detection, the VLMs did not match the classical RGB classifiers; their F1 scores (0.69–0.75) fell below the best RGB classifier (Random Forest, 0.76) and below a trivial always-present baseline (F1 = 0.77), and they carried false positive rates of 73–93% on bloom-absent images, against 27–40% for the RGB classifiers. The VLMs reached high recall by labeling most scenes as bloom-positive, which makes them operationally unreliable in this configuration. For severity classification, the VLMs assigned 60–70% of their predictions to the "moderate" category regardless of actual conditions and identified at most one of the two severe blooms, whereas the classical classifiers tracked the ground-truth distribution and delivered two to nearly three times the exact-match accuracy (0.44–0.59 vs. 0.20–0.225). The strongest method across all metrics was the multi-spectral SVM (F1 = 0.833, false positive rate 27%, accuracy 0.795). Switching the same SVM from RGB to multi-spectral features raised accuracy from 0.675 to 0.795, a 12-percentage-point gain that measures the spectral information carried by red-edge and shortwave infrared bands that are accessible through multi-spectral sensors but unavailable to standard VLM vision encoders. Feature-importance analysis showed that the multi-spectral classifiers ranked chlorophyll-specific indices, the Normalized Difference Chlorophyll Index (NDCI) and the Floating Algae Index (FAI), among their top predictors, the same signatures used in established operational algorithms, while the RGB classifiers relied on red-channel variability and green-dominant pixel fractions because RGB inputs cannot compute those indices. Two compounded limitations therefore constrain off-the-shelf VLMs for aquatic remote sensing: the limited spectral information available through standard RGB channels and a mismatch between the land-dominated training distributions of these models and aquatic optical conditions. Domain-specific classifiers operating on multi-spectral data remain the more suitable tools for continued development of HAB monitoring and water-quality retrieval. [ABSTRACT FROM AUTHOR] |
| Copyright of Remote Sensing is the property of MDPI and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Highlights: What are the main findings? Commercial vision-language models (VLMs; GPT-4o, GPT-5.5, Claude Sonnet 4.6) flagged harmful algal blooms with 73–93% false positive rates on bloom-absent satellite images, below the best RGB classifier and a trivial always-present baseline. The VLMs collapsed 60–70% of severity predictions into "moderate" regardless of actual conditions and identified at most one of the two severe blooms. A multi-spectral SVM (10 Sentinel-2 bands) achieved the strongest overall detection (F1 = 0.833, 79.5% accuracy, 27% false positive rate), outperforming every other method tested. Multi-spectral feature importances ranked the chlorophyll indices NDCI and Floating Algae Index among the top predictors, the same spectral signatures used in NOAA's operational algorithms. What are the implications of the main findings? The 12-percentage-point accuracy gain from RGB to multi-spectral features within the same SVM quantifies the spectral information gap, carried by the red-edge and shortwave infrared bands, that constrains RGB-only models and VLM vision encoders. For aquatic monitoring that requires spectral discrimination, domain-specific classifiers on multi-spectral data remain the right tool, and VLMs cannot substitute for them in spectral analysis. VLMs may serve a complementary role as narrative generators for non-specialist audiences, but their inability to access spectral bands is a hard boundary for current architectures. In recent years, there has been growing interest in applying vision–language models (VLMs) to quantitative remote sensing. This study evaluates whether three commercial VLMs (GPT-4o, GPT-5.5, and Claude Sonnet 4.6) can detect and classify the severity of harmful algal blooms (HABs) from Sentinel-2 satellite imagery of western Lake Erie and compares them against classical machine learning classifiers (Random Forest (RF), Support Vector Machine (SVM), and eXtreme Gradient Boosting (XGBoost)) trained on both a three-band red, green, blue (RGB) composite representation of the imagery and a 10-band multi-spectral reflectance representation. Forty bloom events identified from the National Oceanic and Atmospheric Administration (NOAA) Harmful Algal Bloom Operational Forecast System (HAB-OFS) severity assessments were assembled into the evaluation dataset, spanning seven bloom seasons (2019–2025). For binary bloom detection, the VLMs did not match the classical RGB classifiers; their F1 scores (0.69–0.75) fell below the best RGB classifier (Random Forest, 0.76) and below a trivial always-present baseline (F1 = 0.77), and they carried false positive rates of 73–93% on bloom-absent images, against 27–40% for the RGB classifiers. The VLMs reached high recall by labeling most scenes as bloom-positive, which makes them operationally unreliable in this configuration. For severity classification, the VLMs assigned 60–70% of their predictions to the "moderate" category regardless of actual conditions and identified at most one of the two severe blooms, whereas the classical classifiers tracked the ground-truth distribution and delivered two to nearly three times the exact-match accuracy (0.44–0.59 vs. 0.20–0.225). The strongest method across all metrics was the multi-spectral SVM (F1 = 0.833, false positive rate 27%, accuracy 0.795). Switching the same SVM from RGB to multi-spectral features raised accuracy from 0.675 to 0.795, a 12-percentage-point gain that measures the spectral information carried by red-edge and shortwave infrared bands that are accessible through multi-spectral sensors but unavailable to standard VLM vision encoders. Feature-importance analysis showed that the multi-spectral classifiers ranked chlorophyll-specific indices, the Normalized Difference Chlorophyll Index (NDCI) and the Floating Algae Index (FAI), among their top predictors, the same signatures used in established operational algorithms, while the RGB classifiers relied on red-channel variability and green-dominant pixel fractions because RGB inputs cannot compute those indices. Two compounded limitations therefore constrain off-the-shelf VLMs for aquatic remote sensing: the limited spectral information available through standard RGB channels and a mismatch between the land-dominated training distributions of these models and aquatic optical conditions. Domain-specific classifiers operating on multi-spectral data remain the more suitable tools for continued development of HAB monitoring and water-quality retrieval. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 20724292 |
| DOI: | 10.3390/rs18132147 |
