Real-Time Agricultural Insights: An Advanced Raspberry Pi Thermal-Multispectral Camera with Onboard Computer Vision

BINA Pro multi-modal computer vision system (originally developed by DurUntash Lab, San Diego, CA)

The conventional approach to agricultural imaging often involves a significant time lag: capturing valuable field data through manual or automated means, only to process and analyze it later in the laboratory. This inherent delay has always felt like a bottleneck. With the readily available power of the Raspberry Pi, sophisticated machine and deep learning algorithms, and robust computer vision libraries like OpenCV and PlantCV, the question arises: why not integrate these elements into a single, intelligent system capable of performing the entire workflow – from image acquisition to feature extraction and real-time analysis – directly in the field?

While visible-band Raspberry Pi cameras offer an accessible and cost-effective entry point, their capabilities are inherently limited by the information contained within RGB images. Recognizing this constraint, I embarked on a project to develop a low-cost yet high-quality multimodal camera system built around the Raspberry Pi (Model: BINA Pro, DurUntash Lab, San Diego, CA).

This system goes beyond simple multispectral imaging (avoiding the limitations of basic blue filters in DIY imagers) by integrating true multispectral (VIS-NIR), thermal imaging, a light sensor, and a rangefinder into a unified platform. Furthermore, it allows direct connection to commercial meteorological sensors like the CS ClimaVUE50 via its SDI-12 port, enriching the data stream with comprehensive environmental context.

At the heart of the system lies a user-friendly graphical user interface (GUI) with embedded computer vision algorithms and crop models. This intelligent software automatically generates and displays three distinct image layers, each offering a range of selectable options:

• Layer 1 (Multispectral): Provides access to individual spectral bands (Red, Green, Near-Infrared – filter dependent) or a composite view (R+G+NIR).

  
  

• Layer 2 (Index): Enables the visualization of crucial vegetation indices such as Red NDVI, GNDVI, Normalized Red, Normalized Green, and Normalized NIR, as well as a segmented canopy layer highlighting plant area.

  
  

• Layer 3 (Thermal + Microclimate): Integrates thermal data with microclimate information to display critical plant stress indicators like ΔT (temperature difference), CWSI (Crop Water Stress Index), estimated actual Transpiration, and Stomatal Conductance.

Beyond image analysis, the system is also engineered to measure plant canopy height and automatically calculate key agricultural metrics including canopy cover percentage, crop coefficient (Kc), and estimated crop evapotranspiration (ETc). The final output of the imager is a well-organized CSV file containing timestamped and geotagged data, facilitating seamless integration with other data analysis pipelines and GIS platforms.

This is an evolving project, and my current efforts are focused on refining the onboard image segmentation algorithms to enhance their accuracy and robustness. To date, the primary application area has been water and crop loss management, providing valuable insights for optimizing irrigation and identifying stress early. However, the potential applications extend far beyond this, and I am actively expanding its capabilities. I am confident that researchers in plant phenomics, particularly those focused on high-throughput phenotyping, will find this integrated system to be a powerful and versatile tool.

For a more detailed overview of the system's features and its development history, I have prepared a presentation which you are welcome to watch below.