This article chronicles a decade of research (2012-2021) dedicated to advancing thermal sensing for crop monitoring. My journey began with the humble infrared thermometer (IRT) and culminated in the development of a sophisticated computer-vision system.
While this endeavor has inspired many – from seasoned growers to fellow researchers – it's crucial to emphasize that complexity isn't always the optimal solution. Sometimes, a simple yet reliable system based on infrared thermometry proves more effective for most growers than a highly sophisticated computer vision system.
A simple yet reliable system based on infrared thermometry proves more effective for most growers than a highly sophisticated computer vision system.
Researchers may benefit from more complex systems. However, they must be cognizant of the inherent limitations of even the most advanced computer vision algorithms. Investing hundreds, or even thousands, of hours in developing computer vision algorithms for a single research project may not always be the most prudent use of resources.
The trajectory of Generative AI suggests that it will soon be capable of handling the most intricate computer vision tasks, effectively supplanting most, if not all, mediocre computer vision algorithms currently developed within academic institutions. Therefore, researchers should strategically allocate their time and financial resources, prioritizing areas that will not be readily overtaken by AI.
The trajectory of Generative AI suggests that it will soon be capable of handling the most intricate computer vision tasks, effectively supplanting most, if not all, mediocre computer vision algorithms currently developed within academic institutions.
Undoubtedly, computer vision and robotics are domains poised for significant advancements driven by larger corporations. It is my contention that academic institutions and even smaller (and in some cases, larger) AgriTech companies will soon find themselves unable to compete effectively with AI agents offered as services by industry giants like Google, OpenAI, and their counterparts. Consequently, I believe they should concentrate their efforts on mastering and disseminating the fundamental principles of precision agriculture – areas less susceptible to AI disruption.
It is my contention that academic institutions and even smaller (and in some cases, larger) AgriTech companies will soon find themselves unable to compete effectively with AI agents offered as services by industry giants like Google, OpenAI, and their counterparts.
The Evolution of Crop Imaging
Conventional crop imaging typically involves a multi-step process. First, the hardware is meticulously set up and tuned. Subsequently, images are captured and preprocessed to prepare them for analysis. This often includes steps such as noise reduction and geometric corrections.
Next, image processing techniques are employed to extract relevant features from the images, such as color, texture, and spectral signatures. Finally, these extracted features are analyzed using statistical methods, machine learning (ML), or deep learning (DL) algorithms to gain insights into crop health and performance.
Imaging Platforms & Techniques
Imaging platforms can vary significantly, ranging from indoor systems utilized on conveyors to ground-based systems employed on robots or unmanned ground vehicles (UGVs). Aerial platforms, such as drones and aircraft, as well as satellite-based systems, also play a crucial role in agricultural imaging.
A diverse array of imaging techniques are available, including:
RGB (visible): Captures color information.
NIR Multispectral: Measures light reflected in near-infrared wavelengths.
Hyperspectral: Captures narrow bands across the electromagnetic spectrum.
Thermal: Detects infrared radiation emitted by objects.
Fluorescence: Measures the emission of light after excitation by another source.
Multimodal: Combines multiple imaging techniques for comprehensive analysis.
Examples of Imaging Applications
The applications of imaging in agriculture are numerous and diverse. These include:
Precision Irrigation: Monitoring plant water status in real-time to optimize irrigation schedules.
Disease Detection: Early detection of diseases before visual symptoms appear, enabling timely intervention and mitigating crop losses.
Fruit Ripening: Monitoring fruit ripening in storage facilities to ensure optimal harvest timing and quality.
Livestock Management: Weight monitoring in precision livestock farming to improve animal health and productivity.
Robotics: Fruit and plant detection for robotic harvesting and other automated agricultural tasks.
Phenotyping: Seed shape analysis in high-throughput phenotyping for crop improvement programs.
Challenges of Imaging in Agriculture
Several challenges impede the widespread adoption and effective utilization of imaging technologies in agriculture. These challenges include:
Hardware Limitations:
The unavailability of affordable, reliable, and standardized hardware for continuous field measurements poses a significant obstacle.
Ensuring the robustness and weatherproofing of hardware is crucial for reliable operation in field conditions.
Maintaining the cleanliness of camera lenses is essential for accurate image acquisition.
High power consumption can limit the deployment and operational duration of imaging systems.
Sensor Integration:
The frequent need to integrate additional non-imaging sensors, such as soil moisture sensors or weather stations, can increase system complexity and cost.
Changes in hardware configurations can adversely affect image processing and modeling efforts, requiring recalibration and adjustments.
Multimodal Imaging:
Detecting specific targets, such as leaves or lesions, within thermal images can be challenging.
Accurate image registration and fusion across different imaging modalities are critical for effective multimodal analysis.
Automation:
The imaging process often lacks automation, requiring manual intervention and potentially delaying data analysis.
Image processing and computer vision techniques can be complex and require specialized expertise, which may not be readily available to many growers.
Access to user-friendly and affordable software packages for image processing and analysis remains a challenge.
The Solution: An All-in-One Multimodal Imaging System
To address these challenges, my team and I developed an all-in-one hardware and software package specifically designed and optimized for agricultural applications.
System Development (2012 – 2021)
The development of this system spanned several years, evolving from initial prototypes to the current integrated platform.
Early Stages: The journey began with single-pixel infrared thermometers (2012-2014), followed by the integration of thermal cameras into the system (2015).
Lepton Module Integration: Low-cost Lepton modules were subsequently incorporated to enable thermal-RGB imaging (2016-2017).
Heimann Sensor Integration: Higher-resolution Heimann sensors were then integrated to enhance the system's imaging capabilities (2018).
All-in-One System: Finally, a fully integrated system combining thermal and multispectral imaging was developed (2019-2021).
All-in-One Computer Vision System
Hardware Components
The system is designed to be portable and battery-operated, facilitating easy deployment and operation in the field. A Raspberry Pi serves as the computational core, providing the necessary processing power for image analysis and data management.
Multimodal Sensors: The system incorporates both thermal (FLIR Lepton 3.5) and multispectral sensors, enabling comprehensive data acquisition.
Microclimate Unit: A dedicated microclimate unit measures environmental parameters such as air temperature, humidity, wind speed, and radiation.
GPS Receiver: A GPS receiver geo-tags images and sensor data, providing accurate location information.
Software Features
The system is equipped with a user-friendly graphical user interface (GUI) for intuitive system control and data visualization.
Embedded Crop Models: The software incorporates a suite of embedded crop models, including NDVI, CWSI, transpiration, and crop ET, to provide real-time insights into crop health and performance.
Computer Vision Algorithms: Advanced computer vision algorithms are integrated into the system, enabling automated image processing tasks such as target detection, background removal, and image registration.
Data Storage and Export: The system efficiently stores raw images, processed data, and model outputs in a structured format for subsequent analysis and interpretation.
Key System Features
Low-cost and Open-Source: The system is built using affordable and readily available open-source components, making it accessible and cost-effective.
Multimodal Approach: The combination of thermal and multispectral imaging provides a more comprehensive and informative assessment of crop conditions.
IoT-Enabled: The system is designed to be IoT-enabled, facilitating wireless data transmission and remote monitoring.
Comparison: Commercial vs. DIY Imagers
Commercial imaging systems often rely on expensive proprietary hardware and software, limiting their flexibility and accessibility. These systems typically have high power requirements, restricting their suitability for outdoor applications, particularly in unattended and continuous measurement scenarios.
Moreover, they can be difficult to integrate into existing agricultural systems and may only provide raw images, necessitating the use of cloud computing or in-lab image processing using specialized PC software. Compatibility with most wireless sensor networks and LPWAN IoT solutions may also be limited.
In contrast, DIY all-in-one imagers offer several advantages. They rely on inexpensive and readily available camera modules and sensors, facilitating easy integration and customization. These systems are typically more affordable, power-efficient, and well-suited for outdoor applications. Their open-source nature promotes accessibility and facilitates community-driven development and innovation. Furthermore, they can be easily integrated with other sensors and seamlessly incorporated into LPWAN wireless sensor networks.
The Importance of Simplicity and Practicality
While sophisticated computer vision systems offer powerful capabilities, they can be complex, time-consuming to develop, and may not always be necessary for all applications.
For Growers: A simple, reliable IRT-based system might be more suitable for practical on-farm applications, providing valuable insights into crop water status and enabling timely irrigation adjustments.
For Researchers: While complex systems can be valuable for research, it's crucial to consider the time and resources required for development and the potential limitations of current algorithms. Focusing on robust and reliable data collection methods and sound experimental design is paramount, regardless of the complexity of the imaging system.
A simple, reliable IRT-based system might be more suitable for practical on-farm applications, providing valuable insights into crop water status and enabling timely irrigation adjustments.
The Rise of AI and the Future of Agricultural Imaging
The rapid advancement of Generative AI is poised to revolutionize the field of computer vision, significantly impacting the landscape of agricultural imaging.
Impact on Research: AI-powered solutions may eventually surpass many of the computer vision algorithms currently developed by researchers in academia. This shift necessitates a recalibration of research priorities.
Focus on Fundamentals: Researchers should prioritize investing time and resources in understanding and teaching the fundamental principles of precision agriculture, areas less likely to be directly impacted by AI. This includes a deep understanding of crop physiology, soil science, and environmental factors, which are crucial for effective decision-making in agriculture regardless of the technological advancements.
The Rise of AI-as-a-Service: Larger companies like Google and OpenAI are likely to dominate the development and provision of advanced AI-powered solutions for agriculture. This shift towards "AI-as-a-Service" will likely reshape the agricultural technology landscape. While this presents both opportunities and challenges, it underscores the importance of developing a skilled workforce capable of effectively utilizing and adapting these AI-powered tools to specific agricultural contexts.
Conclusion
This article explores the development of a low-cost, all-in-one multimodal imaging system for crop monitoring. While imaging technology offers valuable insights, it's crucial to recognize that it may not always be the most suitable solution for every agricultural application. For growers, a simpler, more practical approach, such as a reliable IRT-based system, might be more beneficial for everyday farm management.
Furthermore, the rapid advancement of Generative AI will significantly impact the field of computer vision. Researchers should prioritize investing time and resources in understanding and teaching the fundamental principles of precision agriculture, such as crop physiology, soil science, and environmental factors. These core principles will remain crucial for successful agricultural practices, regardless of the advancements in AI-powered technologies.
By focusing on these fundamental areas and strategically selecting imaging technologies that best suit specific needs and constraints, researchers and growers can effectively leverage technology to improve agricultural sustainability and productivity while avoiding unnecessary complexity and resource expenditure.
References
Osroosh, Y., Peters, R.T., Campbell, C., Zhang, Q., 2016. Comparison of irrigation automation algorithms for drip-irrigated apple trees. Computers and Electronics in Agriculture, 128: 87–99.
Osroosh, Y., Peters, R.T., Campbell, C., 2016. Daylight crop water stress index for continuous monitoring of water status in apple trees. Irrigation Science, 34(3): 209–219.
Osroosh, Y., Peters, R.T., Campbell, C., Zhang, Q., 2015. Automatic irrigation scheduling of apple trees using theoretical crop water stress index with an innovative dynamic threshold. Computers and Electronics in Agriculture, 118: 193–203.
Osroosh, Y., Peters, R.T., Campbell, C., 2015. Estimating potential transpiration of apple trees using theoretical non-water-stressed baselines. Journal of Irrigation and Drainage Engineering, 141(9): 04015009.
Osroosh, Y., Peters, R.T., Campbell, C., 2015. Estimating actual transpiration of apple trees based on infrared thermometry. Journal of Irrigation and Drainage Engineering, 141(8): 04014084.
Osroosh, Y. et al., 2019. Detecting fruit surface wetness using a custom-built low-resolution thermal-RGB imager. Computers and Electronics in Agriculture, 157: 509-517.
Osroosh, Y. et al., 2018. Economical thermal-RGB imaging system for monitoring agricultural crops. Computers and Electronics in Agriculture, 147: 34–43.
Mohamed, A.Z., Osroosh, Y., Peters, R.T., Bates, T., Campbell, C., Ferrer-Alegre, F., 2020. Monitoring water status in apple trees using a sensitive morning crop water stress index. Irrig. and Drain. 1–15.
Can Canopy Measurements Determine Soil Moisture? (Part 1-2), 2016.
Comments