top of page
Search

From Fields to Factory: Adapting a Low-Cost Thermal Imaging System for Food Quality Control

Writer's picture: Y. Osroosh, PhDY. Osroosh, PhD
Jul 19, 20218 min read

Updated: Aug 30, 2024

A Few Years Ago, I Had a Project...

A few years ago, I embarked on a project to explore the potential of low-cost thermal camera modules in machine vision and automation within the baked food industry. This project was followed by an intriguing visit to a "dog treat" factory in Joplin, MO. Our goal was to develop a basic setup to demonstrate to potential clients and gauge their reactions.


I pondered the best approach to this problem and considered whether our low-cost system, originally developed for agricultural applications (Osroosh et al., 2018; Osroosh and Peters, 2019), could be adapted for this new purpose. While I worked on developing a new imaging system, I also explored the possibility of using commercially available cameras designed for machine vision applications.


Machine Vision in Industry vs. Agriculture

Machine vision-based industrial automation and machine vision in agriculture share similarities but also have distinct differences. For instance, in high-throughput phenotyping, we combine various sensors and imaging systems to collect extensive data on plants. This data is typically processed afterward.


In contrast, food factories operate in an industrial setting, requiring systems designed to handle harsh environments, fast-moving targets, and stringent regulations. These systems prioritize speed, durability, and compliance with safety standards.


My System: A Novel Approach

While combining thermal and RGB images to separate targets from backgrounds is not a new concept, my system offers a unique value proposition. I was one of the first to develop such a system for continuous monitoring of plant canopies at an affordable price. While my system might seem simplistic compared to industrial counterparts, it's worth noting that these industrial systems are often not suitable for continuous agricultural monitoring due to their complexity and cost.


Development Challenges

While it might seem simple to use the thermal-RGB camera I developed in my research (Osroosh et al., 2018) in a food processing facility, the reality is much more complex. Even if we could successfully adapt this system, it would likely result in a product that's not suitable for agricultural applications. Alternatively, we could outsource the development or purchase a ready-made solution from industrial equipment manufacturers, potentially saving time, labor, and capital while ensuring higher reliability. Here are some of the specific challenges I've identified in adapting my imaging system for the food industry:


1. Ambient Temperature Range

The sensors and electronics in this system have specific operating temperature limits. There's no guarantee that the current design can withstand the harsh conditions of a food processing factory, especially when installed near hot conveyors or ovens. This is particularly critical for ensuring reliable operation in environments with extreme temperatures.


2. Target Temperature Range

The high temperatures encountered in food processing limit the available thermal module options. Lepton FLIR modules (V2.5) have a limited range of 0-50°C (according to blackbody test results), insufficient for monitoring baked food. Newer models offer a wider range (up to 400°C) but compromise accuracy (±10°C). Heimann sensors, while offering a broader range (-20 to >1000°C) and higher accuracy (±2°C), may have other limitations that need to be considered.


3. Speed (Frame per Second) and Response Time

The thermal and RGB camera modules used in this system have limitations in terms of frame rate, which can be insufficient for scanning fast-moving targets on conveyors. If the frame rate is too slow, the camera may capture the same object at multiple points within a single frame, resulting in blurred or distorted images. This is similar to the blurriness that occurs when a handheld camera is moved while taking a photo.


The camera modules' speed at their highest resolution (required for accurate measurements) does not exceed 10-20 frames per second. To address this, one option is to install the camera system closer to the target. However, this requires modifications to the processing line and a heat/vibration-resistant enclosure or platform. Even with this approach, the camera modules may not tolerate the extreme heat.


Response time is another important factor. The RGB and thermal modules in this system use different technologies. The Heimann thermal sensor is a thermopile array sensor, meaning it uses hundreds of sensors to capture data. These sensor outputs (pseudo-pixels) are not measured simultaneously. If the module takes 100 milliseconds to complete a frame, a moving target will be distributed over more pixels, appearing cut or elongated. For instance, in a dog food factory, I recorded speeds as high as 0.9 meters per second in the oven area. At this speed, the target moves approximately 9 centimeters in 100 milliseconds, which is several times the size of dog biscuits.


4. Processing Power

The Raspberry Pi 3, the SBC used in this design, may not have sufficient processing power for higher resolution images and frame rates. Increased processing demands can lead to longer processing times, resulting in fewer frames processed in real-time and potentially slowing down the entire system. The newer Raspberry Pi 4 might offer improved performance in this regard.


5. Resolution

To obtain accurate measurements at a safe distance from heat sources and reflected radiation, a higher resolution is necessary. Unfortunately, the thermal modules I'm using do not allow for this. To increase the thermal module frame rate to a higher level (>20Hz), I had to decrease the resolution from 80x60 pixels to 80x16 pixels (which is possible with the Heimann sensor). This reduction in resolution impacts image quality and detail.


6. Cleaning

Camera lenses must be kept clean to ensure accurate image capture. While this was not a significant concern in the original agricultural application, it is essential in a food processing facility. Continuous airflow can help to prevent dust and debris from accumulating on the lenses.


7. Lighting

The RGB light sensor and white balancing features can be beneficial in outdoor environments to compensate for varying lighting conditions (e.g., passing clouds). However, indoor food processing facilities require consistent, uniform illumination. Unwanted shadows, insufficient lighting, or inappropriate lighting can complicate image processing and increase errors. Higher speeds necessitate brighter lighting sources to ensure adequate exposure time.


8. Background

The color of the conveyor belt can affect image processing. To simplify the process, it may be necessary to modify part of the conveyor to create a consistent background color.


9. Mechanical Shock

Continuous vibrations from the production line can damage the system over time. A vibration-resistant design is crucial to ensure the system's longevity and reliability in this environment.


10. Enclosure and Mounting Platform

This project extends beyond camera development and may require modifications to the production line to accommodate a suitable mounting platform. The platform must be resistant to heat stress, humidity, and mechanical shock. It should also provide a controlled light source over the target. Additionally, thermal measurements can be affected by reflected longwave radiation (heat) from surrounding objects. To ensure accurate measurements, the mounting platform should isolate the samples and protect the camera from reflected radiation. Longwave radiation can negatively impact thermal image processing.


11. Standards and Regulations

Adherence to safety and quality standards is paramount in the food industry. This related article, "Machine Vision Challenges and Applications in the Food and Beverage Industry," emphasizes the importance of these standards.


Lepton vs Heimann Thermal Modules

Based on my research, commercially available "radiometric" thermal modules with high resolution include FLIR, Heimann, and Melexis. This comparison focuses on FLIR and Heimann sensors, highlighting their key specifications for the intended application:



Resolution

Melexis offers the MLX90640 with a resolution of 32x24 pixels, but it was not available for individual sale when I last checked. Heimann's highest resolution module is the HTPA80x64d (5,120 pixels). FLIR provides the Lepton 2.5 and 3.5, with the Lepton 3.5 offering a resolution of 160x120 pixels (19,200 pixels).


Speed (Frame Rate)

The effective frame rate for both the HTPA80x64d and Lepton 3.5 is below 9 Hz. This is an export-compliant frame rate due to the military applications of thermal modules. While the Lepton's frame rate cannot be changed, the HTPA80x64d's frame rate can be increased by reducing pixel and ADC resolutions to 80x16. However, this comes at the cost of reduced image quality.


Target Temperature Range

The HTPA80x64d offers an impressive temperature range of -20 to 1000°C. The Lepton allows for adjusting the target temperature range using low/high gain settings. However, this impacts accuracy. High gain (-10 to 140°C) offers +/- 5°C or 5% accuracy, while low gain (up to 450°C) offers +/- 10°C or 10% accuracy. Region-specific calibration can improve accuracy but is costly and time-consuming.


Noise

The Lepton, resembling a camera module, includes integrated digital thermal image processing for noise reduction and other enhancements. The Heimann sensor requires external processing for calibration and improvements. In terms of noise, the Lepton generally produces better results.


Shutter (Exposure Time)

The Lepton features a mechanical shutter for auto-calibration, while the Heimann sensor is shutterless. I encountered shutter failures during my experiments with Lepton modules. The Lepton's shutter speed is unknown, but it takes approximately half a second to complete FCC. A higher shutter speed is necessary to avoid blurry images, but the module does not allow for manual adjustment. Given the Lepton's low frame rate, I anticipate low shutter speeds as well.


FLIR recommends using a shuttered version of the OEM camera for most applications. Uncooled microbolometer-based cameras can drift with temperature changes, requiring periodic re-normalization (FFC). Shutterless versions can be used in certain situations, but an initial FFC is necessary. Without specific details about the intended use, FLIR recommends a shuttered version.


ESD Sensitivity

The Lepton module is highly susceptible to electrostatic discharge (ESD). In contrast, the Heimann sensor offers ESD protection of at least 2.0 kV, making it more resistant to damage from static electricity.


Industrial Alternatives to Low-Cost Thermal Camera Modules

Beyond low-cost thermal camera modules, a variety of established technologies address food quality control, process monitoring, and control in the food industry. These options encompass both imaging and non-imaging techniques and cater to online and offline quality checks on processing lines of various scales. Here's a breakdown of some promising alternatives:


1. Industrial Machine Vision Systems (Diverse Technologies):

Industrial machine vision systems leverage various technologies to analyze and inspect objects. Here are some resources showcasing their capabilities:


2. Industrial Thermal and RGB Camera Systems (Building a Machine Vision System):

This approach allows you to build a customized machine vision system using commercially available industrial cameras. Here are some resources to get you started:


3. Line Scan Cameras:

These cameras capture a single line of pixels at a very high speed, making them ideal for inspecting fast-moving objects on conveyor belts. They require conveyor speed as input to assemble captured slices into a complete image.


4. Laser-Based Industrial Machine Vision Systems:

Laser triangulation appears to be a popular choice in the food industry for obtaining 3D profiles of objects. Its potential advantages include lower cost and higher robustness compared to area or line scan cameras.


5. Machine Vision Software:

Many software packages can handle processing of both thermal and RGB images for machine vision applications. Here are some prominent options:


Conclusion

Adapting a low-cost thermal camera system designed for agriculture to a food processing environment presents significant challenges. While it offers potential for a more affordable solution, further research and development are necessary to address these challenges and determine the feasibility of this approach.


References

Khanal, S., Fulton, J., Shearer, S., 2017. An overview of current and potential applications of thermal remote sensing in precision agriculture. Comput. Electron. Agr., 139: 22–32.


Osroosh, Y. et al., 2019. Detecting fruit surface wetness using a custom-built low-resolution thermal-RGB imager. Computers and Electronics in Agriculture, 157: 509-517.


Osroosh, Y. et al., 2018. Economical thermal-RGB imaging system for monitoring agricultural crops. Computers and Electronics in Agriculture, 147: 34–43.


0 comments

Comments

bottom of page