Defect detection has become an essential demand for the production line to check whether the products meet standards. Although we consider that problem through image processing, we always have different approaches to detect what we need for every product in different industrial areas.
In this blog post, we will introduce some technologies for defect detection in X-ray images – a usual and popular case used in many factories.
1. Mask RCNN Neural Network
In recent years, mask RCNN has proven its efficiency, performance, and accuracy in solving object detection and object segmentation problems. With the development of the backbone net such as Resnet50, Resnet101 and so on, Mask RCNN will be state-of-the-art in those fields. That’s why we have used it for visible fault detection.
When considering the defects of objects from X-ray images, we’ve divided them into two cases:
- Cases that can use Mask RCNN: The defects of the objects need to be visible, big “enough,” have almost the same particular pattern, and don’t need to be pre-processed. We can label these cases and train them with Mask RCNN if we have enough datasets for the objects.
- Cases that cannot use Mask RCNN: In some cases, the defects could be very difficult to determine, whether they’re too small, invisible, have unclear patterns, or need image preprocessing.
For example, there is a defect in a casting wheel under X-ray. If the defect is visible and has a particular pattern, we can train it with Mask RCNN to detect that defect.
2. Edge Detection
Edge detection is an image processing technique for finding the boundaries of objects within images. It works by detecting discontinuities in brightness. Edge detection is used for image segmentation and data extraction in areas such as image processing, computer vision, and machine vision.
Edge detection includes a variety of mathematical methods that aim at identifying edges and curves, at which the image’s brightness changes sharply. In our project, we focus on two algorithms:
- Sobel Edge Detection: this is a very common edge extraction filter, which is an approximation to a derivative of an image. We can use the Sobel filter for both directions x and y. In the image processing field, using Sobel edge detection means we do a convolution with 2 matrixes for x direction and y direction.
The advantage of Sobel edge detection lies in its simplicity because of the approximate gradient calculation and ability to consider in two directions for different purposes. However, this filter cannot produce accurate edge detection with very thin and smooth edges, and the accuracy is affected by noise.
Here is an example of using Sobel edge detection for the x and y direction.
| Sample image | Sobel in X direction | Sobel in Y direction |
 |
 |
 |
The Sobel edge detection can be used to determine the position of the object’s edges in images to measure the distance or check the edges to detect any cracks or discontinuing.
- Canny Edge Detection: This is an upgrade from Sobel Edge Detection which shows better results by enhancing the computation’s complexity. This filter consists of 5 steps in the whole process.
With Canny Edge Detection, image processing on X-ray becomes better with some advantages:
- Noise will be removed to get a better input image.
- The edge signal can be enhanced to get better results, and the effectiveness can be adjusted using parameters.
However, because this is a multiple-stage filter, it will take time due to its complex computation and it cannot be implemented in real-time applications.
Here is an example of using Canny Edge detection with the same image above. We can easily see that the edges of any part are present better and thinner than using Sobel Edge. In reality, after putting the X-ray image through a threshold filter to clear the details, we can use Canny Edge to cover small objects inside the device and detect if there are any abnormal details.
3. Thresholding
Thresholding is a type of image segmentation where we change the pixels of an image to make it easier to analyze. In thresholding, we convert an image from color or grayscale into binary, i.e., one that is simply black and white. Most frequently, we use thresholding as a way to select areas of interest of an image, while ignoring the parts we are not concerned with.
In the OpenCV Library, there are two kinds of thresholding: simple and adaptive.
- Simple Thresholding: If the pixel value is smaller than the threshold, it is set to 0, otherwise it is set to a maximum value. There are at least 5 algorithms for threshold. For X-ray images, we will use BINARY and TOZERO thresholds to detect the faults in products.
- BINARY: this filter will check if the pixel is over or under a threshold value. If this value is bigger, the pixel will be changed to white. Otherwise, it is black. For X-ray images, this is a useful filter to check if there are any cracks that can appear clearly big enough white streak or white spot.
- TOZERO: this filter has the same comparison mechanism as BINARY, but the rules for the pixel over the threshold value are different. Any pixel under this value will be changed to black, the others will remain unchanged. That means the filter will still keep the region which has unclear transition between gray and white, so it will help us treat cracks or some fault details in the unclear gray-white region.
Here is an example of applying those thresholds, using the same parameters to control the quality of the output image (remember to convert the image to gray before using threshold).
| Sample image |  |
| BINARY threshold |  |
| TOZERO threshold |  |
In this example, both filters help us check the basic detail of the object. However, we found that TOZERO threshold will be more useful than BINARY if we want to detect any defects in the gray pixel.
- Adaptive Thresholding is applied to deal with different lighting conditions, which is the tough case for normal thresholding. With adaptive thresholding, the algorithm determines the threshold for a pixel based on a small area around it. So, we get different thresholds for different areas of the same image, from which we will get a better result.
There are two kinds of adaptive thresholding: THRESH MEAN (mean value of pixel in an area) and THRESH GAUSSIAN (gaussian weighted sum). Let’s check out an example where we apply the adaptive thresholding for the sample image above using the same parameters.
| THRESH MEAN |  |
| THRESS GAUSSIAN |  |
In this example, the adaptive filter is a great threshold to save more detail than the normal threshold. To get a better result, we have to adjust the parameters.
Besides these filters from OpenCV, we also have other algorithms such as Niblack, Otsu, and Sauvola to threshold the detail we want and they appear in SKIMAGE.
4. Conclusion
As mentioned earlier, every product has multiple approaches to deal with defects checking problems in the image field. What we have done is an approach for this kind of X-ray sample. To adapt to the project’s requirements, we should combine different techniques and adjust the parameters till we get a better detection result. We hope this article has provided you with useful techniques to handle fault detection on devices from X-ray images.
References
Mask RCNN Lib: ayoolaolafenwa/PixelLib: Visit PixelLib’s official documentation https://pixellib.readthedocs.io/en/latest/ (github.com)
Image enhancing: https://towardsdatascience.com/measuring-enhancing-image-quality-attributes-234b0f250e10
Line detection: https://www.geeksforgeeks.org/line-detection-python-opencv-houghline-method/
Image thresholding: OpenCV: Image Thresholding
Edge detection: Edge Detection Using OpenCV | LearnOpenCV #
SKIMAGE filter: Thresholding — skimage v0.19.2 docs (scikit-image.org)
Sample images: http://dmery.ing.puc.cl/index.php/material/gdxray/