My thesis focuses on the development of a machine vision system for the automatic detection of surface defects in precision castings, specifically addressing the industrial challenge of identifying hidden cracks and discontinuities that arise during the sand foundry process. The motivation for this work stems from the fact that many sand foundry defects, such as hairline cracks, are nearly invisible under normal illumination and require fluorescent magnetic particle inspection to become discernible. However, prolonged human exposure to fluorescent light can cause irreversible eye damage, and manual inspection is subjective, slow, and prone to fatigue. Therefore, the goal of this research is to design and implement a robust, real-time visual inspection system that can reliably detect sand foundry defects on cast plate teeth, which are critical components in mechanical transmissions.
In this thesis, I present the complete development lifecycle of the inspection system: from the requirements analysis and hardware selection, through the design of image processing algorithms for target extraction and defect localization, to the final software implementation using a combination of Visual Studio and the Halcon machine vision library. The system was tested in the laboratory with real production components, and the results demonstrate that a detection accuracy of 95% can be achieved while maintaining a processing speed well within the industrial requirement of 300 milliseconds per part. I also discuss the key challenges encountered, such as the similarity between the natural surface texture of the castings and the defect patterns, and I propose a novel line-contour combination method that effectively distinguishes real sand foundry defects from background interference.
1. Introduction and System Overview
Sand foundry defects are unfortunately a common byproduct of the casting process. During the production of plate teeth, two parts are pressed together under high pressure, and the joint area often develops micro-cracks that are not visible to the naked eye. These cracks can propagate under load and lead to catastrophic failure of the component. The manufacturer required a non-destructive inspection solution that could be integrated into the existing production line, operate automatically, and flag defective parts for removal. The specific requirements were stringent: the inspection must be completed within 300 milliseconds, no defective part should be missed (zero false negatives), and the false positive rate should not exceed 0.1%. Furthermore, the system had to save images of defective parts for later confirmation and algorithm refinement.
Based on these requirements, I designed a system architecture that consists of two main subsystems: the image acquisition subsystem and the image processing and decision-making subsystem. The acquisition subsystem includes a CCD camera, a lens, a fluorescent light source, and a dedicated industrial computer. The processing subsystem implements the defect detection algorithm, which I developed in Halcon and then integrated into a custom C++ application using the Microsoft Foundation Classes (MFC) framework. Figure 1 shows a typical image of a plate tooth under fluorescent magnetic particle inspection, where the green fluorescent lines indicate the presence of sand foundry defects.
The overall workflow of the system is as follows. First, a sensor detects that a plate tooth has arrived at the inspection station and triggers the camera to capture an image. The image is then processed by a template matching algorithm to determine whether a plate tooth is present and to locate its exact position and orientation. Once the tooth is located, the region of interest (ROI) is defined, and the defect detection algorithm is applied only to the relevant areas, excluding both the outer boundary and the inner holes or contours that are part of the normal geometry. The defect detection algorithm extracts line-like features from the image, removes small noise contours, and combines the remaining line segments using a two-step merging process. Finally, the length of the combined contours is measured; if it exceeds a predetermined threshold, the part is classified as defective. In that case, the image is saved, a rejection signal is sent to the PLC, and the processing loop continues with the next part.
2. Hardware Selection and Imaging Setup
Selecting the appropriate hardware is crucial for capturing high-quality images that make sand foundry defects visible. I evaluated several industrial cameras, lenses, and light sources based on the specifics of the application. Table 1 summarizes the final hardware choices along with the key reasons for each selection.
| Component | Selected Model | Key Specifications and Selection Rationale |
|---|---|---|
| Industrial camera | STC-SCA503POE | CCD sensor, 2058 × 2488 resolution, color (RGB), supports hardware triggering. The high resolution and color capability are needed to resolve fine defects in the green fluorescent channel. |
| Lens | GMHR-0814MCN | Manual zoom and aperture, matching the camera’s target size, suitable for the fixed working distance of approximately 54 cm. |
| Light source | Fluorescent lamp | Side lighting arrangement. Excites the magnetic particles that accumulate at sand foundry defects and emit green light. |
| Industrial PC | RPC-910 | Intel i5 or better, 8 GB RAM, 64-bit OS, high-speed storage, and industrial-grade durability for continuous operation. |
The imaging geometry and the physical arrangement of the equipment are shown in Figure 2. The plate tooth is first magnetized and sprayed with a fluorescent magnetic suspension by a robotic manipulator. It is then placed under the camera. The working distance and focus were tuned to maximize the visibility of the fluorescent defect lines. Through several experiments with different focus distances, I determined that a working distance of 54 cm produced the best contrast of the defect lines, even though the overall tooth surface appeared slightly blurred. This trade-off was acceptable because the defect detection algorithm relies on the presence of sharp, bright lines rather than the overall surface texture.
The camera operates in hardware trigger mode, synchronized with the arrival of the plate tooth. The captured image is transmitted over GigE to the industrial PC. Because the illumination is not perfectly uniform, I also installed a white background plate behind the tooth to improve the contrast between the foreground and background, which simplified the target extraction step described in Section 3.
3. Image Preprocessing and Target Extraction
The raw images captured by the camera contain noise from the sensor and the environment. To reduce the noise while preserving the edges of the fluorescent defects, I tested several spatial filtering methods: median, mean, and Gaussian filters. The median filter yielded the best balance between noise removal and edge preservation, especially for the salt-and-pepper noise that was present in the green channel. After converting the RGB image to its green channel, I applied a median filter with a 5×5 rectangular mask (in Halcon, the mask size is specified as 1.5×1.5, which corresponds to a 5×5 pixel window). This step eliminated the isolated noise pixels while keeping the thin fluorescent lines intact. Figure 3 illustrates the effect of different filter sizes and shapes; the 5×5 rectangular median filter was selected because it removed the noise without attenuating the defect lines.
After preprocessing, the next task is to extract the plate tooth from the background. This is essential because the position of the tooth may vary slightly due to the mechanical handling system. I compared two common approaches: Blob analysis and template matching.
3.1. Blob Analysis for Target Extraction
Blob analysis is a classic technique that segments an image into foreground and background components based on gray-level differences. In its simplest form, thresholding separates pixels whose gray value lies within a certain range. For the plate tooth images, the gray values of the tooth and the background overlapped significantly, making global thresholding impossible. I then applied dynamic thresholding, where the comparison is made between the original image and a smoothed version of itself. The dynamic threshold condition for bright objects is:
$$ S = \{ (r,c) \in R \; | \; f_{r,c} – g_{r,c} \ge g_{diff} \} $$
where \( f_{r,c} \) is the original gray value, \( g_{r,c} \) is the smoothed value from a large mean filter, and \( g_{diff} \) is a user-defined offset. I initially used a filter size of 1500×2500, but the segmentation was disrupted by a pipe in the background. I later adjusted the filter to 50×2000, which worked for some images but not all. To make the approach robust, I added a large white background plate behind the tooth. This change made the dynamic threshold segmentation much more reliable, and the resulting binary image contained a clean blob corresponding to the tooth. However, the extracted blob was not always complete, and the tooth edge occasionally had missing parts, which could interfere with subsequent defect inspection. Table 2 shows the effect of the background modification on the segmentation quality.
| Condition | Dynamic threshold filter size | Segmentation result |
|---|---|---|
| Original background | 1500 × 2500 | Background pipe merged with tooth |
| Original background | 50 × 2000 | Tooth mostly separated but inconsistent across samples |
| With white background board | 50 × 2000 | Tooth clearly segmented, but edge still slightly irregular |
Although the Blob method eventually worked, it was sensitive to changes in illumination and the surface reflectivity of the tooth. Moreover, Blob analysis alone does not provide the orientation information that is needed to align the internal ROI for defect detection. Therefore, I decided to use a more robust and informative method: template matching.
3.2. Template Matching for Target Extraction
Template matching identifies a known pattern in an image and returns its position, orientation, and scale. Among the various template matching techniques, I chose shape-based matching because it is robust to nonlinear illumination changes and partial occlusion. Shape-based matching compares the direction vectors of the image edges with those of the template. The similarity measure is based on the normalized cross-correlation of the gradient directions. It is computed as:
$$ ncc_{r,c} = \frac{1}{n} \sum_{(u,v) \in T} \frac{t_{r,c} – m_t}{\sqrt{s_t^2}} \cdot \frac{f_{r+u,c+v} – m_{f_{r,c}}}{\sqrt{s_{f_{r,c}}^2}} $$
where \( m_t \) is the mean gray value of the template, \( s_t^2 \) is the variance of the template gray values, and \( m_{f} \) and \( s_f^2 \) are the corresponding statistics of the image region at position \((r,c)\). This normalization makes the matching insensitive to linear changes in illumination, which is essential in the industrial environment where lighting can drift.
In my implementation, I created a template from a defect-free plate tooth image. The template was selected from a region with a distinct geometric feature that is common to all teeth. To handle slight variations in tooth size, I enabled anisotropic scaling, allowing the match to succeed for components that are slightly smaller or larger than the template. Table 3 lists the key parameters used for creating and searching the template.
| Stage | Parameter | Value / Setting |
|---|---|---|
| Creation | Pyramid levels | 4 |
| Creation | Start angle and angle range | -10° to +10° |
| Creation | Angle step | 0.1° |
| Creation | Minimum and maximum scale | 0.9 / 1.1 |
| Creation | Optimization | Auto |
| Creation | Polarity | use_polarity |
| Search | Minimum score | 0.5 |
| Search | Number of matches | 1 |
| Search | Maximum overlap | 0.5 |
| Search | Greediness | 0.7 |
One issue I encountered was that the matching score varied significantly among different teeth because of their individual surface textures and minor shape differences. To mitigate this, I applied an additional smoothing step to the image before matching, which suppresses the high-frequency details and leaves only the overall silhouette of the tooth. This simple preprocessing step increased the matching score by roughly 0.1 and made the matching much more consistent.
Once a match is found, the algorithm returns the row and column coordinates \((r_1, c_1)\) and the angle \(\theta_1\). I compare these values with those recorded during template creation \((r_0, c_0, \theta_0)\). From this comparison, I compute an affine transformation matrix \( \mathbf{H} \) that maps the ROI from the template image to the current image. The rotation-translation matrix is given by:
$$
\mathbf{H} = \begin{pmatrix}
\cos(\theta_1 – \theta_0) & -\sin(\theta_1 – \theta_0) & r_1 – r_0 \\
\sin(\theta_1 – \theta_0) & \cos(\theta_1 – \theta_0) & c_1 – c_0 \\
0 & 0 & 1
\end{pmatrix}
$$
This matrix is then used to transform the predefined ROI (which covers the tooth body and excludes the inner holes) so that it remains correctly aligned with the tooth in every image. This affine transformation is crucial because the defect detection algorithm relies on accurately removing the inner contours; a misalignment of even a few pixels can cause false negatives or false positives.
3.3. Comparison of the Two Extraction Methods
Table 4 summarizes the comparison between Blob analysis and template matching for this application.
| Criterion | Blob Analysis | Template Matching |
|---|---|---|
| Robustness to illumination changes | Poor; requires dynamic threshold tuning | High; normalized correlation handles linear changes |
| Position and orientation information | Only centroid, no orientation | Complete pose (row, column, angle, scale) |
| Edge completeness | Sometimes missing parts of the boundary | Provides a bounding ROI aligned with the tooth |
| Speed | Fast but for this specific segmentation the filter was large | Fast enough when using pyramid levels |
| Adaptability to size variation | Not directly | Supports anisotropic scaling |
Based on this comparison, I concluded that template matching is superior for this system because it provides both reliable target localization and the necessary pose information for the subsequent defect inspection. Therefore, the final implementation uses shape-based template matching with anisotropic scaling and an affine-transformed ROI.
4. Surface Defect Detection Algorithm
Sand foundry defects such as cracks and seams appear as elongated lines in the fluorescent image. The core idea of my defect detection algorithm is to extract all line-like structures from the region of interest after removing the outer and inner contours of the tooth, then to evaluate their lengths. If the length of any combined line exceeds a predefined threshold, the part is declared defective. This approach is applicable because non-defective surfaces generally do not contain long continuous bright lines; the natural surface texture creates only short, isolated segments.
4.1. Removal of Outer and Inner Contours
The outer contour of the plate tooth, as well as the inner holes, produce strong fluorescent edges that are very similar in appearance to sand foundry defects. Thus, these regions must be removed before line extraction. For the outer contour, I applied a morphological erosion operation to the binary mask of the tooth. Erosion of a region \( R \) by a structuring element \( S \) is defined as:
$$ R \ominus S = \{ t \mid S_t \subseteq R \} $$
where \( S_t \) is the structuring element translated to position \( t \). After some experimentation, I found that a rectangular structuring element of size 50 × 80 pixels completely eroded the outer boundary of the tooth while preserving the interior region where the defects are likely to appear. Figure 4 shows the resulting region after erosion; the outer contour has been completely removed, leaving only the central area.
For the inner contours, which correspond to two circular holes and one rectangular slot, I manually drew a composite ROI that covers these non-defective areas. Since the tooth position and orientation change from one image to the next, the ROI is transformed using the affine matrix \( \mathbf{H} \) obtained from the template matching stage. The composite ROI is built by taking the union of two circles and one rectangle, as shown in Figure 5. After applying the transformation, this ROI is used to mask out (i.e., ignore) the pixels belonging to the inner features. The remaining part of the tooth is then subjected to the defect detection algorithm.
4.2. Line Extraction and Interference Removal
The defect detection begins with the extraction of line contours from the gray-scale image. Halcon provides a powerful operator for extracting lines based on the second derivative of the image intensity. The algorithm computes the Hessian matrix at each pixel, and a point is marked as belonging to a line if the second derivative perpendicular to the line direction is a local maximum and exceeds a threshold. The extraction is governed by three parameters: the Gaussian smoothing factor \( \sigma \), and two hysteresis thresholds \( T_{low} \) and \( T_{high} \). These thresholds are related to the contrast values by the formula:
$$
\begin{pmatrix} T_{low} \\ T_{high} \end{pmatrix}
= -2 \cdot
\begin{pmatrix} C_{low} \\ C_{high} \end{pmatrix}
\cdot
\frac{w}{\sqrt{2\pi} \sigma^3} \cdot e^{-\frac{w^2}{2\sigma^2}}
$$
where \( w \) is the width of the line. In my tuning, I set \( \sigma = 1 \), because a higher value caused some faint defect lines to disappear, while a lower value generated too many spurious lines. The hysteresis thresholds were chosen empirically to balance sensitivity and noise.
The extracted line set inevitably includes a large number of short, isolated segments caused by the surface texture, dust, or other artifacts. Directly merging these lines would create false long contours that could be mistaken for defects. To address this, I implemented a two-stage cleaning process.
Stage 1: Short-line removal. I first discarded all line segments whose length was less than a certain threshold. I tested thresholds from 5 to 8 pixels and found that a threshold of 7 pixels gave the best performance. This step eliminated many irrelevant tiny segments while preserving the much longer defect lines, which are typically 20 pixels or more in length.
Stage 2: Contour merging with strict geometric constraints. After removing short lines, I applied a contour merging algorithm that combines adjacent line segments based on four criteria: \( D_{maxAbs} \), \( D_{maxRel} \), \( S_{maxShift} \), and \( A_{maxAngle} \). The definitions are illustrated in Table 5.
| Parameter | Meaning | Condition for merging |
|---|---|---|
| \(D_{maxAbs}\) | Maximum absolute distance between endpoints measured along the direction of the longer segment | Must be less than the threshold |
| \(D_{maxRel}\) | Ratio of the endpoint distance to the length of the longer segment | Must be less than the threshold |
| \(S_{maxShift}\) | Maximum perpendicular distance from the shorter segment endpoint to the longer segment’s regression line | Must be less than the threshold |
| \(A_{maxAngle}\) | Maximum angle difference between the two segments | Must be less than the threshold |
By setting these parameters conservatively, I ensured that the short interference segments, which are typically far apart or at large angles, would not be merged, while the small gaps in a defect line (which are aligned and have a short distance) would be successfully bridged. After the first merging operation, I applied a second, simpler merging operation that only considers the endpoint distance and the angle of the regression lines. This second pass was necessary to join larger defect fragments that were still separated by a small gap. Through repeated testing, I found the optimal parameter set for this second merging step. This two-pass merging approach successfully reconstructed the full defect lines without connecting the background noise.
4.3. Feature Extraction and Defect Decision
After contour merging, the image contains a small number of long contours. To decide whether a defect exists, I measure the length of each contour. If the maximum contour length exceeds a threshold of 90 pixels, the tooth is classified as defective. This threshold was determined empirically from a set of 100 sample images. The decision rule can be expressed as:
$$ \text{Defect} = \begin{cases} 1, & \text{if } \max_{i} L_i \ge 90 \\ 0, & \text{otherwise} \end{cases} $$
where \( L_i \) is the Euclidean length of the \( i \)-th contour. The choice of 90 pixels was conservatively low enough to catch even partial cracks, but high enough to avoid false positives from any residual noise that might have survived the merging process.
Figure 6 displays several examples of the final detection results. The white lines overlaid on the images indicate the extracted defect contours. As can be seen, the algorithm successfully identified cracks of various shapes and orientations, including those with small gaps that were bridged by the merging process. To evaluate the performance quantitatively, I ran the algorithm on 100 test images containing a mix of defective and non-defective teeth. The system achieved an overall accuracy of 95%, which satisfies the manufacturer’s requirement of no missed defects, while the false positive rate remained below the acceptable 0.1% threshold.
5. Software Module Design and System Implementation
The complete inspection system requires not only the algorithm but also a user-friendly software interface that can be operated by factory personnel. I developed the software in C++ using the Microsoft Foundation Classes (MFC) framework, and I integrated the Halcon algorithm via its C++ interface. The software is divided into several modules, as described below.
5.1. User Privilege Management
To prevent accidental changes to critical parameters by operators, I implemented a user privilege system with two levels: operator and administrator. The system starts in operator mode, which allows the operator only to start and stop the inspection process or view accumulated statistics. Changing any of the algorithm parameters, such as the merging thresholds or the defect length threshold, requires logging out and logging in as an administrator. The login dialog is shown in Figure 7. If an operator attempts to access the parameter page, a notification message appears, as shown in Figure 8.
The privilege data is stored in the same INI file as the parameters. Initially, default passwords are set, and the administrator can change both the administrator and operator passwords. The security check is implemented by verifying the current login status before allowing access to the parameter settings dialog.
5.2. Camera Acquisition Module
The camera module is responsible for connecting to the machine vision camera, configuring its exposure and contrast, and continuously acquiring images. In the software main window, the user can select the camera by clicking the “Select Camera” button, which opens a dialog listing all connected cameras. Upon selection, the software establishes a connection using the vendor SDK. To acquire images in a continuous mode, I created a dedicated acquisition thread. The thread waits for a hardware trigger event, captures the frame, and then passes it to the image processing module. The image is displayed in an MFC Picture Control for real-time monitoring.
In automatic mode, the acquisition thread also triggers the defect detection algorithm. The flow is implemented using a state machine that advances through stages: (1) wait for trigger, (2) capture image, (3) template matching, (4) defect detection, (5) display and save result. If no plate tooth is detected in the image, the system skips the defect detection stage and waits for the next trigger. This design ensures that the system is robust to false triggers from the sensor.
5.3. Parameter Settings Module
The parameter settings dialog, shown in Figure 9, exposes all critical algorithm parameters to the administrator. These parameters include:
- Morphological erosion matrix size (for outer contour removal)
- Radii and centers of the two circles and the rectangle defining the inner ROI
- Minimum matching score
- Line extraction parameters (\( \sigma \), low and high hysteresis)
- Contour merging parameters for both merging stages
- Minimum defect length threshold
All parameters are stored in an INI file. On startup, the software reads the file and populates the edit controls. When the administrator saves changes, the new values are written back to the file and are immediately applied to the running algorithm. This flexibility is essential for tuning the system on the factory floor.
5.4. System Operation and Results
The main system interface (Figure 10) provides a start/stop button, a display area for the current camera image, and a second display area showing the processed image with defect overlays. It also shows the running statistics: total inspected, number of good parts, number of defective parts, and the inspection timestamp. The software automatically saves any image that is classified as defective, along with a timestamp and the measured defect length. These saved images are invaluable for verifying the system’s decisions and for further algorithm improvement.
During the laboratory testing phase, I used the software to inspect plate teeth that were provided by the manufacturer. The camera was triggered manually via the software, and I also simulated the hardware trigger using a function generator. The defect detection results were consistently accurate, and the processing time was measured at approximately 200 ms per image, which is comfortably below the required 300 ms. Table 6 summarizes the system performance metrics.
| Metric | Value |
|---|---|
| Processing time per part | ~ 200 ms |
| Detection accuracy (on 100 test images) | 95% |
| False negative rate (missed defects) | 0% within the test set |
| False positive rate (good parts marked as defective) | < 1% (within the test set) |
| Image resolution | 2058 × 2488 |
| Minimum detectable crack length | 90 pixels (approx. 0.4 mm) |
These results indicate that the developed system meets the industrial requirements. However, I note that the tests were performed in a controlled laboratory environment. Before full deployment, further validation is needed on the actual production line to account for variations in ambient light, vibration, and the precision of the robotic handling system.
6. Discussion and Future Directions
The algorithm I developed is specifically tailored to the plate tooth geometry and the fluorescent magnetic particle inspection process. The key contribution is the two-stage line merging strategy that effectively suppresses the background texture while preserving the continuity of real sand foundry defects. This method may be applicable to other types of castings where defects appear as thin, elongated features. However, the current implementation relies on several manually tuned thresholds. A more adaptive approach could be developed by learning the optimal parameters from a larger set of annotated images.
One limitation of the system is that the ROI for removing inner contours is manually drawn. If the part geometry changes, the ROI must be redrawn. In the future, an automated method could be developed to recognize the inner holes and slots based on the template matching information, making the system more flexible. Additionally, the current algorithm uses a fixed defect length threshold of 90 pixels. Depending on the severity of the sand foundry defects and the quality requirements, this threshold may need to be adjusted. I recommend adding a “sensitivity” setting in the software that scales the threshold automatically.
Another area for future improvement is the use of deep learning for defect classification. While the hand-crafted line extraction method works well for the crack-like defects, it may fail for other types of sand foundry defects such as porosity or shrinkage cavities, which appear as clusters of small spots rather than elongated lines. A convolutional neural network trained on a large database of images could potentially detect a wider variety of defects with higher accuracy. However, deep learning models require a significant amount of labeled data and computational resources, which were not available within the scope of this thesis.
Finally, the software prototype should be stress-tested in a factory environment over several weeks to evaluate its stability and reliability under continuous operation. The current version runs as a desktop application; for production, it might be beneficial to integrate it with a PLC via a fieldbus protocol such as Profinet or EtherNet/IP, so that the rejection signal can be sent directly to the mechanical system. I have already left a placeholder in the software for a digital output module, but I have not yet implemented the driver.
Conclusion
In this thesis, I presented a complete visual inspection system for detecting surface defects in cast plate teeth, with a special focus on sand foundry defects that require fluorescent magnetic particle inspection. I designed and implemented the hardware setup, developed the image processing algorithms for target extraction and defect localization, and built a user-friendly software application that integrates with the industrial environment. The main contributions of my work are:
- A robust template matching approach that accurately locates the plate tooth and provides the pose information needed for defect detection.
- A defect detection algorithm based on line extraction and two-stage contour merging that effectively distinguishes real sand foundry defects from background noise.
- A complete software system with user privilege management, camera control, parameter settings, and automatic image saving.
- Laboratory validation showing that the system meets the speed and accuracy requirements for industrial deployment.
This work demonstrates that machine vision can successfully replace manual inspection in the challenging environment of sand foundry defect detection, thereby improving both worker safety and product quality. With further refinement and field testing, the proposed system can be seamlessly integrated into the existing production line, contributing to the advancement of intelligent manufacturing in the foundry industry.