In the manufacturing industry, the surface quality of castings directly affects the reliability and service life of mechanical equipment. Traditional manual visual inspection is inefficient and highly subjective, failing to meet the requirements of modern intelligent manufacturing. Through an in-depth analysis of the characteristics of sand foundry defects, I propose a series of deep-learning-based methods for automatic defect detection, model compression, and embedded deployment. The first part of this research introduces an improved real-time detection transformer called FSD-DETR, which adopts a lightweight StarNet backbone, an enhanced frequency channel attention module, a novel feature aggregation block, and a pixel-wise IoU loss. Experimental results on a self-built casting surface defect dataset demonstrate that the proposed model achieves 96.1% mAP@50 with only 13.09M parameters and 37.1 GFLOPs, which is significantly lighter and more accurate than the baseline. In the second part, I adopt the LAMP pruning strategy combined with channel-wise knowledge distillation to further reduce the model size. The final pruned and distilled model retains 96.1% mAP@50 while decreasing parameters to 7.4M and computational cost to 21.9 GFLOPs. The optimized model is successfully deployed on an NVIDIA Jetson Nano embedded device with a PyQt5-based visual detection system, enabling real-time detection at 397 FPS. The proposed approach offers a practical solution for high-precision and real-time detection of sand foundry defects in complex industrial environments.
1. Introduction
Casting is one of the most fundamental manufacturing processes for producing metal components with complex geometries. The surface quality of castings is a critical factor that influences their mechanical properties, fatigue life, and safety performance. In practice, various types of sand foundry defects, such as blowholes, shrinkage cavities, cracks, inclusions, and scratches, can appear on casting surfaces due to unstable process parameters, mold design issues, or material properties. These defects not only degrade the aesthetic quality of the final products but also significantly reduce their structural integrity. Therefore, reliable surface inspection is essential to ensure the quality of castings.
Traditional inspection methods include penetrant testing, eddy current testing, and ultrasonic testing. These non-destructive testing techniques are often time-consuming, require specialized equipment, and are difficult to automate. Visual inspection by human workers remains widely used in small and medium-sized foundries, but its accuracy and consistency are affected by fatigue, experience, and environmental factors. With the rapid development of industrial cameras and deep learning, automatic visual inspection based on convolutional neural networks and vision transformers has become a promising alternative for detecting sand foundry defects. However, existing deep learning detectors often suffer from a trade-off between detection accuracy and model complexity. Many high-precision models are too large to run on embedded devices with limited computational resources, which prevents real-time deployment on production lines.
To address these challenges, I propose a complete solution for surface defect detection of industrial castings. My contributions are fourfold. First, I construct a large-scale dataset of sand foundry defects using a ZED2 stereo camera in real industrial environments. The dataset contains eight typical defect categories and goes through a rigorous data cleaning and augmentation process. Second, I design an improved detection model named FSD-DETR based on RT-DETR. The model introduces a lightweight StarNet backbone, a frequency channel attention mechanism, a geometry-aware feature aggregation module, and a PIoU loss function. This model achieves high detection accuracy while maintaining a small parameter count and high inference speed. Third, I further compress the FSD-DETR model using the LAMP pruning strategy and compensate for potential performance degradation with channel-wise knowledge distillation. Finally, I deploy the optimized model on an NVIDIA Jetson Nano platform and develop a visual detection system based on PyQt5 for practical industrial use.
The rest of my thesis is organized as follows. Related work is introduced in Section 2. Section 3 describes the dataset construction process. Section 4 details the architecture of FSD-DETR. Section 5 presents experiments and analysis. Section 6 discusses model pruning and knowledge distillation. Section 7 introduces the embedded deployment and visual detection system. Conclusions are drawn in Section 8.
2. Related Work
2.1 Traditional Non-destructive Testing Technologies
Non-destructive testing (NDT) techniques are widely used to detect surface and internal defects without damaging the workpiece. Penetrant testing uses capillary action to draw a liquid penetrant into surface-breaking defects. Eddy current testing employs electromagnetic induction to identify surface and near-surface anomalies. Ultrasonic testing relies on high-frequency acoustic waves to detect internal cracks and inclusions. These methods have their own advantages and limitations. For instance, ultrasonic testing provides high sensitivity to internal defects but requires a coupling medium and skilled operators. Eddy current testing is fast and contactless but is only suitable for conductive materials. Penetrant testing is simple but only works for open surface defects. In addition, these NDT methods usually require manual interpretation of signals or images, making the inspection process subjective and difficult to automate. As a result, there is a growing demand for automated visual inspection solutions that can integrate seamlessly into modern casting production lines.
2.2 Traditional Object Detection Methods
Traditional object detection methods rely on handcrafted features and shallow machine learning classifiers. Common approaches include threshold segmentation, edge detection, morphological operations, and local binary patterns. These features are fed into classifiers such as support vector machines, AdaBoost, or Bayesian networks. For example, some researchers have used improved support vector machines with feature selection algorithms to recognize steel surface defects. Others have employed cascade classifiers to detect casting defects. However, the performance of these methods heavily depends on the design of handcrafted features and the choice of parameters. In complex industrial environments where lighting conditions, surface textures, and defect morphologies vary widely, traditional methods often suffer from poor generalization and low robustness.
2.3 Deep Learning-based Object Detection Methods
Deep learning has revolutionized the field of industrial defect detection. Two-stage detectors, such as Faster R-CNN and Mask R-CNN, first generate region proposals and then perform classification and regression. These models usually achieve high accuracy but are computationally expensive. Single-stage detectors, such as SSD and YOLO, directly predict object categories and locations in one forward pass, making them much faster. YOLO-based models have been extensively adopted for surface defect detection due to their real-time capability. However, many YOLO variants require anchor boxes and non-maximum suppression, which introduce additional hyperparameters and reduce robustness.
The emergence of Detection Transformer (DETR) models provides a new paradigm for end-to-end object detection. DETR eliminates the need for handcrafted anchors and NMS by treating detection as a set prediction problem. RT-DETR is a real-time DETR variant that balances speed and accuracy by using a hybrid encoder and an efficient decoder. In my work, I choose RT-DETR as the baseline. Despite its advantages, RT-DETR still contains a relatively large backbone and a significant number of parameters, which limits its deployment on resource-constrained embedded devices. Additionally, the original loss function in RT-DETR is not optimal for detecting small and irregular sand foundry defects. Consequently, I propose a series of modifications to enhance its feature extraction capability, detection accuracy, and efficiency.
3. Dataset Construction
3.1 Data Acquisition
To support the research on sand foundry defects detection, I collected a large number of casting surface images from actual industrial production lines. The images were captured with a ZED2 stereo camera, which provides high-resolution synchronized image streams. The camera was mounted at different angles to capture diverse views of the casting surfaces. The original images were recorded in 1280×1280 resolution. During data collection, I considered multiple defect categories: incompleteness, porosity, shrinkage depression, cracks, blowholes, protrusions, scratches, and slag inclusions. These categories represent the most common sand foundry defects encountered in real casting processes. The detailed parameters of the ZED2 camera are listed in the following table.
| Parameter | Value |
|---|---|
| Sensor type | 1/3” 4MP CMOS |
| Sensitivity | 1900 Mv/Lux-sec |
| Pixel size | 2 μm × 2 μm |
| Output format | YUV 4:2:2-UYVY (8 bits) |
In the first stage, I captured more than 15,000 raw images. The images were then sorted and randomly split into training, validation, and test sets with a ratio of 7:2:1. This self-built dataset is named the Casting Surface Defect Dataset (CSDD). All experiments in this thesis are based on this dataset unless otherwise stated.
3.2 Data Cleaning
Raw images often contain redundant, blurred, or overexposed samples that could degrade the training performance. To construct a high-quality dataset, I performed several data cleaning steps. First, I used perceptual hashing (pHash) to identify duplicate or highly similar images. The Hamming distance between hash values was computed to measure image similarity. If the distance was smaller than a threshold of 5, the image pair was considered a duplicate and one of them was removed. Second, I evaluated image sharpness using the Laplacian variance method. Images with a Laplacian variance below 60 were considered blurry and discarded. Third, I combined the fast Fourier transform (FFT) to analyze the frequency domain of each image. By masking the low-frequency region and computing the average high-frequency energy, I obtained a frequency-based sharpness score. Images with a score below 6 were removed. Finally, I manually reviewed the remaining images and discarded those with barely visible defects or severe quality loss.
For images with local overexposure, I applied an illumination correction method based on the QuickShift image segmentation algorithm. The process consisted of converting the image to grayscale, detecting bright regions with an intensity threshold, refining the mask by QuickShift superpixels, and applying mean filtering to the affected areas. This step reduced the effect of strong reflections and improved the visibility of the underlying surface details. After the multi-stage cleaning process, 4,000 high-quality images were retained as the foundation of the CSDD dataset.
3.3 Data Augmentation
To simulate the variability of real industrial environments, I applied three data augmentation strategies to the cleaned images. The first strategy is horizontal flipping. By mirroring the images, I increased the diversity of defect orientation and helped the model learn direction-invariant features. The second strategy is color jittering, which randomly adjusts brightness, contrast, and saturation. The brightness factor, contrast factor, and saturation factor were set to 0.3, 0.4, and 0.5, respectively. Color jittering simulates different illumination conditions and improves the model’s robustness to lighting variations. The third strategy is a slight perspective transformation. In this method, the four corners of an image are randomly shifted within a scale range of 0.1 of the image width or height. This operation simulates camera angle changes and geometric distortions that may occur on a real production line. After augmentation, the total dataset size increased to 4,168 images.
3.4 Dataset Annotation
To train the object detection model, I annotated all images using the open-source tool LabelImg. I chose the YOLO format because of its compatibility with many modern detection frameworks. In the YOLO format, each image corresponds to a text file in which every line represents a defect instance. The line contains the class index, the normalized center coordinates, and the normalized width and height. The distribution of defect labels in the CSDD dataset is summarized below.
| Defect Type | Incompleteness | Porosity | Shrinkage | Crack | Blowhole | Protrusion | Scratch | Slag Inclusion |
|---|---|---|---|---|---|---|---|---|
| Number of Labels | 1535 | 2124 | 659 | 873 | 501 | 746 | 686 | 815 |
The annotation process ensured that all defect instances were correctly localized and labeled. The resulting dataset provides a reliable benchmark for evaluating different detection algorithms for sand foundry defects. In addition, the dataset is diverse enough to support research on small object detection, fine-grained classification, and robustness to industrial noise.
4. FSD-DETR Model
4.1 Overall Architecture
FSD-DETR follows the general framework of RT-DETR, which consists of a backbone network, a hybrid encoder, and a transformer decoder. The backbone extracts multi-scale features from the input image. The hybrid encoder consists of an attention-based intra-scale feature interaction module and a cross-scale feature fusion module, which generates enhanced multi-scale representations. The decoder performs set prediction using learnable object queries. In my design, I introduce three major improvements to the baseline. First, I replace the original ResNet18 backbone with a lightweight StarNet network and embed a frequency channel attention mechanism into the feature extraction stage. The resulting backbone module is called Fstar Block. Second, I design a novel feature aggregation module named FRA to enhance multi-scale feature fusion with geometric perception capability. Third, I use PIoU loss instead of GIoU to improve the regression accuracy of small and irregular defects. These modifications jointly contribute to a significant improvement in detection performance while reducing model complexity.
4.2 Fstar Block with Frequency Channel Attention
The StarNet backbone adopts a simple yet effective structure based on star operation. The star operation performs element-wise multiplication on features from different subspaces, which implicitly projects inputs into a high-dimensional non-linear feature space. This property enables StarNet to achieve strong feature representation with a small number of parameters. In my implementation, I introduce the frequency channel attention (FCA) mechanism after the depthwise convolution branch in each StarNet block. FCA uses the discrete cosine transform (DCT) to decompose the input feature map into frequency components. The feature map is first divided along the channel dimension, and each sub-feature is transformed with DCT. Frequency selection is then applied to preserve the most informative components. After the DCT, the frequency features are squeezed by global average pooling, passed through a bottleneck with a reduction ratio r, and followed by a sigmoid activation to obtain channel weights. The weights are applied to the original feature map to emphasize important channels and suppress irrelevant background information. This mechanism significantly improves the model’s ability to detect small sand foundry defects in cluttered scenes.
4.3 FRA Feature Aggregation Module
The Feature-aware Relation Aggregation (FRA) module is designed to enhance multi-scale feature fusion. FRA contains two 1×1 convolution layers at the input and output ends. These convolutions reduce the channel dimension and enable efficient feature interaction. Between them, I use a DSConv layer, which stands for dynamic snake convolution. DSConv adapts its receptive field according to the geometric shape of the target by applying iterative offsets along the x-axis and y-axis. This property makes the convolution kernel more flexible when capturing elongated or curved defect structures. The DSConv operation is implemented by splitting the weights into a variable quantized kernel and two distribution shift components: kernel distribution shift and channel distribution shift. The offsets are generated iteratively to ensure continuity of the convolutional kernel. A residual connection is added around the FRA module to preserve the original information and avoid gradient vanishing. The FRA module improves the model’s capability to perceive fine-grained details and long-range dependencies, which is beneficial for detecting defects with varying scales and shapes.
4.4 PIoU Loss
The GIoU loss used in RT-DETR has a weak penalizing effect when the predicted bounding box and the ground-truth box have little or no overlap. This behavior can lead to unstable training in multi-object scenarios. To overcome this limitation, I adopt the Pixel-wise IoU (PIoU) loss. PIoU incorporates angle information and pixel-level overlap into the IoU computation, enabling a more accurate evaluation of the difference between two oriented boxes. For a directed rectangle \(b\) represented by \((c_x, c_y, w, h, \theta)\), a pixel point \(p_{i,j}\) can be projected to the center line. The distance from the pixel to the center is denoted by \(d_{i,j}\). The binary membership of a pixel inside the box is approximated by a kernel function \(K(d,s)\), which is a smooth step function defined as:
$$
K(d,s) = 1 – \frac{1}{1 + e^{-k(d-s)}}.
$$
The membership function for a pixel \(p_{i,j}\) with respect to box \(b\) is computed as:
$$
F(p_{i,j}|b) = K(d_{i,j}^{w}, w) K(d_{i,j}^{h}, h).
$$
Then the intersection area between two directed boxes \(b\) and \(b’\) is approximated by:
$$
S_{b \cap b’} \approx \sum_{p_{i,j} \in B_{b,b’}} F(p_{i,j}|b) F(p_{i,j}|b’).
$$
The union area is:
$$
S_{b \cup b’} \approx w \times h + w’ \times h’ – S_{b \cap b’}.
$$
Finally, the PIoU value is:
$$
PIoU(b,b’) = \frac{S_{b \cap b’}}{S_{b \cup b’}}.
$$
The PIoU loss is defined as:
$$
\mathcal{L}_{piou} = \frac{-\sum_{(b,b’) \in M} \ln PIoU(b,b’)}{|M|},
$$
where \(M\) denotes the set of positive samples. This loss function provides more effective gradient feedback for small and thin defects, which are common in sand foundry defects detection.
5. Experimental Results and Analysis
5.1 Experimental Setup and Evaluation Metrics
All experiments were conducted on a workstation with an Intel Xeon 8362 CPU, 45GB of RAM, and an NVIDIA RTX 3090 GPU with 24GB memory. The operating system was Ubuntu 22.04, and the deep learning framework was PyTorch 2.3.0 with CUDA 12.1. The input images were resized to 640×640. The models were trained for 120 epochs with a batch size of 8 and an initial learning rate of 0.0001. I used the AdamW optimizer without loading pretrained weights. To ensure fair comparison, all models were trained under the same data split and augmentation pipeline.
I used the following metrics to evaluate the detection performance: Precision (P), Recall (R), mean Average Precision at IoU threshold 0.5 (mAP@50), mean Average Precision averaged over IoU thresholds from 0.5 to 0.95 (mAP@50:95), model parameters, GFLOPs, and inference speed in frames per second (FPS). Precision and recall are defined as:
$$
P = \frac{TP}{TP + FP},
$$
$$
R = \frac{TP}{TP + FN},
$$
where \(TP\), \(FP\), and \(FN\) denote true positive, false positive, and false negative counts, respectively. The average precision for one class is the area under the precision-recall curve:
$$
AP = \int_0^1 P(R) \, dR.
$$
Then mAP is computed as:
$$
mAP = \frac{1}{N} \sum_{i=1}^{N} AP_i,
$$
where \(N\) is the number of classes.
5.2 Comparison of Backbone Networks
To verify the effectiveness of the lightweight backbone, I replaced the original ResNet18 with several lightweight backbones, including EfficientViT, MobileNet, RepViT, UniRepLKNet, and StarNet. The comparison results are shown in the following table.
| Backbone | Parameters (M) | mAP@50 (%) | mAP@50:95 (%) |
|---|---|---|---|
| ResNet18 | 19.9 | 94.4 | 65.3 |
| EfficientViT | 10.7 | 91.8 | 59.4 |
| MobileNet | 11.3 | 90.7 | 60.6 |
| RepViT | 13.3 | 91.8 | 61.7 |
| UniRepLKNet | 12.7 | 91.0 | 59.8 |
| StarNet | 12.0 | 95.0 | 65.7 |
The results show that StarNet achieves the best mAP@50 and mAP@50:95 among all lightweight backbones while maintaining a relatively small parameter count. Compared to ResNet18, StarNet reduces parameters by 39.7% and improves mAP@50 by 0.6%. Therefore, I select StarNet as the backbone of FSD-DETR.
5.3 Comparison of Attention Modules
I integrated different attention mechanisms into the StarNet backbone to investigate their contributions to detection performance. The compared attention modules include CA, GC, ECA, and FCA. The experimental results are listed below.
| Model | Parameters (M) | mAP@50 (%) | mAP@50:95 (%) |
|---|---|---|---|
| StarNet | 12.00 | 95.0 | 65.7 |
| +CA | 12.03 | 91.9 | 60.8 |
| +GC | 12.12 | 95.5 | 65.4 |
| +ECA | 12.00 | 95.8 | 65.8 |
| +FCA | 12.03 | 95.7 | 66.5 |
FCA provides the highest mAP@50:95 of 66.5% and a competitive mAP@50 of 95.7%. Compared with the original StarNet, FCA improves mAP@50:95 by 0.8%. The parameter increase is negligible. Therefore, FCA is chosen for the proposed Fstar Block.
5.4 Comparison of Loss Functions
I evaluated several IoU loss functions in the RT-DETR framework, including GIoU, CIoU, DIoU, EIoU, SIoU, and PIoU. The results are shown in the following table.
| Loss Function | mAP@50 (%) | mAP@50:95 (%) |
|---|---|---|
| GIoU | 94.4 | 65.3 |
| CIoU | 94.6 | 65.9 |
| DIoU | 94.9 | 65.5 |
| EIoU | 95.2 | 66.1 |
| SIoU | 95.2 | 66.2 |
| PIoU | 95.2 | 66.4 |
The PIoU loss achieves the highest mAP@50:95 value, demonstrating its superiority in precise localization. Although its mAP@50 is the same as EIoU and SIoU, the improvement in mAP@50:95 indicates that PIoU helps the model generate higher-quality bounding boxes. This is particularly important for sand foundry defects, which often have irregular boundaries and small geometric sizes.
5.5 Hyperparameter Analysis
I conducted a series of experiments to investigate the influence of batch size, learning rate, and training epochs on the final FSD-DETR model. The results are summarized as follows.
| Group | Batch Size | Learning Rate | Epoch | Training Time (h) | mAP@50 (%) | mAP@50:95 (%) |
|---|---|---|---|---|---|---|
| 1 | 4 | 0.0001 | 120 | 4.920 | 95.2 | 65.9 |
| 2 | 8 | 0.0001 | 120 | 2.936 | 96.1 | 67.4 |
| 3 | 16 | 0.0001 | 120 | 2.384 | 95.5 | 66.2 |
| 4 | 8 | 0.001 | 120 | 3.021 | 93.0 | 61.3 |
| 5 | 8 | 0.0005 | 120 | 2.918 | 94.7 | 65.1 |
| 6 | 8 | 0.00005 | 120 | 3.120 | 95.6 | 65.5 |
| 7 | 8 | 0.0001 | 100 | 2.392 | 95.6 | 66.3 |
| 8 | 8 | 0.0001 | 150 | 3.614 | 95.9 | 67.0 |
Batch size of 8 offers the best trade-off between training speed and accuracy. A learning rate of 0.0001 is optimal for stable convergence. Moreover, 120 epochs are sufficient to achieve the best performance without overfitting. These hyperparameters are used in all subsequent experiments.
5.6 Comparison with Mainstream Detectors
I compared FSD-DETR with several state-of-the-art object detection models, including YOLOv5m, YOLOv8m, YOLOv10m, YOLO11m, and RT-DETR-L. The comparison results on the CSDD dataset are shown below.
| Model | Parameters (M) | GFLOPs (G) | Precision (%) | Recall (%) | mAP@50 (%) | mAP@50:95 (%) | FPS |
|---|---|---|---|---|---|---|---|
| YOLOv5m | 25.05 | 64.0 | 91.7 | 85.8 | 91.9 | 61.8 | 103.5 |
| YOLOv8m | 25.84 | 78.7 | 93.4 | 87.3 | 92.7 | 63.2 | 87.2 |
| YOLOv10m | 16.46 | 63.5 | 89.3 | 80.7 | 89.3 | 61.6 | 87.6 |
| YOLO11m | 20.04 | 67.7 | 91.6 | 84.8 | 92.1 | 61.9 | 87.7 |
| RT-DETR-L | 32.00 | 103.5 | 93.4 | 89.7 | 94.0 | 64.7 | 53.5 |
| FSD-DETR (Ours) | 13.09 | 37.1 | 94.8 | 93.2 | 96.1 | 67.4 | 92.3 |
FSD-DETR achieves the highest precision, recall, mAP@50, and mAP@50:95 while maintaining the second-highest FPS. Compared to RT-DETR-L, the parameters and GFLOPs are reduced by 59.1% and 64.1%, respectively, while mAP@50 is improved by 2.1%. The proposed model also surpasses all YOLO variants in accuracy, demonstrating its superiority for sand foundry defects detection.
I also evaluated the per-class detection results for all eight defect categories. The following table presents the mAP@50 values for each defect type.
| Defect | Incompleteness | Porosity | Shrinkage | Crack | Blowhole | Protrusion | Scratch | Slag Inclusion |
|---|---|---|---|---|---|---|---|---|
| YOLOv5m | 96.4 | 77.4 | 93.4 | 92.8 | 96.8 | 93.4 | 86.3 | 98.4 |
| YOLOv8m | 96.6 | 79.3 | 96.2 | 93.7 | 97.1 | 93.7 | 87.0 | 98.1 |
| YOLOv10m | 95.0 | 74.5 | 90.9 | 87.0 | 95.3 | 91.2 | 82.7 | 98.1 |
| YOLO11m | 97.3 | 78.0 | 91.9 | 94.2 | 96.9 | 94.6 | 85.1 | 99.0 |
| RT-DETR-L | 98.3 | 83.9 | 96.3 | 93.5 | 97.3 | 93.7 | 89.8 | 99.5 |
| FSD-DETR | 98.7 | 86.5 | 98.5 | 97.6 | 97.3 | 95.9 | 95.0 | 99.4 |
FSD-DETR achieves the best or near-best results for all categories. In particular, it significantly improves the detection of porosity, shrinkage, crack, and scratch defects. These defect types often have small areas or elongated shapes, which are difficult for existing detectors. The experimental results confirm the effectiveness of the proposed feature aggregation and loss function improvements.
5.7 Ablation Study
To analyze the contribution of each component, I performed an ablation study. The baseline is RT-DETR with ResNet18. H1 indicates the replacement of the backbone with StarNet. H2 indicates the addition of FCA. H3 indicates the addition of FRA. H4 indicates the replacement of the loss function with PIoU. The results are shown in the following table.
| Model | H1 | H2 | H3 | H4 | Parameters (M) | GFLOPs (G) | mAP@50 (%) | mAP@50:95 (%) |
|---|---|---|---|---|---|---|---|---|
| Baseline | 19.88 | 57.0 | 94.4 | 65.3 | ||||
| A | √ | 12.00 | 31.8 | 95.0 | 65.7 | |||
| B | √ | √ | 12.03 | 31.8 | 95.7 | 66.5 | ||
| C | √ | √ | 13.06 | 37.1 | 95.9 | 66.7 | ||
| D | √ | √ | 12.00 | 31.8 | 95.8 | 66.4 | ||
| E | √ | √ | √ | 13.09 | 37.1 | 96.2 | 66.7 | |
| FSD-DETR | √ | √ | √ | √ | 13.09 | 37.1 | 96.1 | 67.4 |
Each modification brings a consistent improvement in accuracy. The final FSD-DETR model achieves a 1.7% improvement in mAP@50 and a 2.1% improvement in mAP@50:95 while reducing parameters by 34.2% and GFLOPs by 34.9%. These results demonstrate that the combination of lightweight backbone, frequency attention, feature aggregation, and PIoU loss yields a highly effective detector for sand foundry defects.
5.8 Generalization Experiments
To verify the generalization ability of FSD-DETR, I conducted experiments on the public NEU-DET steel surface defect dataset. This dataset contains six types of surface defects with complex textures and backgrounds. The results are shown below.
| Model | Crazing (%) | Patches (%) | Inclusion (%) | Pitted Surface (%) | Rolled-in Scale (%) | Scratches (%) | mAP@50 (%) |
|---|---|---|---|---|---|---|---|
| YOLOv5 | 47 | 92.2 | 78.2 | 77.1 | 44.9 | 71 | 65.7 |
| Faster-RCNN | 37.6 | 85.3 | 80.2 | 81.5 | 54 | 89.2 | 71.3 |
| Retina-Net | 45.9 | 91.1 | 84.2 | 74.7 | 43.5 | 81.6 | 70.2 |
| Multi-Scale YOLOv5 | 34.5 | 89.8 | 76.8 | 82.5 | 61.6 | 86.8 | 72.0 |
| Fine-YOLO | – | – | – | – | – | – | 73.1 |
| NHD-YOLOx | – | – | – | – | – | – | 74.3 |
| RT-DETR-r18 | 24.3 | 91.1 | 78.8 | 75.2 | 59.9 | 86.7 | 69.3 |
| FSD-DETR | 51.5 | 84.9 | 81.7 | 76.7 | 61.8 | 91.1 | 74.6 |
FSD-DETR achieves the highest mAP@50 of 74.6%, outperforming all compared models. The improvement over the baseline RT-DETR-r18 is 5.3 percentage points. This demonstrates that the proposed model generalizes well to different industrial surface defect datasets and is not overfitted to a specific dataset.
I also evaluated FSD-DETR under different lighting conditions on the CSDD dataset. The model shows consistent performance under normal, left-side, and right-side illumination, which confirms its robustness to lighting variations. Such robustness is essential for real industrial environments where lighting changes frequently.
6. Lightweight Model Compression via Pruning and Knowledge Distillation
6.1 Model Pruning
Although FSD-DETR is already lightweight, further compression is needed for deployment on resource-constrained embedded devices. Model pruning is an effective technique to reduce redundancy in neural networks. I compared four pruning strategies: L1 pruning, Group Hessian pruning, Group Taylor pruning, and Layer-adaptive Magnitude-based Pruning (LAMP).
L1 pruning removes weights with small L1 norms. The L1 norm of a weight vector \(w\) is:
$$
\|w\|_1 = \sum_{i=1}^{n} |w_i|.
$$
Group Hessian pruning uses second-order derivative information to evaluate the importance of weight groups. The Hessian matrix is defined as:
$$
H = \frac{\partial^2 \mathcal{L}}{\partial w^2}.
$$
Group Taylor pruning uses a second-order Taylor expansion of the loss function:
$$
\mathcal{L}(w + \Delta w) \approx \mathcal{L}(w) + \nabla \mathcal{L}(w)^T \Delta w + \frac{1}{2} \Delta w^T H \Delta w.
$$
LAMP pruning assigns a score to each weight based on its magnitude and the cumulative magnitude of larger weights in the same layer. The score for weight \(u\) is:
$$
score(u, w) = \frac{w(u)^2}{\sum_{v \geq u} w(v)^2}.
$$
Weights with lower scores are pruned first. This strategy ensures that important layers retain more weights while redundant layers are pruned more aggressively.
I applied these pruning methods to the FSD-DETR model with pruning coefficients \(P\) equal to 1.2, 1.4, and 1.6. The pruning coefficient is defined as the ratio of the original computational cost \(F\) to the pruned computational cost \(F’\):
$$
P = \frac{F}{F’}.
$$
The experimental results are summarized in the following table.
| Pruning Method | P | Parameters (M) | GFLOPs (G) | mAP@50 (%) | FPS |
|---|---|---|---|---|---|
| Baseline | – | 19.9 | 57.0 | 94.4 | 90.1 |
| FSD-DETR | – | 13.1 | 37.1 | 96.1 | 92.3 |
| L1 | 1.2 | 10.7 | 30.4 | 94.6 | 138.9 |
| L1 | 1.4 | 9.1 | 26.0 | 92.7 | 333.3 |
| L1 | 1.6 | 7.6 | 22.2 | 92.8 | 233.0 |
| Group Hessian | 1.2 | 9.9 | 30.1 | 96.0 | 175.4 |
| Group Hessian | 1.4 | 8.8 | 26.1 | 95.9 | 370.4 |
| Group Hessian | 1.6 | 7.9 | 22.9 | 95.0 | 238.1 |
| Group Taylor | 1.2 | 9.7 | 30.0 | 96.0 | 181.8 |
| Group Taylor | 1.4 | 8.5 | 26.0 | 95.4 | 153.8 |
| Group Taylor | 1.6 | 7.4 | 22.3 | 95.4 | 322.6 |
| LAMP | 1.2 | 9.6 | 30.0 | 96.1 | 166.7 |
| LAMP | 1.4 | 8.3 | 25.8 | 96.1 | 370.4 |
| LAMP | 1.6 | 7.4 | 21.9 | 96.0 | 400.0 |
LAMP pruning achieves the best overall trade-off. At \(P=1.6\), the model retains 96.0% mAP@50 with only 7.4M parameters and 21.9 GFLOPs. The inference speed is increased to 400 FPS, which is more than four times faster than the original FSD-DETR. Therefore, I select LAMP with \(P=1.6\) as the final pruning strategy.
6.2 Knowledge Distillation
Pruning can slightly degrade the detection accuracy. To recover the performance, I adopt channel-wise knowledge distillation (CWD). In this framework, the original FSD-DETR serves as the teacher model, and the pruned FSD-DETR serves as the student model. CWD aligns the channel-wise feature distributions between the teacher and the student using KL divergence. The student model is trained to mimic the teacher’s feature representations, which helps it retain the most discriminative information after pruning.
The experimental results are shown in the following table.
| Model | Parameters (M) | GFLOPs (G) | mAP@50 (%) | FPS |
|---|---|---|---|---|
| Teacher (FSD-DETR) | 13.1 | 37.1 | 96.1 | 92.3 |
| Student (FSD-DETR-LAMP) | 7.4 | 21.9 | 96.0 | 400.0 |
| Student + CWD | 7.4 | 21.9 | 96.1 | 397.0 |
After CWD distillation, the student model recovers to 96.1% mAP@50 while maintaining a negligible loss in inference speed. The final model is both compact and accurate, making it suitable for real-time detection on embedded devices such as Jetson Nano.
7. Embedded Deployment and Visualization Detection System
7.1 Requirements and Hardware Platform
To deploy the lightweight model in an industrial environment, I selected the NVIDIA Jetson Nano as the embedded computing platform. The Jetson Nano offers a good balance between computational capability, power consumption, and cost. It integrates a 128-core NVIDIA Maxwell GPU and a quad-core ARM Cortex-A57 CPU. The platform provides 4GB of LPDDR4 memory and supports CUDA, cuDNN, and TensorRT. The detailed hardware specifications are listed below.
| Item | Specification |
|---|---|
| GPU | 128-core NVIDIA Maxwell |
| CPU | Quad-core ARM Cortex-A57 @ 1.43 GHz |
| Memory | 4GB 64-bit LPDDR4 25.6GB/s |
| Storage | 16GB eMMC 5.1 |
| AI Performance | 473 GFLOPS |
| Camera Interface | 2 MIPI CSI-2 DPHY lanes |
| USB | 4 USB 3.0, USB 2.0 Micro-B |
For image acquisition, I used an IMX219 camera module with a CMOS sensor. Its main parameters are shown below.
| Parameter | Value |
|---|---|
| Optical size | 1/4 inch |
| Resolution | 8 megapixels (3280×2464) |
| Frame rate | 1080p @ 30 FPS |
| Interface | MIPI CSI-2 |
The Jetson Nano was installed with JetPack SDK, which includes CUDA, cuDNN, and other necessary libraries. The model was converted and optimized for inference using PyTorch. The final pruned model was loaded directly into the PyTorch environment for deployment.
7.2 System Architecture
I designed a visual inspection system based on PyQt5. The system architecture is composed of four functional modules: system initialization, data input, detection execution, and result output. The system initialization module loads the model configuration and sets the confidence threshold and IoU threshold. The data input module supports two modes: loading a local image or opening a camera stream. The detection execution module runs the FSD-DETR model on the input image and produces bounding boxes, class labels, and confidence scores. The result output module displays the annotated image and saves the detection results to local files.
7.3 User Interface and Functional Verification
The main interface of the detection system includes a login window and a main detection window. The login window provides user registration and authentication to ensure system security. The main detection window contains buttons for parameter adjustment, model configuration, image loading, camera activation, detection execution, and result saving. Two image display panels show the original image and the annotated result side by side. A text panel logs the detection process and outputs the number of detected objects for each defect category.
In the actual detection test, the system successfully detects sand foundry defects on the embedded Jetson Nano platform. The annotation boxes are displayed with their corresponding classes and confidence values. The system also allows users to save the visualized results and a structured detection report. The saved files are stored in a specified directory for quality traceability. The exit process of the system is carefully designed to release camera resources and delete temporary files after confirmation.
The deployment results confirm that the pruned and distilled FSD-DETR model runs efficiently on the embedded platform. The inference speed is sufficiently high for real-time quality inspection. The integration of the PyQt5 interface improves the usability of the system, enabling operators without programming skills to perform defect detection easily. This demonstrates the engineering practicality of the proposed algorithm.
8. Conclusion
In this thesis, I have presented a comprehensive research work on the detection of sand foundry defects using deep learning techniques. I first constructed a large-scale dataset of casting surface defects by collecting images from real industrial production lines and applying rigorous cleaning and augmentation procedures. The resulting CSDD dataset covers eight common defect types and provides a solid basis for model training and evaluation.
I then proposed an improved detection model named FSD-DETR. The model integrates a lightweight StarNet backbone, a frequency channel attention module, a geometry-aware feature aggregation module, and a PIoU loss function. Extensive experiments show that FSD-DETR achieves 96.1% mAP@50 with only 13.09M parameters and 37.1 GFLOPs, outperforming many mainstream detectors while maintaining high inference speed. The generalization experiments on the NEU-DET dataset further demonstrate that the proposed model has strong adaptability to different industrial surface defect scenarios.
To satisfy the requirements of embedded deployment, I applied LAMP pruning and channel-wise knowledge distillation. The pruned and distilled model retains the same detection accuracy while reducing parameters to 7.4M and computational cost to 21.9 GFLOPs. The inference speed reaches 397 FPS, which greatly exceeds the real-time requirement. The final model was successfully deployed on an NVIDIA Jetson Nano and integrated into a PyQt5-based visualization system. The system supports both image and real-time video detection, providing a practical tool for intelligent quality inspection in casting production.
Future research directions include exploring more advanced model compression techniques such as low-bit quantization and neural architecture search. In addition, multi-modal information fusion, such as combining visual images with depth data or acoustic signals, can be investigated to further improve the detection of complex sand foundry defects. Online learning mechanisms can also be developed to continuously adapt the model to new production conditions. These efforts will help bring intelligent defect detection systems into broader industrial applications, ultimately improving the automation and intelligence level of the casting industry.