In my research, I focus on the detection of sand foundry defect on industrial castings. The casting industry is one of the fundamental pillars of equipment manufacturing, and the surface quality of castings is directly related to the service performance and operational safety of mechanical products. In real manufacturing environments, casting defects such as incomplete shapes, blowholes, shrinkage cavities, cracks, gas pores, protrusions, scratches, and slag holes are commonly observed. Traditional manual inspection methods are inefficient, subjective, and difficult to maintain stable quality. Therefore, an intelligent, automated, and reliable sand foundry defect detection method is urgently needed. Deep learning has brought a new technical route for visual inspection, but many existing models still suffer from redundant parameters, high computational cost, and unsatisfactory performance on small or low-contrast defects. My dissertation proposes a lightweight and high-precision detection framework for sand foundry defect, aiming to solve the contradiction between accuracy, speed, and model complexity in industrial deployment.
In this study, I present a complete pipeline that includes dataset construction, model design, model compression, and embedded system deployment. The core contribution is an improved detection model named FSD-DETR, which is based on the real-time detection transformer framework. I also introduce a pruning and distillation strategy to further reduce the model size while preserving detection accuracy on sand foundry defect images. Finally, I deploy the optimized model on an embedded platform and implement a visual inspection system with a graphical user interface. The proposed method is validated on both a self-built casting surface defect dataset and a public steel surface defect dataset. My experiments demonstrate that the proposed approach achieves a good balance between detection precision, inference speed, and computational efficiency, which is meaningful for online quality control in foundry production.
1. Dataset Construction for Sand Foundry Defect Detection
High-quality data are essential for training a robust sand foundry defect detector. I used a ZED2 stereo camera to collect casting surface images from multiple angles in an industrial environment. The camera provides high-resolution images with 1280×1280 pixels, which are suitable for capturing fine defect details. The dataset includes eight typical sand foundry defect categories: incomplete, blowhole, shrinkage, crack, gas pore, protrusion, scratch, and slag hole. These defects are diverse in shape, size, and contrast, making the detection task challenging and representative of real industrial conditions.
To ensure data quality, I applied a systematic cleaning process. First, perceptual hashing was used to remove duplicate or highly similar images, and a Hamming distance threshold was set to identify redundant samples. Second, blur detection based on Laplacian variance and fast Fourier transform was employed to eliminate low-quality images. Third, the QuickShift image segmentation algorithm was used to correct local overexposure caused by reflective regions on the casting surface. These steps effectively reduced the amount of noise and redundancy in the raw image set. After cleaning, 4,000 high-quality images were retained for annotation and model development.
To improve the robustness of the sand foundry defect detection model, I applied several data augmentation techniques, including horizontal flipping, color jittering, and slight perspective transformation. These operations simulate variations in viewpoint, illumination, and surface appearance, which are common in real production lines. After augmentation, the total number of images was increased to 4,168. The dataset was split into training, validation, and testing sets with a ratio of 7:2:1. All defect instances were manually annotated using the LabelImg tool, and the bounding boxes were represented in YOLO format. The distribution of defect labels in the CSDD dataset is summarized in the following table.
| Defect Type | Number of Labels |
|---|---|
| Incomplete | 1,535 |
| Blowhole | 2,124 |
| Shrinkage | 659 |
| Crack | 873 |
| Gas pore | 501 |
| Protrusion | 746 |
| Scratch | 686 |
| Slag hole | 815 |
The table reveals that the distribution of sand foundry defect types is not balanced. Blowholes and incomplete defects are more frequent, while gas pores and scratches are less common. This imbalance increases the difficulty of training a generalizable detector. In my model design, I chose robust feature extraction and loss functions to alleviate this problem. The constructed dataset provides a reliable basis for evaluating the effectiveness of the proposed FSD-DETR model.
2. Baseline RT-DETR and Its Application to Sand Foundry Defect Detection
The real-time detection transformer (RT-DETR) is a modern object detection framework that reformulates detection as a set prediction problem. It uses a transformer encoder-decoder architecture and bipartite matching to eliminate the need for anchor boxes and non-maximum suppression. The RT-DETR-r18 variant uses ResNet18 as the backbone network, which produces multi-scale feature maps at different resolutions. These features are processed by a hybrid encoder composed of an attention-based intra-scale feature interaction module and a cross-scale feature fusion module. The decoder then uses object queries to predict class labels and bounding boxes. Although RT-DETR achieves competitive accuracy and speed, its parameter scale and computational overhead remain high for embedded deployment in sand foundry defect detection systems. I therefore propose several structural improvements to reduce model size and enhance feature representation.
The overall architecture of my FSD-DETR model consists of three main components: a lightweight backbone network, a hybrid encoder, and a transformer decoder. The backbone extracts multi-scale features from sand foundry defect images. The hybrid encoder enhances semantic representation through global attention and cross-scale fusion. The decoder performs final object classification and localization. In the following sections, I describe each improvement in detail, including the Fstar Block, the FRA feature aggregation module, and the PIoU loss function.
3. FSD-DETR Design for Sand Foundry Defect Detection
3.1 StarNet Backbone and Fstar Block
In order to reduce the computation cost of the baseline model, I replaced the ResNet18 backbone with StarNet. StarNet is a lightweight neural network architecture that uses star operations, which are based on element-wise multiplication of features from different subspaces. The star operation maps input features into a high-dimensional nonlinear feature space in an implicit way, similar to polynomial kernel expansion. This design enables the network to achieve strong feature representation capability while maintaining a compact structure. Experimental results show that StarNet can significantly reduce both parameters and FLOPs compared with ResNet18, while improving the detection accuracy for sand foundry defect.
To further enhance the feature extraction ability in complex backgrounds, I introduced a frequency channel attention mechanism (FCA) into the StarNet backbone. The improved module is named Fstar Block. The FCA mechanism uses discrete cosine transform to convert feature maps from the spatial domain to the frequency domain. It first splits the input feature map along the channel dimension into several sub-feature maps, then applies DCT to extract frequency components. A frequency selection strategy is used to preserve representative frequency components. After reducing the channel dimension through global average pooling, a fully connected layer produces channel-wise weights via the sigmoid function. These weights are applied to the original features through element-wise multiplication. This process strengthens important channels and suppresses irrelevant background information, which is valuable for detecting small and low-contrast sand foundry defect.
| Backbone Network | Parameters (M) | mAP@50 (%) | mAP@50:95 (%) |
|---|---|---|---|
| ResNet18 | 19.9 | 94.4 | 65.3 |
| EfficientViT | 10.7 | 91.8 | 59.4 |
| MobileNet | 11.3 | 90.7 | 60.6 |
| RepViT | 13.3 | 91.8 | 61.7 |
| UniRepLKNet | 12.7 | 91.0 | 59.8 |
| StarNet | 12.0 | 95.0 | 65.7 |
The table shows that StarNet outperforms the other lightweight backbones in sand foundry defect detection. Compared with ResNet18, StarNet reduces parameters by nearly 40% and improves mAP@50 by 0.6%. This demonstrates that the star operation can serve as an efficient alternative to traditional residual blocks in industrial defect detection tasks.
I further compared different attention mechanisms integrated into the StarNet backbone. The results are summarized below.
| Model | Parameters (M) | mAP@50 (%) | mAP@50:95 (%) |
|---|---|---|---|
| StarNet | 12.00 | 95.0 | 65.7 |
| StarNet + CA | 12.03 | 91.9 | 60.8 |
| StarNet + GC | 12.12 | 95.5 | 65.4 |
| StarNet + ECA | 12.00 | 95.8 | 65.8 |
| StarNet + FCA | 12.03 | 95.7 | 66.5 |
The comparison shows that the FCA module provides the best mAP@50:95 improvement with minimal additional parameters. The frequency-domain modeling capability of FCA helps the model focus on sand foundry defect regions with rich texture and edge information, which is especially useful for crack and scratch detection.
3.2 FRA Feature Aggregation Module
Multi-scale feature aggregation is critical for detecting sand foundry defect with large scale diversity. I designed a feature-aware relation aggregation (FRA) module that enhances geometric perception. The FRA module uses two 1×1 convolution layers at the input and output stages to compress features and reduce computation. Between these layers, a DSConv module is used. DSConv is an efficient convolutional operator that decomposes a standard convolution into a variable quantized kernel and distribution shifts. The distribution offsets allow the convolution kernel to adaptively adjust its receptive field according to the geometric shape of the target. This property is advantageous for detecting elongated defects such as cracks and scratches in sand foundry defect images.
The DSConv module uses an iterative offset strategy to maintain the continuity of the kernel. For a kernel \(K\), the positions along the x-axis can be expressed as:
$$
K_{i\pm c} =
\begin{cases}
(x_{i+c}, y_{i+c}) = (x_i + c, y_i + \sum_{i}^{i+c} \Delta y_{i+c}) \\
(x_{i-c}, y_{i-c}) = (x_i – c, y_i + \sum_{i-c}^{i} \Delta y_i)
\end{cases}
$$
Similarly, the positions along the y-axis are defined as:
$$
K_{j\pm c} =
\begin{cases}
(x_{j+c}, y_{j+c}) = (x_j + \sum_{j}^{j+c} \Delta x_{j+c}, y_j + c) \\
(x_{j-c}, y_{j-c}) = (x_j + \sum_{j-c}^{j} \Delta y_j, y_j – c)
\end{cases}
$$
By using these offset equations, the DSConv kernel can adapt to curved or elongated structures. The FRA module also adopts a residual connection to add the input features to the module output. This residual design prevents gradient degradation and improves feature flow. In my experiments, adding the FRA module to FSD-DETR improved mAP@50 by about 0.9%, while increasing the computational cost only slightly.
3.3 PIoU Loss Function
Boundary box regression is a key component in sand foundry defect detection. The original RT-DETR uses GIoU loss, which may not provide sufficient gradient information when the predicted box and ground-truth box have small or no overlap. This problem is particularly serious for small defects that occupy only a few pixels. To solve this problem, I introduced the Pixels-IoU (PIoU) loss function. PIoU considers the pixel-level intersection area between two oriented rectangles and introduces angle information into the overlap calculation.
For a pixel \(p_{i,j}\), the function that determines whether it lies inside an oriented rectangle \(b\) is defined as:
$$
F(p_{i,j}|b) = K(d_{i,j}^{w}, w) K(d_{i,j}^{h}, h)
$$
where the kernel \(K(d, s)\) is defined as:
$$
K(d, s) = 1 – \frac{1}{1 + e^{-k(d-s)}}
$$
The intersection area between two oriented rectangles \(b\) and \(b’\) is approximated by:
$$
S_{b \cap b’} \approx \sum_{p_{i,j} \in B_{b,b’}} F(p_{i,j}|b) F(p_{i,j}|b’)
$$
The union area is then computed as:
$$
S_{b \cup b’} \approx w \times h + w’ \times h’ – S_{b \cap b’}
$$
The PIoU metric is defined as:
$$
PIoU(b, b’) = \frac{S_{b \cap b’}}{S_{b \cup b’}}
$$
Finally, the PIoU loss is written as:
$$
L_{piou} = – \frac{\sum_{(b,b’) \in M} \ln PIoU(b,b’)}{|M|}
$$
I compared PIoU with several popular IoU-based loss functions including GIoU, CIoU, DIoU, EIoU, and SIoU. The experimental results are shown in the following table.
| Loss Function | mAP@50 (%) | mAP@50:95 (%) |
|---|---|---|
| GIoU | 94.4 | 65.3 |
| CIoU | 94.6 | 65.9 |
| DIoU | 94.9 | 65.5 |
| EIoU | 95.2 | 66.1 |
| SIoU | 95.2 | 66.2 |
| PIoU | 95.2 | 66.4 |
From the table, PIoU achieves the highest mAP@50:95 among all compared loss functions. This indicates that PIoU improves the localization quality of the model, which is crucial for detecting small and closely packed sand foundry defect.
4. Experiments and Analysis
All experiments were performed under a unified environment. I used an Ubuntu 22.04 operating system, an Intel Xeon processor, 45 GB of memory, and an NVIDIA RTX 3090 GPU with 24 GB memory. The software environment included CUDA 12.1 and PyTorch 2.3.0. During training, no pre-trained weights were loaded. The input images were resized to 640×640. I trained the models for 120 epochs with a batch size of 8 and used the AdamW optimizer. The initial learning rate was set to 0.0001. These hyperparameters were selected based on a series of ablation experiments.
To evaluate the performance of sand foundry defect detection models, I used the following metrics. Precision and recall are defined as:
$$
P = \frac{TP}{TP + FP}
$$
$$
R = \frac{TP}{TP + FN}
$$
The average precision is the area under the precision-recall curve:
$$
AP = \int_{0}^{1} P(R) dR
$$
The mean average precision is the average over all classes:
$$
mAP = \frac{1}{N} \sum_{i=1}^{N} AP_i
$$
In addition, I used parameters and GFLOPs to measure model complexity, and FPS to measure inference speed. These metrics provide a comprehensive view of the model performance in industrial sand foundry defect detection scenarios.
4.1 Hyperparameter Analysis
I conducted hyperparameter experiments to determine the best training configuration. The results are listed in the following table.
| Group | Batch Size | Learning Rate | Epochs | Training Time (h) | mAP@50 (%) | mAP@50:95 (%) |
|---|---|---|---|---|---|---|
| 1 | 4 | 0.0001 | 120 | 4.920 | 95.2 | 65.9 |
| 2 | 8 | 0.0001 | 120 | 2.936 | 96.1 | 67.4 |
| 3 | 16 | 0.0001 | 120 | 2.384 | 95.5 | 66.2 |
| 4 | 8 | 0.001 | 120 | 3.021 | 93.0 | 61.3 |
| 5 | 8 | 0.0005 | 120 | 2.918 | 94.7 | 65.1 |
| 6 | 8 | 0.00005 | 120 | 3.120 | 95.6 | 65.5 |
| 7 | 8 | 0.0001 | 100 | 2.392 | 95.6 | 66.3 |
| 8 | 8 | 0.0001 | 150 | 3.614 | 95.9 | 67.0 |
From the results, the optimal configuration is group 2, where batch size is 8, learning rate is 0.0001, and the number of epochs is 120. This configuration yields the highest mAP@50 and mAP@50:95 while maintaining a reasonable training time. I used this configuration for all subsequent experiments.
4.2 Comparison with Mainstream Models on CSDD
To validate the effectiveness of FSD-DETR, I compared it with YOLOv5m, YOLOv8m, YOLOv10m, YOLO11m, and RT-DETR-L. The comparison results are shown in the following table.
| Model | Parameters (M) | GFLOPs (G) | Precision (%) | Recall (%) | mAP@50 (%) | mAP@50:95 (%) | FPS |
|---|---|---|---|---|---|---|---|
| YOLOv5m | 25.05 | 64.0 | 91.7 | 85.8 | 91.9 | 61.8 | 103.5 |
| YOLOv8m | 25.84 | 78.7 | 93.4 | 87.3 | 92.7 | 63.2 | 87.2 |
| YOLOv10m | 16.46 | 63.5 | 89.3 | 80.7 | 89.3 | 61.6 | 87.6 |
| YOLO11m | 20.04 | 67.7 | 91.6 | 84.8 | 92.1 | 61.9 | 87.7 |
| RT-DETR-L | 32.00 | 103.5 | 93.4 | 89.7 | 94.0 | 64.7 | 53.5 |
| FSD-DETR (Ours) | 13.09 | 37.1 | 94.8 | 93.2 | 96.1 | 67.4 | 92.3 |
The results clearly show that FSD-DETR achieves the highest mAP@50 and mAP@50:95 among all compared models. In addition, FSD-DETR has the smallest parameter count and computational cost among the compared models except for a few YOLO variants, while still providing a high FPS. This demonstrates that the proposed improvements effectively reduce model complexity and improve the detection accuracy of sand foundry defect.
4.3 Per-Class Detection Performance
I also analyzed the detection performance for each defect category. The per-class mAP@50 results are shown in the following table.
| Model | Incomplete (%) | Blowhole (%) | Shrinkage (%) | Crack (%) | Gas Pore (%) | Protrusion (%) | Scratch (%) | Slag Hole (%) |
|---|---|---|---|---|---|---|---|---|
| YOLOv5m | 96.4 | 77.4 | 93.4 | 92.8 | 96.8 | 93.4 | 86.3 | 98.4 |
| YOLOv8m | 96.6 | 79.3 | 96.2 | 93.7 | 97.1 | 93.7 | 87.0 | 98.1 |
| YOLOv10m | 95.0 | 74.5 | 90.9 | 87.0 | 95.3 | 91.2 | 82.7 | 98.1 |
| YOLO11m | 97.3 | 78.0 | 91.9 | 94.2 | 96.9 | 94.6 | 85.1 | 99.0 |
| RT-DETR-L | 98.3 | 83.9 | 96.3 | 93.5 | 97.3 | 93.7 | 89.8 | 99.5 |
| FSD-DETR (Ours) | 98.7 | 86.5 | 98.5 | 97.6 | 97.3 | 95.9 | 95.0 | 99.4 |
FSD-DETR achieves the best detection accuracy on most defect categories, especially on cracks and scratches, which are usually more difficult to detect due to their elongated shapes and low contrast. The improvement in these categories confirms that the FRA module and PIoU loss are effective for geometric feature modeling and precise boundary regression in sand foundry defect detection.
4.4 Ablation Study
To evaluate the contribution of each proposed component, I conducted a series of ablation experiments. The baseline model is RT-DETR with ResNet18. The improved components include H1: StarNet backbone, H2: FCA attention, H3: FRA module, and H4: PIoU loss. The results are shown below.
| Model | H1 | H2 | H3 | H4 | Parameters (M) | GFLOPs (G) | mAP@50 (%) | mAP@50:95 (%) |
|---|---|---|---|---|---|---|---|---|
| Baseline | 19.88 | 57.0 | 94.4 | 65.3 | ||||
| A | √ | 12.00 | 31.8 | 95.0 | 65.7 | |||
| B | √ | √ | 12.03 | 31.8 | 95.7 | 66.5 | ||
| C | √ | √ | 13.06 | 37.1 | 95.9 | 66.7 | ||
| D | √ | √ | 12.00 | 31.8 | 95.8 | 66.4 | ||
| E | √ | √ | √ | 13.09 | 37.1 | 96.2 | 66.7 | |
| FSD-DETR | √ | √ | √ | √ | 13.09 | 37.1 | 96.1 | 67.4 |
The ablation results show that every component contributes positively to the final performance. The combination of all components achieves a good balance between accuracy and efficiency. The final FSD-DETR model reduces parameters by 34.2% and FLOPs by 34.9% compared with the baseline, while improving mAP@50 from 94.4% to 96.1%.
4.5 Generalization on NEU-DET Dataset
To verify the generalization ability of my method beyond sand foundry defect images, I tested FSD-DETR on the public NEU-DET steel surface defect dataset. This dataset contains six types of surface defects: crazing, patches, inclusion, pitted surface, rolled-in scale, and scratches. I compared my model with several existing detectors. The results are shown in the following table.
| Model | Crazing (%) | Patches (%) | Inclusion (%) | Pitted Surface (%) | Rolled-in Scale (%) | Scratches (%) | mAP@50 (%) |
|---|---|---|---|---|---|---|---|
| YOLOv5 | 47.0 | 92.2 | 78.2 | 77.1 | 44.9 | 71.0 | 65.7 |
| Faster R-CNN | 37.6 | 85.3 | 80.2 | 81.5 | 54.0 | 89.2 | 71.3 |
| RetinaNet | 45.9 | 91.1 | 84.2 | 74.7 | 43.5 | 81.6 | 70.2 |
| Multi-Scale YOLOv5 | 34.5 | 89.8 | 76.8 | 82.5 | 61.6 | 86.8 | 72.0 |
| Fine-YOLO | – | – | – | – | – | – | 73.1 |
| NHD-YOLOx | – | – | – | – | – | – | 74.3 |
| RT-DETR-r18 | 24.3 | 91.1 | 78.8 | 75.2 | 59.9 | 86.7 | 69.3 |
| FSD-DETR (Ours) | 51.5 | 84.9 | 81.7 | 76.7 | 61.8 | 91.1 | 74.6 |
FSD-DETR achieves the highest mAP@50 of 74.6%, which is 5.3% higher than the RT-DETR-r18 baseline. The results confirm that the proposed model has strong generalization ability and is not limited to a specific type of industrial surface defect. The knowledge learned from sand foundry defect images can be transferred to other surface inspection tasks.
5. Model Pruning and Knowledge Distillation for Sand Foundry Defect Detection
Although FSD-DETR is already lighter than the original RT-DETR-r18, further compression is needed for deployment on embedded devices such as the Jetson Nano. I therefore studied model pruning and knowledge distillation techniques. Model pruning removes redundant parameters and channels in the network, while knowledge distillation transfers rich semantic information from a large teacher model to a compact student model. The combination of these two techniques can significantly reduce the model size and improve inference speed while preserving detection accuracy.
5.1 Pruning Methods
I compared four pruning strategies: L1 pruning, Group Hessian pruning, Group Taylor pruning, and LAMP pruning. The L1 norm of a weight vector is defined as:
$$
\|w\|_1 = \sum_{i=1}^{n} |w_i|
$$
Group Hessian pruning uses second-order derivative information to evaluate the sensitivity of each weight group. The Hessian matrix is defined as:
$$
H = \frac{\partial^2 \mathcal{L}}{\partial w^2}
$$
Group Taylor pruning uses a second-order Taylor expansion of the loss function to estimate the importance of each group:
$$
\mathcal{L}(w + \Delta w) \approx \mathcal{L}(w) + \nabla \mathcal{L}(w)^T \Delta w + \frac{1}{2} \Delta w^T H \Delta w
$$
LAMP pruning evaluates weight importance using a layer-adaptive score. The LAMP score for a weight \(w(u)\) is computed as:
$$
score(u, w) = \frac{w(u)^2}{\sum_{v \geq u} \bigl(w(v)\bigr)^2}
$$
In my experiments, the pruning coefficient \(P\) is defined as:
$$
P = \frac{F}{F’}
$$
where \(F\) is the computational cost before pruning and \(F’\) is the computational cost after pruning. I tested \(P = 1.2\), \(P = 1.4\), and \(P = 1.6\). The results are shown in the following table.
| Method | P | Parameters (M) | GFLOPs (G) | mAP@50 (%) | FPS |
|---|---|---|---|---|---|
| RT-DETR-r18 (original) | – | 19.9 | 57.0 | 94.4 | 90.1 |
| FSD-DETR (unpruned) | – | 13.1 | 37.1 | 96.1 | 92.3 |
| L1 pruning | 1.2 | 10.7 | 30.4 | 94.6 | 138.9 |
| L1 pruning | 1.4 | 9.1 | 26.0 | 92.7 | 333.3 |
| L1 pruning | 1.6 | 7.6 | 22.2 | 92.8 | 233.0 |
| Group Hessian | 1.2 | 9.9 | 30.1 | 96.0 | 175.4 |
| Group Hessian | 1.4 | 8.8 | 26.1 | 95.9 | 370.4 |
| Group Hessian | 1.6 | 7.9 | 22.9 | 95.0 | 238.1 |
| Group Taylor | 1.2 | 9.7 | 30.0 | 96.0 | 181.8 |
| Group Taylor | 1.4 | 8.5 | 26.0 | 95.4 | 153.8 |
| Group Taylor | 1.6 | 7.4 | 22.3 | 95.4 | 322.6 |
| LAMP | 1.2 | 9.6 | 30.0 | 96.1 | 166.7 |
| LAMP | 1.4 | 8.3 | 25.8 | 96.1 | 370.4 |
| LAMP | 1.6 | 7.4 | 21.9 | 96.0 | 400.0 |
The table shows that LAMP pruning achieves the best balance between accuracy and speed. At \(P = 1.6\), the model parameters are reduced to 7.4M and the computational cost is 21.9 GFLOPs, while mAP@50 remains at 96.0%. This demonstrates that LAMP can effectively identify important channels and preserve the structural knowledge of sand foundry defect detection features.
5.2 Knowledge Distillation
To compensate for the slight accuracy loss caused by pruning, I adopted the channel-wise distillation (CWD) method. In this framework, the original FSD-DETR model serves as the teacher model, and the LAMP-pruned model serves as the student model. CWD aligns the channel-level feature distributions between teacher and student models. The student model learns to mimic the teacher model’s feature responses, thereby recovering the representational ability lost during pruning. The experimental results are summarized below.
| Model | Parameters (M) | GFLOPs (G) | mAP@50 (%) | FPS |
|---|---|---|---|---|
| Teacher (FSD-DETR) | 13.1 | 37.1 | 96.1 | 92.3 |
| Student (FSD-DETR-LAMP) | 7.4 | 21.9 | 96.0 | 400.0 |
| Student + CWD | 7.4 | 21.9 | 96.1 | 397.0 |
After CWD distillation, the accuracy of the pruned model is fully recovered to 96.1% while the inference speed remains as high as 397 FPS. This result confirms that knowledge distillation is a powerful tool for optimizing the trade-off between model size and detection performance in sand foundry defect detection.
6. Embedded Deployment and Visual Detection System
To make the proposed algorithm practical in real industrial environments, I deployed the optimized model on an NVIDIA Jetson Nano embedded platform. Jetson Nano is a low-power edge computing device equipped with a 128-core NVIDIA Maxwell GPU and a quad-core ARM Cortex-A57 CPU. It provides approximately 0.5 TFLOPS of FP16 computing power, which is sufficient for running lightweight deep learning models. The detailed hardware parameters are shown in the following table.
| Item | Specification |
|---|---|
| GPU | 128-core NVIDIA Maxwell |
| CPU | Quad-core ARM Cortex-A57 @ 1.43 GHz |
| Memory | 4 GB 64-bit LPDDR4, 25.6 GB/s |
| AI Performance | 473 GFLOPS |
| Storage | 16 GB eMMC 5.1 |
| Camera Interface | 2 MIPI CSI-2 DPHY lanes |
| USB Ports | 4 × USB 3.0, USB 2.0 Micro-B |
For image acquisition in the embedded system, I used the IMX219 camera module. This CMOS sensor has a resolution of 8 million pixels and supports 1080p video at 30 frames per second. It is connected to Jetson Nano via the MIPI CSI-2 interface and provides stable image data for real-time defect detection.
The deployment process included several steps. First, I formatted a microSD card and burned the JetPack system image. JetPack includes CUDA, cuDNN, and other necessary libraries for deep learning inference. After the system was installed, I configured the Python environment, installed PyTorch and OpenCV, and adapted the pruning and distillation scripts to the embedded platform. The final model was exported and loaded into the detection system.
I developed a visual defect detection system based on the PyQt5 framework. The system supports user login, parameter adjustment, model configuration, image detection, camera detection, and result saving. The main interface is divided into two image display areas. The left side shows the original casting image before detection, and the right side displays the image with highlighted detection results. The system also provides a log area to show detection information such as image file names, defect classes, confidence values, and the total number of detected targets.
When a user clicks the “Load Image” button, the system reads a casting image from local storage and displays it in the input area. After clicking “Start Detection”, the system runs the lightweight model to identify sand foundry defect regions. The detection results are drawn on the image as colored bounding boxes with class labels and confidence scores. For example, an “incomplete” defect may be labeled with a confidence score of 0.93, while a “protrusion” defect may have a score of 0.91. The log panel shows both the detected class names and their counts, such as incomplete: 3 and protrusion: 1.
The system also supports real-time detection through a camera. When the camera is activated, the video stream is displayed on the interface, and each frame is processed by the detection model. This functionality is useful for online inspection on a production line. In addition, the system provides a “Save Result” button that stores the annotated detection image and a structured detection report to a local directory. This feature enables traceability and post-analysis of sand foundry defect data.
I verified the system in a real operating environment. The pruned FSD-DETR model after CWD distillation achieved a detection speed of about 397 FPS on a high-end GPU, and on Jetson Nano it still maintained a high enough frame rate for real-time inspection. The system operated stably when processing images and video streams, and the detection results were consistent with the ground-truth annotations. The visual interface was simple and intuitive, allowing operators without programming experience to use the system easily.
7. Conclusion and Future Work
In my dissertation, I presented a comprehensive study on sand foundry defect detection using deep learning. I proposed a lightweight and high-precision detection model called FSD-DETR, which integrates a StarNet backbone, a frequency channel attention mechanism, a feature-aware relation aggregation module, and a PIoU loss function. The model significantly reduces the parameter count and computational cost compared with the baseline RT-DETR while improving the detection accuracy. Experiments on the self-built CSDD dataset showed that FSD-DETR achieves 96.1% mAP@50 with only 13.09M parameters and 37.1 GFLOPs. The model also demonstrated strong generalization ability on the public NEU-DET dataset, achieving 74.6% mAP@50.
To further satisfy the requirements of embedded deployment, I used LAMP pruning and CWD knowledge distillation to compress the model. The final pruned and distilled model has only 7.4M parameters and 21.9 GFLOPs, while maintaining 96.1% mAP@50. This result proves that the combination of pruning and distillation is effective for building lightweight sand foundry defect detection systems.
I also deployed the optimized model on a Jetson Nano platform and developed a PyQt5-based visual inspection system. The system supports image detection, video detection, parameter adjustment, and result management. It provides a user-friendly interface and stable runtime performance, making it suitable for industrial quality control.
In future work, I plan to explore more advanced model compression techniques such as low-bit quantization and structural reparameterization. I also intend to investigate multi-modal information fusion, such as combining image data with 3D point clouds or acoustic signals, to improve the robustness of sand foundry defect detection under complex working conditions. Furthermore, online learning and active learning strategies can be introduced to adapt the model to new casting types and evolving production environments. These directions will bring industrial intelligent inspection closer to full automation and real-time adaptability.