Edge AI Defect Detection System: Real-Time Quality Control on ARM Chips

1. The Challenge: Real-Time Requirements for Production Line Quality Control

In high-speed automated production lines, the core contradiction of quality inspection lies in the trade-off between "speed" and "precision." Traditional machine vision systems based on rule-based image processing algorithms are fast enough, but their accuracy often falls short when confronting complex defect types such as surface scratches, color anomalies, and micro-cracks. Deep learning-based defect detection models, on the other hand, offer high accuracy but typically require GPU-level computational resources, resulting in excessive inference latency and prohibitive per-unit deployment costs that make it difficult to scale across every inspection station.

Our client is a precision electronic component manufacturer whose production line runs at 120 products per second through each inspection station. This means the inspection time window for each product is only 8.3ms. The existing manual visual inspection covered only 15% of products (sampling inspection), and was further limited by operator fatigue, with miss rates visibly climbing during later shifts. The client had evaluated placing GPU servers alongside the production line, but the hardware cost for a single inspection station exceeded NT$500,000, and with cooling and maintenance requirements, full deployment across all 40 inspection stations was not feasible.

The client's requirements were crystal clear: achieve 100% inspection of every product with inference latency under 10ms and accuracy of at least 99%, all within a hardware cost limit of NT$3,000 per station. This locked the deployment target onto ARM Cortex-M series microcontrollers -- an extremely constrained environment with only a few hundred kilobytes of memory.

2. Technical Approach: TinyML Defect Detection Architecture

Deploying deep learning models on microcontrollers -- known as TinyML -- is one of the most challenging research directions in edge AI in recent years. Lin et al. presented MCUNet at NeurIPS 2020^[1], which pioneered the possibility of running ImageNet-level neural networks on microcontrollers with only 256KB of memory, with the core innovation being the simultaneous search for optimal network architecture and inference scheduling.

2.1 Dataset Construction and Labeling Strategy

High-quality defect detection begins with high-quality training data. The MVTec AD dataset published by Bergmann et al.^[6] is the standard benchmark in the industrial defect detection field, covering 15 different categories of industrial products with corresponding defect samples. We used this as a methodological reference to construct a proprietary product defect dataset for our client.

During a three-week data collection phase, we installed high-speed industrial cameras on the client's production line, continuously capturing over 5 million product images at 200 frames per second. Through close collaboration with the quality control team, we defined 12 defect types and established a three-tier labeling process: junior annotators marked suspected defect regions, senior QC personnel confirmed defect classifications, and algorithm engineers verified labeling quality. The final effective dataset contained 42,000 labeled images, with defective samples comprising approximately 8%.

2.2 Base Model Selection

Given the extreme memory constraints of microcontrollers, we started with the MobileNet architecture proposed by Howard et al.^[3] as our baseline. MobileNet's depthwise separable convolution design reduces the computational cost of standard convolutions by 8-9x, making it one of the most suitable convolutional architectures for resource-constrained environments.

However, the smallest version of standard MobileNet V1 (width multiplier 0.25) achieves only about 50% classification accuracy on ImageNet, which is far from sufficient for the 99% accuracy requirement of industrial defect detection. Therefore, rather than directly using an existing architecture, we combined the architecture search approach from MCUNet to perform a targeted neural architecture search (NAS) on the client's defect classification task, with the search space constrained within the target hardware's memory and computational budget.

3. Model Compression Pipeline: From PyTorch to ARM Cortex-M

Even after architecture search, the optimal model still exceeded the target hardware's memory budget. Deep Compression by Han et al., published at ICLR 2016^[2], proposed a three-stage compression pipeline of "pruning -- quantization -- Huffman coding," providing us with a systematic compression methodology.

3.1 Structured Pruning

We first performed structured pruning on the fully trained model. Unlike unstructured pruning (zeroing out individual weights), structured pruning directly removes entire convolutional kernels or channels, ensuring that the pruned model remains a standard dense neural network without requiring specialized sparse computation hardware. We employed a gradient-based importance assessment method, progressively pruning channels that contributed least to the final classification, achieving a 60% reduction in model size with only a 0.4% drop in accuracy.

3.2 Post-Training Quantization

The pruned model still uses 32-bit floating-point numbers to represent weights and activations. Sze et al. provided an in-depth analysis of the impact of various quantization strategies on model performance and hardware efficiency in their survey in the Proceedings of the IEEE^[5]. We adopted an INT8 symmetric quantization strategy, compressing each weight and activation from 32 bits to 8 bits, resulting in a further 4x reduction in model size.

The key challenge during quantization is calibration -- determining the quantization range (scale and zero-point) for each layer to minimize quantization error. We used approximately 1,000 representative images as the calibration dataset, covering all defect types and normal products. After quantization, the accuracy loss on the validation set was only 0.2%, well within the acceptable range.

3.3 Deployment to TensorFlow Lite Micro

TensorFlow Lite Micro (TFLM), published by David et al. at MLSys 2021^[7], provides a lightweight inference engine specifically designed for microcontrollers. TFLM's core design principle is zero dynamic memory allocation -- all memory required during inference is fully determined at compile time, which is critical for bare-metal environments without an operating system.

We converted the quantized model to TensorFlow Lite FlatBuffer format and integrated it into the client's firmware development environment. The final deployed model size was 198KB (including model weights and inference engine code), with peak memory usage of 187KB, well within the resource budget of the ARM Cortex-M7 (512KB Flash / 256KB SRAM).

4. Deployment Results

After a two-month parallel testing period on the client's production line (with AI detection results and manual visual inspection results recorded simultaneously and cross-validated), the system went live. The MLPerf Tiny benchmark proposed by Banbury et al.^[4] provided us with a standardized performance measurement methodology. Below are the measured results from the production environment:

• Inference Latency: Single image inference time of 7.8ms (average), with a maximum of 9.4ms, consistently below the 10ms threshold. At a production line speed of 120 units per second, the system has ample time margin.
• Detection Accuracy: Overall accuracy of 99.2%, with a defect recall rate of 99.5% and precision of 99.7%. By comparison, the average accuracy of manual visual inspection was 94.3%, dropping to 89.7% after 4 consecutive hours of work.
• False Positive Rate: 0.3%, meaning an average of 3 out of every 1,000 non-defective products were incorrectly flagged as defective. These false positives enter a secondary manual review process and do not cause material losses.
• Hardware Cost: The embedded hardware cost per inspection station is NT$2,800 (including MCU development board, industrial camera module, and cabling), far below the NT$500,000+ cost of the GPU-based approach.
• Power Consumption: Power consumption of only 0.8W per inspection station, requiring no additional cooling system and suitable for 24/7 continuous operation.

5. Scaled Deployment and Continuous Learning Mechanism

5.1 Factory-Wide Deployment Strategy

Based on the pilot station validation results, the client decided to deploy the system across all 40 inspection stations factory-wide. We designed a standardized deployment toolchain: through an OTA (Over-The-Air) firmware update mechanism, model deployment or updates at a single station can be completed within 15 minutes without affecting production line operations. The full factory deployment was completed within two weeks.

5.2 Continuous Learning and Model Iteration

Defect types in industrial environments are not static -- new raw material batches, process parameter adjustments, and environmental condition changes can all produce novel defect patterns. We established a "edge collection -- cloud training -- edge deployment" continuous learning loop:

1. Edge Collection: Each inspection station automatically collects low-confidence images (model output probability between 0.3-0.7), marking them as samples pending review.
2. Cloud Labeling and Retraining: Pending review samples are uploaded weekly to a cloud platform, where QC personnel label them and add them to the training dataset. The model undergoes incremental training and compression every two weeks.
3. OTA Model Update: After offline validation (accuracy no lower than the current version), the new model is pushed to all stations via the OTA mechanism.

Over the six months since launch, the model has undergone 12 iterative updates, with defect type coverage expanding from the initial 12 to 19 types, and accuracy improving slightly from 99.2% to 99.4%.

5.3 Economic Benefits

The annual benefit assessment following factory-wide deployment shows that quality-related costs (including customer complaint handling for missed defective products, return losses, and QC labor) decreased from NT$24 million annually to NT$12 million, a 50% reduction. The total hardware investment for all 40 stations was NT$112,000, and including development and integration costs, the overall payback period was approximately 4.5 months.

More importantly, the 100% full inspection capability enabled the client to offer stricter quality commitments to their end customers, directly driving the signing of two large orders whose contract value far exceeded the investment in the quality system itself.