A tier-1 automotive supplier came to us with a painful problem: their manual quality inspection line was catching only 78% of surface defects on stamped metal parts. Every missed defect meant a potential recall, and their six human inspectors were burning out at a rate of 30% turnover per year.
We built an AI-powered visual inspection system using YOLOv8 and deployed it on NVIDIA Jetson AGX Orin modules at the edge. In production, it runs at 60 frames per second, detects 23 defect types with 99.5% accuracy, and has reduced false positives to under 0.3%. The client recovered the hardware cost in 11 weeks.
This post walks through the architecture, the training pipeline, the deployment decisions, and the mistakes we made along the way. If you're evaluating AI for quality inspection in manufacturing, logistics, or even code review, the same patterns apply.
- Model Architecture: YOLOv8m (medium) trained on 12,000 labeled images covering 23 defect classes. Input size 640x640, output bounding boxes and confidence scores.
- Edge Hardware: NVIDIA Jetson AGX Orin 64GB module per camera station. TensorRT-optimized model achieves 60fps inference with FP16 precision.
- Data Pipeline: Labeled with Roboflow, augmented with Albumentations (random brightness, noise, rotation). Synthetic defects generated with Blender for rare classes.
- Integration: RTSP cameras -> GStreamer pipeline -> OpenCV preprocessing -> TensorRT inference -> MQTT to MES (Manufacturing Execution System).
- 1
Data Collection and Labeling
We installed 6 Basler ace 2 cameras (12MP, global shutter) with controlled ring lighting to capture 50,000 images of good and defective parts. Each image was labeled by two independent annotators using Roboflow's polygon tool. Disagreements were resolved by a senior quality engineer.
- Capture at least 200 images per defect class, more for rare defects.
- Use consistent lighting and camera angles across all training data.
- Label with tight bounding boxes - loose boxes hurt mAP.
- 2
Training with YOLOv8 and Data Augmentation
We used Ultralytics YOLOv8m with a custom dataset of 12,000 images after cleaning. Training ran on a single NVIDIA A100 for 300 epochs with early stopping. We applied heavy augmentation: random brightness (±30%), Gaussian noise (σ=0.05), rotation (±15°), and mosaic augmentation.
- Use Albumentations for on-the-fly augmentation during training.
- Set mosaic=1.0 for the first 100 epochs, then reduce to 0.5.
- Monitor validation mAP every 10 epochs - stop if no improvement for 30 epochs.
- 3
Optimization with TensorRT for Edge Deployment
The trained PyTorch model was exported to ONNX, then converted to a TensorRT engine using FP16 precision. We used the Jetson's DLA (Deep Learning Accelerator) for additional throughput. The final engine runs at 60fps with batch size 1, consuming only 15W per camera.
- Export with dynamic axes for variable batch sizes if needed.
- Calibrate INT8 if you need even lower latency - we stayed with FP16 for accuracy.
- Profile with trtexec --loadEngine=model.engine --warmUp=100 --duration=30
- 4
Deployment and Integration with MES
Each Jetson runs a Python daemon that captures frames via GStreamer, preprocesses (resize, normalize), runs inference, and publishes results to an MQTT broker. The MES subscribes to defect alerts and triggers a rejection arm on the conveyor belt.
- Use ZeroMQ or MQTT for low-latency messaging - avoid HTTP polling.
- Add a watchdog timer that reboots the Jetson if inference stalls for >5 seconds.
- Log all predictions to a local SQLite database for offline analysis.
Invest in Synthetic Data for Rare Defects
Some defect classes (e.g., hairline cracks, micro-porosity) had fewer than 50 real samples. We used Blender to render 3D models of the part with procedurally generated defects, then composited them onto real backgrounds. This boosted recall on those classes from 55% to 91%.
- Use domain randomization: random lighting, camera angles, and backgrounds.
- Validate synthetic data quality by having a human review a random sample.
Don't Trust Out-of-the-Box Models
We tested a pre-trained YOLOv8n on the factory floor without fine-tuning. It detected 12% of defects - essentially useless. Even fine-tuning on generic industrial datasets (like MVTec AD) only got us to 68%. You need factory-specific data with the exact lighting, angles, and defect morphologies.
Build a Feedback Loop for Continuous Improvement
We added a 'human review' station where an operator validates every flagged defect and can correct false positives/negatives. Those corrected images are automatically added to the next training cycle. Over 8 weeks, the false positive rate dropped from 1.2% to 0.3%.
- Store all inference results with timestamps and image thumbnails.
- Retrain the model weekly using only the last 7 days of reviewed data.
Account for Environmental Drift
Factory lighting changes throughout the day (sunlight through skylights, shadows from moving equipment). We added a calibration step: a reference part with known defects is run every hour. If inference confidence drops below a threshold, the system alerts maintenance to clean cameras or adjust lighting.
Plan for Edge vs. Cloud Tradeoffs
We initially considered streaming all video to the cloud for inference. At 60fps per camera, that's 1.2 Gbps - not feasible with the factory's network. Edge inference on Jetson cost $4,500 per station (one-time) vs. $2,000/month per camera for cloud GPU. Edge won on latency and total cost of ownership.
- Calculate bandwidth: 1920x1080 @ 60fps = 1.2 Gbps raw. Even with H.264, it's ~50 Mbps per camera.
- Edge inference is mandatory for real-time (<100ms) requirements.
Ignoring Class Imbalance in the Loss Function
Our first model had 99% overall accuracy but missed 80% of 'scratch' defects because scratches made up only 2% of the training data. We switched to focal loss (γ=2) and oversampled minority classes by 3x. That fixed the recall imbalance.
Using Default NMS Thresholds
YOLOv8's default NMS IoU threshold (0.7) caused multiple overlapping boxes for the same defect. We tuned it to 0.5 and added soft-NMS to reduce false positives on overlapping defects like 'dent with crack'. This improved mAP by 4 points.
Not Testing with the Exact Production Lighting
We trained on images taken with prototype lighting (diffuse LEDs). On the actual factory floor, the client used fluorescent tubes that created harsh shadows. The model's accuracy dropped to 82% on the first day. We recollected 2,000 images under production lighting and retrained - accuracy returned to 98%.
Skipping a Pilot Phase
The client wanted to deploy to all 12 stations at once. We convinced them to start with 2 stations for 4 weeks. That pilot revealed a conveyor belt vibration issue that caused frame misalignment. We added a mechanical stabilizer and updated the preprocessing pipeline before rolling out to the remaining stations.
Choosing the Right Model Size and Hardware
We tested YOLOv8n (nano), YOLOv8s (small), and YOLOv8m (medium) on the Jetson AGX Orin. Nano ran at 120fps but had 93% accuracy - too many false negatives. Medium ran at 60fps with 99.5% accuracy. Large (YOLOv8l) hit 99.7% but only achieved 35fps, which didn't meet the 50fps requirement for the conveyor belt speed.
The Jetson AGX Orin is overkill for many applications. For simpler defect detection (e.g., missing components on a PCB), a Jetson Orin NX 16GB ($699) can run YOLOv8s at 80fps. We chose the AGX Orin because the client needed 23 classes and high-resolution input (1920x1080 downsampled to 640x640).
If you're deploying hundreds of cameras, consider the Jetson Orin Nano ($249) with YOLOv8n for low-complexity tasks. The tradeoff is accuracy - we measured a 5% drop in mAP compared to the medium model on this dataset.
AI Quality Inspection Is Ready for Production
The system we built for that automotive supplier has been running for 9 months with zero unplanned downtime. It catches defects that humans miss, runs 24/7, and has paid for itself three times over in scrap reduction. The key was not the model architecture - it was the data pipeline, the edge deployment strategy, and the continuous feedback loop.
If you're considering AI for quality inspection, start small. Pick one production line, one camera, and the most common defect class. Build the full pipeline from camera to MES, measure the impact, and iterate. The technology is mature enough that the risk is no longer in the AI - it's in the integration and operations.
IRPR has built similar systems for clients in automotive, electronics, and food processing. If you want to skip the learning curve and get to production faster, we can help. Our fixed-price engagements deliver a working MVP in 8-12 weeks, including hardware selection, model training, and edge deployment.
The IRPR engineering team ships production software for 50+ countries. Idea → Roadmap → Product → Release. 200+ products live.
About IRPR