In a 2024 McKinsey survey, only 5% of manufacturing AI pilots reached production scale. That's not a data problem—it's an engineering problem. The gap between a Jupyter notebook and a real-time inference pipeline running on a factory floor is wider than most teams anticipate.
We've built several of these systems at IRPR: from defect detection on injection molding lines to predictive maintenance on CNC machines. This post walks through the technical decisions that made those deployments succeed—latency budgets, edge hardware selection, model optimization, and continuous monitoring.
If you're a CTO or engineering manager tasked with moving an AI pilot to production, this is the blueprint. No fluff. Just specifics: model versions, hardware specs, and numbers you can use to set internal SLAs.
Why Most Factory AI Projects Stall
The most common mistake is underestimating the edge environment. Factory floors are noisy—electrically, thermally, and vibrationally. A GPU box that works in a server room can fail within a week in a 40°C enclosure next to a press machine. We've seen teams burn months trying to run a full ResNet-152 on a Jetson Nano when a MobileNetV3 with pruning could deliver 95% of the accuracy at 10x lower latency.
Another failure mode: no deployment playbook. Many teams spend 80% of their time training models and 20% on deployment infrastructure. In production, the ratio should be reversed. You need a pipeline that automates model packaging, hardware-specific compilation (TensorRT for NVIDIA, OpenVINO for Intel), and A/B testing on the line.
Data drift is the silent killer. A defect detector trained on clean images from a lab will fail when factory lighting changes due to time of day, or when a new resin batch introduces subtle texture variations. Without automated drift detection and retraining triggers, your model's accuracy will degrade within weeks.
Steps to Deploy AI on the Factory Floor
- 1
1. Define Your Latency Budget
Start with the production line speed. If the line runs 60 parts per second, you have 16.67 ms to capture, process, and decide on each part. Account for camera exposure, image transfer, and any mechanical actuation (reject arms). We typically aim for 15 ms end-to-end, leaving a 1.67 ms safety margin.
- Measure baseline with a simple edge device like a Raspberry Pi 4 (runs about 2-3 FPS with a lightweight model)
- Scale up hardware until latency fits within budget (NVIDIA Jetson Orin NX runs MobileNetV3 at ~100 FPS)
- 2
2. Choose Edge Hardware Strategically
Don't default to the most powerful GPU. Power, thermal, and cost constraints matter. For a simple binary OK/NOK classifier, a $150 Intel Movidius stick on a thin client can suffice. For multi-class defect detection, a Jetson Orin NX or Nano is appropriate. We've shipped systems on all three.
- NVIDIA Jetson Orin NX: 40 TOPS, ~30W, $399
- Intel Movidius Myriad X: 1 TOPS, ~2W, $79 (for simple classifiers only)
- Raspberry Pi 4 + Coral TPU: 4 TOPS, ~15W, $125
- 3
3. Optimize the Model for Inference
Train in PyTorch or TensorFlow, then export to ONNX. Compile with TensorRT (NVIDIA) or OpenVINO (Intel). Quantize to FP16 or INT8. On a recent IRPR build, we took a PyTorch ResNet-50 (25 MB, 8 ms on GPU) and after TensorRT INT8 quantization, we got 2.3 ms at 99.2% of original accuracy.
- Use torch.onnx.export with dynamic axes for batch processing
- Apply quantization-aware training if accuracy drop exceeds 0.5%
- 4
4. Build an Edge Pipeline with Kubernetes or Docker Compose
Even on a single Jetson, containerize inference, camera capture, and data streaming. Use MQTT for lightweight event passing to a cloud aggregator. Store raw images (or just embeddings) in a local buffer for later analysis. We use Redis for temporary storage of inference results.
- Deploy with Docker Compose on Jetson (NVIDIA provides JetPack runtime)
- Stream telemetry to a central ML pipeline stack (we use Feast for feature store, MLflow for model registry)
- 5
5. Implement Continuous Monitoring and Retraining
Set up drift detection on feature distributions (Feldman–Cousins or simple PSI). When drift crosses a threshold (e.g., PSI > 0.2), trigger a retraining job on the cloud with recent labeled data. Automatically deploy the new model via an OTA update mechanism (e.g., AWS IoT Greengrass component update).
- Use Evidently for monitoring, or write a simple Python script reading Redis streams
- Typical retraining cycle: every 2-4 weeks initially, then less frequent as model stabilizes
- Latency Budget: End-to-end time from image capture to actuator command. For a 60 ppm line, budget under 16 ms.
- Throughput: Number of parts processed per unit time. Must match line speed; we aim for 20% headroom.
- Model Drift: Statistical change in input or prediction distribution. Measure weekly with PSI or KS test; trigger retraining at threshold.
- Cost per Inference: Total hardware amortization + cloud compute + electricity per inference. Benchmarking target: $0.001-$0.005 for high-volume lines.
Common Mistakes in Factory AI
Ignoring Edge Constraints
Running a full GPU server on the factory floor is possible but expensive and fragile. We've seen teams deploy $10k workstations only to have them fail due to vibration or dust. Design for the edge: fanless enclosures, IP65 ratings, and wide-temperature components.
Treating AI as a One-Off Project
Factory AI is not a set-it-and-forget-it system. Models drift, hardware degrades, production lines change. You need a continuous training pipeline, model versioning, and monitoring. Treat it like a product feature, not a research project.
Neglecting Data Versioning
When a model fails in production, you need to replay the exact input images. We use DVC to version training data and store inference results in Parquet files with timestamps. Without that, debugging is impossible.
Final Thoughts
Deploying AI on the factory floor is an engineering discipline, not a data science exercise. It requires cross-functional collaboration between ML engineers, DevOps, and manufacturing controls teams. The systems we've built at IRPR consistently hit 99%+ uptime by adhering to edge-first architecture, containerized pipelines, and automated drift monitoring.
If your team is planning a factory AI deployment and needs experienced engineers who have shipped these systems before, we'd be happy to help. Book a call with our team to discuss your specific line constraints and latency budgets.
The IRPR engineering team ships production software for 50+ countries. Idea → Roadmap → Product → Release. 200+ products live.
About IRPR