Computer Vision Inference Pipeline on Azure
A computer-vision inference pipeline on Azure spanning event-driven ingestion, GPU inference, and human-in-the-loop review with active learning. It scales on queue depth and tracks drift as visual inputs change.
Overview
A computer-vision pipeline turns images or video frames into structured results, such as labels, bounding boxes, or defect flags. Use this architecture for use cases like manufacturing quality inspection, document processing, content moderation, or visual search, where volumes are high and inference needs GPUs. It combines automated inference with human review for low-confidence cases.
Components
- Azure Blob Storage: stores incoming media and processed results.
- Event Grid: triggers processing as soon as media arrives.
- Azure ML / Azure Container Apps: run the GPU inference service for custom models, scaling with load.
- Azure AI Vision: provides pretrained models for common tasks like OCR and tagging when a custom model is unnecessary.
- Cosmos DB: stores extracted results, confidence scores, and review status.
- Azure Monitor: tracks throughput, latency, and accuracy signals.
Data Flow
Media uploaded to Blob Storage raises an Event Grid event that enqueues a processing task. The inference service pulls the item, preprocesses it (resize, normalize, frame-extract for video), runs the model on GPU, and writes structured output with confidence scores to Cosmos DB. High-confidence results pass straight through; low-confidence ones are routed to a human review queue. Reviewer corrections are stored as new labeled data for retraining, closing the active-learning loop.
Scaling and Resilience
The inference service scales on queue depth and GPU utilization; batching frames improves GPU throughput and lowers cost per image. Event-driven, queue-backed design absorbs bursts and lets failed items retry without losing work. Make processing idempotent so retries do not duplicate results. Run across zones and keep a dead-letter queue for items that repeatedly fail. Define throughput and latency SLOs per workload tier.
Security
Images can contain personal or confidential information; restrict access, encrypt at rest and in transit, and apply retention limits. Use managed identities instead of stored keys. Track lineage from source media to result for audit. Monitor for data drift as cameras, lighting, or product lines change, since vision models degrade quietly when inputs shift. Control who can deploy new models and require validation before rollout.
Trade-offs and Alternatives
Pretrained Azure AI Vision services are fast to adopt but limited to common tasks; custom models on GPU give higher accuracy for specialized domains at greater cost and effort. Human-in-the-loop review raises quality and produces training data but adds latency and labor, so reserve it for low-confidence cases. Real-time per-frame inference is expensive for video; sample frames or run batch processing when latency allows. Edge inference is an alternative when bandwidth or privacy rules out sending media to the cloud.