VisionChain: Taming Foundational Vision Models for use in the Real World

Abstract

Since LLMs came into the spotlight via ChatCPT, there has been an explosion of tooling which make it possible to deploy specialist NLP applications without the need to train a model for a specific problem. This has greatly increased the accessibility of developing advanced Deep Learning powered applications. Despite the tools and technology being available, the same has not occurred for Computer Vision. VisionChain seeks to close this gap.

Keywords

• eol>Vector Database
• Computer Vision
• Object Detection
• Grounding Dino
• Zero-Shot
• Heuristics

1. Problem Statement

2. Proposed Solution

A simple, user-friendly API to make developing and evaluating Computer Vision systems on top of Foundational Computer Vision Models extremely simple and fast. This can be achieved via a combination of retrieval-augmented detection and post-processing heuristics.

3. Demo

The demo will involve the real-time improvement of a foundational Object Detection model on the realistic industrial dataset zerowaste (https://ai.bu.edu/zerowaste/). The demo will begin with the raw output of GroundingDino. Performance will then be improved by the retrieval-augmented detection approach, and finally by the addition of a few post-processing heuristics.

References