How do you distinguish one food delivery package from another if your vision is impaired? The Search app’s Smart Camera is the perfect way to solve this! It’ll recognize and tell you not just the name of the product but also the expiry date, nutrition facts, and other important information.
In this report, we’ll tell you how the Smart Camera works and what tasks it helps to solve, as well as discuss the peculiarities of using the Camera as a blind person and how it adapts to such scenarios.
On top of that, we’ll talk about the technologies that allow the Smart Camera to detect and track objects or text from a video feed, why it uses a proprietary object tracker, and why we can’t use the readily available ARkit/ARCore. Furthermore, we’ll discuss how neural networks know when the text in the image is cropped and what changes we had to make in the client-side business logic to implement the text recognition scenario for blind users.
