One of the reasons why we love Image Recognition and AR is that they can be used for so many different experiences on so many target objects, from photos to posters to product labels. (These customer projects perfectly prove that.)
However, a well-working Image Recognition or AR experience starts with selecting the right targets – in Catchoom's terminology, the right reference images – for your experiences.
This is why we've collected some tips to help you prepare the right collection of images for our CraftAR Image Recognition solutions. We'll also present you your new best friend, our extended Image Quality Rating.
Foreword: how to recognize everyday objects...
One of CraftAR's specialties is that our recognition engine not only works great on flat objects, like photos or print images, but it works very well on 'non-planar', in other words, non-flat objects, too.
Such non-flat objects may be bottle labels, cans, packaged goods, unique building fronts, a graphic T-shirt, just to name a few. As long as they have a distinctive pattern, they can be recognized. (Read our guidance regarding what kind of images and objects can be recognized well, and which don't.)
With CraftAR, companies & developers can let their users interact with tangible, everyday objects and link all sorts of surprising digital experiences to them. For example, scanning a bottle of wine to see reviews or check out alternatives in the online store.
But how can you make your app recognize a physical object?
Now that's how 'reference images' come into the picture.
What are 'reference images' and why do they matter?
When you are developing an app with Image Recognition functionalities inside, you need to represent your objects –your items – via 'reference images' in order to make them recognizable. In the CraftAR platform, you can group those items into so called 'collections'.
Then, you can link those reference images to specific events, such as redirecting the user to a web page, content or AR experiences. Finally, you can synchronize these collections with your apps via our APIs, as often as you wish, being able to keep your database up-to-date.
Even if Catchoom's recognition seemed more accurate than the market average in tests, even in difficult conditions, good reference images are crucial to boost the performance and keep your users happy.
Let's see some recommendations to help you prepare the perfect collection of images for your experiences.
Top tips for preparing your collections of reference images
1. Include various sides to let your users 'play around' with your objects.
In most cases, one reference image is enough, as our engine is invariant to size and rotation of the actual real-life object, and it can handle different angles to certain extent, too.
If you want to get a photo or print advertisement recognized in a magazine, it's enough to use the original jpeg or png image file. Even in cases of non-flat objects, say, cans or bottles, one reference image of their representative frontal face tends to give good results on their own.
However, if you want to use physical objects that have different sides, such as a book, a box of cereals or other kind of consumer packaged goods, it's wise to upload corresponding shots besides frontal image(s).
Reference images uploaded for the same item, a makeup box.
Top row: frontal images of the two most characteristic sides of the box.
Bottom row: aiding the recognition by adding additional reference images of the most typical viewpoints,
considering that this is how many of the users may see the box in real life.
It may also be useful in situations when your users may approach your objects from very different viewpoints, such as an exhibition stand or a unique storefront.
Adding reference images from different sides and typical viewpoints helps the users successfully interact with the objects while minimizing the instructions given to them.
2. In case of outdoor structures, add reference images showing various conditions.
Our CraftAR Image Recognition engine performs quite well when illumination is suboptimal. However, in case of outdoor structures such as unique buildings, storefronts, signs or sculptures, it is advised to include various reference images that cover extreme ambient conditions, such as rainy, sunny, cloudy, gloomy etc.
Besides this, different angles can also be included in the collection of images, as we've just explained above.
Images depicting different 'moments' of the same building and its unique front.
This way, you can ensure that your users can enjoy a reliable experience 'all year around'.
3. Use the original image files whenever possible.
We generally recommend to use reference images that predominantly show the actual image or object, focusing on its distinctive pattern.
For instance, if you are going to an expo and want to get your corporate pop up banner recognized, it's recommended to upload the original – jpeg or png – design to the CraftAR content management system as the principal reference image. (Even if you might add photos from additional angles, as explained earlier.)
Left: reference image uploaded to CraftAR;
Right: the actual pop-up banner people can scan with their phone at the expo
to learn more about the company's offerings.
Actually, our technology is so reliable that as long as irrelevant elements of the background do not take up more than 20% of the reference image, the recognition will still work in real life. Nevertheless, it's better to rule this problem out.
4. If you need to take a photo of the object, don't forget to crop it.
Sometimes you cannot avoid taking a photo of the item to use it as a reference image.
For example, you're working at a museum that wants to create an app so that visitors can scan & learn more about the paintings. In such cases, it's important to crop the photo and get rid of the unnecessary details.
Instead of taking Van Gogh's painting off the wall (right),
you can use a cropped photo of the painting (left) as the reference image when developing the museum's app.
5. Stay focused! Don't use blurred images.
Blurry reference images negatively affect the performance of the real-life recognition. If you need to take a photo of the object, pay attention to the right focus.
Similarly, if you use digital images, don't scale it up excessively – as it might get distorted – and don't add 'noise'. Read more about the recommended reference image quality, dimensions and formats.
6. Look up best practices for your specific use cases.
While there are some general guidelines, as explained above, it is important to do your homework for your specific project. Working with print magazines or advertisements offers different technical challenges than getting your logo recognized on your business card. Not to mention working with three-dimensional product packaging.
Here you can find some further recommendations, tailored to common use cases:
- Preparing reference images for Print & Publishing.
- Preparing reference images for Drink Bottles and other cylindrical objects.
- Preparing reference images for Logo Recognition.
- Preparing reference images for Exhibitions & Events.
7. Check your reference images with CraftAR's Image Quality Rating.
Now comes the best part: we help you take most of the guesswork out of Image Recognition.
Until now, we only offered an Image Quality Rating for Augmented Reality experiences, but from now on, you can also check whether your reference images are appropriate for recognition purposes in normal conditions.
The Image Recognition item view inside our CraftAR platform includes an orientative 5-star rating. (It is also available via our CraftAR Management API.) The score is based on whether the image has a strong visual pattern and the presence of such a pattern in the entire image. Low scores indicate few features or large textureless areas.
Left: this reference image reached a high score, as it has a distinctive pattern all over the image area.
Right: this reference image was given a lower quality value, because the distinctive pattern covers only a small area when compared to the patternless grey margin. This is why we mentioned the importance of cropping.
This reference image, showing a piece of wooden floor, has very little visual pattern.
Therefore, its rating is very low, as it's a poor candidate for recognition.
Note that the rating does not take the repetitiveness of the pattern into account, which may also negatively affect the recognition performance. Hence, the rating feature mainly serves to rule out reference images that will surely not work well. It's generally wise to eventually double check the recognition on the actual physical object, too, making sure that it will deliver the experience the way it is expected.
Getting rid of images that don't work well will help you save time and effort and make your job more efficient, besides getting the most value out of our plans.
It's always been easy to integrate our fast and accurate CraftAR Image Recognition solutions into your apps, but now it just became even more...predictably good. Don't hesitate, see it for yourself:
Disclaimer: we do not have official affiliations with most of the brands and artworks presented in the article. We meant to use them as examples of the possibilities of the technology. Copyrights: The Art Institute of Chicago, Benefit, Hard Rock Cafe, Winewoo.