Predicting Christmas Presents with AI
What did you get for Christmas? Maybe AI has the answer.
Artificial intelligence can do a lot of things. Playing video games better than professionals, creating beautiful works of art, and writing stories. But can AI predict what you’re getting for Christmas?
Classifying images is something AI does exceptionally well. A type of model known as a convolutional neural network performs exceptionally on image tasks.
The field of computer vision is constantly advancing and improving. As a result, image classification is becoming an easy task, fueled by increasingly large datasets and more extensive models.
But all of these models and experiments have the same assumption. The image shows the object you are trying to classify.
This assumption is not the case for a Christmas gift. Yes, the object you’re trying to identify is in the image. But not precisely. There is some material in the way. The gift is wrapped. The wrapping obscures the object making it hard to predict and a more exciting Christmas.
This article is broken down into three sections.
- Problems with gift prediction.
- Gathering Data.
- Developing a model.
You may already be thinking of the problems with predicting what’s in a Christmas present. I’ve outlined a few of the big ones for this project below.
Too many options
Predicting Christmas gifts from images of the wrapped present is a difficult task for many reasons. But there is one huge reason which makes this task near impossible for AI (and difficult for us humans)
Gifts are often wrapped in boxes with shapes independent of the shape of the gift within.
This structure means that different gifts can be within the same box. AI does not like identical boxes with different content. How on earth would anyone determine the difference.
This problem brings me to the second reason this particular task is difficult. I am only using the images of wrapped presents.
I don’t know about you, but when I’m given a Christmas gift, I immediately notice the weight of the gift. Simply passing me the gift provides me with additional information about what is inside.
The gift is too light — Yup, no coal in there.
Even something like a quick shake (maybe check to make sure shaking won’t break the gift first) will provide you with more information about what the gift may or may not be.
The Christmas Present Prediction Problem
So to adjust this problem into something which works well with AI, I’m going to reduce the problem into something more feasible.
Instead of a specific gift, can I predict the type of gift from a set of gifts? This change will dramatically reduce the problem complexity.
Besides, even if I could build a model to predict specific gifts, is predicting the object correct? Some people would say the brand of the gift matters a lot too.
Maybe I can tell that you’re getting underwear, but is it a brand new pair from a luxury brand? Or are you getting a pair from the bulk bin?
So predicting categories of gifts seems like cheating. And well, it is. But as you’ll see as I gather the data, this isn’t exactly a well-established problem.
Thus, problem reduction is key even to get this project off the ground.
Data
Despite all the fancy models, the most important aspect of AI is data. You need lots of it, and you need good quality.
While you have probably heard “more data is always better”, what is happening here is that with more data, a larger amount is of good quality.
But since Christmas gift prediction is not a long-established field, I’ll only be able to gather a relatively small dataset. So data quality is of utmost importance.
Gathering Data
Give Christmas presents a quick Google search. You’ll find lots of images. In fact, you’ll find millions of images.
Millions of perfectly wrapped Christmas gifts of all shapes and sizes. Pristine folds and beautiful bows.
But do those photos tell you what’s inside? No.
Despite all those images of nicely wrapped presents, they don’t show you what’s inside. And rightly so, that would spoil the surprise.
Yet, to build a model to solve my problem, I need to know what’s inside. I need to know what is in those pretty boxes.
Labeling Data
The problem with these images of gifts is that they mainly show people a nicely wrapped present.
Those stacks of gifts at the mall of perfectly wrapped gifts next to Santa? They’re likely all empty boxes. Perhaps they’re full, but the point is we don’t know for sure.
But I need labels. I need to know what’s in the box to train a model to predict what’s in future presents. So to find the information I need, I need to look elsewhere for my data.
YouTube to the Rescue
Ah, YouTube, the place where you can find videos about literally anything and everything.
Here is where I realized my solution. What I’m looking for isn’t wrapped gifts. It’s videos of people wrapping gifts. So I get a nice view of the gift before wrapping and a nice view of the final product.
And good news, YouTube has a lot of gift wrapping videos. A lot.
Here is where I’ve gathered the data for my model. First, I carefully take screen captures of the presents before and after wrapping.
However, because of the nature of the dataset, I will not be sharing the data. Properly distributing the data would require the permission of all of the creators, many of whom are not active. But their wrapping videos remain.
The Dataset
There I was, going through videos, carefully using screen capture to pull out images of presents and images of the gift inside and taking extra precautions only to select the gift in the image.
The plan is to gather images in pairs. The first image represents the actual gift and the second image is of the final wrapped present. Both images are given a numerical ID so that the labels can be modified using the gift images.
With the final set of gifts and wrapped gifts. I still need to go through and label each of the images. This process will determine the categories that I want to use for prediction. The aim is to divide the images into rough categories with enough images of each type to build a model.
The final dataset contains 64 images of different presents. The process took a few hours-only videos where a gift is identified before wrapping are used. Additionally, I wanted images with an unobstructed view of the gifts.
In many cases, this required going through videos frame by frame to get clear images of the presents not blocked by arms, hands, or other presents.
Needless to say. It was a painful process. So I have 64 images, and I’m going to make that work.
Categories of Presents
There are limitless different gifts you can give someone. Unfortunately, for AI, I need to define what type of gifts I’m trying to predict.
I want to strike a balance between too general or too specific categories.
I decided on toys, books, cloth-based, and other.
- Toys contain all things related to children’s toys.
- Books are books and other related reading material.
- Cloth-based includes gifts such as socks, sweaters, and stuffed animals.
- Other contains electronics, bottles, perfumes, and other miscellaneous items.
Data Augmentation and Manipulation
With so few images, I am using data augmentation and manipulating images to produce more models to train a model. These augmentations effectively simulate viewing the object from different points of view, perhaps different angles or lighting.
The first type of augmentation I use is reflections and rotations. Flipping over vertical lines, horizontal lines, and rotating creates 4x the number of images.
Additionally, I apply gray-scaling and resizing to each image to simplify the training process. The grayscale forces the model to learn the classes based on shape, whereas the resizing ensures that all my screen captures are consistent sizes for the model to process.
Creating a Model & Predicting Presents
As you may have guessed, I’ll be using a branch of AI called computer vision to predict the contents of Christmas gifts.
In particular, I’ll be using a deep learning model called a convolutional neural network (CNN). These networks are ideal for classifying images. They use several different types of layers within them.
The CNN uses convolutional layers, pooling layers, and dense layers. At the end of the model, a class is predicted from four classes.
How well does AI guess?
As with most AI problems, there is a plethora of metrics available you should check out with classification problems. But, for this problem, I’m just using accuracy.
You’ll also notice several other design choices, such as a learning scheduler, weighting classes, and an Adam optimizer. Each of these small choices affects the overall performance.
As you can see, the model was quite bad. Deep learning models cannot learn well from the data without enough data. The accuracy maxes out at 50% on the test set despite the decreasing loss.
Slightly better than random with four classes but certainly not great. The toys make up 40% of the classes, skewing the results. The remaining performance can effectively be chalked up to randomness, considering the size of the data set.
However, Christmas gifts are supposed to be mysterious. Presents will intentionally be in boxes that don’t match the shape of the gift.
Therefore most of the structural features of the wrapped presents that a CNN can learn won’t always be directly related to the shape of the gift.
For these reasons and many more predicting presents is a difficult task for AI. Therefore, I don’t expect this problem to be solved by AI for some time, if at all.
Wrap Up
The final model did not perform too well. But you can probably already think of ways to improve the model.
Adding more data would surely improve the model. Unfortunately, there aren’t enough images for a deep learning model to predict future Christmas gifts effectively. But the trade-off is that there are more well-defined categories of images to determine with more presents.
Adding other attributes about the presents should also improve the performance. No person guesses a gift based solely on looking at it from afar. You pick it up, give it a shake, feel the weight. All of these things would most definitely help with predictions.
But as you’ve seen, AI needs data. And predicting presents is not a well-established task. So, for the time being, predicting what’s in a Christmas gift is left to the kids on Christmas morning.
Merry Christmas!
If you’re interested in reading articles about novel data science tools and understanding machine learning algorithms, consider following me on Medium. I always include code in my articles that you can apply to your work!
If you’re interested in my writing and want to support me directly, please subscribe through the following link. This link ensures that I will receive a portion of your membership fees.
For those interested in optimizing deep learning models, here are several articles I’ve written on the subject. These techniques will help you squeeze out even more performance from your deep learning models.
