This App Identifies the Movie You’re Watching and Lets You Buy What You See


Tyler Cooper thinks that movies are
the most beautiful commercials on
the planet.
That’s the idea that drives his
company, TheTake, which launched
its eponymous mobile app on
Thursday. The app uses Shazam-
like technology to automatically
identify movie scenes and then
pinpoint products in those scenes,
so that you can instantly buy them.

Let the app listen to a movie you’re
watching, and if it can recognize
the scene, it will identify this movie
moment on your phone or tablet
screen, along with annotations for
different items contained in the
scene, from suits to hats to
sunglasses. Then it points you to
retail partners where these
products can be purchased.
And, just in case that $1000
Balenciaga dress is a bit too pricey
for you, you may also get options
for similar items available at
different price points. “Our goal is
to make professional content
shoppable,” Cooper says.
TheTake is yet another (rather
ambitious) effort to join the
widespread movement towards
mobile shopping. According to one
report, 70 percent of consumers
have made a purchase via
smartphone in the last six months,
up from 59 percent in 2013, and
Cooper wants a piece of this pie.
The idea isn’t entirely new. In late
2011, eBay unveiled a new tool in
its iPad app, Watch with eBay, that
ostensibly let TV fans purchase
items from any show they were
watching, using their zip code
and local cable programming
schedules. It soon faded from view,
but Cooper believes his product will
be different.

Essentially, TheTake is a massive
classification and indexing project.
First, the company had to build an
audio recognition system as well as
computer vision technology for
identifying products in scenes.
Then, using this technology, it had
to start building a database of
movie scenes and products in those
scenes. The team began with this
year’s movies, and it’s now working
backwards through previous years.
When new movies arrive, the team
can toss these into the mix as well.

Fashioning such technology is no
simple task—and these types of
things don’t always work as
advertised. eBay’s users, for one,
ended up rather dissatisfied with
the company’s new tool, expecting
that the app would include more
dynamic features it did not. But
according to Cooper, TheTake was
able to automate the indexing of
movies, and he says the company is
focusing on hit films. “It turns out
if you look at 90 percent of box-
office hits in a year, that amounts
to just 90 to 140 movies or so,”
Cooper explains. “It’s a really
addressable market.”

At launch, TheTake includes more
than 100 movies in its database, but
Cooper says this will rapidly
expand. The system can “ingest”
movies at a rate of about
ten library titles a week. If you try
to identify a film and the movie
isn’t in the database, you can
request inclusion, and the company
will work to index the most-
requested flicks.

Though TheTake is sticking to
movies for now, the company says
it could ultimately handle TV, music
videos, and other professional
multimedia content. “We eventually
want to be this ubiquitous piece of
technology,” Cooper says, “that lets
people shop any video.”


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s