Audio identification company, mufin, develops software for automatic content recognition and music recommendation technology using digital signal processing. Mufin products are licensed as white label software solutions for the business-to-business sector.
Since 2011, Germany-based mufin has offered specialised solutions for second screen applications and interactive advertising. Audio recognition technology enables mobile applications to synchronise with TV programs and detect commercials in real time.
Challenge
The increased popularity of smartphones and tablets has not only changed everyday life by making information and online contacts constantly available, it has also changed our relationship to television. According to a Nielsen study, 84% of American smartphone and tablet owners use their mobile devices while they watch television.
In many cases mobile second screens are simply used in parallel to the TV although unrelated to the subject of the program being watched. Most often people simply check emails and visit social media sites such as Facebook or Twitter.
The second screen can draw the viewer’s focus away from the television; advertisers have to fight harder than ever before to get the audience’s attention.
Approach
Using audio identification technology it is possible to connect television and mobile devices to create a synchronized multi-screen advertising. Employing second screen applications in this way opens up all kinds of new opportunities: companies can provide news and additional information on their brand, viewers can use their mobile device to fill out polls or collect bonus points.
An excellent example of a company using this approach is TVSMILES. This Berlin-based startup uses mufin audio recognition technology to identify commercials and provide related games and quizzes. Their app allows viewers to play for ‘Smiles’ which can then be redeemed in exchange for attractive bonuses.
It also acts as a bridge between TV advertising, digital marketing and the point-of-sale by creating a platform for dialogue marketing and enabling viewers to interact directly with the TV commercial.
The technology
How does the audio device recognise commercials? A mobile device with mufin’s recognition technology records audio from the television and analyses it in real time. A highly-compressed summary known as an acoustic fingerprint is generated from the recorded audio material. This is subsequently compared to the fingerprints in the database in order to find the most similar reference fingerprint.
Each reference fingerprint contains additional information such as station and timestamp. Using this data it is possible to achieve continuous synchronisation. Sometimes only a program or selection of commercials need to be identified, in this case the process takes place directly on the mobile device due to the relatively low volume of data and short search time.
Results
The TVSMILES app has already been downloaded over 1 million times and has earned over 50,000 five star ratings, making it one of the highest ranking apps in Germany. Between 300,000 and 400,000 users are active each month, with as many as 100,000 per day. Since its launch, up to several hundred million ‘Smiles’ have been collected by app users.
These figures demonstrate the potential of this technology to create more effective TV advertisements, as well as providing direct interaction with the target group and benefiting advertisers through additional branding.
Investors also believe in this business case. TVSMILES recently received $7,000,000 for Series A and plans to move into its first foreign markets this year.