Many folk will have seen stories or posts floating around the internet claiming that someone was talking to someone else about X one day and they suddenly started received adverts about it on their phone, the assumption being that the phone listened in on the conversation, picked out the keywords and sent an ad on that basis.
One likely alternative explanation is that the person experiencing this had just primed, or sensitised, themselves to that ad. We see and blank out thousands of ads every day, at least consciously, but that doesn’t mean we don’t see them. And by talking about a thing we are then primed (self-primed?) to consciously notice it if it does cross our attention path soon after (my cog psych knowledge is not that good; there are probably some really good experiments and mechanisms around to explain this… eg stuff).
Another possible contributory factor is that the models are getting better at prediction. You do a sporty thing at a particular location (your phone knows where you are) and talk about different deodorant products afterwards. You then spot a deodorant ad. Your phone has been listening to you. Or maybe your phone (or the services or networks it is connected to) spotted you were at a sporty location, there was no phone activity for an hour and a half, (not even jiggling around, as detected by the gyros, so you left your phone somewhere; or maybe the signal died when you put it in a locker) so maybe you were doing something sporty, so maybe: worth a shot at advertising a deodorant?
Now the phone may or may not be being used to listen to you in the audible sense of hearing your spoken conversations (it’s certainly being used to “listen” to your actions in web tracking ways, for example), and the webcos et al. tend to protest that they don’t. But they don’t make life easy for themselves with the sorts of things they do announce they can do.
For example, in a recent blog post on the Google AI blog, Real-time Continuous Transcription with Live Transcribe, there’s this:
Today, we’re announcing Live Transcribe, a free Android service that makes real-world conversations more accessible by bringing the power of automatic captioning into everyday, conversational use. Powered by Google Cloud, Live Transcribe captions conversations in real-time…
Okay, so if you have a network connection, your phone could transcribe any audio it heard in real time. Google are celebrating that fact. There is no technology blocker if access to the microphone and internet connection are available, and the microphone is in range of the conversation.
Potential future improvements in mobile-based automatic speech transcription include on-device recognition, …
So they also want to be able to do it on the phone…
The world is full of such apparent contradictions. On the one hand, conspiracy theories about what the tech giants (increasingly, rather than “the state”, as in the case of China) are doing; on the other, announcements by the same companies about what they don’t (as a matter or policy), can (technically), and do want to (technically) do.
Which comes down to a question of trust that the policies they operate under are: a) sound; b) followed; c) not not followed.
Here are some more possible contradictions…
We trust the Amazon store as a place to shop, right? Like a supermarket or department selling branded goods. But hold on a minute… For a start, it’s increasingly a market, and just like a free market or a car boot sale, buyer beware. At scale. Why? Well, Amazon Warned Apple of Counterfeit Products in 2016 and is now Warning Investors that Counterfeit Products are a Problem; there are plenty of other stories about counterfeit products on Amazon out there.
Something else to note about Amazon is that they are like a supermarket in a certain respect: they sell products from a wide range of own brand items, although you may not realise it. One way of trying to track down what brands they own is to look at the WIPO Trademark database.
(I built a trademarks by company explorer once, using OpenCorporates data and OpenRefine, but I suspect it’s rotted by now. Maybe worth revisiting, along with something that mines companies in a corporate grouping and grabs trademarks associated with all of them?)
So how about another Google story — Advancing research on fake audio detection:
When you listen to Google Maps driving directions in your car, get answers from your Google Home, or hear a spoken translation in Google Translate, you’re using Google’s speech synthesis, or text-to-speech (TTS) technology. …
Over the last few years, there’s been an explosion of new research using neural networks to simulate a human voice. These models, including many developed at Google, can generate increasingly realistic, human-like speech.
While the progress is exciting, we’re keenly aware of the risks this technology can pose if used with the intent to cause harm. Malicious actors may synthesize speech to try to fool voice authentication systems, or they may create forged audio recordings to defame public figures.
…
We’re taking action. When we launched the Google News Initiative last March, we committed to releasing datasets that would help advance state-of-the-art research on fake audio detection. Today, we’re delivering on that promise…
On the one hand, the Goog is trying to create ever authentic voices. On the other, so are the bad guys. (Google, by implication, is not a bad guy).
By releasing some of their data, they hope to encourage third parties to create systems to distinguish real voices from machine generated ones.
The way research works, of course, is that the folk (at Google…) who create machine generated voices will presumably try to improve their creations to avoid detection by the new improved machine generated voice detectors on the grounds of “improving customer experience”…
In their defense:
As we published in our AI Principles last year, we take seriously our responsibility both to engage with the external research community, and to apply strong safety practices to avoid unintended results that create risks of harm.
So I wonder, will Google add something inaudible to its machine generated voices that flag out a voice as machine generated, even one sent over a low pass filtered phone connection, in a spirit of responsibility? This would make it trivially easy to detect a Google generated voice and prevent lazy bad guys from using it to fool other machines.
Finally, and this may seem like more Google bashing, but hey, this is just a sample pulled from today’s feeds (I’m on catch up), how do these big cos develop the trust that supports a belief that they can be trusted to formulate and adhere to sound policies that are not just for corporate benefit but also, at the very least, do no harm to the rest of society, if not actually benefitting it? By being good (corporate) citizens in wider society? Google pays more in EU fines than it does in taxes. Erm…?
OUseful.Info, the blog... 