On the video chat, a woman waves and reaches out to a baby, bouncing on its mother’s lap. The picture is so lifelike — the baby has its fingers in its mouth — it’s as if the woman thinks she can touch the people on the other side of the screen.
It isn’t a hologram, but it seems close.
In the throes of the coronavirus pandemic, Zoom became a lifeline. Now Google is trying to reinvent video chat with computer vision and machine learning to create realistic 3D images you can talk to without donning a special headset or glasses. The result is what Google calls a “magic window” that connects people across cities and countries. Right now it’s being tested at Google offices in the San Francisco Bay Area, New York and Seattle, but the world’s largest search provider will run trials with a few business partners later this year.
Called Project Starline, the initiative is one of the marquee announcements at the Google I/O developer conference, which made its return on Tuesday after skipping a year because of COVID-19. In 2020, people cocooned themselves at home to fend off the fast-spreading virus. Hospitals overflowed, and loved ones were lost. Tech executives announcing new products weren’t on people’s minds.
Now much of the world is in a different place. Vaccination distribution is continuing in many countries, and more of the world is opening up. The search giant, too, hopes to turn the page. It kicked off the three-day I/O developer fest in a fashion that seems almost throwback: It was live.
True to the times, the new normal was a little different. The keynote was broadcast from a courtyard at Google’s headquarters in Mountain View, California, rather than at the Shoreline Amphitheatre, the nearby outdoor venue that’s hosted the confab since 2016. Instead of a large crowd of developers, the audience was a small group of Google employees. It was a step away from the glossy pretaped productions that Apple and Samsung have delivered in the pandemic era, even if it was still streamed.
“Over the past year, we’ve seen how technology can be used to help billions of people through the most difficult of times,” CEO Sundar Pichai said in a statement ahead of his keynote speech. “It’s made us more committed than ever to our goal of building a more helpful Google for everyone.”
For Google, the chaos extended well beyond the pandemic. The company has weathered a host of controversies over the past 12 months. It’s the target of three major antitrust lawsuits, including a landmark case by the US Department of Justice and another by a bipartisan coalition of nearly 40 states. Google’s ouster of prominent artificial intelligence researchers sparked outrage across the industry. Pichai, who’s been CEO since 2015, was hauled before Congress three times in the last year to defend the company in front of lawmakers concerned about everything from disinformation to alleged anti-conservative bias.
“They’re focused on the product story, and that’s the story they want to tell,” said Bob O’Donnell, founder of Technalysis Research. “They’re eager to show they’re pushing forward.”
‘We have to move past this’
One area of Google’s business — artificial intelligence — neatly captures the company’s complicated year. Google is a powerhouse in the field, and its I/O announcements put that prowess on display. But in the background, thorny issues underscore struggles that Google would like to move past.
One of the biggest unveilings at I/O is the announcement of a new campus in Santa Barbara, California, focused on quantum computing. It will be host to hundreds of Google employees and include a quantum data center, research labs and a chip manufacturing facility.
“This is a longer-term bet on an interesting technology that might be able to solve important problems that no classical computer can solve,” says Jeff Dean, Google’s AI chief.
Another highlight is a technology for Google Search called the “multitask unified model,” or MUM, which Google calls a “milestone” in understanding information. The goal of MUM is to decipher complex questions that involve several steps and layers. For example, if someone asks Google how to prepare for climbing two different mountains, the search engine could in the future give that person information on the elevation of each mountain, as well as details on training and what gear to buy.
There’s also a new service called Derm Assist, which uses AI to help people learn more about possible skin, hair and nail conditions. Using phone cameras, people upload pictures of an ailment and answer a series of questions. The software then matches the pictures to a database of 288 known conditions and gives them information on what the condition might be.
As helpful as that may sound, the service raises potential privacy concerns. Sending pictures of a worrisome mole to Google may be a little too creepy for some people. But Karen DeSalvo, Google’s chief health officer, says people are already coming to Google with their health concerns, with almost 10 billion Google searches a year related to hair, skin and nail issues. Still, she acknowledges the trust barriers, especially as people become increasingly wary of big tech.
“It’s something we really think about,” DeSalvo says. “What we hope over time is that, as people see the information getting better-quality, as they’re seeing that they’re getting navigated to authoritative sources, they’re going to increasingly trust us.”
Though Google has made strides in AI, the division has been roiled for months by high-profile terminations and resignations over ethics and diversity. In December, Timnit Gebru, one of the few prominent Black women in the field, announced on Twitter that she had been fired over a research paper that called out risks for bias in AI, including in systems used by Google’s search engine.
The fallout led to Google’s firing of Margaret Mitchell, who founded the company’s ethical AI unit and co-led it with Gebru, after an investigation over data security. Samy Bengio, who managed Gebru and Mitchell and voiced support for them, resigned last month.
“The reputational hit is a real thing,” Google’s Dean said in his first interview since the controversy became public in December. “But we have to move past this, and we are deeply committed to doing work in the space and feel it’s a really important area.”
A slew of app updates
Putting the spotlight back on products also means highlighting changes to Google products that billions of people use every day, including the Android mobile operating system, Maps and Search.
At I/O, the company trumpeted a new milestone for Android: It’s now running on 3 billion devices globally, another sign of dominance for the most popular mobile software in the world. Android powers almost nine out of every 10 smartphones on the planet.
Other changes to Android include a major design makeover, the biggest aesthetic change to the platform since 2014. The new look includes a color extraction feature, which generates a color palette for phones by pulling similar hues from their wallpaper. Another feature now lets people unlock their cars using their phone as a digital key. It will also let people send their keys to other phones when they lend out their car. Google is partnering with BMW to debut the tool, and it will be available first on Google’s Pixel phones and Samsung’s Galaxy phones.
Google also updated its Maps app. One change is a direct result of the pandemic: a feature that tracks how crowded neighborhoods and other large areas are. Google reckons you’ll want to know if a farmer’s market, for example, has attracted a rush of people so you can avoid it if you want. Another feature adds more context to a map based on the time of day. So it won’t show you, say, a closed breakfast place if you look at the map at 9 p.m.
The tech giant also introduced a product update to its most iconic service — its search engine. Now results will include a label called “About this result” that provides users in the US more context about that source, in an attempt to combat misinformation. Google says it’s working with Wikipedia to provide background on websites, including short descriptions. People can also see when the site was indexed and whether or not your connection to the site is secure.
“This allows you to see, from a given piece of information, more about the source,” says Liz Reid, a vice president of engineering for Google Search. “So you can really understand: Why is this source the one that’s telling you this information? And how much do you trust the source?”
The change comes as Google and other tech giants face intense scrutiny over false information being spread on their platforms. When it comes to Google’s services, the video site YouTube often gets the brunt of criticism for spreading misinformation and conspiracies, but shady search results can also be the culprit. In March, Pichai appeared virtually before Congress alongside Facebook CEO Mark Zuckerberg and Twitter chief Jack Dorsey to testify about the danger of misinformation on tech platforms.
“Staying ahead of these challenges and keeping users safe and secure on our platforms is a top priority,” Pichai said at the time.
A closeup on privacy
The search giant, which has long been criticized for its data practices, is also touting its efforts in privacy and security at I/O.
Many of Google’s privacy changes are coming to Android 12. The most intriguing one is called Private Compute Core, which cordons off some data processing like speech recognition and machine vision from the rest of the operating system. The processing is done locally on the device and unconnected from the network, keeping the data more private. It powers Google features like Live Caption, which generates captions in real time, and Now Playing, which recognizes music playing in the area, similar to Shazam.
Another new feature allows people to limit what they share with apps, like only disclosing their approximate location instead of their exact whereabouts. The distinction could be useful, for example, if someone doesn’t want to share their precise location with a weather app.
Asked why Google hasn’t introduced a similar feature, Sameer Samat, vice president of product management for Android, suggested the company is working on it. “Who says we’re not?” he replied. When pressed, he said the company has “nothing to announce” right now.
Jen Fitzpatrick, a senior vice president in charge of core experiences and infrastructure across Google, says the last year, in which people across the globe have come to rely on technology to stay connected while physically apart, has made privacy more urgent.
“The pandemic has accelerated a lot of trends in terms of the amount of time people are spending online, on their various devices,” Fitzpatrick says. “It just reinforces how important it is to give users experiences that are safe.”
Now Google will have to prove that privacy is a part of its normal.
