Amazon’s AWS cloud compting service on Wednesday morning kicked off its machine learning summit via virtual transmission.
The morning’s keynote talk was lead by Swami Sivasubramanian, AWS’s vice president of AI and machine learning; Yoelle Maarek, vice president of research and Science for Alexa Shopping; Bratin Saha, vice president of machine learning at AWS; with a guest appearance by Ashok Srivastava, who is Chief Data Officer at software maker Intuit.
Sivasubramanian lead off with a talk about machine learning being “one of the most transformative” technologies in a generation. He cited a stat that more than 100 papers in machine learning are published each day. “Machine learning is going mainstream,” said Sivasubramanian. More than 100,000 customers use AWS for machine learning, said Sivasubramanian, citing examples such as pharma giant Roche and The New York Times.
Sivasubramanian offered examples of “working backward from the customer” in the development of ML. The first example was how a system can “learn with less data.” Accessing and annotating data is “too tedious” as ML becomes mainstream, said Sivasubramanian. He cited the example of the NFL wanting to manage its library of video assets from football games. Another customer, an 80-year-old pizza company, wanted to ensure every pizza has the same amount of cheese to maintain quality. The company used AWS to get an imaging system for pizza inspection. The solution was what’s called “few-shot learning,” where machine learning is supplied with only a limited number of examples.
Few-shot learning is used for custom data labeling in the Amazon Rekognition product, Sivasubramanian noted. The NFL, for example, assigns custom labels to things such as players and jerseys in videos. Amazon’s service for defect inspection, Amazon Lookout for Vision, also uses Rekognition.
Sivasubramanian cited the desire to replicate a real factory setting. So, the team that developed Lookout for Vision built a replica of a factory to try the few-shot approach in the real world.
Sivasubramanian’s next example was understanding “irregular text” with machine learning, where, for example, text is blurred. Accuracy goes way down, he noted. That’s important for real-world instances such as transcribing doctors’ handwritten notes.
The traditional language model approach of guessing with the first few letters of a text runs into problems when there is little context. So, the AWS team invented something called “SCATTER,” for “Selective Context Attentional Scene Text Recognizer.”
SCATTER sends an image through additional processing that has a decoder that can choose to employ either contextual or merely visual information. The SCATTER technology lead to a 3.7% improvement in text recognition on benchmark character recognition tasks, a big improvement, Sivasubramanian said. SCATTER is now used in AWS’s automatic text extraction service.
Sivasubramanian then brought up Maarek, to talk about “giving Alexa a sense of humor.” Maarek referenced Alan Turing’s 1950 paper on “Computing Machinery and Intelligence,” in which the mathematician argued against presumptions about computers.
“Think of debuggers,” said Maarek, which is an example of how a computer “thinks about its own thought,” something people thought computers wouldn’t do but that Turing said they would. “Already, Turing was looking at having a sense of humor being a really hard challenge” in computer science, said Maarek.
Rather than trying to make Alexa be funny, said Maarek, the challenges was, “We want to look backward, ask whether customers are funny, and how should the machine respond to it,” explained Maarek.
That lead to the challenge of “detecting humor when customers are the one being funny.” To train the system, Maarek and team looked at humorous customer comments on Amazon. “We actually discovered tons of funny questions” in product comments, she said, such as “can you hack into the Matrix” via the Nintendo Switch video game machine, as one customer asked. Maarek proceeded to explain the joke…
Another example was customer sarcasm, she said. Will a luxury drink cooler “make me fly,” a customer asked. Said Maarek, “Sarcasm: funny.” Another type of humor is “the superiority theory of humor,” such as asking whether the Amazon Show appliance will cook breakfast. That is an example of a human showing their superior understanding of the simple machine, she explained. Someone asked of the Hutzler Banana Slicer, “Will it bend the other way?” Another example: “If a unicorn farts in the woods and no one is around, does it make a sound?” (pertaining to the product Unicorn Meats.)
Maarek said the team built a deep learning model, employing notions about humor such as subjectivity, and using embeddings. “We took into account domain bias, to make sure we didn’t over-fit our model.” As a result, the team was able to present a paper with high degrees of humor accuracy at last year’s SIGER conference.
Then, the team moved on to how to detect with speech in Alexa. Would customers appreciate, she asked, Alexa understanding the humor? Or did customers want to feel superior? Maarek cited humorous user utterances toward Alexa, such as “Alexa, can you burp?”
“You will see a ton of toilet humor,” she said, “It’s part of a very important area of humor, relief humor.”
“Alexa, what is your blood type?” was among the things that get asked. Some examples, she noted, are not so much funny as playful. Such utterances are examples of both personification and superiority on the part of humans. “We defined playfulness,” she said. Playfulness means, “the customer doesn’t expect Alexa to take this request literally,” and Alexa should not add anything to the shopping list of the user.
Maarek said she and the team had to go back to researching the teachings of Aristotle, Kant, Schopenhauer, and other great thinkers regarding humor, to understand all the forms of humor. Surveying all the forms of humor helped the team understand the matter of what users will enjoy from Alexa. The question became, Will users enjoy it if Alexa understands their humor?
The team started with “personification,” where people relate to Alexa as a personality, as a point of conjecture to explore the problem. They recruited a hundred college students in a blind question-asking exercise, talking to an entity they didn’t know was Alexa (The entity was named “Shirley,” a play on the movie Airplane.)
The students’ questions were examined by a custom version of Google’s BERT transformer neural net. It employed sentiment analysis and such. “We got a pretty good model,” she said, “to detect these funny personification utterances on the fly.”
The team went to a speed-dating site to scope out questions people ask when trying to be funny or playful. That lead to a survey of personification questions that people ask “Do you think as good as a woman?” is one kind of question that gets asked.
The result was that the team determined that human questioners enjoy it when Alexa responds to their playfulness. “They really want to have fun not at Alexa but with Alexa,” was the conclusion of the research.
Sivasubramanian then came back on stage and talked about an industrial example of ML use: Amazon’s automated fulfillment centers. The company installed 800 sensors on equipment in a fulfillment center in Germany. “We learned a lot,” said Sivasubramanian, including how to reduce faults, and how to better understand the optimal range from a sensor to a wireless gateway. Amazon is going to install “tens of thousands” of its Monitron sensors in the coming months across its fulfillment facilities, said Sivasubramanian.
Sivasubramanian moved on to talk about “horizontal” use cases, where customers don’t have much in the way of ML skills. That includes “embedding” what’s called “autoML,” where customers don’t need to know about model design or tuning. The tech is then used for things such as customer service, document recognition, etc. Sivasubramanian called out domain-specific models for healthcare, such as medical note transcription.
Sivasubramanian called out Amazon SageMaker, the company’s development environment, which is the way the company brings machine learning frameworks such as PyTorch and TensorFlow to data scientists.
Then Sivasubramanian moved on to deploying machine learning at scale, and he invited up Saha. Saha made the point that customers have increased their model deployment from “just a few” models in the early days to “thousands” per customer. SageMaker, he noted, now supports hundreds of billions of predictions per month. “From a dozen models to millions of models and hundreds of billions of predictions in just a couple years,” was how Saha summed up the progress.
Saha cited Lyft as a customer. They used SageMaker to reduce model training time for “Level 5” ADAS (self-driving.)
iFood, a leading food delivery company in LatAm, used SageMaker to reduce the travel distance of delivery staff.
Saha cited examples of “using SageMaker in our daily lives.” That includes when you order from Amazon, he said. Amazon needed to integrate and manage tools for fulfillment. That includes “monitoring millions of global shipments annually.” That lead an internal Amazon team to build a computer vision system to scan items at fulfillment centers. That project wasn’t able to handle new requests in production. So, they had to develop ML models offline. That process took three to six months. With SageMaker, said Saha, the team was able to reduce model deployment to two weeks and were able to reduce “prediction latency” by 50%, he said.
“We are building SageMaker along three vectors,” said Saha. “Infrastructure, tools, and ML industrialization.” Saha talked about things built on top of SageMaker, including AWS Inferentia, which is used by customers such as Snap, Autodesk, and Condé Nast for lower cost and higher performance of inference.
Saha talked about the Habana Gaudi-based chips that will be coming to EC2 instances this year. He also called out AWS’s own home-grown “Trainium” chip for ML training, also coming later this year.
Saha moved to talking about the problem of deploying multiple endpoints, one for each ML model. The solution was SageMaker’s “multi-model endpoints,” which allows one to “host hundreds of thousands of models on a single endpoint.” That leads to optimization of prediction accuracy and throughput, he said.
The next technology was SageMaker Clarify, a tool to gain insight into why ML models produce certain predictions. Saha cited the use by SageMaker of “Shapley values,” running experiments on the model or data set to see how predictions improve. Amazon was able to make those Shapley tests run ten times faster than existing open-source implementations, he said.
A third vector, he said, was industrialization of machine learning. “We asked ourselves, How did software go from a niche to an industry?” It involves tools from software such as IDE and CI/CD. SageMaker is the first IDE for ML, he noted. Another analogous tool is the CI/CD tool. Those are rarely available in ML, he noted.
So, Amazon built SageMaker Pipelines, the first purpose-built CI/CD service for machine learning, he said. The tool lets one roll back and troubleshoot ML code at each time step. Customers such as 3M are using Pipelines to let them scale. “With just a few clicks, you can create an entirely automated workflow that reduces months of coding to just a few hours,” said Saha.
Another industrialization avenue is training everyone on ML. Every engineer who joins Amazon has learn ML, he noted. There is a new MOOC, on Coursera, for deep learning, in conjunction with DeepLearning.ai. The course, Practical Data Science, is for those ready to implement ML in the enterprise. (Coursera’s founder Andrew Ng is a speaker later today at the Summit.)
ZDNet‘s Larry Dignan chatted with Ng and Saha about the course.
Andrew Ng, co-founder of Coursera, and DeepLearning.ai, talks with AWS’s Swami Sivasubramanian.
Amazon AWSSivasubramanian came back on to introduce Intuit’s chief data officer, Srivastava. Srivastava said Intuit’s “mission” is “powering prosperity around the world.” He cited stats of tax returns filed via TurboTax (48 million) and “Mint users empowered to make smart money decisions” (over 25 million.)
Srivastava said a lot of that was a result of “AI and machine learning at scale.” Intuit has been working with Amazon AWS since 2018. Intuit has put 250 “AI assets” into production, and is running 2,059 “AI tasks” in production. Intuit has been able to file 600 AI patents in the U.S. in recent years, he said.
Srivastava talked about what he called the AI “hierarchy of needs.” The actual AI model is at the top of the pyramid, the stuff Intuit wanted its developers to focus on, and stuff that is “nonproductive” is the Ml infrastructure below it, and the data infrastructure below that. Those were the things were Intuit relied on Amazon. Srivastava said the company has seen a 60% increase in mobile app deployments. “The benefits to your AI teams are immense,” he said. Intuit has increased by 50% its number of deployed ML models.
Srivastava reviewed Intuit’s machine learning platform, built on top of SageMaker, Amazon Connect, etc. That includes ML Ops Intuit has built, and CI/CD tool chains. And then on top of that, the company has built its “virtual expert platform,” which can field questions such as an Intuit customer’s home renovation, using natural language processing, digital assistants, etc. to connect the user to a subject-matter expert. Srivastava used the example of TurboTax Live, which poses questions about a person’s work and such to figure out one’s taxes. “Through the magic of the virtual expert platform, you can get connected to the right expert,” a human consultant, he said. That is made possible by machine learning that does routing, he noted.
Sivasubramanian, closing out the keynote, noted in broad sweeps how machine learning is “transforming entire business processes.” That’s happened by “barriers to entry being lowered.”
And that’s a wrap!
