Chip industry is going to need a lot more software to catch Nvidia’s lead in AI

The semiconductor industry is in the midst of a renaissance in chip design and performance improvement, but it will take a lot more software to catch up with graphics chip titan Nvidia, an industry conference Tuesday made clear. 

The Linley Fall Processor conference, which is taking place as a virtual event this week and next week, is one of the main meet-and-greet events every year for promising young chip companies. 

To kick off the show, the conference host, Linley Gwennap, who has been a semiconductor analyst for two decades, offered a keynote Tuesday morning in which he said that software remains the stumbling block for all companies that want to challenge Nvidia’s lead in processing artificial intelligence. 

“Although several chip vendors and cloud-service vendors have developed impressive hardware for AI acceleration, the next hurdle is the software,” said Gwennap

“Software is the hardest word,” quipped Gwennap, referring to the struggles of competitors. 

He noted how companies either don’t support some aspects of popular AI frameworks, such as TensorFlow, or how some AI applications for competing chips may not even compile properly.  

“To compete against deep software stacks from companies such as Nvidia and Intel, these vendors must support a broad range of frameworks and development environments with drivers, compilers, and debug tools that deliver full acceleration and optimal performance for a variety of customer workloads.” 

Nvidia, which dominates the training operations that build neural networks, has a lead of over a decade with a software system called CUDA. Luminaries in the AI field, who work with Nvidia chips to build their neural networks, have repeatedly expressed the view that the field of AI needs competition to break Nvidia’s hold on the science.

The use of AI is spreading from cloud computing data centers where it has traditionally been developed to embedded devices in automobiles and infrastructure. Vendors such as the UK’s Imagination and Think Silicon, a division of chip equipment giant Applied Materials, are pushing the boundaries in low-power designs that can go into power-constrained devices, such as battery-powered, microcontroller gadgets. 

The stakes seem suddenly higher since Nvidia announced last month that it intends to buy Arm Plc for $40 billion. Arm makes the intellectual property at the heart of all the chips made by all the challengers in the chip industry. Hence, Nvidia’s software is poised to gain even greater sway. 

Also: Nvidia makes a clean sweep of MLPerf predictions benchmark for artificial intelligence

Companies wanting to take market share from Nvidia have each had to build their own software to replicate some of what Nvidia does, but none of the offerings have yet closed the gap, said Gwennap. 

“There are some open efforts that are going on, they are not getting a huge amount of traction,” he observed. “It’s been up to most of the companies to develop their own alternatives, and that’s why it’s been taking so long.”

The conference featured a host of companies, new and established, that each has some innovation that can edge out Nvidia on raw performance in some AI tasks. They include startups such as Tenstorrent, Brainchip, and SiFive. Established companies including Intel and Google are also participating.

Illustrating the breadth and depth of innovation, Global Foundries, a contract chip maker that produces processors for tons and tons of chip companies, described how it is improving the physics underlying chips to create better transistors, the fundamental building block of all chips. 

Despite that progress, Nvidia continues to rack-up impressive performance with its parts, both old and new, as evinced by the latest AI benchmark test results, released by the MLPerf industry organization on Wednesday.

Software, said Gwennap, remains the sticking point to close the gap. 

“Intel and Qualcomm have good software stacks, they have heavily invested in software and have substantial resources, and we are seeing some progress from the well-funded startups, making good progress, but yes, it’s going to take time,” said Gwennap, when asked what efforts could give Nvidia substantial competition. 

“Developing this software can take months to years depending on the size and expertise of the team.”

Companies that are selling their own computing systems, including Cerebras Systems and Graphcore, have built brand-new software programs to optimize how neural networks are processed by their chips. However, those individual efforts may Balkanize the use of the new chips. 

Nvidia’s CUDA, by contrast, presents artificial intelligence designers with a consistent platform on which they can focus their efforts.

It may take some combination of all the different efforts to mount a credible challenge to Nvidia, Gwennap suggested.  

“I think there’s going to be some consolidation at some point to help some of these companies increase their software investment,” said Gwennap, referring to the startups with their varying software efforts. 

Whatever it takes, software is essential, said Gwennap. Without a complete set of software programs, the software stack, “the customer prospects for an AI accelerator are limited.”

The conference, now in its fifteenth year, has over 1,000 attendees this year, Gwennap told ZDNet, which is more than three times as many attendees as in prior years, when the event was held at hotel ballrooms in the Silicon Valley area.

The conference continues Wednesday and Thursday and will have a second set of presentations next Tuesday, Wednesday, and Thursday, including a keynote address from Google on Tuesday. 

By ZDNet Source Link