What it takes to create a GPT-3 product

When Open-AI introduced GPT-3 last year, it was met with much enthusiasm. Shortly after GPT-3’s release, people started using the massive language model to automatically write emails and articles, summarize text, compose poetry, create website layouts, and generate code for deep learning in Python. There was an impression that all types of new businesses would emerge on top of GPT-3.

Eight months later, GPT-3 continues to be an impressive scientific experiment in artificial intelligence research. But it remains to be seen whether GPT-3 will be a platform to democratize the creation of AI-powered applications.

Granted, a disruptive technology might need more time to create a sustainable market, and GPT-3 is unprecedented in many respects. But developments so far show that those who stand to benefit the most from GPT-3 are companies that already wield much of the power in AI, not the ones who want to start from scratch.

GPT-3 from a scientific standpoint

As far as research in natural language processing is concerned, GPT-3 is not a breakthrough. Like other language models that are based purely on deep learning, it struggles with common sense and isn’t good at dealing with abstract knowledge. But it is remarkable nonetheless and shows that you can still move the needle on NLP by creating even larger neural networks and feeding them more data than before. GPT-3 surpassed its predecessor in size by more than two orders of magnitude and was trained on at least 10 times more data.

The result was a language model that could perform zero-shot and few-shot learning. This essentially means that you can use GPT-3 for many applications without writing any code, without spending time and expensive resources retraining it, and without making any tweaks to the architecture. And for many applications, you just need to show the AI model one or two examples of the output you expect, and it starts to perform the task on new input with remarkable accuracy.

This performance led to speculation that GPT-3 would enable developers to create AI-powered apps without extensive knowledge of deep learning. And this would eventually lead to a new generation of entrepreneurs who would create new businesses on top of GPT-3.

But that’s not how the business of artificial intelligence works.

GPT-3 from a business standpoint

OpenAI’s decision to commercialize GPT-3 was largely due to the company’s need for sustainable funding. The AI research lab is burning a lot of cash to train its AI models and cover the salaries of its scientists. And it couldn’t continue operating on donations from founders and backers. It needed a sustainable source of income. And part of it will come from renting its huge language model to other companies.

One benefit of delivering GPT-3 as a cloud service is that it removes the technical and financial challenges of running the AI model. Instead of going through the pains and costs of setting up a server cluster that can run GPT-3, developers can directly use the language model through APIs and pay as they use it.

But while OpenAI’s GPT-3 service abstracts away the complexity of sitting up language models, it doesn’t remove the other challenges that go with building successful products:

• Prove that you’re solving a real problem that people are struggling with
• Prove that you’re solving it at least 10 times better than others in the market
• Prove that you can deliver your services at scale
• Have a roadmap for profitability, where the cost of acquiring a customer is lower than the average revenue per customer
• Have a business model that can’t be copied by competitors

That last point is important. Consider, for example, Amazon. Creating an Amazon clone is not impossible. So why hasn’t any other product dethroned the ecommerce giant? Amazon has built a hefty “moat” around its platform through network effects: Buyers continue to go to Amazon because that’s where the sellers are. Sellers continue to sell their wares on Amazon because that’s where the buyers are. So, no matter how good an Amazon clone you create, unless you can bring a critical mass of buyers and sellers to your platform, you won’t be able to snatch the competition away from the “everything store” in a profitable and sustainable way.

Building profitable applications on GPT-3

When it comes to launching machine learning-powered products, competition is defined not only by network effects and features, but also by AI factories:

• You must have a solid infrastructure that consolidates the data needed to train your AI models
• You must have the means to collect fresh quality data to continuously learn from users’ interactions with your product and fine-tune your algorithms

For instance, Amazon has a lot of historical data on customer purchases. This allows the company to develop machine learning algorithms that can provide relevant suggestions to customers or make its supply chain more efficient. These algorithms improve sales, optimize shipping and delivery, and reduce operating costs, giving Amazon the edge over other ecommerce platforms. This advantage brings more users to Amazon. Users in turn generate more data, providing Amazon with even more opportunities to learn, improve its algorithms, and try new AI-powered features. As long as Amazon can maintain this cycle, it will continue to dominate the ecommerce landscape.

Companies that want to compete with Amazon not only have to replicate its products, but they must also overcome Amazon’s huge data barrier.

This is where GPT-3 becomes a bit problematic. The language model is its own AI factory. A zero-shot learning system delivered as a cloud API service is not meant to learn new things. Everything it has comes out of the box, and you can’t change the underlying AI model. And if OpenAI improves GPT-3 over time (which it probably will), it will immediately deliver the upgraded model to all API clients at the same time.

The language model levels the ground for everyone. Any application you build on GPT-3 can easily be cloned by another developer.

This brings me to my main point: You can’t count on GPT-3 to build an entire product and working business model without taking extra measures to solidify your position in the market. On the other hand, GPT-3 can be a good platform to add crucial pieces to applications that already have a sizable market share.

GPT-3 startups

Since GPT-3’s beta launch, we’ve seen a number of new startups that aim to use the language model to create different applications. Alex Schmitt, investor at Cherry Ventures, has compiled a nice list of GPT-3 projects, including several products ranging from automated web- and ad-copy generation to resume-writing and website creation.

While some of these applications look like they solve a real problem, I don’t see most of them having a sustainable business model.

For instance, one company markets its application as “the first GPT-3 powered resume builder” that auto-completes resume content as you type. This could be a useful application, but I can’t see how it could be a sustainable business.

For one thing, one of the main sources of revenue for cloud-based applications is recurring revenue from subscribed users. And accordingly, the GPT-3 resume generator has several monthly subscription plans. But people don’t need to write resumes every month, so the product will probably have a very high churn rate as most users will leave in the first month after signing up.

Moreover, I don’t see why Microsoft, which already has an exclusive GPT-3 license and owns LinkedIn, wouldn’t add this feature to its Office suite for free or at a low cost if it proves to be a real game-changer.

Another example is Copysmith, a company that uses GPT-3 to “Write ads, descriptions, metadata, landing pages, blog posts, and more in seconds.” This is an area that GPT-3 could have promising results. I don’t think GPT-3 would be a good tool for writing in-depth analyses and op-eds about complicated topics. But it can certainly take on simpler tasks such as assisting in writing web copy.

You provide Copysmith with a prompt such as the name of your company, the target audience, and a short description, and it generates text for ads, product descriptions, marketing text, and more.

It is worth noting that the text GPT-3 generates is not perfect and still needs some polishing, but it could improve productivity and reduce costs. For instance, one person can use the tool to do the work of several content writers and result in lower staff costs (to the chagrin of content writers). The monthly pricing includes $19, $60, and $500 plans, which provide approximately 500, 2,500, and 20,000 pieces of original content respectively. So, it would be suitable for individual companies as well as large content mills that serve several customers.

But again, given the low entrance barrier of using GPT-3, I don’t see why other companies wouldn’t copy the same model (Headlime is another company that provides similar features). Alternatively, large content mills could create their own in-house version of the tool with little effort.

One company that seems to have gained traction is OthersideAI, which uses GPT-3 to generate email messages. You provide OthersideAI with key bullet points, and it generates a full email for you. The AI also uses the email chain and past correspondence as input to further personalize the tone of the email.

There’s no pricing plan available for OthersideAI, and it is currently in private beta phase.

The company has raised $2.6 million in funding to create its application, which means it is on to something. But what’s interesting is that the OthersideAI also knows that just putting a good user interface on top of GPT-3 is not a good business strategy. The company tweeted on January 22: “We need your help teaching our email generator to write a more extensive variety of emails.”

GPT-3 is a closed model and you can’t train it, which means there’s more going on under the hood than a simple facade over the GPT-3 API. To be clear, you can improve GPT-3’s performance by providing it one or more sample prompts and responses. The language model maps these new examples to its vast corpus of encoded knowledge and applies it to your new prompt to create a more specialized output. And the company is probably banking on this as its differentiating factor.

So, OthersideAI is creating a moat around its product by gathering a large corpus of example emails. It can then use this data in different ways to improve GPT-3 beyond its basic configuration. For instance, the company might create a much simpler machine learning model that matches the user input to the most relevant example and then feeds both to GPT-3 to generate the new email. This could be a working strategy. But it can also be a dangerous path that will engage the company in the troubles of handling private user data.

One of the benefits of OthersideAI is that it can be integrated into different email applications. But it will be interesting to see if products such as Gmail or Microsoft Outlook will consider adding similar features in the future.

Where does GPT-3 fit in the application ecosystem?

As these cases show (I suggest you explore the full list of projects on Schmitt’s website), GPT-3 is not a hassle-free path to creating a startup. I certainly expect the language model and its successors to improve existing applications and become a good platform for creating in-house tools for creativity. (I’m particularly interested in seeing how Microsoft will integrate GPT-3 into Office, Teams, Dynamics, and Bing.)

But creating profitable new businesses and products on top of GPT-3 needs careful planning to fend off copycats and competitors. And interestingly, the way to do it is not much different from what other AI companies are doing: gather quality data, learn new things, and create better machine learning models.

Ben Dickson is a software engineer and the founder of TechTalks. He writes about technology, business, and politics. 

This story originally appeared on Bdtechtalks.com. Copyright 2021

By VentureBeat Source Link