Although the possibilities that are glimpsed for ChatGPT and generative Artificial Intelligence (AI) are many, the high cost that it can entail for organizations is a topic that is not always talked about. However, it is necessary to go deeper in order to control its cost.
Typically, it is the big companies like Microsoft, Meta and Google that are investing heavily to develop their technological advantage by leveraging the AI. However, if the margin for AI applications is typically smaller than the margins for Software as a Service, due to the high cost of computing, it could dampen the current boom.
Even when the software is built or trained, it still requires a lot of computation to run large language models because it does billions of calculations every time it returns a response to a prompt. By comparison, serving applications or web pages requires much less computation.
These calculations also require specialized hardware. While traditional processors can run models of machine learning, They’re slow. Most of the training now takes place on graphics processors, or GPUs, which were initially intended for 3D gaming but have become the standard for AI applications because they can do many simple calculations simultaneously.
AI training models
Analysts estimate that the process of training a language model such as OpenAI GPT-3 could cost more than 4 million dollars. The most advanced language models could multiply this number.
Organizations that build great language models they should be cautious because it costs a lot
Meta’s largest LLaMA model released last month, for example, used 2048 Nvidia A100 GPUs to train on 1.4 trillion tokens (750 words is about 1000 tokens), which took about 21 days.
It took about a million GPU hours to train. With AWS-exclusive pricing, it would cost more than $2.4 million. And with 65 billion parameters, it’s smaller than current GPT models in OpenAI, such as ChatGPT-3, which has 175 billion parameters.
Organizations that build great language models they should be cautious when they retrain the software, which helps improve their capabilities, because it costs a lot.
More use, higher pay
To use a trained machine learning model to make predictions or generate text, engineers use the model in a process called “inference,” which can be much more expensive than training as it may need to be run millions of times to become popular.
A product like ChatGPTwhich is estimated to have hit 100 million monthly active users in January, could have cost OpenAI $40 million to process the millions of messages people entered into the software that month.
Costs skyrocket when these tools are used billions of times a day. Financial analysts anticipate that the chatbot Bing AI from Microsoftwhich runs on an OpenAI ChatGPT model, needs at least $4 trillion in infrastructure to deliver answers to all Bing users.
how could i change
It is not clear whether AI computing will remain expensive as the industry develops. Companies that make entry-level models, semiconductor manufacturers, and start-ups see opportunities in lowering the price of running AI software.
More and more companies will focus on the development of smaller and more specific models to optimize costs
Nvidia, which has about 95% of the market for AI chips, continues to develop powerful versions designed specifically for machine learning, but improvements in chip power across the industry have slowed in recent years.
Some companies have focused on the high cost of AI as a business opportunity and more and more companies will focus on developing smaller and more specific models. These are cheaper to train and operate, rather than language models which, like ChatGPT, manage to attract much of the attention.
Meanwhile, OpenAI announced last month that it is lowering the cost for enterprises to access its GPT models. Now he charges a fifth of a cent for about 750 words of production.
initial image | Negative Space
