Qualcomm CEO: Phones Equipped for 100-Billion-Parameter Models Arriving by 2028

Qualcomm's CEO says some AI companies are building phones to continuously run 100-billion-parameter models by 2028.

Some AI companies are building phones to run 100 billion parameter models continuously by 2028, says Qualcomm's CEO.

Become a special member of Arjan Ambassador in order to benefit from and access special facilities and different departments in Arjan Ambassador.

Qualcomm CEO Cristiano Amon says some of the leading AI companies are building phones that can run large language models with 100 billion parameters continuously. According to him, these devices may arrive by 2028; However, RAM memory capacity limitation, its high cost and thermal challenges are still important obstacles for implementing such models on smartphones.

In an interview with the Fireside Alpha podcast, Amon stated that he could not reveal the names of the companies in question, but some of the leading companies in the field of artificial intelligence are developing phones with this capability. He did not provide technical details or a specific roadmap to achieve this goal.

The biggest obstacle to running large language models on smartphones is their required memory capacity. Running a 100 billion parameter model with 4-bit quantization requires at least 50 GB of memory; without taking into account the memory required by the operating system, programs and the temporary memory of the input text of the model.

Even with 2-bit quantization, the model itself will require about 25 GB of memory. These figures are higher than the RAM capacity common in smartphones, and considering the limited supply and the increase in the cost of memory, it seems difficult to achieve such a capacity.

Last year, some Android phones were released with up to 24GB of LPDDR5X RAM; But the memory crisis has caused a number of phone manufacturers to abandon the release of models with more RAM.

Running large models continuously has another problem: the heat generated. Heavy processing can increase the temperature of the device, and as a result, the chip performance will automatically decrease to prevent overheating; A process called thermal performance loss.

One of the possible solutions is to reduce the accuracy of the numbers used to represent the model parameters. This method, called quantization, reduces the amount of memory required. However, reducing the precision to less than 4 bits can significantly reduce the quality of language models; Especially when doing reasoning tasks.

Another solution is to use the combination of experts (MoE) architecture. In this architecture, only part of the network is activated to process each token. In this way, active specialists can be kept in RAM and other parts of the model can be placed on device storage to be loaded when needed.

The next solution is to use smaller and distilled models. 20-30 billion parameter models trained by distillation method can achieve a quality close to 100 billion parameter models on smartphones. This approach may be more commercially viable because it requires less memory.

I am a linguistics student and as much as I am interested in "words", I also enjoy exploring the world of technology and science.