Running summarizing tools on a new article

Check the paper below, then start its synthesis.


Infinite-parameter Large Language Model

Fei Ding

DOI: 10.33774/coe-2024-7khtw (Computer Science)
License: not stated (all rights reserved)

Abstract: In the standard transformer architecture, in creasing model parameters leads to linear growth in computational cost and activation memory. To address this issue, we propose a novel Infinite Parameter Large Language Model (IP-LLM) architecture that decouples model size from computational cost and de vice memory. Existing large language models Figure 1: Parameters A, B, C, and D store knowledge are all fixed-parameter models, while human knowledge is infinite and expands daily. Finite parameters are inherently limited in their capac ity to accommodate this boundless knowledge. Our IP-LLM architecture can potentially ac commodate infinite knowledge, resolving this issue and laying the foundation for realizing a truly omniscient and omnipotent artificial gen eral intelligence in the future.Our architecture surpasses MOE in performance while requiring significantly less memory.

Published on 29 Jul. 2024