Researchers at UC Santa Cruz achieve a breakthrough with an open source LLM that only uses a mere 13 watts of power when running on custom hardware.
Researchers at the University of California, Santa Cruz have made a breakthrough by creating a large language model (LLM) running on custom hardware that only sips a mere 13 watts, which is the equivalent of a modern LED light bulb. The researchers say this new LLM is 50 times more efficient that other LLMs running on traditional hardware, and is capable of competing with established models such as Meta’s Llama.
Modern neural networks make use of matrix multiplication, which is a technique where words are represented by numbers and stored in matrices, which are then multiplied with one another to create language.
