Nucleus AI plans to leverage its AI research to build an OS aimed at hyper-optimizing supply and demand for agriculture.
California-based Nucleus AI, a four-member startup with talent from Amazon and Samsung Research, today emerged from stealth with the launch of its first product: a 22-billion-parameter large language model (LLM).
Available under an open-source MIT license and commercial license, the general-purpose model sits between 13B and 34B segments and can be fine-tuned for different generation tasks and products. Nucleus says it outperforms models of comparable size and will eventually help the company build towards its goal of using AI for transforming agriculture.
“We’re starting with our 22-billion model, which is a transformer model. Then, in about two weeks’ time, we’ll be releasing our state-of-the-art RetNet models, which would give significant benefits in terms of costs and inference speeds,” Gnandeep Moturi, the CEO of the company, told VentureBeat.The new Nucleus AI model
Nucleus started training the 22B model about three and a half months ago after receiving compute resources from an early investor.
The company tapped existing research and the open-source community to pre-train the LLM on a context length of 2,048 tokens and eventually trained it on a trillion tokens of data, covering large-scale deduplicated and cleaned information scraped from the web, Wikipedia, Stack Exchange, arXiv and code.
