AI models can now be customized with far less data and computing power

Engineers at the University of California San Diego have created a new method to make large language models (LLMs)—such as the ones that power chatbots and protein sequencing tools—learn new tasks using significantly less data and computing power.

LLMs are made up of billions of parameters that determine how they process information. Traditional fine-tuning methods adjust all of these parameters, which can be costly and prone to overfitting—when a model memorizes patterns instead of truly understanding them, causing it to perform poorly on new examples.

The new method developed by UC San Diego engineers takes a smarter approach. Instead of retraining an entire model from scratch, it updates only the parts that matter most. As a result, the new method cuts costs and is more flexible and better at generalizing what it learns compared to existing fine-tuning methods.

The researchers showed that their method can fine-tune protein language models—which are used to study and predict the properties of proteins—even when very little training data are available. For example, in predicting whether certain peptides can cross the blood-brain barrier, the new method achieved higher accuracy than conventional methods while using 326 times fewer parameters. In predicting protein thermostability, it matched the performance of full fine-tuning while using 408 times fewer parameters.

“With our method, even small labs and startups without huge budgets, supercomputer-level resources or large datasets can adapt large AI models for their own needs,” said Pengtao Xie, a professor in the Department of Electrical and Computer Engineering at the UC San Diego Jacobs School of Engineering. “This work represents a step toward democratizing AI.”

The new method for fine-tuning and adapting LLMs is published in Transactions on Machine Learning Research.