AI has dreamt up a blizzard of new proteins. Do any of them actually work? But researchers hope that the real prize will be a revolution for the field
Emerging protein-design competitions aim to sift out the functional from the fantastical
On a Saturday morning in mid-August, Alex Naka embarked on what he describes as “a little hackathon” in his girlfriend’s kitchen. Powered by his laptop, some coffee and, at one point, about 80 cloud-based artificial intelligence (AI) processors, he generated scores of computer-engineered proteins designed to block a cell receptor that is mutated in some tumours.
Naka — who on weekdays is a protein engineer at a medical technology company in Alameda, California — entered his ten most promising creations into a newly launched protein-design competition and watched them climb to the top of the leader board.
The contest, run by a biotechnology start-up firm called Adaptyv Bio in Lausanne, Switzerland, is one of at least five to have popped up over the past year or so. Most of the people entering the competitions are wielding AI tools such as AlphaFold and chatbot-inspired ‘protein language models’ that have exploded both in popularity and in power. Three of the researchers behind some of these tools were awarded this year’s Nobel Prize in Chemistry for their efforts. The accolades come, in part, from the hope that newly created proteins could serve as more-effective drugs, industrial enzymes or laboratory reagents.
But the boom in designer proteins has mostly sown confusion, say scientists. People are churning them out faster than they can be made and tested in labs, making it hard to tell which approaches are truly effective.
Contests have driven key scientific advances in the past, particularly for the field of protein-structure prediction. This latest crop of competitions is drawing people from around the world into the related field of protein design by lowering the barrier to entry. It could also quicken the pace of validation and standards development and perhaps help to foster community. “It will push the field forward and test methods more quickly,” says Noelia Ferruz Capapey, a computational biologist at the Centre for Genomic Regulation in Barcelona, Spain.
But the competitions will have to overcome some hurdles, say scientists, such as identifying which problems to tackle and working out how to select winners objectively. Getting the formula right is important. “These competitions can do damage” to a field if they are not executed properly, says Burkhard Rost, a computational biologist at the Technical University of Munich in Germany.
Competitive by design
The protein-design contests are partly inspired by a 30-year-old competition that helped to kick off the revolution in biological AI. Since 1994, the Critical Assessment of Structure Prediction (CASP) has been challenging scientists to predict the 3D shape of proteins from their amino-acid sequences. Winners of the competition — founded by computational biologist John Moult at the University of Maryland in Rockville and Krzysztof Fidelis, a computational biologist at the University of California, Davis — are determined by comparing the computational predictions with unpublished structural models.
In 2018, London-based DeepMind (now Google DeepMind) won CASP with its first version of the protein-structure-prediction tool AlphaFold. Its next iteration, AlphaFold 2, performed so well in 2020 that Moult declared the problem of predicting simple protein structures largely solved. The competition has since shifted its focus to other emerging challenges, such as predicting the structure of multiple interacting proteins in a complex.
Now, many hope that these competitions can push the protein-design field forwards just as CASP helped to spur a revolution in protein-structure prediction. “There would have not been an AlphaFold had it not been for CASP,” says Rost. “We need these competitions to do the job right and motivate people.”
In June, Rost and several of his colleagues won the Protein Engineering Tournament run by Align to Innovate, an international, open-science non-profit organization. The event included two parts. First, participants tried to predict the properties of different enzyme variants. The best-performing teams in this round then re-engineered an enzyme that breaks down starch, with the best designs determined by lab experiments. A 2025 tournament is now in the works.
A Winter Protein Design Games contest that announced its winner in April was run by Liberum Bio, a biotechnology company in Kitchener, Canada, and Rosetta Commons, a collaboration of mostly academic scientists that maintains protein modelling tools. The contest tasked entrants with re-engineering an existing protein — a plant-virus enzyme used widely in protein purification — to make the molecule more efficient.
Two other contests asked participants to come up with entirely new proteins. Adaptyv’s was looking for proteins capable of attaching to a growth hormone receptor called EGFR that is overactive in many cancers. The 90 entrants submitted more than 700 designs.
And in Bits to Binders, researchers are vying to create small proteins that could be used in a T-cell cancer therapy. Run by the BioML Society, a graduate-student-led group at the University of Texas at Austin, it attracted 64 teams from 42 countries — including Nigeria, Colombia, Iran and India. Around 18,000 designs are now being tested, with results due in early 2025. “We were quite surprised with the turnout,” says co-organizer Clayton Kosonocky, a biochemistry PhD student at the university.
Newcomers welcome
Julian Englert, chief executive and co-founder of Adaptyv, says that many of the participants in its contest work in protein engineering and design. However, the competition has also received promising entries from people with no professional experience in biology. An entrant from Iran made his predictions using a gaming computer because he didn’t have access to more-powerful systems.
Englert says that the high-quality entries from people who aren’t established researchers reminds him of the garage-tinkering origins of Apple, Microsoft and other tech giants. “It would have taken them two years of studying and joining a lab to get to the point where they can get started. Here they can do it over a weekend.” He imagines a future in which freelance protein designers vie for bounties set by companies, academic labs and others seeking a custom molecule.
The competitions can save time in other ways, too. Getting quick experimental results from contest organizers was a big motivation for Michael Heinzinger, a machine-learning scientist at the Technical University of Munich who was part of the winning team with Rost. “Otherwise we would have had to put in the time to write a grant,” he says. “For me, the prize is saving time.”
In terms of actual prizes, the Align to Innovate tournament didn’t offer one, but some others do. The winners of Bits to Binders will get a 3D-printed trophy of their design and some merchandise from the biotech company that is conducting the experiments, called LEAH Laboratories in Eagan, Minnesota. There will also be opportunities for collaboration.
Adaptyv, which sells automated lab testing of molecules created by protein designers, offered a handful of free experiments and some of its own swag. And the winners of the Rosetta Winter Games split US$5,000.
But the standout is the recently launched Evolved 2024 contest, in which the first-place team will take home a $25,000 credit for Amazon Web Services, along with credits from other companies (including OpenAI) worth thousands of dollars. Its sponsors include Lux Capital, a venture-capital firm in New York City that has invested more than $1.5 billion in tech firms, and EvolutionaryScale, a biology AI start-up also in New York City, that has attracted $142 million in investment.
Beyond big cash prizes
Choosing who will reap these rewards isn’t always straightforward. The Evolved 2024 contest — more of a hackathon, in which teams work on broad-brush problems such as predicting drug efficacy and safety — will be judged subjectively by a panel of experts. But even for the contests with more clear-cut protein-design goals, “it’s not trivial to figure out who wins,” says Erika DeBenedictis, a biological engineer and founder of Align to Innovate. Her organization’s tournament gauged designs on the basis of their activity, stability and how well (or even whether) they could be made. “When you design a protein, there are a lot of ways it can fail,” she says.
And if competitions are to move the needle in protein design, they will need to address the challenges that the wider field is tackling, say scientists. Unlike structure prediction, protein design can have wildly different criteria from task to task. The approaches to craft a particular type of enzyme might not translate to other proteins, such as vaccine components.
The competitions could prove counterproductive if they send the field down a rabbit hole — for example, by judging designs too narrowly, warns Rost. Researchers could also fail to reap the full benefits of protein-design competitions if contestants keep quiet about their approaches, says Anthony Gitter, a computational biologist at the University of Wisconsin–Madison. “If teams aren’t communicating their methods, there’s not much opportunity to learn about what work and what fails.”
So far, this doesn’t seem to be happening. Most of the competitions encourage or even require participants to describe their approaches. The contests could also help to bring together some of the disparate fields that have been drawn to protein design — from biochemistry labs that pioneered protein engineering methods to machine-learning scientists who cut their teeth in natural-language processing, says Gitter. “People organizing the competitions, to have maximum impact on the field, should think really hard about how to create a community.”
When the Adaptyv competition results were announced in late September, Naka was disappointed. Although all ten of his entries looked strong, none of his designs worked in the lab. Only 5 of the 147 designs that were tested actually bound to the target molecule. And more than 50 of them couldn’t even be made.
That actually isn’t bad: past efforts to design EGFR binders have had much lower success rates. “It’s par for the course in protein engineering — you have to be ready to fail a lot,” Naka says. The winners were Martin Pacesa and Lennart Nickel, structural biologists at the Swiss Federal Institute of Technology (EPFL) in Lausanne, who posted a preprint describing their approach and made its code open-source (M. Pacesa et al. Preprint at bioRxiv https://doi.org/nmfm; 2024). Adaptyv has now launched a second competition that builds on the first.
Naka wishes he had started working on his entries earlier. He describes his hackathon as “type 2 fun” — painful at the time, but enjoyable in retrospect. Through the competition, he forged connections with like-minded scientists, including Gitter. “It feels like it lowered the barrier to entry and let a lot of new people participate in protein design,” he says. “I’ll definitely be participating in similar events in the future.”
Nature 634, 532-533 (2024)
doi: https://doi.org/10.1038/d41586-024-03335-z
This story originally appeared on: Nature - Author:Ewen Callaway
