Rosetta Commons community members submit public comment on safety considerations for chemical and biological AI models

Members of the Rosetta Commons community recently submitted a public comment to the U.S. AI Safety Institute, housed within the National Institute of Standards and Technology, in response to a Request for Information (RFI) on Safety Considerations for Chemical and Biological AI models. This initiative reflects Rosetta Commons’ ongoing effort to engage with biosecurity research and contribute to relevant policy development processes.

The RFI sought information about current and future practices for the responsible development and use of chemical and biological (“chem-bio”) AI models, defined as “AI models that can aid in the analysis, prediction, or generation of novel chemical or biological sequences, structures, or functions.”

The public comment draws on perspectives shared by scientists during a biosecurity-oriented roundtable discussion at Summer RosettaCon in Washington, USA (August 2024), a workshop conducted at European RosettaCon in Copenhagen, Denmark (November 2024), and a survey administered in person at European RosettaCon and then virtually with the broader Rosetta Commons community.

Key messages of the public comment include include:

• We expect advances in protein structure prediction and design research to enable tangible improvements in public and environmental health—from curing diseases to addressing climate change. Computational biomolecular prediction and design tools will a critical role in enhancing public health security, for example, by improving our ability to prevent, detect, and respond to pandemics. However, as with any biotechnology, these advances are accompanied by the risk that they could be used deliberately or accidentally in a manner that threatens public health or the environment. A fundamental challenge in developing sound science policy in this domain, in particular, is the current lack of empirical evidence regarding the actual risks posed by in silico research. While there is broad agreement that policies should be evidence-based, we currently lack robust mechanisms to gather and evaluate data that would help in understanding where risks may lie.
• Strengthening biosecurity measures at the stage of nucleic acid synthesis should be a priority. A computationally designed molecule can only cause harm if it is physically produced. Therefore, while identifying and mitigating the risks of model misuse is important, strengthening biosecurity measures at the synthesis stage is crucial. This is also an area where developers of chem-bio AI models could make significant technical contributions.
• Policies concerning chem-bio AI models should be developed with meaningful involvement of the scientific community, as even non-binding government guidance can have far-reaching effects on research through the incentives and precedents it creates. Working in collaboration with scientists who understand both the technical capabilities and limitations of these tools will help ensure that government actors are better equipped to identify and address legitimate security concerns while mitigating the risk of policies that place unduly burdensome barriers on beneficial research.

The full text of the submission is here and on the regulations.gov docket.

Note: This comment was prepared by individuals associated with Rosetta Commons and supported by written feedback from researchers in this community and the broader biomolecular structure analysis, prediction, and design communities. However, the opinions expressed therein are not official positions of Rosetta Commons or its signatory institutions.

By Samuel Curtis, January 2024