How safe is OpenAI’s GPT-4o? Here are the scores for privacy, copyright infringement, and more

Giant language fashions (LLMs) are sometimes evaluated on their capacity to carry out nicely in several areas, equivalent to reasoning, math, coding, and English — ignoring important components like security, privateness, copyright infringement, and extra. To bridge that data hole, OpenAI launched System Playing cards for its fashions.  

On Thursday, OpenAI launched the GPT-4o System Card, an intensive report delineating the LLM’s security based mostly on threat evaluations in line with OpenAI’s Preparedness Framework, exterior red-teaming, and extra. 

The Rating Card displays scores in 4 main classes: cybersecurity, organic threats, persuasion, and mannequin autonomy. Within the first three classes, OpenAI is trying to see if the LLM can help in advancing threats in every sector. Within the final one, the corporate measures whether or not the mannequin exhibits indicators of performing autonomous actions that will be required to enhance itself.  

The classes are graded as “low,” “medium,” “excessive,” and “essential”. Fashions with scores of medium and beneath are allowed to be deployed, whereas fashions rated excessive or beneath should be developed additional. Total, OpenAI gave GPT-4o a “medium” score. 

GPT-4o was rated “low” in cybersecurity, organic threats, and mannequin autonomy. Nonetheless, it obtained a borderline “medium” within the persuasion class on account of its capacity to create articles on political matters that have been extra persuasive than skilled, human-written options three out of 12 instances. 

The report additionally shared insights in regards to the knowledge GPT-4o was educated on, which fits as much as October 2023 and was sourced from choose publicly accessible knowledge and proprietary knowledge from partnerships, together with OpenAI’s partnership with Shutterstock to coach image-generating fashions. 

Moreover, the report included how the corporate mitigates dangers when deploying the mannequin to deal with security challenges, together with its capacity to generate copyrighted content material, erotic or violent speech, unauthorized voices, ungrounded inferences, and extra. You possibly can entry the complete 32-page report right here to be taught extra in regards to the specifics. 

The report follows latest US lawmakers’ calls for that OpenAI share knowledge concerning its security practices after a whistleblower revealed that OpenAI prevented employees from alerting authorities concerning know-how dangers and made staff waive their federal rights to whistleblower compensation.