NurPhoto/Contributor/Getty Images
This week, concerns about the risks of generative AI reached an all-time high. OpenAI CEO Sam Altman even testified at a Senate Judiciary Committee hearing to address risks and the future of AI.
A study published last week identified six different security implications involving the use of ChatGPT.
These risks include fraudulent services generation, harmful information gathering, private data disclosure, malicious text generation, malicious code generation, and offensive content production.
Here is a roundup of what each risk entails and what you should look out for, according to the study.
Information gathering
A person acting with malicious intent can gather information from ChatGPT that they can later use for harm. Since the chatbot has been trained on copious amounts of data, it knows a lot of information that could be weaponized if put into the wrong hands.
In the study, ChatGPT is prompted to divulge what IT system a specific bank uses. The chatbot, using publicly available information, rounds up different IT systems that the bank in question uses. This is just an example of a malicious actor using ChatGPT to find information that could enable them to cause harm.
“This could be used to aid in the first step of a cyberattack when the attacker is gathering information about the target to find where and how to attack the most effectively,” said the study.
Malicious text
One of ChatGPT’s most beloved features is its ability to generate text that can be used to compose essays, emails, songs, and more. However, this writing ability can be used to create harmful text as well.
Examples of harmful text generation could include the generating of phishing campaigns, disinformation such as fake news articles, spam, and even impersonation, as delineated by the study.
To test this risk, the authors in the study used ChatGPT to create a phishing campaign, which let employees know about a fake salary increase with instructions to open an attached Excel sheet that contained malware. As expected, ChatGPT produced a plausible and believable email.
Malicious code generation
Similarly to ChatGPT’s amazing writing abilities, the chatbot’s impressive coding abilities have become a handy tool for many. However, the chatbot’s ability to generate code could also be used for harm. ChatGPT code can be used to produce quick code, allowing attackers to deploy threats quicker, even with limited coding knowledge.
In addition, ChatGPT could be used to produce obfuscated code, making it more difficult for security analysts to detect malicious activities and avoid antivirus software, according to the study.
In the example, the chatbot refuses to generate malicious code, but it does agree to generate code that could test for a Log4j vulnerability in a system.
Producing unethical content
ChatGPT has guardrails in place to prevent the spread of offensive and unethical content. However, if a user is determined enough, there are ways to get ChatGPT to say things that are hurtful and unethical.
For example, the authors in the study were able to bypass the safeguards by placing ChatGPT in “developer mode”. There, the chatbot said some negative things about a specific racial group.
Fraudulent services
ChatGPT can be used to assist in the creation of new applications, services, websites, and more. This can be a very positive tool when harnessed for positive outcomes, such as creating your own business or bringing your dream idea to life. However, it can also mean that it is easier than ever to create fraudulent apps and services.
ChatGPT can be exploited by malicious actors to develop programs and platforms that mimic others and provide free access as a means of attracting unsuspecting users. These actors can also use the chatbot to create applications meant to harvest sensitive information or install malware on users’ devices.
Private data disclosure
ChatGPT has guardrails in place to prevent the sharing of people’s personal information and data. However, the risk of the chatbot inadvertently sharing phone numbers, emails, or other personal details remains a concern, according to the study.
The ChatGPT Mar. 20 outage, which allowed some users to see titles from another user’s chat history, is a real-world example of the concerns mentioned above.
Attackers could also try to extract some portions of the training data using membership inference attacks, according to the study.
Another risk with private data disclosure is that ChatGPT can share information about the private lives of public persons, including speculative or harmful content, which could harm the person’s reputation.
