AhnLab Security Insight_Attackers vs. Defenders: The Future of AI Security

[jasonX]

 

Attackers vs. Defenders: The Future of AI Security

 

AI technology is advancing faster than ever before. Generative AI is becoming increasingly sophisticated and is now driving innovation across various industries beyond everyday conversational services. Starting with ChatGPT, technologies like Microsoft's VASA-1, HeyGen, Sora AI, and Suno AI have emerged, enabling the creation of videos where people appear to speak and move using just a single photo and audio file. These technologies can also convert given videos into 40 languages and over 300 different voices, and they can even be used to produce movies or compose music. However, with the advancement of these technologies, there is also an increase in cases of AI misuse, presenting new challenges for the security industry. 

This report will introduce the types of threats generated by AI, cases of AI misuse, and AhnLab's strategies for enhancing security using AI.

1. The Potential Risks of Generative AI
Cyber threats stemming from AI manifest in various forms, many of which have already materialized and are causing significant damage. There are four major potential risks arising from generative AI.
 

First, hallucination refers to errors that occur during the process of generating information based on the data the AI has been trained on. AI may sometimes produce information that is either nonexistent or inaccurate, potentially leading users to make poor decisions and raising concerns about reliability.

Bias emerges when the inherent biases in training data cause problems. AI learns patterns and makes predictions based on training data, but if this data contains stereotypes or discriminatory elements related to gender, race, or religion, it increases the likelihood of generating biased outputs.

Copyright issues arise when AI incorporates copyrighted materials without authorization during large-scale data training, potentially infringing on the rights of content creators. The legal status of AI-generated content in relation to copyright remains unresolved and is a critical issue that must be addressed in the era of widespread AI adoption.

Finally, data privacy breaches are another significant concern. If the data used for AI training contains personal or sensitive information, its exposure or misuse can lead to serious consequences. This could result in the unauthorized use of personal information and severe violations of privacy rights.



2. Attacks on Generative AI
Threat actors are abusing the widespread commercialization of generative AI and the various ways it's being adopted by businesses and individuals to attack generative AI in multiple ways. In a broad sense, attacks on generative AI include adversarial prompting, data poisoning, and model reversing.

First, adversarial prompting can be divided into jailbreaking and prompt leaking. Jailbreaking involves disabling or bypassing the ethical constraints of AI by exploiting vulnerabilities through carefully crafted prompts designed to circumvent its built-in rules. As a result, AI can operate in dangerous or illegal ways.

Prompt leaking, on the other hand, refers to the theft of prompts used in generative AI. Prompts are instructions or commands that guide the AI to perform tasks accurately. If system prompts are leaked through code-sharing platforms like GitHub, threat actors could exploit them to distort AI or undermine its reliability. This is particularly concerning for large language models like GPT models, as leaked prompts could significantly undermine data security.

The second attack type, data poisoning, involves deliberately injecting malicious or distorted data into the AI training process to manipulate its outputs. A well-known example is Microsoft's AI chatbot "Tay". Released in 2016, Tay learned and interacted based on user-provided data. However, some users fed Tay inappropriate content, including profanity, racism, and sexism, causing the chatbot to adopt and spew highly offensive and hateful remarks. As a result, Microsoft was forced to shut Tay down just 16 hours after its launch. This incident highlights the critical importance of high-quality training data and how attackers can abuse AI systems.

The third type, model reversing, is a technique where attackers perform repeated queries on an AI model to analyze its functioning and extract its training data. By studying the model's responses, threat actors can infer its internal structure and training data, potentially exposing confidential or sensitive information, including personal data. This attack method exploits security vulnerabilities in AI models, posing significant threats to data protection and model security.



3. Cyber Threats Advanced by the Application of AI
Threat actors are not only targeting AI systems but also rapidly incorporating AI into their attack campaigns. AI-powered attacks include malware creation, phishing attacks, deepfakes, and vulnerability analysis and hacking. These attacks have reached unprecedented levels of sophistication and scale, making them significantly more dangerous than traditional methods.
 

First, AI is being actively utilized to write malware and scripts. Attackers leverage AI to generate large volumes of obfuscated code designed to evade detection, enabling them to bypass existing security systems. Notably, AI significantly enhances the efficiency and speed of creating ransomware or malware aimed at stealing personal information. Recently, there has been an increase in cases where AI is used to search for images containing sensitive information, such as passwords or cryptocurrency wallet addresses, which are then abused for malicious purposes.

In addition, AI allows attackers to execute phishing attacks with greater precision. AI-generated phishing email content appears highly natural, as if written by a person. While traditional phishing emails could often be identified through specific words or awkward sentence structures, AI-powered phishing creates messages that are virtually indistinguishable from legitimate business emails or notifications since they are based on large language models. The advancement of AI has caused a more than 40-fold increase in phishing emails compared to the past, significantly enhancing their realism and impact, which underscores AI's profound influence on cyberattacks.

Deepfake technology allows for the creation of realistic fake videos using just a few images and voice samples. The quality of deepfakes continues to improve, and they are increasingly being abused for crimes such as spreading fake news, causing social unrest, producing fraudulent video content, and conducting voice phishing. For example, attackers can now fabricate videos of celebrities making false statements or use a person's voice to commit financial fraud, creating significant risks of social and financial disruption.

 
 

4. AhnLab's Security Enhancement Strategy Using AI
Given the sophistication and complexity of AI-driven attacks, traditional security approaches are insufficient for effective defense. Consequently, AI holds immense importance not only in countering AI-enabled attacks but also in addressing emerging threats.
 

AhnLab is actively applying AI technology to enhance its solutions and platforms. The company's AI capabilities are rooted in filing technology, which classifies malicious and legitimate files. Additionally, AhnLab trains its learning models using a wide range of data, including behavior-based analysis, URL information, and event logs. This technology is integrated into all major products, such as V3, V3 Mobile, EDR, MDS, TIP, and XDR, significantly improving overall threat detection, analysis, and response capabilities. Furthermore, AhnLab is advancing its security technology in multiple areas, including malware and phishing email detection and response, prediction of potential threats, and comprehensive risk management.

Attackers vs. Defenders: The Future of AI Security

Related Article: Cybersecurity in the AI Era: Evolve Faster Than the Threats or Get Left Behind


Data and info derived from AhnLab with permission