Indirect prompt injection (IPI) is an evolving threat vector targeting users of complex AI applications with multiple data sources, such as Workspace with Gemini. This t…
Posted by Adam Gavish, Google GenAI Security TeamIndirect prompt injection (IPI) is an evolving threat vector targeting users of complex AI applications with multiple data sources, such as Workspace with Gemini. This technique enables the attacker to influence the behavior of an LLM by injecting malicious instructions into the data or tools used by the LLM as it completes the user’s query. This may even be possible without any input directly from the user.IPI is not the kind of technical proble…
We are opening our advanced Client-Side Security tools to all users, featuring a new cascading AI detection system. By combining graph neural networks and LLMs, we've reduced false positives by up to 200x while catching sophisticated zero-day exploits.
Thank you to everyone who participated in the Cybersecurity Framework Profile for Artificial Intelligence (Cyber AI Profile) Workshop in January! The input we received on the Preliminary Draft during this workshop has been invaluable and is informing the development of the next draft of the NIST Cyber AI Profile. We are working toward publishing a full workshop summary soon that captures themes and highlights from the event. In the interim, we would like to share a preview of what we heard… Bac…
Cloudflare AI Security for Apps is now generally available, providing a security layer to discover and protect AI-powered applications, regardless of the model or hosting provider. We are also making AI discovery free for all plans, to help teams find and secure shadow AI deployments.
Cloudflare AI Security for Apps is now generally available, providing a security layer to discover and protect AI-powered applications, regardless of the model or hosting provider. We are also making AI discovery free for all plans, to help teams find and secure shadow AI deployments.
Posted by Nathan Parker, Chrome security team Chrome has been advancing the web’s security for well over 15 years, and we’re committed to meeting new challenges and opportunities with AI. Billions of people trust Chrome to keep them safe by default, and this is a responsibility we take seriously. Following the recent launch of Gemini in Chrome and the preview of agentic capabilities, we want to share our approach and some new innovations to improve the safety of agentic browsing. The primary ne…
Posted by Lyubov Farafonova, Product Manager, Phone by Google; Alberto Pastor Nieto, Sr. Product Manager Google Messages and RCS Spam and Abuse; Vijay Pareek, Manager, Android Messaging Trust and Safety As Cybersecurity Awareness Month wraps up, we’re focusing on one of today's most pervasive digital threats: mobile scams. In the last 12 months, fraudsters have used advanced AI tools to create more convincing schemes, resulting in over $400 billion in stolen funds globally.¹ For years, Android …
Posted by Elie Bursztein and Marianna Tishchenko, Google Privacy, Safety and Security TeamEmpowering cyber defenders with AI is critical to tilting the cybersecurity balance back in their favor as they battle cybercriminals and keep users safe. To help accelerate adoption of AI for cybersecurity workflows, we partnered with Airbus at DEF CON 33 to host the GenSec Capture the Flag (CTF), dedicated to human-AI collaboration in cybersecurity. Our goal was to create a fun, interactive environment, …
Posted by Adam Gavish, Google GenAI Security TeamWith the rapid adoption of generative AI, a new wave of threats is emerging across the industry with the aim of manipulating the AI systems themselves. One such emerging attack vector is indirect prompt injections. Unlike direct prompt injections, where an attacker directly inputs malicious commands into a prompt, indirect prompt injections involve hidden malicious instructions within external data sources. These may include emails, documents, or…
The NICE Workforce Framework for Cybersecurity ( NICE Framework) was revised in November 2020 as NIST Special Publication 800-181 rev.1 to enable more effective and rapid updates to the NICE Framework Components, including how the advent of emerging technologies would impact cybersecurity work. NICE has been actively engaging in conversations with: federal departments and agencies; industry; education, training, and certification providers; and international representatives to understand how Ar…
The rapid proliferation of Artificial Intelligence (AI) promises significant value for industry, consumers, and broader society, but as with many technologies, new risks from these advancements in AI must be managed to realize it’s full potential. The NIST AI Risk Management Framework (AI RMF) was developed to manage the benefits and risks to individuals, organizations, and society associated with AI and covers a wide range of risk ranging from safety to lack of transparency and accountability.…