Prompt Injection in JetBrains Rider AI Assistant

WithSecure discovered a prompt injection vulnerability in JetBrains Rider AI Assistant which could allow an attacker to exfiltrate data. This is similar to an issue found in GitHub Copilot Chat, a VS Code Extension, by Johann Rehberger. Prompt injection attacks aim to alter the LLM's intended behavior, leading to unexpected or malicious outputs. The consequences vary depending on the application. In this case, an injection attack might cause the LLM to generate output that, when rendered in the AI Assistant Chat, enables data exfiltration.

Details

JetBrains Rider AI Assistant provides AI-powered features for software development, leveraging the JetBrains AI Service. This service connects users to different LLMs transparently within the IDE, offering functionality like code suggestions, explanations, and refactoring based on the context of the user’s project.

The vulnerability arises from indirect prompt injection. If an untrusted code snippet containing a malicious prompt injection payload is passed to the AI Assistant (for example, when asking for an explanation), the injected instructions will be executed by the underlying LLM.  

A proof-of-concept demonstration showing this exploit, where the AI Assistant leaks code to an attacker-controlled server, can be viewed here:

The impact of this vulnerability is real, but in many cases, it may not be as serious as it initially appears. This type of vulnerability relies on the user invoking the AI Assistant on untrusted code snippets they’ve introduced into the IDE—often from sources like Stack Overflow or other public forums. In these cases, an attacker could more easily embed a traditional backdoor directly in the code, allowing them to execute their own code within the application context or on the developer’s workstation, potentially persisting into production. In contrast, prompt injection is limited to manipulating the LLM’s responses within the IDE.

Prompt injection remains a challenging issue to completely prevent, but the following mitigations can reduce exposure:

• Disable Rendering of Hyperlinks and Images: To avoid potential exfiltration through outbound requests, restrict the AI Assistant from rendering any hyperlinks or images.
  
• Domain Allowlisting: If rendering URLs or images is necessary, ensure only trusted domains are allowed in outbound requests to limit external connections.

This aligns with the mitigation Microsoft implemented for GitHub Copilot Chat, where markdown images are no longer rendered.