Links and materials for Living off Microsoft Copilot

This is a post with all of the links and additional materials for a talk I gave at BlackHat USA 2024 titled Living off Microsoft Copilot.

Table of Contents

• Demos:

• LOLCopilot

• 15 Ways to Break Your Copilot

• Johann Rehberger @wunderwuzzi23

• Pliny the Prompter @elder_plinius and the BASI com …

• Mark Russinovich @markrussinovich

Slides and demos

Here they are (split into two parts due to size): part 1, part 2.

All of the demos are up on YouTube.

Demos:

• RCE - getting Copilot to search for, analyze and exfiltrate sensitive data via Bing search results, by Tamir Ishay Sharbat - video

• RCE - getting Copilot to manipulate banking information while keeping original file references for trustworthiness, by Tamir Ishay Sharbat - video

• RCE - getting Copilot to lure its users to our malicious phishing website, by Gal Malka - video

• Post-compromise - abusing Copilot with powerpwn to automate spear phishing for all of your victim’s collaborators, by Lana Salameh - video

• Post-compromise - manually using copilot to craft a malicious spear phishing email, by Lana Salameh - video

• Post-compromise - abusing Copilot to bypass DLP and MIP, accessing sensitive content without leaving a trace, by Tamir Ishay Sharbat - video

Tools and research

LOLCopilot

Is an offensive security tool that allows you to abuse Copilot to live of the land of O365. It allows you to use Copilot via an API, automates data gathering (whoami++) and crafts spear phishing emails in the compromised user’s style to all of their collaborators.

The whoami module takes whoami to a whole new level - find top collaborators, documents, password resent emails.

The Spear Phishing module finds all of your collaborators, for each it find the latest interaction you’ve had with them and crafts the perfect response to get them to believe your phish. Copilot will write a message in your style learning from your inbox.

15 Ways to Break Your Copilot

Plugins are a way for AI to actually DO things on your behalf, which makes them extra dangerous. Check out my other BH talk this year for more info on Copilot Studio, the platform behind Copilot M365 plugins.

Other people’s work

Johann Rehberger @wunderwuzzi23

Johann is the best when it comes to AI app hacking. He found the first AI RCE AFAIK. I really recommend you check out his blog.

Pliny the Prompter @elder_plinius and the BASI community

Pliny has broken any LLM out there. If you're into jailbreaking check them out.

Mark Russinovich @markrussinovich

Mark has published really fundamental AI security work. I especially recommend checking out his threat model for AI apps and his work on jailbreaking (Crescendo, Skeleton Key).

Using Teams for trustworthy phishing

Attackers can use Teams to send a message across tenants, luring their victims to provide sensitive info or execute malware. Check out TeamPhisher.