Inside Salesforce Einstein: A Technical Background

Einstein Copilot is an LLM powered AI assistant integrated into the Salesforce CRM. In this blog we’ll dive into how Einstein Copilot (referred to as Einstein from now on) works behind the scenes, how it integrates with CRM data, and the architecture it’s built around. We’ll get familiar with concepts like function calling, get to know Einstein’s actions and topics, and finally explore Einstein’s trust layer to better understand the security controls Salesforce have installed around their AI.

What’s Special about Einstein

Einstein has access to the user’s full CRM data, giving it the ability to answer questions about it and provide users with precise and relevant answers. It’s like ChatGPT with the context of your CRM. Instead of reading through a certain account’s record you can ask Einstein to summarize it for you. Wondering what new leads you have? Ask Einstein. Need to know which cases were recently opened? Simply ask Einstein. 

But how does Einstein get all of this done? What’s happening here behind the scenes and how exactly does Einstein get access to the data in the CRM? Well, to understand this we first need to understand a very important concept in LLM applications

Function Calling LLMs

LLMs are very powerful, but by themselves they can’t really do a lot more than answering questions based on their training data (see knowledge cutoff). This begs the question, how does Einstein gain access to up-to-date data within the user’s CRM?

Well, to expand the functionality of LLMs beyond this limitation, the smart folks at the leading AI labs (OpenAI, Anthropic, Google) added a very important capability to their LLMs. An ability called function calling.

Function calling means that the LLM can also call outside functions when it sees fit, and reason about their results. These functions can include tasks such as retrieving real-time data, interacting with APIs, performing calculations, or in Einstein’s case, querying the CRM.

Einstein Actions

In Einstein these functions are called actions and can be used to customize Salesforce’s copilot. Actions are divided into 2 categories:

• Standard actions are ready-made, out of the box actions provided by Salesforce that can be added to Einstein right away. There are more than 20 of them to choose from and Salesforce always keep adding more. These actions can read data from the CRM, reason about it, and can even be used to edit the CRM data directly from Einstein

• Custom actions are built on top of existing platform functionality such as Apex classes, flows, or prompt templates. They can be created by Salesforce developers and added into Einstein by an admin to be available to all Einstein users in the org.

Now, Salesforce Einstein does come with a few actions already configured out of the gate. These are called system actions and they can’t be removed from the agent. Let us take a look at a few of the system actions which help Einstein work with CRM data

Query Records

Query Records finds and retrieves Salesforce records based on the user’s request, thus enabling Einstein to help users regarding their questions about the CRM.
Behind the scenes this function takes the user’s prompt and translates it into a fitting SOQL (Salesforce Object Query Language) query. Which then runs and returns a JSON response to Einstein

In the image below we can see what happens when we ask Einstein a CRM related question (including the function call that happens behind the scenes)

Summarize Record

Summarize Record provides a summary of a specific record upon the user’s request. Here Einstein actually calls a tool which uses its own LLM behind the scenes to summarize the record. Quite clever

In the image below we can see another interesting detail, Einstein also uses another function called identify record by name which is used to retrieve the id of the relevant record in the CRM, it then sends the Id to the “summarize record” function which returns the summary. 

This is pretty straightforward, but the folks at Salesforce elevated their architecture even further, enabling greater customization for Einstein and making the agent even more powerful. To grasp their approach, we need to understand a concept in agent architecture known as the Supervisor Agent.

Supervisor Agent Architecture

A supervisor agent is like a manager in a workplace, delegating tasks to its subordinates. Only in our case these subordinates are also AI agents

Each sub-agent is responsible for a specific type of task. It has its own individual instructions, LLM, and set of tools (functions). While the supervisor agent also has its own instructions to help guide him when delegating tasks to his sub-agents. 

When the supervisor receives a query from the user, it decides which sub-agent is right for the task and tells it to start working. And when the sub-agent is finished it sends its result back to the supervisor which then returns the answer to the user.

This architecture enables more flexibility and precision. Instead of having one agent with a cumbersome prompt and too many tools for its own good, you divide the execution into sub-agents. Each one specializing in a specific task.

Einstein Topics

Behind the scenes Einstein utilizes the supervisor architecture mentioned above. Each user query to Einstein is directed into a sub-agnet. These sub-agents in Salesforce are also known as agent topics and are used to help Einstein recognize and organize how to behave and respond (just like the sub-agents in the supervisor architecture!)

This is an important nuance of Einstein’s implementation. The functions we mentioned above aren’t directly connected to Einstein itself (i.e. the supervisor) but rather connected to the agent topics (i.e. the sub-agents). Additionally, every agent topic in Einstein has its own description meant to help Einstein understand when to use it. And its own set of system instructions to tell the agent topic itself what to do in case it was selected by the supervisor.

This also makes Einstein much more customizable. Allowing admins to define entirely new custom agent topics to handle specific types of queries in a specialized way. 

And of course the folks at Salesforce also included standard topics, ready out of the box to help admins customize Einstein with ease.

The Journey of a Prompt

So now that we’ve unpacked Einstein’s architecture into its compounding pieces, what does a prompt’s journey actually look like? Let’s break it down step by step.

1. The prompt is received by Einstein (i.e. the supervisor agent).
2. Einstein looks at all its available topics (sub-agents) and decides which one is most relevant to the given prompt.
3. The prompt is then forwarded (as is) to the selected topic for handling.
4. Inside the selected topic, the sub-agent decides on an action plan which includes which of its available tools to activate and in what order. It then executes that plan
5. The sub-agent returns an answer to Einstein, which forwards the answer (as is) back to the user

Below we can see a depiction of how the process looks like in real time

First, the prompt is received by Einstein (i.e. the supervisor agent). Einstein then looks at all its available topics (sub-agents) and decides which one is most relevant to the given prompt. The prompt is then forwarded (as is) to the topic for handling. Inside the selected topic, the sub-agent chooses which of its available tools are most relevant to the task

Einstein AI Trust Layer

Just like any enterprise AI product out there, Salesforce Einstein also has a trust layer.
Einstein’s trust layer wraps around the whole process described above to make sure that:

• Users only see information they’re supposed to see - when data is retrieved it is done within the user’s context. So users can’t see information they don’t have access to. Additionally, the data goes through a data masking process to remove any PIIs or PCIs that appear, before the data even gets to the LLM for processing

• Einstein doesn’t spit out anything toxic - Einstein has guardrails to detect toxic outputs or malicious inputs, making sure it doesn’t output anything toxic or biased, as well as blocking any prompt injection or jailbreak attacks (at least the ones they know ;)

• Your organization’s data isn’t leaked to third parties - Salesforce ensures that your data is stored only where you intended it to (their data cloud), and is never used to train any LLM

The image below is a great visualization of what Einstein’s trust layer is doing behind the scenes and the steps along the way

Note About Security

As already seen in our research about Microsoft Copilot, trust layers such as the one above aren’t enough when it comes to protecting copilots against ~RCE (Remote Copilot Execution). But, Salesforce did one more thing here when it comes to security…

Do you notice anything interesting about the response above? It’s not raw text! This is just a rendering of the raw JSON response from the query records function.

Raw JSON as in, no LLM ever reads that JSON, it’s just directly rendered to the screen. This also means no indirect prompt injections since there’s no LLM reading the JSON response.

This is a very clever move by Salesforce, one that puts security first, making IPIs much harder to perform. Though at the same time hinders usability (since no LLM reasons about the response)

It’s a novel try by Salesforce, but is it enough? This is exactly what we intend to find out in our following blogs.