Streaming APIs play a very important role when it comes to integration. The reason why I say so is because back a few decades, we had the pull technology. It consisted of a client server architecture. The client had to pull all the time for the changes from the server. Technically speaking, there was always a service running behind the scene that constantly looked for updates and If there was an update, it would try to pull the change and update the changes to the system.
That worked fine back in the days. Now, in a Salesforce space, if you’re going to use that approach that’s pretty inefficient because Salesforce has a Streaming API which is pretty much based on event driven architecture and is based on a publisher subscriber model.
Explore More: What Is the Salesforce Bulk API and When Should Call Centers Use It Instead of REST API?
What is Streaming API?
Streaming API allows systems to send and receive updates instantly when something changes using push technology. Each of these changes is called an event. It can be lead creation, status change, etc.
It enables streaming of event notification using push technology and provides a subscription mechanism for receiving that event in real time. It maintains a persistent connection that continuously sends updated data until the connection is terminated.
It is mostly used when we need to consume a constant flow of live data, real time data. By using this Streaming API, we can send and receive event notifications within a Salesforce platform, or we can send event notifications from Salesforce to external apps, or from external applications to Salesforce.
Architecture Overview
The publisher–subscriber model consists of three core components:
- event producer/publisher
- event bus
- event consumer/subscriber
Event producer produces an event when change happens and sends a message to the event bus. For example, Salesforce can create an event when a new lead is added.
An event bus is a channel that holds and delivers the message that’s been produced by the producer or the publisher
An event consumer receives a message from the event bus, it takes that information and does whatever action it wants to do.
Suppose a Lead is marked as Do Not Call in Salesforce and is published. The salesforce sends a message about it to the event bus that carries it to the subscriber (CRM). It removes the lead from active campaigns. From that point onward, agents will no longer call that lead.
Use Case: Pushing Contact Changes to External System
Let’s say you wanted to push contact change from Salesforce to another system like an external dialer or server. to a separate component which is like a server and Salesforce is a publisher.
When any change happens to a contact or contact information, Salesforce publishes an event. This event can be triggered using tools like trigger, flow or process builder, change data capture
Once the event is published, it must be pushed out to the external system. To do that, it needs to support a communication protocol called CometD, which is used to receive streaming events from Salesforce. CometD can be implemented using C#, Java, or Node.js.
Once implemented, the external system receives the event, it can then process the data and perform any action it needs for example, updating its database or adjusting a calling list.
In case the external system is temporarily down and misses the event Replay ID becomes useful. Replay ID is a unique identifier assigned to each event. It allows the system to request past events that were already published. So if something was missed, the system can reconnect and retrieve the missing updates.
Replay ID has 2 options. Minus one means it’s only broadcasting the latest message, it will not contain information about the past message. Whereas minus two that means it contains information about the latest and the past. That might be useful when you’re troubleshooting or even from a certification point of view.
First Generation Streaming API
In the first generation of Salesforce Streaming API, there were mainly two options: Push Topic Events and Generic Streaming Events. These methods are older, but they still work today.
Push Topic Events
In push topic events, you control your stream that goes out using a SOQL query. The push topic events are very handy because your fields that will be part of your stream will be governed by your SOQL query. Whatever field you specify only that will be part of it. For example, if you have an outbound call center you specify new leads name and email, updated contacts, status changes (Interested, Not Interested, Do Not Call), and campaign membership updates in your query, only those fields will be pushed when system updates or changes.
It also contains a record operation which tells the receiver whether change is an update, deletion, creation or undeletion. So Push Topic gives you control, but you must define and manage the SOQL query carefully.
Generic Streaming Events
If you wanted to push the event message with an arbitrary string value then Generic Streaming Event is good for you. In it, you just send a custom message that can be a simple notification that a campaign has started. It is used to send system-wide messages to connected applications like pause campaign A, activate evening dialing mode, etc. It gives you flexibility, but it does not automatically track Lead or Contact field changes like Push Topic does.
Second Generation Streaming API
Salesforce later came up with the second generation Streaming API:
- Change Data Capture
- Platform Event
They’re very efficient as compared to the first generation.
Change Data Capture (CDC)
If you’re using Change Data Capture, something changes in Salesforce space that gets pushed. You don’t have to use a SOQL query to specify fields. Actually, the event message contains header fields with information about the change so you don’t really have to use the SOQL query.
If you want to enable Change Data Capture, follow the following steps:
- You go to Setup and search for Change Data Capture,
- Select the object (like Lead, Contact, account or opportunity), and activate it.
After that, Salesforce starts publishing changes for that object.
After that, Salesforce starts publishing changes for that object. For example, a new Lead is created, a contact updates phone number, a lead is marked as Do Not Call, etc. CDC notifies the system immediately without writing complex queries.
Platform Events
Platform Event is a new version of generic streaming. With Platform events, you can customize your message.
In Salesforce space, you can create a Platform Event in the following way:
- Go to Setup and search for Platform Events.
- Create a new Platform Event and add custom fields to it.
- Once you do that, you can use your Apex code or the Process Builder to push the changes through those fields to the platform event.
With platform events, you can push high volume data with 72 hours retention period which means if a system was down, it can retrieve missed events within that time.
Out of four streaming options: push topic, generic, Change Data Capture and Platform Event, you might have to use either one based on your business needs. As an architect or as a senior developer it becomes your responsibility to understand when to use what.
How Salesforce Streaming API Enables Real-Time Sync in Outbound Call Centers
In outbound call centers, salesforce is the main system where all the data is stored about leads, contacts, campaign information, and call dispositions.
On the other hand, dialer systems, AI voice agents, or CTI integration are external systems that make calls, handle voice conversations, or manage telephony functions.
As these two systems are essential part of call centers but are separate. To keep every interaction up to date and customized these two systems must sync with each other and this happens via integrations.
In this case, the salesforce streaming API plays an important role. Due to it, dialers and other external systems do not have to use pull technique to see if a new Lead or Contact has been created. Rather Change Data Capture or Platform Events allow Salesforce to push the update immediately.
Suppose a new lead comes in salesforce, it sends message via event bus to external systems like dialer and it can then queue the lead for an outbound call without polling, delays, or batch jobs
After an agent or AI voice bot completes a call and updates the disposition such as Interested, Not Interested, Do Not Call, or Follow Up Later, the dialer can publish an event back to Salesforce so the corresponding record gets updated automatically.
FAQS about Salesforce Streaming API
1. What is the Salesforce Streaming API and how does it work?
Streaming API allows systems to send and receive updates instantly when something changes using push technology. It consists of event producer/publisher, event bus, and event consumer/subscriber. Event producer produces an event when change happens and sends a message to the event bus. For example, Salesforce can create an event when a new lead is added. An event bus is a channel that holds and delivers the message that’s been produced by the producer or the publisher. An event consumer receives a message from the event bus, it takes that information and does whatever action it wants to do.
2. What is the difference between pull technology and Streaming API in Salesforce?
In pull technology, the client needs to request data from the server periodically and keep asking whether data is available or not. In pull technology, there is a chance that we make excessive calls to a server and maybe the server slows down. Whereas in the salesforce streaming API, we subscribe to a server once, and whenever the data is available at the server side it will automatically send it to the client side.
3. What is Replay ID in the Salesforce Streaming API?
Replay ID is a unique identifier assigned to each event. It allows the system to request past events that were already published. So if something was missed, the system can reconnect and retrieve the missing updates. Replay ID has 2 options. Minus one means it’s only broadcasting the latest message, it will not contain information about the past message. Whereas minus two that means it contains information about the latest and the past.
4. How does Streaming API help in outbound call centers?
In outbound call centers, Due to streaming API, dialers and other external systems do not have to use pull technique to see if a new Lead or Contact has been created. Rather Change Data Capture or Platform Events allow Salesforce to push the update immediately. With streaming API, the exchange of data happens in real time between salesforce and external systems via event bus.
