Nowadays, we know that the use of microservices is widespread across various projects and companies. Among the many challenges this architectural model brings, one significant issue is managing data consistency in operations involving more than one service and potentially multiple databases.
Within this scenario of distributed systems and the need to handle transactional context, there's a pattern that fits perfectly: the SAGA Pattern.
Although many have heard of it, not everyone may fully grasp it. However, once understood, attempting to implement it can bring forth questions and challenges.
There's plenty of material available online about this pattern, but my aim here is to contribute in some way to your understanding.
Origin and concept Well, the original idea of SAGA actually emerged in 1987 through a paper by two computer scientists from Princeton University. Initially, the aim was to support transactions or long-lived operations, which they termed LLTs (Long Lived Transactions).
More recently, sagas have been applied to help solve consistency problems in modern distributed systems, as documented in a 2015 article on distributed sagas.
Since then, SAGA has evolved and has become well-suited for situations where you need to handle operations involving multiple microservices that require data consistency.
In a monolithic model, you perform a transaction in the database and save everything. In the microservices model, however, this transaction is "virtualized" because the operation itself consists of transactions that occur in each participating service.
The idea is simple: each microservice executes an operation altering its database. By the end of the process, all databases will be updated and consistent.
If a failure occurs during the execution of the process in any service, this process triggers compensatory actions to be executed in the services that operated beforehand.
It's worth noting here that for SAGA, conditional or partial failures don't matter.
Let me explain. If in an e-commerce system a failure occurs in the payment service due to the buyer's card being invalid or something similar, the system could still attempt to complete the transaction using other cards the customer may have registered on the platform or other available payment methods (here, I'm not discussing whether this functionality makes sense or not).
Only after all system attempts to complete the transaction are exhausted should it "notify" others to initiate compensatory actions. The SAGA pattern focuses on definitive success or definitive failure.
SAGA can also be seen as another name for eventual consistency. But in a way, it exposes the problem, in that we need to think about how to handle operations in your system if they don't succeed, which is where compensatory actions come into play.
So, if I were to summarize the pattern in an informal way in one sentence, I'd say it's basically a stack concept but with an "Undo (CTRL+Z)" functionality.
And obviously, whenever we talk about SAGA, we need to mention the two strategies for its implementation (Orchestration and Choreography).
Orchestration In the orchestrated model, there's a central component responsible for triggering all services involved in a SAGA. If successful, the agent activates the next service. If it fails, this agent orchestrates compensatory actions. This component is also known as the Saga Execution Coordinator (SEC).
In this model, knowledge about the flow is removed from the microservices. Another component understands the flow and coordinates the operation's execution so that microservices focus solely on performing their operations.
Here, we shouldn't confine ourselves to synchronous communication strategies. It's quite common to use a Message Broker to support the flow, allowing services to be triggered upon receiving a message from a topic they are interested in and then posting back a new event as a result of their execution. This triggered event enables the orchestrator to activate the next process to be executed.
It's also worth noting that the orchestrator agent can be used as a kind of state machine for the various SAGAs that pass through it.
