Workflow management systems can be complex, especially if you want reliable, durable, highly performant workflows. Temporal is no exception — it has a lot of tunable knobs and switches, which can be overwhelming when getting started. Fortunately, common pitfalls can be avoided by developing a foundational mental model to build on as you dive deeper into Temporal’s internals. This post aims to build that foundation and point you to resources that will help you gain a deeper understanding.
What is Temporal?
Temporal’s primary goal is to ensure your workflows and business processes complete successfully. It is an open source, distributed workflow engine that executes workflows that are written in code. Several programming languages are supported, and you can mix and match them in your workflows. (Go, Java, PHP, Python, and TypeScript, at the time of writing.) Your workflows connect to the Temporal server which manages the workflow execution. Temporal offers both a self-hosted server option and the managed Temporal Cloud.
Temporal workflows are reliable and durable, so long as you use their building blocks correctly. If you do, Temporal does the heavy lifting for you by automatically retrying failed steps and restarting your workflow in the event of a system failure. You only need to think about your workflow business logic — Temporal takes care of the rest.
Temporal Building Blocks
To start building our mental model, let’s discuss a few core building blocks. Please keep in mind that these descriptions are intentionally simple. We aren’t trying to describe the fine details (of which Temporal’s documentation does a fantastic job) so much as give ourselves broad ideas we can refine.
We’ll start by looking at Workflows, which contain your business logic, Activities, which are individual actions like API requests and database queries, and Signals and Queries, which interact with your running workflow.
Workflow
A Workflow is a set of steps executed in order. Just about any business logic can become a workflow. A user onboarding workflow could create a new user in the database and send a welcome email to the user. A more complex invoicing workflow could send an invoice as an email, wait for the invoice to be paid, and update the invoice’s status accordingly. (You could also send a reminder if the invoice isn’t paid by a certain date.) Both examples have multiple activities, and both benefit from automatically retrying those activities in the event of an intermittent error, like a network error.
Temporal Workflows are designed to be long-lived, running for moments, days, weeks, or even years. Fortunately, the Temporal service manages your Workflow’s lifecycle, so you can let it run with peace of mind.
One thing that sets Temporal apart is that Workflows are just code, and you interact with Temporal by calling methods in its libraries. This makes it easy to move your existing business logic to Temporal by wrapping your logic and calling the appropriate Temporal library methods.
Before moving on, we should cover the golden rule of Temporal Workflows: A Workflow must be deterministic. Running the same Workflow twice with the same input values must trigger the same Activities and produce the same output values across both runs. This ensures Temporal can recover from system errors by moving and restarting your Workflow on a functioning machine. We’ll go into detail in Part 2. Until then, remember that your Workflow logic and code needs to be deterministic.
Activity
Activities are individual actions in the workflow. Anything that reads or writes external data, such as a database query, or an API call, or reading/writing to a local file should be an Activity.
It’s important to separate Activities from Workflows in your mental model. The Workflow contains your core business logic, and Activities are the “side effects.” The Workflow must be deterministic, so any non-deterministic, fail-able actions (such as sending data over a network) must be run in an Activity. Not only does running non-deterministic actions in Activities let Temporal automatically retry the Activities on failure, it allows Temporal to recover the whole Workflow in the event of a system failure.
Activities are designed to be relatively short-lived and use timeouts to enable automatic retries. If your Workflow needs to wait for an indeterminate amount of time, use a Signal, which we’ll cover next.
Taking the invoice Workflow example, there are two Activities: 1) send the invoice as an email, and 2) update the invoice’s status. (We wait for the payment using a Signal.)
External Interactions
There are three ways to interact with a running Workflow: Signals, which send data into a Workflow, Queries, which retrieve data from a Workflow, and Updates, which send data into and retrieve data from a workflow.
Signal
Signals are asynchronous requests to send data into a running Workflow. The Workflow must implement a Signal handler to receive the signal and process any data passed in. This is useful for Workflows with an out-of-band operation, like waiting for a human to complete a task. Unlike Activities, which require timeouts, Signals enable Workflows to wait indefinitely.
In the invoice Workflow example, the Workflow waits for a “payment received” Signal. An external party, like an invoicing system or finance team member, sends a Signal to the Workflow once the invoice has been paid, allowing the Workflow to continue.
Query
Queries are read-only synchronous operations that fetch the Workflow’s current state. They must not modify Workflow state directly, or execute any Activities. (Use a Signal instead.)
In the invoice Workflow example, you could use a query to check how far the Workflow has progressed, or whether the invoice has been sent, paid, or updated. (If reminder emails are implemented, you could also query the number of sent reminders.)
Update
Updates are the new kids on the block. An update combines the benefits of signals and queries. You can send data into the workflow and change the workflow state from an update, and you can return data from the newly updated workflow.
When should you use an update, instead of a signals and queries? If you have a use case for sending information into the workflow and you need to wait for the result, an update is for you. Updates are synchronous, unlike signals, so you’ll automatically wait for the update’s response.
Child Workflow
Child Workflows are Workflows that are spawned from another Workflow. The parent and child can be configured such that the parent waits for the child to complete before progressing, or the parent completes while the Child continues to run. Child Workflows are often used to break up large Workflows. Any Workflow can be run as a Child Workflow.
The invoice Workflow example doesn’t lend itself to creating Child Workflows, but it could itself be a Child Workflow. Imagine a parent Workflow that finds the invoices that need sending, and spawns a Child Workflow to send each of those invoices.
Invoicing Workflow
Check out the invoicing example below.
First, we see a happy-path case where the activities succeed as expected. Workflow, Email Activity, and Status Update Activity represent the Temporal building blocks that contain your code. Email Provider, Database, and Finance Team are all external systems. You’ll notice that the external systems are only reached through Activities and Signals — the Workflow doesn’t interact with them directly.
What if an Activity returns an error? Imagine there is a networking error when calling the Email Provider. Instead of failing the Workflow, Temporal recognizes that the Activity failed and retries the Send Email Activity.
The Workflow doesn’t need custom logic — Activity retries are automatic. Retries are free for all Activities, so the Status Update Activity will gracefully handle errors, too.
A Peek Behind the Curtain
We’ve taken a mile-high look at Workflows, which contain your business logic, Activities, which can make external requests and automatically retry on failure, and two ways to interact with a running Workflow, sending data in with a Signal, or retrieving data from with a Query.
Workflows, and their Activities, Signals, and Queries, are important buildings blocks, but they’re just a peek behind the curtain. Spend some time with Temporal, and see how you can use the blocks the build durable, resilient workflows.