A workflow engine is software that runs a multi-step process, remembers exactly where it got to, and can resume from that point after a crash. The scheduling and the visual builder are the parts people notice. The durable state is the part that makes it a different category of tool from a cron script.
Most definitions of a workflow engine describe the business use case: approvals, routing, notifications, tasks moving from one person to the next. That is accurate and it explains nothing about why you would need one. The engineering answer is narrower and more useful. A workflow engine exists because long-running processes fail halfway through, and something has to know which half already happened.
Table of contents
- The one-sentence definition, and the word that carries it
- What you get beyond persistence
- The two families, and why the distinction matters when choosing
- When a cron script is genuinely the right answer
- The costs nobody puts in the comparison table
- A practical way to choose
- How this fits the rest of the stack
- FAQ
The one-sentence definition, and the word that carries it
A workflow engine executes a defined sequence of steps, persists the state of every step, and decides what runs next based on that state.
The load-bearing word is persists. A shell script executing five steps in order also has state, but it lives in the process memory. Kill the process at step three and the state is gone. Rerun it and steps one and two happen again, which may mean two invoices, two emails, or two charges.
A workflow engine writes step three’s completion to a database before moving to step four. Restart it and it picks up at four. That single property is the reason the category exists, and it is why every serious engine has a database behind it.
What you get beyond persistence
Durable state is the foundation. Five things are built on top of it, and each one is something you would otherwise write yourself, badly, in the middle of a project that was about something else.
- Retries with backoff, per step. The step that calls a flaky API retries six times over an hour. The step that charges a card does not retry at all. That is a per-step policy, not a global one.
- Idempotency boundaries. The engine knows a step completed, so a replay does not repeat it. This is what stops a retry from becoming a duplicate.
- Waiting without burning a process. A workflow that waits three days for an approval should not be a process sitting in memory for three days. The engine parks the state and wakes it on the event.
- Observability per execution. Which run, which step, what input, what error, how long. Not a log line in a stream of thousands.
- Versioning. Workflows in flight keep running the definition they started with, while new runs use the new one. Changing a script mid-flight has no equivalent concept.
You can build all five. Teams do, usually starting with a retry loop and a status column, and eighteen months later they have an undocumented workflow engine with one maintainer.
The two families, and why the distinction matters when choosing
The tools that call themselves workflow engines split into two groups that solve different problems, and comparing across the line produces bad decisions.
Visual and integration-first. The workflow is a graph you assemble in a browser, wired to third-party services. The unit of work is a connector call. n8n and similar tools live here. The strength is that the person defining the workflow does not have to be the person who deploys it.
Code-first and durable-execution. The workflow is a function in your language, and the engine makes its execution durable. Temporal, Restate, Inngest, and Laravel Workflow live here. The strength is that the workflow is versioned in your repository and tested like the rest of your code.
There is a third group, the BPMN engines like Camunda and Zeebe, aimed at modelled business processes with human approval steps. And a fourth, the data pipeline schedulers like Airflow and Argo Workflows, which are optimised for DAGs of batch jobs rather than long-lived business state.
Pick the family before you compare products. A team evaluating Temporal against n8n is comparing a durable execution runtime against an integration platform, which is not a comparison so much as a category error.
When a cron script is genuinely the right answer
This is the section most articles about workflow engines skip, because the article is usually published by a company selling one.
A cron script is enough when every one of these is true:
- The job is idempotent. Running it twice does the same thing as running it once.
- The whole job completes in a single execution. There is no waiting on a human or an external event.
- Failure is acceptable until the next scheduled run.
- There are fewer than about five steps and the branching fits in your head.
- Nobody outside the engineering team needs to see or change what it does.
That describes a large share of real automation: nightly exports, cache warming, report generation, cleanup jobs. Putting an orchestration engine in front of those adds a dependency and a failure mode without adding a capability.
The moment one of those five stops being true, the script starts growing a status column, a retry counter, and a comment that says do not run this twice. That is the signal.
The costs nobody puts in the comparison table
A workflow engine is infrastructure, and infrastructure has a running cost that is not the licence fee.
It needs a database, because that is where the durable state lives, and that database is now production-critical. It needs somewhere to run, usually a persistent service rather than a function. It needs backups that cover both the workflow definitions and the credentials the steps use. And it needs an upgrade path, because the engine will release versions and some of them will change node or SDK behaviour.
There is also a debugging cost that is real and rarely mentioned. When a workflow misbehaves, you are debugging your logic, the engine’s execution semantics, and the third-party service at the same time. Good per-execution observability is what makes this tractable, which is why it belongs on the evaluation checklist above the connector count.
A practical way to choose
Three questions, in order, and they eliminate most of the field quickly.
- Does the process wait on something outside itself, for longer than a request timeout? If yes, you need durable execution. If no, a script or a job queue probably covers it.
- Who edits the workflow? If it is the engineering team, a code-first engine keeps it in the repository with tests and review. If it is operations or marketing, a visual builder is worth the trade.
- What happens on a duplicate run? If the answer is anything worse than wasted compute, the idempotency and step-completion guarantees are the feature you are buying, and everything else is packaging.
For the visual, integration-first case, RunxBuild runs n8n as a managed tool: a create form, a plan, a custom domain, autoscaling, and runtime logs, with a managed Postgres alongside it for the durable state the engine needs. For the code-first case, the engine is a service you deploy from a repository like any other, with the database on the same platform. Either way, the state has to live somewhere that survives a restart, and that is the decision to get right first.
How this fits the rest of the stack
A workflow engine is a database with opinions about execution order. That framing makes the cost obvious: you are adding a persistent service and a production database, not a library. Model both before committing. The RunxBuild hosting calculator shows the runtime and the managed database as separate line items, which is the shape the bill actually takes.
Useful related references:
- n8n Loop Over Items: When You Need It and When You Are Fighting the Engine
- n8n Text Classifier Node: Routing Without Writing a Rules Engine
- Sales Workflow Automation: The Steps Worth Automating
- Services on RunxBuild
FAQ
What is a workflow engine in simple terms?
Software that runs a sequence of steps, saves the outcome of each step to a database, and resumes from the last completed step after a failure. The saving is what separates it from a script that runs the same steps.
What is the difference between a workflow engine and a job scheduler?
A scheduler decides when something starts. An engine decides what runs next based on the recorded state of what already ran. A scheduler that restarts a failed job repeats it from the beginning; an engine resumes it.
Do I need a workflow engine for a simple automation?
Usually not. If the job is idempotent, finishes in one execution, and can wait until the next scheduled run after a failure, cron plus a script is the lower-maintenance answer.
Are BPMN engines and durable execution engines the same thing?
No. BPMN engines like Camunda model business processes with human tasks and approvals in a standard notation. Durable execution engines like Temporal make ordinary code resumable. They overlap in the middle and are chosen for different reasons.
Does a workflow engine need its own database?
Yes, in practice. The durable state has to persist somewhere outside the process. Some engines embed one for development, but any production deployment points at a real database and treats it as critical infrastructure.