What is a worker and why do applications use background tasks?

Published on 2026-10-20 by frfrand

Workers explicados con ejemplos: colas, Celery, correos, PDFs, trabajos demorados, reintentos e idempotencia sin bloquear una web.

A worker is a process that is responsible for executing tasks outside the main flow of an application. In web development, it is often used for jobs that take too long to keep a person waiting in front of a page: generating files, processing images, sending notifications, or importing thousands of records.

This does not mean the work disappears. It simply runs in another process, and often on another machine.

The restaurant example

Imagine a restaurant where the person who takes the order must also prepare each dish before attending to the next customer. It would be a disaster: everyone would end up waiting in line, even if they only wanted to ask about the menu.

Something similar happens in an application when an HTTP request has to wait for a heavy operation to finish. The usual approach is to receive the request, record what needs to be done, and delegate the task.

That is when the worker comes into play.

What is the difference between the web, a queue, and a worker?

The web application handles requests. The queue keeps track of pending jobs. The worker picks up jobs and executes the corresponding logic.

For example, a customer clicks "generate report." The web app can create a task and return an identifier in a few seconds. The worker processes it, and when finished, the user receives a notification or can download the file.

Depending on the chosen technology, the queue and execution can be coordinated with Redis, RabbitMQ, or other systems. Each combination offers different guarantees.

A practical case: email sending

If a new user registers, the application can confirm the registration and send a welcome email as a secondary task.

But be careful: if the email does not arrive, what should happen? Is it retried? How many times? And if the email provider received the message, but the worker lost the confirmation?

These questions matter because a task executed twice can cause duplicate effects. Sending two emails can be annoying; charging a purchase twice would be much worse.

What does idempotence mean?

An operation is idempotent when repeating it produces the same relevant effect as executing it once. For example, setting an order status to "processed" can be designed to tolerate repetitions.

On the contrary, "adding 10 € to the balance" is not idempotent if each execution adds another 10 €. When queues can repeat tasks, this detail ceases to be theory.

A common practice is to use unique identifiers to avoid reapplying the same effect if it has already been performed.

What happens if the worker shuts down?

It depends on how messages and acknowledgments are managed. Some configurations acknowledge a task before executing it; others can return it to the queue after an interruption.

There is no universal guarantee of "exactly once" execution just by installing Celery or Redis. You need to understand retries, acknowledgments, timeouts, and how to record results.

What is Celery?

Celery is a well-known tool in the Python ecosystem for defining and executing distributed tasks via workers and a messaging system. It is frequently used with Django, although it is not limited to that framework.

You can define a task that processes a report and send it to the queue from the backend. Workers execute that function when they have capacity.

To operate correctly, you need supervision, error logging, and proper concurrency configuration.

Do more workers mean more speed?

Not always. If all tasks compete for the same database or consume all available memory, adding workers can make the situation worse.

Some jobs are CPU-bound, others are network-bound, and others depend on external services. Before increasing processes, it is advisable to measure how long each task takes and where the bottleneck is.

It is also important to limit retries: if an external provider fails, it makes no sense to launch thousands of attempts per second and worsen the problem.

When do you not need a worker?

If an operation completes in milliseconds and is part of the immediate response, introducing a queue can add unnecessary complexity.

The goal is not to create a larger architecture, but to keep the application available when long-running or unpredictable tasks appear.

And in ArduMaker?

In ArduMaker, you can manage applications and services that use Redis and workers within VPS. In systems with n8n and other background processes, it is also important to separate the component that receives requests from the one that performs delayed tasks.

In summary: a worker executes tasks that should not block the main response. A queue helps coordinate them, but reliability depends on how you design retries, errors, and duplicate operations.

Sources: Celery Tasks Guide · Introduction to Celery Workers.