BjoernHey, I'm Bjoern 👋

I provide actionable advice for robust and scalable task processing with Celery.

Custom Celery task states


Celery tasks are guaranteed to have a state. In this article, I explain the built-in task states and their meaning. And I show you how to leverage this concept to enrich tasks with your own custom state.
Read more ⟶

Parallel Monte Carlo with Dask


Distributing a Monte Carlo simulation across a Dask cluster.
Read more ⟶

Concurrency and Parallelism


Concurrency and parallelism are often used as synonyms. They are not the same thing though. Musings on the difference between concurrency and parallelism and why it matters.
Read more ⟶

Celery task exceptions and automatic retries


To handle exceptions or not, that's the question
Read more ⟶

Three Ideas to Customise Celery logging handlers


Making logging work with my Celery worker has driven me mad in the past. Here is a quick guide to customising your Celery logger (while keeping your sanity).
Read more ⟶

Two Ways to make Celery 4 run on Windows


Celery 4 no longer supports Windows as an operating system. Which means you can no longer run a worker on a Windows machine. This article explores two workarounds.
Read more ⟶

Quandl Timeseries With Celery and Falcon


A step-by-step tutorial on how to build a Celery application with Falcon as the front-of-the-house.
Read more ⟶

Filesystem as Celery message broker


Most Celery related articles start with a three tier app stack: A worker, a producer and a Redis or RabbitMQ message broker. But did you know you can use the filesystem as Celery message broker?
Read more ⟶

Distributed Monte Carlo with Celery chords


Celery chords are one of the six Celery workflow primitives. First, a group of tasks is executed in parallel. As soon as this group has successfully finished, a final callback task is triggered. This makes a chord a perfect tool for a Monte Carlo simulation.
Read more ⟶

How to call a Celery task from another app


What do you do when you need to invoke another app's Celery task? In this article I show you how to integrate Celery tasks outside the realm of your codebase.
Read more ⟶