Task Queues in Python

Celery is the most complete Python task queue: several brokers (RabbitMQ, Redis, Amazon SQS), periodic tasks through Celery beat, retries, rate limits and workflow primitives such as chains and chords. That breadth comes with more configuration and more ways to go wrong. RQ uses Redis only and keeps the model simple: enqueue a function call, run workers, inspect jobs. Dramatiq aims for Celery's reliability with a smaller API, works with RabbitMQ or Redis and retries failed tasks by default.

Huey is a small queue that runs on Redis, SQLite or in memory, with periodic tasks built in, a good fit for small apps. Taskiq is asyncio-native, so tasks can be async functions, and it integrates well with FastAPI and other async frameworks. Whichever you choose, write tasks to be idempotent, because most queues deliver at least once and a task can run again after a worker crash.