A counter anecdata. We transitioned from a postgres job queue to Rabbit. We had never ending problems after that, many of them were misunderstandings, some where just wrong-fit. We migrated because we had some time on our hands and thought we would alleviate some high pressure jobs. Not only did it not solve the problem, but having written all the code that decides when to pull the next message and what to do with it, and how to dead-letter it - just worked great for us on Postgres. It was so easy to understand and doing things like reprocessing just using a standard postgres DB interface was much easier.
Ultimate the entire processing got removed from our team and no longer needs to do these deployments (acquisition transitions)...
I tried very hard to use postgres as a queue, it was robust but slow once I started to push from more than a few processes/servers. Moving to zeromq initially and sqs after solved all my perf issues, and was still solid.
I think that’s the way to go. Start with Postgres and only if there are problems, then think about something more specialized. Same for microservices. Start simple and introduce a service when really needed.
I hate it when people already start out with 10 or more different systems/services for a few messages per second.
from my experience they're actually much lower than people think, especially when using Postgres for things it's not well-suited for like queuing, and the problem is you rarely can just throw more hardware at it to solve the problem and it's really not a good place to be.
I think that using the right tool for the job is important and saves a lot of time in the long run. There are expensive headaches that we have to resolve.
Ultimate the entire processing got removed from our team and no longer needs to do these deployments (acquisition transitions)...