mirror of
https://github.com/jpetazzo/container.training.git
synced 2026-07-21 22:07:13 +00:00
211 lines
4.0 KiB
Markdown
211 lines
4.0 KiB
Markdown
# Message Queue Architecture
|
||
|
||
There are (at least) three ways to distribute load:
|
||
|
||
- load balancers
|
||
|
||
- batch jobs
|
||
|
||
- message queues
|
||
|
||
Let's do a quick review of their pros/cons!
|
||
|
||
---
|
||
|
||
## 1️⃣ Load balancers
|
||
|
||
<pre class="mermaid">
|
||
flowchart TD
|
||
Client["Client"] ---> LB["Load balancer"]
|
||
LB ---> B1["Backend"] & B2["Backend"] & B3["Backend"]
|
||
</pre>
|
||
|
||
---
|
||
|
||
## Load balancers
|
||
|
||
- Latency: ~milliseconds (network latency)
|
||
|
||
- Overhead: very low (one extra network hop, one log message?)
|
||
|
||
- Great for short requests (a few milliseconds to a minute)
|
||
|
||
- Supported out of the box by the Kubernetes Service Proxy
|
||
|
||
(by default, this is `kube-proxy`)
|
||
|
||
- Suboptimal resource utilization due to imperfect balancing
|
||
|
||
(especially when there are multiple load balancers)
|
||
|
||
---
|
||
|
||
## 2️⃣ Batch jobs
|
||
|
||
<pre class="mermaid">
|
||
flowchart TD
|
||
subgraph K["Kubernetes Control Plane"]
|
||
J1["Job"]@{ shape: card}
|
||
J2["Job"]@{ shape: card}
|
||
J3["..."]@{ shape: text}
|
||
J4["Job"]@{ shape: card}
|
||
end
|
||
C["Client"] ---> K
|
||
K <---> N1["Node"] & N2["Node"] & N3["Node"]
|
||
</pre>
|
||
|
||
---
|
||
|
||
## Batch jobs
|
||
|
||
- Latency: a few seconds (many Kubernetes controllers involved)
|
||
|
||
- Overhead: significant due to all the moving pieces involved
|
||
|
||
(job controller, scheduler, kubelet; many writes to etcd and logs)
|
||
|
||
- Great for long requests (a few minutes to a few days)
|
||
|
||
- Supported out of the box by Kubernetes
|
||
|
||
(`kubectl create job hello --image alpine -- sleep 60`)
|
||
|
||
- Asynchronous processing requires some refactoring
|
||
|
||
(we don't get the response immediately)
|
||
|
||
---
|
||
|
||
## 3️⃣ Message queues
|
||
|
||
<pre class="mermaid">
|
||
flowchart TD
|
||
subgraph Q["Message queue"]
|
||
M1["Message"]@{ shape: card}
|
||
M2["Message"]@{ shape: card}
|
||
M3["..."]@{ shape: text}
|
||
M4["Message"]@{ shape: card}
|
||
end
|
||
C["Client"] ---> Q
|
||
Q <---> W1["Worker"] & W2["Worker"] & W3["Worker"]
|
||
</pre>
|
||
|
||
---
|
||
|
||
## Message queues
|
||
|
||
- Latency: a few milliseconds to a few seconds
|
||
|
||
- Overhead: intermediate
|
||
|
||
(very low with e.g. Redis, higher with e.g. Kafka)
|
||
|
||
- Great for all except very short requests
|
||
|
||
- Requires additional setup
|
||
|
||
- Asynchronous processing requires some refactoring
|
||
|
||
---
|
||
|
||
## Dealing with errors
|
||
|
||
- Load balancers
|
||
|
||
- errors reported immediately (client must retry)
|
||
- some load balancers can retry automatically
|
||
|
||
- Batch jobs
|
||
|
||
- Kubernetes retries automatically
|
||
- after `backoffLimit` retries, Job is marked as failed
|
||
|
||
- Message queues
|
||
|
||
- some queues have a concept of "acknowledgement"
|
||
- some queues have a concept of "dead letter queue"
|
||
- some extra work is required
|
||
|
||
---
|
||
|
||
## Some queue brokers
|
||
|
||
- Redis (with e.g. RPUSH, BLPOP)
|
||
|
||
*light, fast, easy to setup... no durability guarantee, no acknowledgement, no dead letter queue*
|
||
|
||
- Kafka
|
||
|
||
*heavy, complex to setup... strong deliverability guarantee, full featured*
|
||
|
||
- RabbitMQ
|
||
|
||
*somewhat in-between Redis and Kafka*
|
||
|
||
- SQL databases
|
||
|
||
*often requires polling, which adds extra latency; not as scalable as a "true" broker*
|
||
|
||
---
|
||
|
||
## More queue brokers
|
||
|
||
Many cloud providers offer hosted message queues (e.g.: Amazon SQS).
|
||
|
||
These are usually great options, with some drawbacks:
|
||
|
||
- vendor lock-in
|
||
|
||
- setting up extra environments (testing, staging...) can be more complex
|
||
|
||
(Setting up a singleton environment is usually very easy, thanks to web UI, CLI, etc.; setting up extra environments and assigning the right permissions with e.g. IAC is usually significantly more complex.)
|
||
|
||
---
|
||
|
||
## Implementing a message queue
|
||
|
||
1. Pick a broker
|
||
|
||
2. Deploy the broker
|
||
|
||
3. Set up the queue
|
||
|
||
4. Refactor our code
|
||
|
||
---
|
||
|
||
## Code refactoring (client)
|
||
|
||
Before:
|
||
```python
|
||
response = http.POST("http://api", payload=Request(...))
|
||
```
|
||
|
||
After:
|
||
```python
|
||
client = queue.connect(...)
|
||
client.publish(message=Request(...))
|
||
```
|
||
|
||
Note: we don't get the response right way (if at all)!
|
||
|
||
---
|
||
|
||
## Code refactoring (server)
|
||
|
||
Before:
|
||
```python
|
||
server = http.server(request_handler=handler)
|
||
server.listen("80")
|
||
server.run()
|
||
```
|
||
|
||
After:
|
||
```python
|
||
client = queue.connect(...)
|
||
while true:
|
||
message = client.consume()
|
||
response = handler(message)
|
||
# Write the response somewhere
|
||
```
|