Conrad Cloudflare

Storage and background work

When to use KV, D1, R2, Durable Objects and Hyperdrive, caching HTML in KV, KV's eventual consistency, R2 without egress fees, and Queues versus Workflows for background work.

From the D1, R2, KV, Durable Object, Workflows, Queues and Hyperdrive parts of your Cloudflare note.

Know

  • KV: global key-value store, reads from the nearest location, eventually consistent (up to about a minute), about one write per second per key; for config, flags and pages that rarely change.
  • D1: SQLite as a service; one primary per database takes all writes one at a time; read replicas only with read replication on (Sessions API keeps a user's own writes visible); db.batch() for one round trip and one transaction; 10 GB per database.
  • R2: S3-compatible object storage; storage and operations are billed, egress is not.
  • Durable Objects: one instance per id worldwide, in one location, one request at a time, strongly consistent SQLite storage, WebSockets and alarms; for coordination, counters, rate limits, idempotency.
  • Hyperdrive: pools warm connections to an existing Postgres or MySQL near the database and can cache reads, so many Workers share a few connections.
  • Queues: guaranteed delivery with retries (three by default) and batching, a dead letter queue for messages that keep failing, at-least-once so consumers must be idempotent; for hand-offs like emails. Workflows: multi-step processes whose steps are saved and retried, can sleep or wait for events for days.

Fact check of your notes

Gotcha· Queues retry three times by default

The note says five. Cloudflare Queues retries a failed message three times unless max_retries is set; after that it goes to the dead letter queue if one is configured, otherwise it is deleted.

Gotcha· A Durable Object is in one place for everyone

"Only on the closest data center": a Durable Object is created near where it is first used and then stays there; requests from everywhere else travel to it. It is the closest location for the first user, not for each user.

Exercise

Exercise
  • For a shop on Cloudflare, assign each piece of data to a product and say why: product catalogue, product images, cart, feature flags, rendered category pages, payment idempotency, order emails, order processing.
  • Write a D1 batch() that creates an order and its lines in one transaction, and a Durable Object counter that rate-limits one user.

Be able to say

Be able to say· Which Cloudflare storage would you use for what?

"KV for data everybody reads and that may be a minute stale, like config or rarely changing pages. D1 for relational data per service, keeping in mind that all writes go to one primary. R2 for files and images, because there are no egress fees. Durable Objects when one thing must be exactly right: a counter, a lock, a chat room, making sure a payment runs once. Hyperdrive when the data stays in an existing Postgres. And for background work, Queues for single hand-offs like emails, Workflows for multi-step processes that wait and retry."

Your notes

Open in Notes

Nothing yet. Notes filed here show up on this page and the section overview.