Conrad DevOps

A/B testing

A/B testing with a tool like Kameleoon on a site with server rendering and a page cache — assigning visitors, avoiding flicker, keeping variants out of the shared cache, and feature flags.

From the Kameleoon part of your Conrad DevOps note.

Know

Monitoring has its own section: Conrad monitoring.

  • A/B testing: both versions ship in one deploy behind a flag; the tool assigns visitors (10% to the new version), keeps them there with a cookie and tracks goals; results need days and statistical significance. With SSR, decide the variant on the server to avoid flicker, and keep variants out of the shared HTML cache or put them in the key. At 100% the same mechanism is a feature flag.

Exercise

Exercise
  • Write down how you would run an A/B test on the product page without breaking the Valkey cache.

Be able to say

Be able to say· How do you run an A/B test on a server-rendered, cached page?

"Both versions ship in one deploy behind a flag, and the tool assigns visitors, keeps them in their variant with a cookie and tracks the conversion goals. Two things matter on SSR. The variant has to be decided on the server, from the cookie or a server-side SDK, otherwise the page flickers from A to B after load. And the HTML cache must not leak a variant: either the variant goes into the cache key, or the page is cached without the experiment and the variant part renders on the client. And I'd keep the testing script's own cost in the measurement."

Your notes

Open in Notes

Nothing yet. Notes filed here show up on this page and the section overview.