Abrantes - Creating Experiments

Basic setup

Create a test object, define variants, then assign and render the variant.

var MyTest = Object.create(Abrantes);

MyTest.variants = [
  function () { /* control stays as the default page */ },
  function () { document.querySelector(".cta").textContent = "Start free trial"; }
];

MyTest.assignVariant("MyTest");
MyTest.renderVariant();
MyTest.persist("cookie");

Learn how variant functions work →

CSS-only design experiments

For visual tweaks, you can let Abrantes add a variant class on <body> and handle the changes in CSS. The class format is {testId}-{variant}, matching the ID you pass to assignVariant.

/* CSS scoped to the variant class */
.MyDesignExperiment-1 .hero-title { font-size: 2.5rem; }
.MyDesignExperiment-2 .hero-title { font-size: 1.7rem; }

var MyDesignExperiment = Object.create(Abrantes);
MyDesignExperiment.variants = [
  function () {}, // control
  function () {}, // variant 1
  function () {}  // variant 2
];
MyDesignExperiment.assignVariant("MyDesignExperiment");
MyDesignExperiment.renderVariant();
MyDesignExperiment.persist("cookie");

Keep the variant functions empty when CSS handles the changes.
You can target the control with .MyDesignExperiment-0 if needed.
Place the CSS in a stylesheet or a <style> block on the page.

If the user is excluded by segmentation or allocation, assignVariant returns -1 and no variant class is added to the page.

Open design-experiment-1.html →

Traffic allocation and segmentation

Use the optional parameters in assignVariant to limit traffic or target a segment.

MyTest.assignVariant(
  "MyTest", // experiment name
  0.5, // traffic allocation: 50% of users
  () => { return window.innerWidth > 768; } // segment: in this example do the test for users with a viewport wider than 768px
);

The traffic allocation is a number between 0 and 1.
segment is a function that returns true or false. If it returns true the user enters the experiment. See examples of segments.

When the user is not in the experiment assignVariant returns -1 and the page remains unchanged. Also the user participation in the experiment is not tracked.

Tracking results

Tracking is handled by plugins. You should use the right plugins for your tracking tools. Call the plugin method after assignVariant.

// Example with the Google Analytics 4 gtag plugin
MyTest.track("test_event");

You can also:

Use the dataLayer to track users if you are using Google Tag Manager or other compatible software. See the add2DL plugin example for more information.
Use the log plugin to capture raw data.
Use Hotjar or Microsoft Clarity to create heatmaps or screen recordings.

Testing the experiment

To make sure all is working properly and avoid having to restart the same experiment later.

1.- Render specific variants using a parameter in the URL by importing instead of assigning:

MyTest.importVariant("MyTest");

and load the page with the URL parameter ?MyTest=x (where x is the variant number). You can share this URLS with your team or using them to test in mobile phones.

2.- Check the pages of the experiment using the Abrantes checker browser extension.

3.- Use the log plugin and the log api to monitor what happens after the experiment is launched.

Advanced patterns

Wait for the DOM

For SPAs or delayed content, wait for elements before rendering changes. waitFor should be used in the variant definitions. You can use more than one waitFor in a variant to wait for different elements.

function () {
  MyTest.waitFor(".pricing-table", () => {
    document.querySelectorAll(".pricing-table .price").forEach(el => {
    el.textContent = "9.99 €";
    })
  })
}

Open waitFor.html →

Cross-site experiments

Assign the variant in one domain and pass it to another domain .

// Landing
MyTest.crossSiteLink("a[data-cross-site]");

// Step 2
MyTest.importVariant("MyTest");
MyTest.renderVariant();

Open landing.html →
Open step-2.html →

Redirect tests

Use different pages for each variant and keep URL parameters.

if (MyTest.variant === 1) {
  MyTest.redirectTo("/pricing-v2");
}

Open redirect/ →

Alpine.js integration

Alpine is a lightweight and easy to use JavaScript framework. The two examples show how to use Alpine state or template rendering to swap variants without fighting reactivity.

Read the Alpine.js guide →

Working with iframes

Do experiments with iframes, even with limitations on the JavaScript you can put in them. Some aplications with forms may only allow you to run JS after the form is submitted.

Example with full JavaScript → Example with limited JavaScript →

Exclude variants

Exclude variants in the middle of an experiment if they are definitely getting bad results. Users already enrolled in the experiment won't be affected.

Open exclude.html →

Seeded experiments

Use when you want to use Abrantes with a fixed seed for reproducible results. It is very useful to create cookieless experiments or cross-device experiments with logged-in users.

Seeded experiments have to be inside async functions and can fetch user information from servers.

Open seed.html →

Start experiments programmatically

Start an experiment when a certain dataLayer event is triggered, instead of starting it on page load. This technique has many advantages when what's being tested is experienced by the user only after the dataLayer event is triggered.

Open listen2DL.html →

Create experiments