Implementing A/B Testing in Shopify Hydrogen with Sanity CMS

Wed Jun 18 2025 - Yena Lam

Implementing A/B Testing in Shopify Hydrogen with Sanity CMS

Table of content

  • I. Why A/B Testing Matters in a Headless Shopify Setup
  • II. Core Benefits of A/B Testing for Hydrogen + Sanity
  • III. Architecture Overview: Hydrogen + Sanity + A/B Testing
  • 1. Sanity Configuration: Content Variants
  • 2. Define A/B Test Group Assignment (SSR)
  • 3. Query Sanity with GROQ Based on Variant
  • 4.Personalization Based on Segments
  • 5. Hydrogen Components: Dynamically Render Content
  • 6. Track Variants via GTM or Server Logging
  • ✅ Testing & Validation
  • MongoDB Analytics: 'ab_segments' Collection
  • Analytics Aggregation Code
  • Sanity Client Setup
  • Sample GTM Container JSON
  • Final Thoughts

As a Shopify development team, we're constantly pushing e-commerce boundaries. With Shopify Hydrogen v2.0 and Sanity CMS, we build blazing-fast, customizable storefronts. But optimization is key for conversions. A/B testing and personalization are vital.

Our headless setup offers immense control. We avoid clunky third-party plugins that compromise performance. Instead, we leverage Hydrogen/Remix's Server-Side Rendering (SSR) and Sanity's flexible content for robust A/B testing directly in our codebase.

This approach means faster loads, seamless UX, and clear insights into what resonates with customers – no extra dev hours required. Let's dive into technical steps for A/B testing and personalization using Hydrogen's SSR magic.

I. Why A/B Testing Matters in a Headless Shopify Setup

Traditional Shopify themes come with built-in limitations for experimentation. You’re often locked into templates and can’t easily run meaningful tests without apps or third-party scripts. By contrast, headless commerce—especially with Shopify Hydrogen—allows for server-side rendering (SSR), dynamic routing, and composable APIs.

Shopify Hydrogen + Sanity CMS.png With A/B testing in this setup, you can:

  • Validate assumptions before making permanent design changes.
  • Test new content or features without affecting all users.
  • Personalize the buyer journey for specific segments.
  • Drive measurable improvements to conversion rates, AOV, and engagement.

II. Core Benefits of A/B Testing for Hydrogen + Sanity

  • Full Control Over UI and Logic: Test layouts, CTAs, or checkout flows at the component level.

  • Dynamic Content from Sanity CMS: Content teams can manage variant A/B copy directly—no redeployment required.

  • Performance-Friendly with SSR: Avoid client-side flickering or layout shifts common with JavaScript-based testing tools.

  • Scalable Personalization: Combine testing with segmentation for a personalized shopping experience.

  • Granular Analytics: Integrate with MongoDB or Google Analytics for durable, queryable test data.

III. Architecture Overview: Hydrogen + Sanity + A/B Testing

Shopify Hydrogen + Sanity CMS 2.png Your stack should consist of:

  • Shopify Hydrogen: React-based front-end framework with SSR support.
  • Sanity CMS: Headless, real-time content platform with GROQ querying.
  • A/B Test Logic (SSR): Group assignment handled server-side to ensure performance and consistency.
  • Analytics Layer: Track variant exposure, conversions, and segments with tools like Google Tag Manager or MongoDB.

Below is a breakdown of each major step in the implementation process:

1. Sanity Configuration: Content Variants

Goal: Define content variants directly in Sanity for A/B or personalized experiences.

// schemas/homepage.ts
export default {
  name: 'homepage',
  title: 'Homepage',
  type: 'document',
  fields: [
    {
      name: 'variant', title: 'Variant', type: 'string',
      options: { list: ['A', 'B', 'Personalized'] },
      description: "Assign this document to a variant test or personalized segment."
    },
    {
      name: 'heroSection',
      type: 'object',
      fields: [
        { name: 'title', type: 'string', title: 'Hero Title' },
        { name: 'description', type: 'text', title: 'Hero Description' },
        { name: 'image', type: 'image', title: 'Hero Image' },
      ],
    },
    {
      name: 'promoBanner',
      type: 'object',
      fields: [
        { name: 'message', type: 'string', title: 'Banner Text' },
        { name: 'ctaLink', type: 'url', title: 'CTA URL' },
      ],
    },
    // Add more sections like testimonials, features, etc.
  ],
  preview: {
    select: {
      title: 'heroSection.title',
      variant: 'variant'
    },
    prepare(selection) {
      return {
        title: `[${selection.variant}] ${selection.title}`,
        subtitle: 'Homepage variant'
      };
    }
  }
};

Best practices:

  • Use clear naming (e.g., title_A, title_B) if separate fields for each variant.

  • Use object-level variants if personalization is more complex.

2. Define A/B Test Group Assignment (SSR)

Centralize variant and visitor logic in lib/ab-utils.ts:

// lib/ab-utils.ts
import { parse, serialize } from 'cookie';
import { v4 as uuidv4 } from 'uuid';

export function getOrCreateVisitorId(cookieHeader?: string) {
  const cookies = parse(cookieHeader || '');
  const id = cookies.visitor_id;
  return id || uuidv4();
}

export function getOrCreateABGroup(cookieHeader?: string) {
  const cookies = parse(cookieHeader || '');
  if (cookies.ab_group === 'A' || cookies.ab_group === 'B') {
    return cookies.ab_group;
  }
  return Math.random() < 0.5 ? 'A' : 'B';
}

export function getCookieHeaders(visitorId: string, abGroup: string) {
  return [
    serialize('visitor_id', visitorId, { path: '/', maxAge: 30 * 24 * 3600, sameSite: 'lax' }),
    serialize('ab_group', abGroup, { path: '/', maxAge: 30 * 24 * 3600, sameSite: 'lax' })
  ];
}

In your loader:

// app/routes/_index.tsx
import { json, LoaderFunction } from '@shopify/remix-oxygen';
import { getHomepageByVariant } from '~/lib/sanity.server';
import { getOrCreateVisitorId, getOrCreateABGroup, getCookieHeaders } from '~/lib/ab-utils';

export const loader: LoaderFunction = async ({ request }) => {
  const cookieHeader = request.headers.get('cookie') || '';
  const visitorId = getOrCreateVisitorId(cookieHeader);
  const abGroup = getOrCreateABGroup(cookieHeader);

  const homepageData = await getHomepageByVariant(abGroup);
  const cookieStrings = getCookieHeaders(visitorId, abGroup);

  return json({ homepageData, variant: abGroup }, {
    headers: { 'Set-Cookie': cookieStrings }
  });
};

3. Query Sanity with GROQ Based on Variant

Query the appropriate variant document:

// lib/sanity.server.ts
import { createClient } from '@sanity/client';

export const sanityClient = createClient({
  projectId: process.env.SANITY_PROJECT_ID!,
  dataset: process.env.SANITY_DATASET!,
  apiVersion: process.env.SANITY_API_VERSION!,
  useCdn: process.env.NODE_ENV === 'production',
  token: process.env.SANITY_TOKEN
});

export async function getHomepageByVariant(variant: string) {
  const query = `
    *[_type == "homepage" && variant == $variant][0] {
      heroSection { title, description, "imageUrl": image.asset->url },
      promoBanner { message, ctaLink }
    }
  `;
  return sanityClient.fetch(query, { variant });
}

By querying variant-specific documents, editors can fully control A, B, or personalized content.

4.Personalization Based on Segments

In lib/ab-utils.ts, derive segments:

export function getUserSegment(request: Request): Record {
  const url = new URL(request.url);
  return {
    referer: request.headers.get('referer') || '',
    country: request.headers.get('x-vercel-ip-country') || '',
    utm_source: url.searchParams.get('utm_source') || '',
    utm_campaign: url.searchParams.get('utm_campaign') || '',
    device: request.headers.get('user-agent')?.includes('Mobile') ? 'mobile' : 'desktop'
  };
}

Merge this with your loader logic to populate analytics logging and optionally fetch personalized content.

5. Hydrogen Components: Dynamically Render Content

// app/routes/_index.tsx
export default function Index({ homepageData, variant }) {
  const { heroSection, promoBanner } = homepageData;

  return (
    <>
      

{heroSection.title}

{heroSection.description}

Test variant: {variant}
{promoBanner && (

{promoBanner.message}

Learn More
)} ); }

Components remain stateless and rely on server passed props.

6. Track Variants via GTM or Server Logging

# A. Server Logging to MongoDB

Extend lib/ab-utils.ts with storage logic:

import { MongoClient } from 'mongodb';

export async function storeSegmentProfile({
  visitorId, variant, segment, path
}: {
  visitorId: string;
  variant: string;
  segment: Record;
  path: string;
}) {
  const client = new MongoClient(process.env.MONGODB_URI!);
  await client.connect();
  const col = client.db(process.env.MONGODB_DB).collection('ab_segments');
  await col.insertOne({ visitorId, variant, segment, path, timestamp: new Date() });
  await client.close();
}

Call this in your loader after setting cookies.

# B. GTM Client-Side

In your root layout:

useEffect(() => {
  const ab_group = getCookie('ab_group');
  const user_segment = getCookie('user_segment');
  window.dataLayer = window.dataLayer || [];
  window.dataLayer.push({ event: 'ab_test_load', ab_group, user_segment });
}, []);

Trigger GA4 conversions similarly on /thank-you.

7. ✅ Testing & Validation

  • Cookie checks: Use DevTools → Application → Cookies.

  • Server logs: Confirm MongoDB documents in ab_segments.

  • GTM debug: Preview GTM container settings.

  • Visual QA: Ensure components render correct variants without flicker.

  • Analytics tests: Track views → visit MQL and conversion flows.

#Folder Structure Suggestion

/app
  /routes
    _index.tsx
/lib
  ab-utils.ts
  sanity.server.ts
/schemas
  homepage.ts

# Required. '.env' Variables

# Sanity
SANITY_PROJECT_ID=...
SANITY_DATASET=production
SANITY_API_VERSION=2023-06-01
SANITY_TOKEN=...

# MongoDB
MONGODB_URI=mongodb+srv://.../mydb
MONGODB_DB=ab_test_analytics

# Shopify Hydrogen
PUBLIC_STORE_DOMAIN=your-shop.myshopify.com
PUBLIC_STOREFRONT_API_TOKEN=...
PUBLIC_STOREFRONT_API_VERSION=2023-07

MongoDB Analytics: 'ab_segments' Collection

Example document:

{
  "visitorId": "uuid-1234",
  "variant": "A",
  "segment": { "utm_source": "google", "country": "VN", "device": "mobile" },
  "path": "/thank-you",
  "timestamp": ISODate("2025‑06‑18T08:00:00Z")
}

Analytics Aggregation Code

import { MongoClient } from 'mongodb';

async function analyzeAB() {
  const client = new MongoClient(process.env.MONGODB_URI!);
  await client.connect();
  const col = client.db(process.env.MONGODB_DB).collection('ab_segments');
  const [stats] = await col.aggregate([
    {
      $facet: {
        visitsByVariant: [{ $group: { _id: '$variant', totalVisits: { $sum: 1 } } }],
        conversionsByVariant: [
          { $match: { path: '/thank-you' } },
          { $group: { _id: '$variant', totalConversions: { $sum: 1 } } }
        ],
        conversionsBySource: [
          { $match: { path: '/thank-you' } },
          {
            $group: {
              _id: { variant: '$variant', source: '$segment.utm_source' },
              count: { $sum: 1 }
            }
          }
        ]
      }
    }
  ]).toArray();
  console.log(JSON.stringify(stats, null, 2));
  await client.close();
}

Sanity Client Setup

// lib/sanity.server.ts
import { createClient } from '@sanity/client';

export const sanityClient = createClient({
  projectId: process.env.SANITY_PROJECT_ID!,
  dataset: process.env.SANITY_DATASET!,
  apiVersion: process.env.SANITY_API_VERSION!,
  useCdn: process.env.NODE_ENV === 'production',
  token: process.env.SANITY_TOKEN
});

Sample GTM Container JSON

Note: Omitted for brevity—this includes cookie-based variables ('ab_group', 'user_segment'), GA4 tag, and conversion trigger on '/thank-you'.

Instructions to Import:

  1. Download the .json file.

  2. In GTM → Admin → Import Container.

  3. Select destination workspace, merge settings.

  4. Test in Preview mode.

Final Thoughts

A/B testing in a Hydrogen + Sanity CMS architecture enables businesses to take full control of their experimentation process, from SSR logic to content delivery and analytics. With the right setup, you gain the flexibility of headless, the editorial power of Sanity, and the confidence of data-backed decisions.

Whether you’re testing headlines, layouts, or entire journeys, this approach ensures speed, consistency, and clarity at scale.

Read more: