Chromatic Visual Testing: Automated Change Detection

Chromatic is a cloud-based visual testing service built by Storybook's core maintainers. It automates visual regression testing by capturing screenshots in the cloud, comparing them against baselines, and flagging visual changes for review. Unlike local snapshot testing (which is slow and platform-dependent), Chromatic runs in parallel across 10–15 browsers and devices, detecting visual regressions in seconds rather than minutes. It integrates seamlessly into GitHub, GitLab, and Bitbucket pull requests, showing a pass/fail status and a link to an interactive review interface where you can approve or reject changes with a single click.

Setting Up Chromatic

First, sign up at chromatic.com with your GitHub account. Then install the Chromatic CLI:

npm install --save-dev chromatic

Add a build script to package.json:

{
  "scripts": {
    "build-storybook": "storybook build",
    "chromatic": "chromatic --project-token=YOUR_PROJECT_TOKEN"
  }
}

Find your project token on the Chromatic dashboard and replace YOUR_PROJECT_TOKEN. Store it as an environment variable for security:

export CHROMATIC_PROJECT_TOKEN="your-token-here"
npm run chromatic

On first run, Chromatic builds your Storybook, uploads it to the cloud, captures screenshots of every story, and stores them as baselines. Subsequent runs compare against these baselines.

How Chromatic Detects Visual Changes

When you push code to a pull request, Chromatic:

1. Builds Storybook in the cloud.
2. Captures screenshots across 10–15 browsers/devices (Chrome, Firefox, Safari, mobile viewports).
3. Compares against baselines using pixel-level diffing.
4. Reports changes as a GitHub check with an interactive review link.

For example, if you change a Button's background color from #3498db to #2ecc71, Chromatic detects it in seconds and shows a side-by-side diff in the review interface. You can approve the change (updating the baseline) or request changes (failing the PR).

Integrating with GitHub Actions

Set up Chromatic in your CI pipeline to run on every pull request. Add a .github/workflows/chromatic.yml file:

name: Chromatic

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  chromatic:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0 # Full history for Chromatic

      - uses: actions/setup-node@v3
        with:
          node-version: '18'

      - run: npm install

      - name: Publish to Chromatic
        uses: chromaui/action@latest
        with:
          projectToken: ${{ secrets.CHROMATIC_PROJECT_TOKEN }}
          onlyChanged: true # Only snapshot changed stories

Set your CHROMATIC_PROJECT_TOKEN as a GitHub secret. Now Chromatic runs automatically on every PR, adding a check result you can click to review changes.

Reviewing and Approving Changes

After Chromatic completes, a check appears in your PR with a link to the review interface. Click it to see:

• List of changed stories: Stories with visual differences are highlighted.
• Side-by-side diffs: Baseline (left), current (right), with a scrubber to compare.
• Approval options: "Accept" to update baseline, "Deny" to fail the build.

For a Button color change, the diff clearly shows the old and new colors. You can:

1. Accept: Click "Accept" to approve the change. Chromatic updates the baseline.
2. Deny: Click "Deny" to request changes. The PR fails the check until the code is fixed.
3. Comment: Leave notes on specific changes for team feedback.

Once you accept all changes, Chromatic automatically passes the check, unblocking the PR merge.

Handling Flaky or Dynamic Content

Some stories contain dynamic content (timestamps, animations) that changes between runs, causing false positives. Chromatic provides options to ignore these:

In your story file, use the chromatic parameter:

export const WithTimestamp: Story = {
  args: { showTime: true },
  parameters: {
    chromatic: {
      disableSnapshot: true, // Skip this story entirely
    },
  },
};

export const Animated: Story = {
  parameters: {
    chromatic: {
      pauseAnimationAtEnd: true, // Pause animations during snapshot
    },
  },
};

export const HighVariance: Story = {
  parameters: {
    chromatic: {
      threshold: 0.5, // Tolerance for pixel differences (0–1)
    },
  },
};

• disableSnapshot: true: Completely skip this story.
• pauseAnimationAtEnd: true: Wait for animations to finish and pause them.
• threshold: 0.5: Allow 50% of pixels to differ (0.2 = 20% tolerance). Default is exact match.

Cross-Browser Testing

Chromatic captures screenshots across multiple browsers. To see browser-specific results, expand the "Browsers" section in the review interface. Each browser gets its own baseline and diff.

For example, if your Button uses a CSS gradient that renders differently in Safari, Chromatic shows that difference. You can decide to accept the variance (cross-browser rendering is normal) or fix it with vendor prefixes.

To configure which browsers Chromatic tests, add a .chromatic.json configuration file:

{
  "browsers": [
    "Chrome",
    "Firefox",
    "Safari",
    "Chrome (Mobile)"
  ],
  "viewport": [
    {
      "width": 1280,
      "height": 720
    },
    {
      "width": 375,
      "height": 667
    }
  ]
}

This limits testing to 4 browsers and 2 viewports, speeding up snapshot runs.

Using Chromatic for Design System Documentation

Chromatic doubles as a design system browser. Share the Chromatic build URL (visible in the PR comment) with non-technical stakeholders. They can view all components and their states without running Storybook locally. This is invaluable for design reviews and feedback loops.

Comparing Chromatic to Local Snapshots

Feature	Local Snapshots (test-runner)	Chromatic
Speed	Slow (sequential)	Fast (parallel, cloud)
Browsers	Single (your OS font rendering)	10–15 browsers + mobile
Setup	Simple (npm test:storybook)	Requires account + token
Cost	Free	Free tier (limited); paid plan ($99+/month)
Review UI	CLI diffs	Interactive web interface
Cross-browser	No	Yes
CI integration	GitHub Actions config	Auto-integrated via action

For small teams or projects, local snapshots are sufficient. For large design systems or teams, Chromatic's speed and cross-browser testing justify the cost.

Key Takeaways

• Chromatic cloud snapshots: Capture screenshots in the cloud across 10–15 browsers; detect visual changes in seconds.
• GitHub/GitLab integration: Chromatic adds a check to PRs with an interactive review link; approve changes with one click.
• Parallel testing: Cloud infrastructure tests faster than local snapshot runs, critical for large component libraries.
• Cross-browser detection: Catch platform-specific rendering bugs (Safari gradients, Firefox spacing) automatically.
• Design system sharing: Share Chromatic build URLs with stakeholders for visual feedback without running Storybook.
• Handle dynamic content: Use disableSnapshot, pauseAnimationAtEnd, or threshold to ignore flaky elements.

Frequently Asked Questions

How much does Chromatic cost?

Chromatic has a free tier (10 snapshots/month) for small projects. Paid plans start at $99/month for 5,000 snapshots. Large teams often use paid plans. Compare pricing at chromatic.com/pricing.

Can I use Chromatic without GitHub?

Yes. Chromatic supports GitHub, GitLab, and Bitbucket. It also works standalone—you can run npm run chromatic locally and review changes via the web dashboard.

How do I handle stories that intentionally look different per theme (dark mode, light mode)?

Create separate stories for each theme:

export const LightMode: Story = {
  args: { theme: 'light' },
};

export const DarkMode: Story = {
  args: { theme: 'dark' },
};

Chromatic captures both, so visual changes to either are detected independently.

Does Chromatic test every component or just changed ones?

By default, Chromatic tests all components. Use the onlyChanged: true option in the GitHub Action to snapshot only stories that changed. This speeds up PRs but may miss regressions in unmodified components that now look different due to shared CSS changes.

Further Reading

• Chromatic Official Documentation
• Chromatic GitHub Action
• Storybook + Chromatic Integration Guide
• Design System CI/CD with Chromatic