Have you ever shipped a bug to the production environment, even though it passed all the unit tests?
If you said yes, visual testing could’ve prevented this. If not yet, that means visual testing can definitely prevent this.
What is Visual Testing?
Automated visual testing evaluates the appearance of your components or web pages, typically by comparing your builds’ visual appearance against its original design.
It’s in essence the same as an “eye-test” that a frontend engineer will do after making changes to code. Tests check if everything is in the correct place and that the output generally looks how it’s supposed to.
Having automated visual testing in your workflow doesn’t just help you detect visual problems in your application in a systematic way, though. It’s also a great way to catch visual bugs earlier and saves time with cross-browser testing by automatically taking care of what you’d normally do manually. Plus, you reduce your risk of deploying a visually bugged application to the production environment.
In the end, visual testing gets you better alignment between your design system and your actual design.
How we use visual testing
Many applications in our analytics platform product at Littledata use the same atomic components. Each time we need to make a change to an atomic component—or to a component that uses an atomic component—we need to check each instance in our host applications where we use the changed component.
As you can guess, this is time-consuming work and very much prone to human error.
From time to time, small details regarding the visual appearance of components are overlooked in our pull requests. That’s why we integrated the Chromatic visual testing tool into our deployment pipeline.
To show a real-world example of the value this tool—and visual testing overall—can bring, I’d like to share how we integrated Chromatic into our pipeline and how we fit visual testing in our workflow at Littledata.
Integrating Chromatic into our pipeline
Setting up the chromatic workflow and integrating it into your deployment pipeline is pretty straightforward. One of the reasons we chose Chromatic is that it’s built for Storybook workflows, which we already use for our design system.
To start setup, sign in on Chromatic’s website however you prefer. In our case, it’s GitHub.
Next, Click “Choose from Github”
The menu on the right side allows you to choose any project you want to start visual testing on.
After you select your project, you’ll be prompted with instructions on how to get your Chromatic build started. At this point, you’re just 2 steps away from having your first Chromatic build!
Next, go over to your projects directory and add the Chromatic library to your project.
If you’re using npm, enter:
npm install --save-dev chromatic
If you’re using yarn, enter:
yarn add --dev chromatic
After you installed the package, you can publish your first deployment by simply running this command:
npx chromatic --project-token=
And voila! You’ll see your first build in your Chromatic dashboard. While that was very easy to accomplish, we don’t want to have to run the same publish command each time we deploy new code. So instead, let’s integrate Chromatic into the deployment pipeline.
In Littledata’s case, we use CircleCI for our CI/CD pipeline.
How to integrate Chromatic with CircleCI
Assuming that you already have your project on CircleCI, you’ll first need to add your Chromatic token as an environment variable to your project to get started.
To do this, open your project’s settings at CircleCI, then select open environment variables. Then, click “Add Environment Variable.”
Enter CHROMATIC_PROJECT_TOKEN as the name and your project’s Chromatic token as the value, then hit “Add Environment Variable.”
Now, we need to add the Chromatic publish step to our pipeline by modifying our .circleci/config.yml file.
Add this job under jobs:
chromatic-deployment:docker:- image: circleci/node:12steps:- checkout- restore_cache:keys:- v1-dependencies-{{ checksum "package.json" }}- v1-dependencies-- run: yarn install- run: yarn chromatic --project-token=${CHROMATIC_PROJECT_TOKEN}
Then add this workflow under your workflows:
chromatic-deploy:jobs:- chromatic-deployment
Please note that you may want to change your Node version depending on your project’s node version. If you run the Chromatic job after you’ve already installed the packages, you don’t need to run the yarn install command in Chromatic deployment, so you can remove that command.
After you have a Chromatic job in your CircleCI pipeline, each time you push new code that changes the layout of your components, you’ll see the CI/CD check-related Chromatic in your GitHub pull requests (like in the below image.)
In the Chromatic dashboard, you can add collaborators that don’t have access to the GitHub repository like product managers, designers, and QAs. To add collaborators, open the Manage section in the left side menu, then click the Collaborate section. You can use a special link to invite them or send an invitation via email.
Once they accept the invitation, they can review your builds, approve or deny the changes in the components.
How Littledata integrates visual testing in our workflows
The first benefit of implementing visual testing for us was the alignment it created between our codebase and the design system, as we’ve had conflicts between them in the past. Creating that single source of truth also pushed us to make decisions on some unclear cases in our components.
Because we already have an atomic shared components library created, we find it helpful to use Storybook to quickly preview any type of change in our components. Having Storybook already set up made it easier to both install Chromatic in our project and integrate it into our pipeline.
The only downside I could mention is that visual testing is not only another step in the work pipeline but something you need to embrace as a process and spend time on. Each time you make a new build, you need to get approval from designers or product managers.
While it might feel like visual testing slows down the process at first, I feel in the long run you’ll see that it pays off—just as we did.
Tip: Want to join us in building the future of ecommerce analytics and empowering entrepreneurs around the world? Check out Littledata’s open positions.
