Skip to product menu
close
  • Recent Launches
    Press Space or Enter to display list of options
EXPLORE ALL PRODUCTS

Recent Launches

New

Payroll software with automated tax payments and filing.

Try now
New

Robotic process automation software to automate high-volume, rule-based tasks.

Try for free
New

Low-code IoT platform and solutions for connected businesses.

Try now
New

Business formation service to launch and grow your businesses.

Try now
New

Privacy-friendly application analytics solution.

Try for free

Sales

 
CRM

Comprehensive CRM platform for customer-facing teams.

CRM
 
Bigin

Simple CRM for small businesses moving from spreadsheets.

Bigin
 
Forms

Build online forms for every business need.

Forms
 
SalesIQ

Live chat app to engage and convert website visitors.

SalesIQ
 
Bookings

Appointment scheduling app for consultations with customers.

Bookings
 
Sign

Digital signature app for businesses.

Sign
 
RouteIQ

Comprehensive sales map visualization and optimal route planning solution.

RouteIQ
 
Thrive

Complete loyalty and affiliate management platform.

Thrive
 
Voice

Cloud Contact Center Software for businesses.

Voice
 
Suites
CRM Plus

Unified platform to deliver top-notch customer experience.

CRM Plus

Marketing

 
Social

All-in-one social media management software.

Social
 
Campaigns

Create, send, and track targeted email campaigns that drive sales.

Campaigns
 
Forms

Build online forms for every business need.

Forms
 
Survey

Design surveys to reach and interact with your audience.

Survey
 
Sites

Online website builder with extensive customisation options.

Sites
 
PageSense

Website conversion optimization and personalisation platform.

PageSense
 
Backstage

End-to-end event management software.

Backstage
 
Webinar

Webinar platform for webcasting online webinars.

Webinar
 
Marketing Automation

All-in-one marketing automation software.

Marketing Automation
 
LandingPage

Smart landing page builder to increase conversion rates

LandingPage
 
Publish

Manage all your local business listings on a single platform.

Publish
 
SalesIQ

Live chat app to engage and convert website visitors.

SalesIQ
 
Sign

Digital signature app for businesses.

Sign
 
Thrive

Complete loyalty and affiliate management platform.

Thrive
 
Voice

Cloud Contact Center Software for businesses.

Voice
 
NEW
LeadChain

Sync, manage, and convert leads across channels seamlessly.

LeadChain
 
NEW
CommunitySpaces

Online community platform for individuals and businesses to grow their network and brand.

CommunitySpaces
 
Suites
Marketing Plus

Unified marketing platform for marketing teams.

Marketing Plus

Commerce and POS

 
Commerce

eCommerce platform to manage and market your online store.

Commerce

Service

 
Desk

Helpdesk software to deliver great customer support.

Desk
 
Assist

Remote support and unattended remote access software.

Assist
 
Lens

Interactive remote assistance software with augmented reality.

Lens
 
FSM

End-to-end field service management platform for service businesses.

FSM
 
SalesIQ

Live chat app to engage and convert website visitors.

SalesIQ
 
Voice

Cloud Contact Center Software for businesses.

Voice
 
NEW
Solo

The all-in-one toolkit for solopreneurs.

Solo
 
Bookings

Appointment scheduling app for consultations with customers.

Bookings
 
Suites
Service Plus

Unified platform for customer service and support teams.

Service Plus

Finance

 
Books

Powerful accounting platform for growing businesses.

Books
 
FREE
Invoice

100% Free invoicing solution.

Invoice
 
Expense

Effortless expense reporting platform.

Expense
 
Inventory

Powerful stock management and inventory control software.

Inventory
 
Billing

End-to-end billing solution for your business.

Billing
 
Checkout

Collect payments online with custom branded pages.

Checkout
 
NEW
Payroll

Payroll software with automated tax payments and filing.

Payroll
 
NEW
Solo

The all-in-one toolkit for solopreneurs.

Solo
 
Practice

Practice management software for accounting firms.

Practice
 
Sign

Digital signature app for businesses.

Sign
 
Commerce

eCommerce platform to manage and market your online store.

Commerce
 
Suites
Finance Plus

All-in-one suite to manage your operations and finances.

Finance Plus

Email and Collaboration

 
Mail

Secure email service for teams of all sizes.

Mail
 
Meeting

Online meeting software for all your video conferencing & webinar needs.

Meeting
 
Writer

Word processor for focused writing and discussions.

Writer
 
Sheet

Spreadsheet software for collaborative teams.

Sheet
 
Show

Create, edit, and share slides with a sleek presentation app.

Show
 
Notebook

Beautiful home for all your notes.

Notebook
 
Cliq

Stay in touch with teams no matter where you are.

Cliq
 
Connect

Employee experience platform to communicate, engage, and build positive employee relations.

Connect
 
Bookings

Appointment scheduling app for consultations with customers.

Bookings
 
TeamInbox

Shared inboxes for teams.

TeamInbox
 
WorkDrive

Online file management for teams.

WorkDrive
 
Sign

Digital signature app for businesses.

Sign
 
Office Suite

Powerful collaborative work platform for teams.

Office Suite
 
Office Integrator

Built in document editors for web apps.

Office Integrator
 
ZeptoMail

Secure and reliable transactional email sending service.

ZeptoMail
 
Calendar

Online business calendar to manage events and schedule appointments.

Calendar
 
Learn

Knowledge and learning management platform.

Learn
 
Voice

Cloud Contact Center Software for businesses.

Voice
 
ToDo

Collaborative task management for individuals and teams.

ToDo
 
Tables

Work management tool to connect people, processes, and information.

Tables
 
FREE
PDF Editor

Collaborative online PDF editing tool.

PDF Editor
 
Suites
Workplace

Application suite built to improve team productivity and collaboration.

Workplace

Human Resources

 
People

Organize, automate, and simplify your HR processes.

People
 
Recruit

Intuitive recruiting platform built to provide hiring solutions.

Recruit
 
Expense

Effortless expense reporting platform.

Expense
 
Workerly

Manage temporary staffing with an employee scheduling solution.

Workerly
 
NEW
Payroll

Payroll software with automated tax payments and filing.

Payroll
 
Shifts

Employee scheduling and time tracking app.

Shifts
 
Sign

Digital signature app for businesses.

Sign
 
Suites
People Plus

Comprehensive HR platform for seamless employee experiences.

People Plus

Security and IT Management

 
Creator

Build custom apps to simplify business processes.

Creator
 
Directory

Workforce identity and access management solution for cloud businesses.

Directory
 
FREE
OneAuth

Secure multi-factor authenticator (MFA) for all your online accounts.

OneAuth
 
Vault

Online password manager for teams.

Vault
 
Catalyst

Pro-code platform to build and deploy your apps.

Catalyst
 
Toolkit

Complete resource for any admin-related lookup queries.

Toolkit
 
Lens

Interactive remote assistance software with augmented reality.

Lens
 
Assist

Remote support and unattended remote access software.

Assist
 
QEngine

Test automation software to build, manage, execute, and report testcases.

QEngine
 
NEW
RPA

Automate manual, tedious, and repetitive tasks easily.

RPA

BI and Analytics

 
Analytics

Modern self-service BI and analytics platform.

Analytics
 
Embedded BI

Embedded analytics and white label BI solutions, tailored for your needs.

Embedded BI
 
DataPrep

AI-powered data preparation service for your data-driven organization.

DataPrep
 
NEW
IoT

Harnessing IoT analytics for real-time operational intelligence.

IoT

Project Management

 
Projects

Manage, track, and collaborate on projects with teams.

Projects
 
Sprints

Planning and tracking tool for scrum teams.

Sprints
 
BugTracker

Automatic bug tracking software for managing bugs.

BugTracker
 
NEW
Solo

The all-in-one toolkit for solopreneurs.

Solo

Developer Platforms

 
Creator

Build custom apps to simplify business processes.

Creator
 
Flow

Automate business workflows by creating smart integrations.

Flow
 
Catalyst

Pro-code platform to build and deploy your apps.

Catalyst
 
Office Integrator

Built in document editors for web apps.

Office Integrator
 
ZeptoMail

Secure and reliable transactional email sending service.

ZeptoMail
 
QEngine

Test automation software to build, manage, execute, and report testcases.

QEngine
 
Tables

Work management tool to connect people, processes, and information.

Tables
 
NEW
RPA

Automate manual, tedious, and repetitive tasks easily.

RPA
 
NEW
Apptics

Application analytics for all apps.

Apptics
 
Embedded BI

Embedded analytics and white label BI solutions, tailored for your needs.

Embedded BI
 
NEW
IoT

Build, deploy, and scale IoT solutions for connected businesses.

IoT
 
DataPrep

AI-powered data preparation service for your data-driven organization.

DataPrep

IoT

 
NEW
IoT

Low-code IoT platform and solutions for connected businesses.

IoT

Search Result

 
CRM Plus

Unified platform to deliver top-notch customer experience.

Try now
CRM Plus
 
Service Plus

Unified platform for customer service and support teams.

Try now
Service Plus
 
Finance Plus

All-in-one suite to manage your operations and finances.

Try now
Finance Plus
 
People Plus

Comprehensive HR platform for seamless employee experiences.

Try now
People Plus
 
Workplace

Application suite built to improve team productivity and collaboration.

Try now
Workplace
 
Marketing Plus

Unified marketing platform for marketing teams.

Try now
Marketing Plus
 
All-in-one suite

Zoho One

The Operating System for Business

Run your entire business on Zoho with our unified cloud software, designed to help you break down silos between departments and increase organizational efficiency.

TRY ZOHO ONE
Zoho One
Zoho Marketplace

With over 2000 ready-to-use extensions across 40+ categories, connect your favorite business tools with the Zoho products you already use.

EXPLORE MARKETPLACE
Marketplace
Skip to main content
Zoho Academy

5 Guidelines for A/B Testing Your Email Campaigns

  • 7 Mins Read
  • Posted on August 19, 2020
  • Last Updated on October 29, 2024
  • By Mason

Out of all the different platforms available, email marketing has been shown to consistently generate among the highest returns on investment. Some businesses get as much as $44 for every $1 spent on their campaigns, and gain 40x as many customers as Facebook and Twitter combined.

This incredible return on investment is due in large part to how relatively quick and inexpensive it is to create and send out mass email campaigns. However, with so many other businesses engaging in email marketing, it can be hard to get and keep your audience’s attention. Without some kind of analysis, it can be almost impossible to tell why one email was successful, and another one wasn’t.

Enter: A/B testing. A/B tests utilize a simple but effective experimental model to determine what changes will get better engagement from your prospects.

If you’re interested in experimenting on your own emails, or if you’ve started testing but aren’t sure where to go now, take a look at these guidelines to get your A/B tests up and running:

Develop a strong hypothesis

One of the major factors that sets apart email A/B testing from other A/B test subjects is the wide variety of success metrics you can choose from. By contrast, web A/B tests share the same variables for success—conversions (usually to sales, sometimes converting to email leads) or clicks. Whichever version of your webpage gets the most is considered successful. But how do you measure which email performed best? When conducting an experiment, you’ll have data on open rates, clicks, subscription changes, drop-off rates, attachment downloads, and conversions.

It’s always important to craft an experimental hypothesis before testing, but it’s especially important with emails. The amount of data available means you’ll have to take extra care to define your specific goals before you start running tests. Otherwise, you can quickly end up lost and frustrated before your A/B testing journey has even begun.

A strong hypothesis consists of three essential parts:

  1. Which variable you’re modifying
  2. The specific outcome you’re expecting (which metric will increase/decrease)
  3. The rationale for why you think the change will have the expected impact

Here’s an example hypothesis:

“Our customers are unsubscribing because they’re receiving too many emails from us. If we send out our promotional email series at a rate of 1 email a week, rather than biweekly, the rate of users unsubscribing will decrease.”

Describing the reason for the change is important. Doing so allows you to evaluate the assumptions you have about your audience, slowly replacing them with facts.

Boost your open rate

Email marketing can seem like a bit of a paradox. You’ve been given a direct line into your prospects’ inbox. At the same time, you’re competing with dozens of other businesses who have the same level of access that you do. In order to win your prospects’ attention, you need to get them to open your email based on whatever you can squeeze into your subject line and preheader text. Increasing your open rate is the foundation to email marketing success. It means you’re getting your virtual foot in the proverbial door.

There are the four main variables you can test to make your unopened emails stand out in your prospect’s inbox. Here they are, along with some suggestions for improving each of them:

Subject line

  • The most common factors affecting the success of your subject line are the length, the content, and whether you’ve included your company name.
  • Ideal length: around 60 characters max for full visibility on mobile, but this can differ somewhat between screens and platforms.

“From” field

  • A common test you can run is if your audience responds better to generic professional branding (sales@zylker.com) versus a person’s name (tyler@zylker.com).
  • There’s a surprising amount of room for creativity here. You can have your email name reference a specific product (shoes@zylker.com), or have it serve as a friendly greeting (hello@zylker.com).

Preheader text

  • Try removing generic salutations that take up space, like “How are you?” Make sure that your preheader text matches the tone of your subject line.
  • Ideal length: mobile preheaders are typically displayed at between 30 to 55 characters, and up to 130 on a desktop browser.

Personalization

  • Get your readers’ attention and add extra flair to your visible text by customizing it with prospect data.
  • You can use personalization elements to include your recipient’s name in the subject line or preheader, or even the number of days since their last purchase.

Establish your testing strategy

Compared to other aspects of online marketing, creating emails is quick, inexpensive, and easy to implement. It’s much easier to be ambitious with your email experiments, compared to your homepage or online store. However, if you’re planning on making significant changes to your emails, like a drastically different design or copy approach, you need a plan. Before you start your first experiment, you need to determine whether your plan is based on strategic testing or optimization testing.

Strategic testing

Strategic testing consists of drastic style or language changes. The idea is that these changes can potentially help you figure out a better long-term communication strategy for connecting with your prospects and significantly increase your engagement.

This plan works best if:

  • You’re either just starting out your email marketing operations
  • You have strong evidence that you’re missing out on much more engagement than your current strategy is yielding

Otherwise, you take the risk of alienating your readers.

Optimization testing

Optimization testing is the far more common testing strategy. It should be used when you’re confident in your overall technique, but want to optimize reader response by tweaking small variables.

Creating a plan

Whichever approach is the better fit for your situation, you need to commit to it, and plan it out carefully. If you choose strategic testing, you should make a detailed document that explains the following:

  • The qualitative differences between your new and current strategy
  • Why you think the new strategy will perform better than the current one
  • The areas you’re making the changes in
  • What metrics you think will improve through this test

For example, you could discover that your audience responds much more strongly to a casual writing style than a professional one. However, if you’re not precise in documenting your approach, you may not be able to replicate your success.

When laying out your testing plan, you should always aim to transition to optimization testing as quickly as possible. That way, you can both fine-tune your new technique and avoid giving your audience whiplash from too many drastic changes.

Avoiding drop-off

In email marketing, you have the advantage of a captive audience, in the form of your subscriber list. This can be a source of stability, compared to the fickleness of social media algorithms and organic traffic. Part of that stability comes from emails being a linear, one-dimensional medium. Unlike your website, where prospects can browse any combination of your pages in almost any order—and still never go where you want them to—you can control what they see and the order they see it in. Prospects receive your emails in exactly the order you send them, and they either open the email or don’t.

This aspect of email marketing is a double-edged sword. The increased control allows for more stable experiments, but the linear nature of email means that changes made to an earlier email can unexpectedly impact the effectiveness of later emails in the series. To avoid accidentally setting off an unwanted chain reaction, you should prioritize testing emails that already have a consistently high level of engagement. That way, you can get faster, more conclusive results to identify any drop-off points before it’s in circulation for too long.

This is why it helps to monitor your campaigns’ performance long before you start experimenting. It helps you to establish a baseline level of engagement that can serve as a benchmark. Even if you’re only making a small change to one email in a series, make sure you’re paying attention to everything downstream. This advice is applicable across the board, but it’s especially important if you’re testing the first email in a series or sending out an educational or content marketing series.

Audience segmentation

One of the benefits of email is the ability to control the demographics of your test subjects. Even with basic data collected from your subscription form or a previous purchase, you can segment your email list into broad demographic groups. From there, you can start with basic forms of personalization, like localization.

When localization is executed correctly, your users shouldn’t be able to notice that you’ve done anything at all. For instance, you can use their city/state information to divide your audience up into geographic territories. That way, instead of sending your whole list an email featuring the #1 product in the entire US, you can send them personalized emails that show the #1 product in their region.

If you want to determine the best time to send out your emails, don’t send them all out simultaneously. For more consistent results, you can send them out in batches. That way, they arrive at the same hour of the day in each of your prospects’ respective time zones.

Make sure your signup forms have 1-2 questions, as well. You can ask about their business or reason for interest in your brand during email signup or checkout. That will also help you do more finely targeted testing.

Email marketing has massive potential to help your business grow, but it can be an uphill struggle if you don’t create a strategy. However, with a solid grasp of the medium and powerful techniques like A/B testing, you can hone your skills and stand out among your competitors.

Related Topics

Zoho products mentioned on this page: