Every month, traditional publishers lose market share to indie authors. We cover it almost every week on our other podcast, Author Update. Traditional publishing has been walking the slow, long path to decline as indie authors make steady gains in market share.
So why are traditional publishers losing to the indies?
Traditional publishing has a HiPPO problem. HiPPO is an acronym from corporate America, and it stands for the Highest-Paid Person’s Opinion. In traditional publishing, decisions are either made by a committee or by the HiPPO, and neither is good at guessing what will work.
Many indie authors don’t fare much better. In fact, they’re often simply relying on their own preferences when choosing covers, blurbs, titles, and categories.
Not so with savvy indies. Those market share gains are happening because a handful of savvy indies don’t rely on their own opinions. Instead, they rely on science. They use the scientific method to discover what works. Those authors are selling tens of thousands of copies, sometimes millions, often quietly.
The specific scientific method we’re talking about today is split testing.
What is split testing? How can you use it to supercharge your sales?
I love interviewing authors who are willing to share their secrets, so I asked Steve Pieper, a USA Today bestselling thriller author who teaches authors about split testing, to share his secrets. He’s the founder of AMMO and Click Testing for Authors.
What exactly is split testing?
Steve: It’s a method you can use to figure out what does and doesn’t work to get people excited about something. That something could be the idea for your next book, or ideas about character, conflict, and setting. It could be the headline for your book advertisement, the image in that ad, or your cover.
We all have strong opinions about what we like. We see something, and we either like or dislike it instantly. Plenty of things fall in the gray area, but in the main, we have opinions. There’s nothing wrong with that, except that our opinions as authors rarely match the opinions of the people we’re hoping to sell books to. In fact, it’s less than 5% of the time.
Testing lets you throw away 19 of 20 ideas that won’t work and keep the one idea in 20 that is likely to sell books. If you’re testing your ideas before writing your next book, that one idea in 20 is the one that will get people excited to read. When they read, they’re more likely to have a satisfying experience.
It’s a way to transcend our own ill-informed opinions and create books worth buying, books people love and share with their friends.
Thomas: Authors often have to fail with their first book before they’re willing to give science a chance.
Most authors believe the rules don’t apply to them. Others have bad taste, but they know what works and what their readers want. They may not know who their readers are, but they know what they want.
One time out of 20, they’re lucky, and they do pick a cover or title that works, but they are rarely lucky across the board. They may have a good cover but not a good title.
When you keep stacking those one-out-of-20s on top of each other, it ends up being a one-in-a-million chance that they get it right relying on their own preferences.
Science is not a destination. It’s a process of testing hypotheses against each other. Instead of saying, “I like this cover, and I don’t like that cover,” split testing allows you to put both covers in front of potential readers and find out which one gets more clicks, which title gets more clicks, and which blurb draws more people into the story.
Even a little testing can break you of preconceived notions. Superstitions spread. The only way to disprove superstitions is to teswt That unique, beautiful cover doesn’t get as many clicks as the cover that clearly fits with all the other covers in the genre.
How often do you guess wrong?
Steve: I’m the poster child for strong and wrong opinions about what people should like. For years, I tried to get my readers to buy from a sales page I liked, and for years they would not. They would only buy from the sales page they liked.
I kept using the pages I liked, thinking maybe their taste had improved since the last test. Usually it was my taste that was out of step with theirs. It’s useful to understand early in the game that you’re on the wrong path. You can keep going, but you’ll waste a lot of money.
We’re not just testing A against B. We need to test a lot of ideas. For advertisements, and for book ideas, we want to test a dozen ideas at a time over a couple of days, and we want a good portion of those ideas to sit a bit outside our comfort zone.
When we’re creating the ideas to test, we’re still subject to our own biases.
Many times, I’ve almost rejected a particular image or hook or headline because I thought, “I don’t like it,” or “I feel a little squeamish about this.” There was nothing wrong with it, but it was outside my personal comfort zone.
It’s shocking how often that variant came back as the one my audience liked best, and it’s shocking how often authors in our community have the same experience. That’s what makes you a believer. You say, “I didn’t think one person would click on that, but most of them clicked on it.”
Testing keeps you from wasting years of your life, and it’s most powerful when you’re working on your next titles. It’s exceptionally rare for any author to build a profitable business on the strength of one or two or even three titles.
The only authors in our community making seven figures a year on the strength of one or two titles are nonfiction authors, and they are extremely rare. For novelists and for the vast majority of nonfiction authors, you need to create products where the same customer wants all of your books.
The worst thing that happens is when authors go off in a vacuum and write five books, then come to us and say, “I’ve got these five books. I think they’re great. I love the covers, I love the stories. Now it’s time to conquer the world with sales.” It’s heartbreaking, but it just doesn’t happen.
What writing faux pas can split-testing prevent?
Thomas: It’s tragic when they spend years writing those books in a vacuum without any feedback from readers. Maybe they had a beta reader or an editor, but not much more than that.
Most authors haven’t done a single split test. That first split test will help you realize you have no empathy for your reader. You have false empathy, which is thinking your reader is like you, and that if you like it, your reader will like it. As soon as you run that first split test, you will be humbled.
I’ve been podcasting forever, and I’d like to think I know which episode titles will do well and which ones won’t. But I don’t. On Author Update, we just had one of our segments go crazy viral, and I told the team, “I don’t know anything about anything.”
For some segments of that show, I sweat and bleed all week. I read the Pope’s entire encyclical on AI to break down what it meant for authors, and it did okay, but it didn’t go gangbusters. Then we did a segment on cuisine in different parts of the world and what food means to those cultures, and we connected it to the World Cup and to reader preferences. That stupid segment I put so little effort into is getting zillions of views, and I’m sitting there thinking “I don’t know anything about anything.”
The only way to learn is to try stuff and experiment. If all of your tests are really similar to each other, it’s hard to find what works, because they’re all basically the same thing. The weirder and more diverse your tests are, the faster you learn and the better sense you’ll get of what makes a reader say, “Yes, finally.”
Everybody has that feeling, and it was probably that feeling that got you into writing. You thought, “Nobody’s writing a book for me, so I’m going to write it myself.”
You have to write that kind of book for your reader. Once you’re writing the book for your reader, they say, “Yes, finally,” and they won’t shut up about your book. They tell all their friends, and they’re all fired up.
What should you test first?
Steve: It depends on where you are in your career. If you’re considering becoming an author, you should split test your ideas for the book you want to write. There’s a lot of resistance to this, because we feel like we’re selling out by doing what’s popular. That’s not the case. It’s still your magic, your voice, your particular take on the subject. You’re just allowing your readers to guide you toward what they want to read.
Thomas: You probably have a dozen. You’re not letting your reader pick what you write. You’re letting your reader pick what you write next.
Maybe you’ll write those other 11, but after you write that first book, you’ll probably get another dozen ideas, and you can test those. It’s a good way to save yourself from accidentally picking an idea whose time hasn’t come. Maybe in 10 years people will be ready for it, but right now it’s just not in the zeitgeist.
You’re not sacrificing artistic vision. You’re making life easier on your future self, because you’ll be selling books people are already primed to want. They have desires in their own hearts that they want fulfilled.
How do you test a book idea before you write it?
Steve: I’ve been doing business online since 2003. I’m a nerd with an engineering background, so I’ve had a testing philosophy from the get-go, and I’ve done all sorts of tests in many scenarios on different platforms.
The fastest, cheapest, most effective way, and the way that works most reliably across the broadest spectrum of products, services, and books, is Meta ads.
Meta works so well for testing because it’s, by far, the best platform to advertise on if you want to sell things. It’s almost three times better than Google in terms of where the world’s biggest and most successful e-commerce businesses spend their money. That’s where customers are. No matter what you think about Facebook and Instagram (and I probably think the same things you’re thinking), that’s where readers are, and that’s where we should test.
How do I begin to set up a split test?
Steve: The quickest way to set up a test is to first create your 10 ideas. In our process, if you’re making an advertisement for your novel, the first thing we’ll have you test is a tagline. A tagline is a pithy synopsis of what’s ahead in the book. “A long time ago in a galaxy far, far away” is a pretty decent tagline, and it resonates pretty well.
Think of a dozen taglines, then run a dozen cheap, quick ads where every ad is a different idea. Put them in front of people who have a high likelihood of being readers in your genre. Those people will either click on the idea because they’re interested, or they won’t click because they’re not.
At the end of a one- to three-day window, for typically $30 to $50 per test, you’ll have a rank-ordered list of your market’s opinions on your ideas.
That’s a beautiful starting point.
It’s not enough, because your best idea still has to be good enough. If it isn’t, you repeat the test. If your top ideas meet a certain threshold, meaning the right number of people who saw the ad cast their vote anonymously by clicking on it, you know that idea has a high probability of serving you well, so you carry it forward into the next test.
There are ins and outs for what not to do, so you don’t get in your own way, but that’s the idea. People on Meta are customers out in the wild. They’re scrolling, trying to escape the coffee line or escaping reality in some way. They scroll past your ad, they stop, and they think, “I’m about ready for my next book, and this sounds like a cool idea. Let me see what this is about.” That’s what you need for signal.
Can’t I just ask my friends?
Thomas: There are several problems with asking friends for their opinions. First, if they’re friends or family, their primary goal is managing their relationship with you rather than maximizing the economic output of your book.
Second, because they’re your friends or your fans, they will give your cover or copy far more attention than that title or blurb will ever get in the wild. We’re not trying to raise an animal that can’t live in the wild. We’re trying to raise an animal that can survive in the wild and thrive. It’s the lion that eats the other lions, not the lion that gets kicked out of the pride.
Your book title has to be tough enough to withstand the kind of competition that Meta is really good at reproducing because it’s such a noisy, chaotic, algorithmically generated environment. All of the things that make it a terrible place to spend time as a social media user make it a great place to spend time as an advertiser.
Steve: It’s where all the customers are. The other problem with asking people you know for their opinion is that you’re only asking them because you know them. They may not even read in your genre.
My wife read half of one of my 12 novels and couldn’t read the rest, because I write dark, gritty thrillers and she wanted to read a romance. Her opinion isn’t relevant to my genre. Asking her to read it was based on our personal relationship, not on any relevance to what readers in my genre want.
Anytime you’re asking a personal friend for input on your business, it’s probably a mistake. You need to be testing to see what your potential customers respond to, because they’re your bosses.
Thomas: This is why Meta is better than Google for testing. Google is a really good place to advertise for certain kinds of products. If you’re a plumber, Google will probably get you more plumbing business than Meta. Your Google ad is looking for somebody who’s already in a certain state of mind to make a certain decision.
With Meta, you can target a certain kind of person who has a certain psychographic and demographic profile and interests aligned with your topic, which is exactly what you want.
If you’re writing a book for moms of toddlers, Meta can give you moms of toddlers. You don’t want moms of teenagers, because they’re dealing with different challenges and they have different questions. Getting specific feedback from that specific audience is really helpful.
This technique is actually an old screenwriter technique. In Hollywood, they call it “writing the poster first.” Before they write a screenplay, they’ll create a one-page pitch version and take it to all of their Hollywood parties and pitch this short cocktail version until somebody is interested and says, “Give me a treatment on that.” Then you write it. You may have a different idea for every party you go to.
Our Patron Toolbox has tools that can help you with creating a blurb or pitch. The tool everyone likes to use is Book to Blurb, where you upload your book, and it creates a pitch, a one-liner, and back cover copy. The more useful tools are the pitch tools, where you answer questions about your protagonist, antagonist, and setting. Different tools ask different questions, and they’ll create initial pitch ideas for you off of a book you haven’t written yet. You just need a vague idea of where you want to go with the story.
Being forced to answer those questions ahead of time may save you a ton of time writing the story. It’ll save you even more time if it turns out that story doesn’t resonate with the audience.
How can AI help with testing ideas?
Steve: Those kinds of tools are amazing for ideation. If you’re using generative tools, and I know there are a lot of philosophical, ontological, and moral questions around them that everybody needs to answer for themselves, they are amazing for producing a large number of pretty high-quality ideas.
They also still have no idea which one of those ideas is best.
Imagine there are 100 true and accurate ways to describe your protagonist, your setting, or your conflict. The real question is which among them are most compelling for readers. The problem authors have with these tools is that we end up with a cornucopia of amazing ideas, we love all of them, and we do the same thing we always do. We pick our favorite.
In that vast expanse of ideas, it’s even more important to pick ones you don’t like and to test a whole batch of them at once. I’ve been doing this for 20-plus years. I am twice as good as average at predicting what’s going to work, and that means I’m only wrong 90% of the time.
I’m always surprised. In fact, I’m almost surprised when I’m not surprised by a test outcome. Once you know your audience over time, you’ll get a sense of it, but you still have to challenge yourself, because attitudes shift underneath you. When you use those tools, test the output to see what real people who don’t give a hoot about you think about your idea.
Thomas: This can be as simple as testing an email subject line. I
am continually humbled when I split-test email subject lines against my team. I think, “I’m Thomas Umstattd. I can write the best email subject line for my own email.” Somebody on my team may have a better one, so I’ll run a split test, and far more often than I’m comfortable with, I lose the stupid split test.
Testing ideas is humbling.
I’ll have a Q&A call with patrons, and people ask me what they should do. I’ll often say, “You need to test this.” Something I’ve started saying more and more is, “When Thomas and your readers disagree, listen to your readers, because that’s where your money comes from.”
There are things that are generally true but not specifically true, and that specific application depends on your book, your genre, and your target readers.
How do you sell when a customer doesn’t need what you’re selling?
Thomas: When I was a kid, my parents got me a copy of the Guinness Book of World Records, and I was a nerdy young teen reading it.
They don’t normally put firsts in the Guinness Book of World Records, but this one had the first person ever to sell a refrigerator to an Inuit. In the ’90s, people would say, “That so-and-so is such a good salesman he could sell a refrigerator to an Eskimo.”
My book didn’t say how he did it, so my whole teenage life, I had this puzzle in my head. How would you convince somebody who lives in an igloo to buy a refrigerator, assuming no deception or shenanigans? You would have to sell it on some other reason.
You could sell it on the heat it generates on the back, or as a status symbol, or on the optimized shelving for storage.
Suddenly you have to start thinking of ways of describing a fridge where the primary feature isn’t the feature you sell. Most people buy a refrigerator to keep their food cold. For an Eskimo in an igloo, keeping food cold is not a feature. They have cold at home.
I still don’t know how the guy did it, and I’m curious what pitch he used. It’s a great mental thought experiment, because it helps you realize that your readers are far more like those Inuit than they are like you. Learning to put your head inside their head, and then testing to see if you’re accurate, is so important.
What’s the most important skill in publishing?
Steve: We all know it takes persistence and a lot of hard work to succeed in any business. That’s a given. The quality that is most important in publishing is humility. It’s not thinking you know what people want and like, because statistically we don’t.
We’re so deep into our genre, we’ve cared about it so much that we’ve written books. By definition, we’re really different from the millions of people out there who just read the books. Our opinions matter to us, but they are not at all representative.
If it were possible to predict successful books and successful marketing processes in advance, trad pub would not have a 20 to 30% success rate. They’d be batting 1,000. Ten out of 10 books would be winners.
They built massive empires on a 20% success rate. You’ve got to be pretty humble about your ability to pick winners when you’re wrong 80% of the time. Humility is even more important for indie authors like me who are wrong 19 times out of 20.
That’s okay, because we can be wrong quickly and cheaply. Tomorrow, literally tomorrow, we can be right and move forward.
Thomas: Being wrong quickly and cheaply is so important. That traditional publisher has a 20% success rate, and depending on the publisher, I’d say it’s a 20% break-even rate. Roughly 80% of their money comes from about 1% of their books.
Steve: Success means to “earn out” and break even on the advance. That’s it.
As an indie author, if you don’t have the persistence to write five books to get one successful book, you’re going to give up on this dream long before you get there.
The default path for people who don’t know what they’re doing is to do all the work of writing a book, then all the work of publishing it, and then try to find a market for it. That path is going to exhaust you long before it funds you.
Thomas: This whole approach is about making failure faster and cheaper.
If all you’ve invested in the idea is 30 minutes with the Patron Toolbox, generating 100 ideas with the pitch tools and picking your 20 favorites to test, you’re not crushed when 19 of them fail utterly. You’ve made failure cheap. One of them might pop, and you find one is getting clicks at a fraction of the cost and at a much higher rate than the others.
Now you’ve got a book idea people are excited about, and you have some confidence in it, so you can start exploring it.
What do you test for books you’ve already written?
Steve: Most authors who want to start testing have already written a book or several.
Tagline
If you’re creating ads for books you’ve already written, the most important element for fiction is a compelling tagline that hooks readers into the story. Ideally, that tagline works across the entire series. Then, pair it with a headline tailored to the specific book you’re promoting.
Testimonials
The third test is for testimonials, or the reader reviews you have for your books. Not all reader reviews are created equal, and you want to know which ones are most compelling to complete strangers who are considering whether to read your book.
Image
Once we know these things, we start testing the image displayed on the advertisement. Our brain has something like 60,000 times the real estate devoted to digesting visual information as it does to decoding language. So much is communicated about vibe, environment, feel, genre, and expectations in a millisecond when someone looks at your image. We can also include winning headlines, taglines, or testimonials as overlays on the image to drive the point home.
Description
The fifth test is for the description, the longer wording. Not everybody reads it, but many readers who eventually buy your books do take the time to read the text in your Facebook ad, and sometimes Instagram.
You have to be careful, because there are ways to cheat. Put a cat video in one of your tests, cats doing something stupid, and you’ll get clicks to the moon. The cat video will beat your tagline or your headline or your cover candidate hands down. Cat videos will not sell any books for you.
How do you ensure you’re running a clean test?
Steve: We’ve found lots of ways to screw these tests up, so you must keep yourself from getting in your own way. If you stay within those guidelines and follow the process, you’ll have a pretty solid advertisement at the end.
We’re doing this quickly and relatively inexpensively, and we don’t want to give the algorithms too many chances to learn and start selecting individuals to see these ads. We want to give the algorithms few enough impressions that they’re still pretty dumb, which makes a more random sample, and a random sample is what you want. You want to know, “What does the average Joe in my market think about these books?”
Stress Test
The final thing we do is a stress test. It’s an idiot check. Did something sneak through all these tests? Did we wind up with an ad that’s actually not likely to work because of false positives? We try to suss those out in the stress test.
Those six tests together are the ones we use to create advertisements for books and series that authors have already written and now hope to get out into the world, make an impact, and make a profit.
What do you test for books you’re currently writing?
Steve: If you’re working on writing a book, it’s different.
Characters
You want to test the protagonist, antagonist, and the conflict.
Tagline
You certainly want to test the tagline you’re thinking of, because a tagline is a compact and powerful way to communicate what the thing is all about, along with the style, the vibe, the angle, and the fun you’re bringing to it.
Setting
You also want to test the setting. Is it a historical setting? Where? Is it a thriller? Which country?
These things matter in ways you don’t really think about. Australian authors writing in Australian settings sometimes have a hard time selling books in the largest English-speaking book market on earth, which is the United States. You want to test that in advance.
Every one of these key elements you test means people will be more excited to experience the book you’re about to write.
Should you be running A/B tests?
Thomas: In an older Novel Marketing episode, I recommended A/B testing to keep it simple, partly because it makes the math easier.
In the internet marketing world, split testing was prominent, but in the publishing world it was a voice crying in the wilderness. I spoke with a vice president for a major publishing company about split testing 15 years ago, and he just blinked at me and said, “That’s a really interesting idea.” He had never even heard of the concept, and he was managing million-dollar budgets.
A/B testing means you test version A and version B to see which performs better. If you want to test a bunch of ideas, you have to create a tournament bracket system. With multivariate testing, you rely more on Facebook’s ever-improving AI engine to test lots of variables quickly.
Do you use A/B tests, multivariate tests, or something else?
Steve: Multivariate testing is when you have two versions of the same page, and you’ve made a number of changes between them. If you change the image, the headline, the subhead, and the call-to-action button on version B, you don’t have an A/B test so much as a multivariate test.
We don’t recommend that, because some of the changes you made could have improved the conversion rate while other changes made it worse. You have no idea which of the ideas you tested is a keeper and which is not. If you’re hoping to make heads or tails of a multivariate test, you can expect to spend roughly 10 times the money, maybe more. In books, the specific words matter, and this is true in marketing in general. If you’re changing a bunch of words from one page to the other, you don’t know which changes helped and which changes hurt.
We don’t call what we do A/B testing, and we don’t call it multivariate testing.
We call it click testing.
With click testing, you run a number of independent tests, sampling totally different random humans for each of these little tests, but you’re doing it all at the same time.
You’re not asking any one person to pick between three ideas. You’re showing each of your ideas to 400 different random people, and they’re almost never the same people. Sometimes there’s unavoidable overlap, and it’s okay.
You run a bunch of independent tests at the same time, which we can get away with because there are tens of millions of people in the reading audience for most novelists, and we’re only asking for a few hundred impressions at a time.
We run random samples of 10 to 12 ideas over a couple of days. At the end, the samples are tallied, and we get a rank ordering from best to worst, based on what percentage of people got excited enough to click.
We’ve done this a lot. I’ve run or consulted on 6,000-plus tests over 50,000 variations.
Thomas: And he still only gets it right one time out of 10. That’s why testing is so important.
Steve: I’m only wrong 90% of the time.
We have a cutoff. It’s got to be better than a certain number. If it is, you have a winner. If not, think of different ideas, or different wording for the ideas that did best. Sometimes just a word change or two is what gets the picture.
It’s super important not to test more than one thing at a time. As soon as you test more than one idea in the same piece of content at the same time, you make it really hard, confusing, and math-intensive to make heads or tails of your test.
Stay away from multivariate testing. The reason we don’t do A/B testing is that if we were wrong only half the time, A/B testing would work. We’re wrong a lot more than that, so we need to test a lot more ideas at a time to have a shot at finding ideas that are really likely to work in a reasonable amount of time.
Don’t think of it as testing different variables. Think of it as testing one idea on several hundred random people, and at the same time testing a different idea on several hundred other random people, all in the same market. Then we can easily compare the statistical results.
That’s how we get around the hard math problem, and it’s how we get around the problem of so many of our ideas being crap and only a small number of them being any good.
Thomas: In some ways you could say it’s an A, B, C, D, E, F, G test where you’re only testing one variable.
A classic multivariate test is when I have three cover ideas and three different titles, and I want to test the combination of those covers and titles. The challenge statistically is that the cover may have 70% of the impact on the overall results and the title only 30%. Now this title is performing better paired with a B-level cover, and doing the math gets really complicated.
What you’re talking about instead is testing a dozen covers against effectively the same target group of people. It’s like having the exact same race, but you have 12 runners on the track. They’re all running the same race, and you just see who gets to the finish line first, rather than trying to compare points between the person doing track and the person doing the long jump, and whether a silver medal is worth the same as a gold medal.
Steve: You still have the problem of understanding the best combination of image, headline, tagline, testimonials, and description. The faster, cheaper, more reliable way to do that is not by doing them all at once. Instead, you do it sequentially.
Each element is important, and you want to move forward with the best headline you have found, then proceed with the best testimonials in the context of that headline, and the best-performing images.
When you stack up five different variables, you’re talking about tens of thousands of required impressions to make a mathematically meaningful determination. An impression is just any time a human views the thing you’re testing. Doing it all at once, in one big soup of variables, is only doable when you have massive amounts of really cheap impressions.
It’s far easier, cheaper, and faster to do it sequentially.
If we go sequentially, every new test after the first one starts with knowledge of what worked in the past. We’re fitting together new elements that perform well in the context of the highest-performing earlier elements.
We’re getting a little nerdy here, but this matters.
It’s especially tempting when people are working on their sales pages, selling directly to readers from their own Shopify store. They want to make a dozen changes on the sales page for the next version, but then you don’t know what made it better and what made it worse.
You have to test one thing at a time. Only one thing at a time. We call that clean data. Keep your powder dry and your data clean.
What mistakes do authors make when they split test?
Dirty Data
Thomas: One mistake is dirtying the data by making the test more complicated than it needs to be, so you’re not sure what you’ve learned.
Insufficient Sample
Another mistake is not getting enough clicks to matter. You run this big test, and as soon as you get a click, you pick the winner. This one got the first click, so it must be the best.
Too Few Ideas
Steve: The most serious mistakes relate to not using a diverse enough field of ideas in your tests. You might find the best mild variation of your first idea, but that may be much worse than the best example of a totally different angle you could take on your story.
We tell people to find four angles and three versions of each angle, so a total of 12 things to start testing.
Liking All Your Testable Ideas
The other really important rule is that you should feel slightly uncomfortable about at least a third of them. I’m not saying be racy or push the rules or violate your own principles. I’m saying include the stuff that makes you think, “I don’t like that. I don’t know any kind of person who would like that.” That’s right, you don’t know any kind of person who would like it. Maybe the whole market you’re writing for will like it.
Stopping Too Early
The second mistake is cutting off tests too early. The only way to know what happens when 400 people see an idea is to let 400 people see the idea. Clicks don’t space themselves evenly. Sometimes you’ll get no clicks in the first 50 views and seven clicks in the next 50. You have to let the whole test pan out.
Testing More than One Variable
Test one variable at a time. There are specific scenarios in which we violate that for a specific reason, but by and large, if you’re testing more than one variable at a time, you’re really likely to be confused.
Ignoring the Results
The last thing is to do what your testing shows that people want. If your goal is to connect with people, give them a great, entertaining story, make a difference in their lives, and also make some money in the process, then you have to put things in front of them that are relevant and meaningful to them. That almost never coincides with the ideas you think are going to work best.
Thomas: There’s a valuable life principle here. Don’t ask questions you don’t want to hear the answers to.
If your approach to split testing is that you want to be validated on the decisions you’ve already made, and you’re going to go with what you did anyway, split testing is going to be a waste of money for you.
The purpose of split testing is not to sell books. These ads Steve and I are talking about are not an effective way of selling books. Depending on whether the book is out already and how close you are to a finished product, it might result in some sales, but that’s not the goal here.
The goal of click testing is to learn.
If you don’t want to learn anything, save your money. Just buy ads and put the book you were going to make anyway in front of people. You’ll get some clicks on it. That’s a better approach than trying to make testing work if you’re not willing to listen to what the data tells you.
We really should have opened with this, because I could totally see an author saying, “I put that idea in there just because it was off the wall. I wouldn’t actually write that book.” Only test ideas you’re willing to write. Only test covers you’re willing to put on your book.
What if they click on a cover you didn’t use?
Thomas: I spoke about split testing covers with a publisher, and he said, “Where would you send them if they clicked on a cover you didn’t end up using?” That was such an obstacle in his head that he rejected the whole idea of split testing. Sending somebody to a landing page where they wouldn’t see the cover they clicked on was a bridge too far for him.
Where do you send people when they click on the ad?
Steve: You’d be surprised how many people are hung up on that. The short answer is that it just doesn’t matter.
You can send them to a “Coming soon” page. You can send them to your Amazon page. A handful of people might have a slightly weird experience if they land on a homepage instead of a book page, but that allows you to serve everybody else much better.
If you’ve already published a book, send them wherever you’re primarily selling it.
Can click testing ads sell books?
Steve: I want to disagree with something you said earlier, which is that these ads are not going to sell books. Those six individual tests- tagline, headline, testimonials, images, description, and finally the stress test- are unlikely to sell a whole lot of books during the short period when you’re actually performing them.
The purpose of that series of tests is to produce complete advertisements that have a high probability of bringing the right reader to your sales page to buy books from you. You’ll have to spend time getting your sales page dialed in, so the experience people have on the ad is not different from the experience they have on the sales page.
The whole point of those six tests is to produce advertisements that are among the best ways to present this book to potential buyers and get them excited to come check out your book page.
We can run a different series of tests to figure out which book you should write. That’s a more important test to run, but most authors don’t think about engaging their audience until they’ve written their books and want to try to sell them. That’s where most people are, and that’s just cognitively how most people arrive.
During the testing process, you’re absolutely correct, Thomas. You’re unlikely to get rich during the two weeks you’re testing these ads. Afterwards, you should have ads that do a really great job of pulling the right people in to view your book and do sell books.
Thomas: The result is ads that are far more effective for far fewer dollars per click. A lot of people say, “Ads don’t work. It costs me $10 to sell a copy of my $4 book. I’m just losing money.” If the first ad you ever built is the ad you’re trying to be profitable with, it’s not going to work.
Here’s one metaphor to leave you with. If you’ve seen the movie The Incredibles, Syndrome is trying to develop a robot that can defeat Mr. Incredible. He keeps developing robots and testing them against different supers, and the robot gets destroyed, so he makes a better robot, and then a better one. The goal is a robot capable of defeating Mr. Incredible, because that would be a good robot.
As an author, you’re taking the same iterative approach, where you take what you learned on the last test and move it forward into the next one.
Where can you learn to do this yourself?
Thomas: We have just scratched the surface, because to do testing well you have to be fluent in advertising and in testing.
I avoid teaching how to run ads because I hate having to update my videos every time Facebook makes an interface change. When I make a course, I want it to be evergreen, and I never want to test that course again. You can’t do that with an advertising course, so I’m happy for people to pay you. I’ve had almost every person who has an ads course for authors on the podcast, and I tell listeners to find who they like best and go through that person’s course.
Steve: You are 100% correct. The bane of my existence is Facebook changes, because our programs go click by click. We teach you the fundamentals of what is most likely to work for each of these elements, realizing that nobody can predict in advance exactly what will work. If it were possible, I wouldn’t be doing podcasts. I’d probably be off on my yacht someplace.
You can’t predict in advance, so you have to do the testing, and you have to do it in a way that doesn’t get you flawed results, in a way that gets you results you can take to the bank. That’s a click-by-click process.
Our introductory course, for every author selling from zero to $3 million per year, is called Click Testing for Authors. We test a dozen ideas in parallel over a one- to two-day interval, move forward with winners only, and end up with high-quality ads.
We were also the crazy people out in the wilderness in 2017 saying, “Amazon isn’t the be-all and end-all. I’m going to sell directly to readers.” If your catalog supports it with a large enough number of high-quality books, selling directly to readers can produce up to twice the revenue for the same advertising dollar.
It’s a powerful way to improve your immediate cash flow, and it’s also the most reliable way to improve your Amazon royalties, believe it or not.
That course is called AMMO, Author Marketing Mastery through Optimization. The first portion of that direct sales course is also Click Testing for Authors. Whether you’ve never sold a book in your career or you’re looking to move into selling directly, the foundation for everything is Click Testing for Authors. It keeps us from making big mistakes, which are expensive and time-consuming, by putting the wrong things in front of people and hoping they click when they usually don’t.
Should you sell direct?
Thomas: Joanna Penn and I have had a slight disagreement about selling direct, but I’m not against selling direct. I’m against selling direct if your library doesn’t support it, and Steve gets it, because he opened with that clarification.
If your problem is that no one is buying your books, selling direct just adds cost and complexity to your life without solving the problem that no one knows about your books.
If you have 5,000 readers who buy everything you write, and you believe the total addressable market in your genre is only about 7,000 readers, your growth potential may be limited. In that case, the solution may not be finding more readers but earning more from the ones you already have.
Selling direct can help you do that. With the same number of sales, you can earn significantly more per book, potentially moving from part-time to full-time authorship. You also gain a stronger, more direct relationship with your readers.
For the author who’s getting sales but thinks, “I’m sharing so much money with Amazon, and I wish I had data on which books people were reading in which order,” there are a lot of benefits to selling direct.
You’ve got to have the readers first, which is why I like that you opened with the ads. The more money you’re making, the more money you can spend on ads and on testing ads.
I wouldn’t recommend selling direct for somebody with one book, unless that book is some humdinger of a nonfiction book that just sells every single year, like the book What to Expect When You’re Expecting. Every year more women get pregnant and want to know what to expect when they’re expecting, and you’re selling books every year.
How many books do you need before you can quit your day job?
Steve: That’s absolutely right, and it is really rare. We have two authors out of 3,500 in our community who have sold more than seven figures worth of a single title. Two authors out of 3,500. Those are not great odds. You’re talking lottery ticket odds. Your catalog has to support it.
If you write romance, it’s typically eight high-quality romance novels. If you write in other genres with word counts greater than 50,000 words, that number is typically five books. I don’t mean you go off in a vacuum, write five books, and then sit there hoping. You have to have reader feedback showing that you’ve been on the right track for that period of time. If you have one or two or even three books, it doesn’t matter which marketing method you use. Our data shows you’re really unlikely to build a profitable business on the strength of one, two, or three books.
Why is that? Bezos has held the price of books constant, particularly ebooks, for maybe 20 years now. $9.99 is about as much as anybody charges for any ebook anywhere, except for trad pub sweetheart deals. The margins on paperbacks aren’t much better, because printing and shipping costs have risen with inflation. Meanwhile, for those two decades, ad costs have doubled twice.
Your cost to advertise to get a new customer will, in every case except about two out of 3,500, grossly exceed the revenue you receive from selling one book.
That’s a really important piece of information as you’re planning your career. If you have interest in five or six different genres, that’s five or six different businesses, each of which you’ll be five products into before you start seeing anything close to a sustainable career-level income.
I bring this up because it is excruciatingly important. You have to have the right expectations for this business. It’s not one book, one and done, mansions and yachts. It’s years of blood, sweat, tears, and testing to make sure you’re on the right track.
It’s close to my heart, because it is heartbreaking when people show up with one or two books they’re excited about, and they’re ready to quit their day job on the strength of those books. But that’s not what the math tells us is likely to happen.
How do you build a successful brand that sells?
Thomas: It’s an important reminder, because a lot of people in this business will promise you that if you use their special method, their secret sauce, anybody can be an author. People take on that belief that they just need to find the right shortcut and it will make them successful.
That leads to one final reminder I want to leave you with. Being successful as an author requires two things. First, you need to make the right promise to your reader, and second, you have to deliver on that promise. As you deliver on that promise consistently, you build a brand.
Everything we’ve talked about today is helpful for making the right promise. If you’re delivering on the wrong promise, nobody wants it. You’re writing a book nobody wants. Figuring out what book to promise, which is what split testing does, tells you that this title, this angle, this character, this setting is the one. That’s really useful.
You still have to deliver on the promise. Good marketing can’t fix a bad book. This is not a shortcut around learning the craft of writing books that readers want to read.
Steve: Click testing in particular, and testing in general, will help you massively to deliver on the right promise as well. We’ve seen that if you take the time to test your titles before you write them, and as you write them, your odds of a financially successful book or series improve by 20 times, up to 100 times.
Make the right promise, then get reader feedback as you’re crafting the answer to that promise. That involves your market at both of those critical steps. If you go off into your cabin in the woods with no internet connection and write that book according to your own sensibilities, it’s unlikely you’ve hit the nail on the head with the real people out in the world who are going to buy your book.
Do the testing to figure out the promise. Do the testing as you’re executing on that promise, to make sure the way you’re executing still resonates, because in the book business, the words matter. If you use the wrong words with the right promise, you’re still going nowhere.
Please engage your market at every step. You don’t have to do our stuff, but do something to test, so that you’re not wandering around in circles wasting time, money, and life energy.
Thomas: If you don’t write the books that people want to read, readers won’t want to read the books you write. It’s just that simple.
It’s easy to develop a sophisticated palate where you’re blending all these genres, when what readers want is right down the middle of the road.
I often see authors who are frustrated because AI-assisted writers are outselling them. The reason usually isn’t that the AI-assisted authors are better writers. It’s that they’re staying in the center of the genre, giving readers what they want, and using AI to produce those books efficiently instead of taking big creative risks.
Meanwhile, the frustrated author is often trying to blaze a trail where no market exists, usually through genre blending. There’s no established road, and there may not be many readers looking to go there.
You might point to the one author who successfully combined two genres and created something new. What you don’t see are the thousands of writers who tried the same thing, wandered into the wilderness, and never found an audience.
It’s risky. Giving readers what they want, putting your sweat and blood and soul into it while giving the reader a seat at the table, will really make a difference if your goal is to make money.
Some of you don’t really care about money. It’s just a hobby, and this whole episode has helped you realize that. You’re an adult, and it’s a free country. If you want to write as a hobby, you can do that, and you can put your books for sale on Amazon. You’ll be a lot happier as a hobbyist if you’re not measuring your success in dollars and cents.
If you are measuring success in dollars and cents, or you need your book to pay the bills, you’ve got to treat your reader like the customer.
Connect with Steve Pieper
Thomas: I highly recommend what Steve is teaching.
Visit StevePieper.com to learn more about Click Testing for Authors and AMMO (Author Marketing Mastery through Optimization).
Featured Patron
B.D. Lawrence, author of Final Jeopardy: A Jake Sledge Mystery.
A woman with a dangerous past, a brother who cannot speak for himself, a murder that does not add up, and a killer who will do anything to keep the truth buried. PI Jake Sledge didn’t ask for this case, but walking away was never an option.
Related Episodes
