Amazon is an extremely innovative company – and can be quite responsive to self-publisher’s concerns – but sometimes it gets things very wrong too. Today is one of those times.
This post is from 11 March 2016. It has not been updated except to clean up broken links but the comments remain open.
I’ve received several reports from writers threatened with having books removed from sale, and heard even more worrying stories from others who had their titles actually removed from the Kindle Store without notice.
What were these authors guilty of? What crime did they commit for Amazon to adopt such heavy handed treatment? Something completely innocuous: the Table of Contents was at the rear of their books instead of at the front.
Yep, that’s it.
We’ll get to what might be the root cause of this crackdown in a moment, but Amazon is claiming that having a TOC in the end-matter instead of the front-matter is a breach of the (ever-changing, 100+ page PDF) Kindle Publishing Guidelines. Amazon says that rear TOCs result in a poor reader experience, and it has very suddenly decided to clamp down heavily on this practice, without notifying the community-at-large, even though moving extraneous front-matter to the end of the text has been fairly standard practice for years.
Some individual authors are receiving Quality Notices warning them that their title will be removed from sale unless the TOC is moved to the front. Normally these notices – which appear to be generated by bots – give us just five days to comply. Other writers are having their buy buttons removed without receiving these notices.
To give you an idea of how disruptive this can be, read the story of author Walter Jon Williams – who had his Nebula-nominated SF novel Metropolitan removed from sale during a BookBub promotion.
Can you imagine?
His buy button was eventually restored a few days later, but Amazon wasn’t finished. After he moved his TOC to the front of the book and uploaded the new version, Amazon then sent an email to all previous purchasers of the book saying that the author had now corrected serious formatting and editorial issues. Walter Jon Williams said that there were no such issues with this book – which has been on sale in one form or another since 2005 when it was originally published by HarperCollins – and the sole change he made was to move the TOC, as requested.
Needless to say, he’s not too happy at this message going out to his readers. (I should say that the author has been remarkably restrained considering the circumstances, I would be hulk-smashing in all directions.)
Note: I don’t know Walter Jon Williams but I’m sure he won’t mind if you would like to show your support by grabbing Metropolitan at Amazon, Barnes & Noble, Apple or Kobo.
Metropolitan is currently #10,436 in the Kindle Store. But, after a BookBub promo, his book should be much higher in the charts than that. Because of the way parts of the Amazon recommendation engine work (such as what is colloquially known as the Popularity List), being off sale for those several days could continue to depress his Sales Rank and his general Kindle Store visibility for some time to come.
I would also imagine that Walter Jon Williams is significantly out of pocket over this incident (the BookBub ad alone would have cost him $570), and he also missed the chance to take a run at the charts. He won’t be able to get another BookBub for this title for a minimum of six months so this is also a considerable lost opportunity.
In fact, this incident may have affected his BookBub numbers to the point where he could struggle to get his next ad. And while there are plenty of hard-working e-book deal sites which deliver good results for authors (and an infinite number of crappy sites), even together they would struggle to approach the power of a BookBub. It really is the only game in town at that level of sales.
This is serious stuff.
Aside from the awful timing in this author’s situation, any momentum in the Kindle Store is hard won. Having a title off sale for a period will cause that book to plummet in the rankings – and it may not recover until the next promotion, which could be months away. Leaving the financial aspect for one moment, when Amazon opts for the nuclear option it can cause a huge amount of stress.
Walter Jon Williams is not alone – he’s merely the most high profile case thus far because he had the incredible bad luck to trigger this clumsy, automated response during a major promo. Others also had their books pulled without any kind of notice, and many more again have received Quality Notices threatening to remove their books from sale unless they move their TOC to the front.
In other words, it’s a sh*tstorm, and one that was completely unnecessary. I’ll get into how this all started, and what I think Amazon was trying to achieve in a moment, but first I think it’s important to get a sense of scale here.
How many authors and books are affected by this? It’s tricky to say.
A straw poll indicates maybe half of my author friends put their TOCs at the back. Certainly anyone who compiles e-books with Caliber could be affected – one of the most popular tools – or anyone who uses Guido Henkel’s formatting method (which I heartily recommend). And anyone that has read Let’s Get Digital or Let’s Get Visible – because it’s how I recommend authors lay out their books; it’s pretty standard advice.
If I had to put a number on it, I would guess tens of thousands of titles are affected – if not more.
When I started self-publishing in 2011, it was considered best practice to put the TOC at the back of the book. The reasoning is fairly simple, and reader-friendly: it gives them more of the actual text to read in the sample. And when you have a non-fiction book like Let’s Get Digital, the TOC can take up considerable space.
That’s a pain for browsers to wade through and where you can lose an on-the-fence purchaser. Readers aren’t inconvenienced by the TOC being at the back, as it can be summoned with the tap of a button anyway. Further, I don’t believe it was contrary to Amazon’s terms and conditions back in 2011 – either way, it certainly has never been enforced. Until now.
What was Amazon actually trying to do? This is where the story gets worse.
One of the quirks of Kindle Unlimited is that we are all fighting for money from a fixed pot, putting us into competition with each other in a way that we aren’t normally. And KU has been plagued by scammers and opportunists – “authors” who who seek to gain an edge with unethical behavior (I hesitate to call them authors because they are often internet marketers who farm the actual writing out to someone else).
Amazon has been very slow to act. Indeed, the switch to a per-page compensation model is widely believed to have been at least partly prompted by these authors publishing junk booklets which were only a few pages long and contained no real content, but still triggered a full KU borrow payout – but that change was a year in coming and Amazon did little to combat these guys in the meantime.
The latest wheeze from this shady crew was to place a message at the start of their KU titles encouraging readers to click through to the end – because this fools Amazon’s system into thinking the entire book has been read, the author of that title then receives an inflated payout from the KU pot, and then honest, hard-working writers who aren’t pulling these cheap tricks on readers have less money to share. It’s a mess. These guys are peeing in the KU pool and Amazon is paying them by the gallon.
And it seems this is what triggered the TOC crackdown.
(Note: in the absence of any comment from Amazon, this is speculation. But, at the very least, the issue needs to be raised and the contrasting responses from Amazon should be highlighted.)
Here’s an example of one such message. They really aren’t hard to spot, it’s right in your face when you open the offending book, sample it, or click Look Inside on Amazon.
Despite how glaringly obvious these messages are – similarly obnoxious text appears in the product description too – and despite how near-identical the phrasing is across the various perpetrators, it seems that the only method Amazon could devise to catch these guys was to crack down hard on anyone who had rear TOCs – whether they were enrolled in KU or not. The worst part is that banning rear TOCs doesn’t even seem like it will solve the problem as these guys could use regular links, footnotes, or other alternative methods to funnel readers to the end of their books.
The real problem here is that Amazon has such a crude page-counting method that it doesn’t actually know which pages were read, or whether a giant chunk was skipped. Which is kind of blinking important when you are operating a hugely popular subscription model where author compensation is supposed to be based on actual pages read.
What I’m saying is Amazon can invent flying delivery robots, but can’t handle a 1990s-level internet marketer scam.
To rub further salt in the wound, Amazon seems to be taking a very relaxed approach to these scammers. Multiple threads on KBoards have examples of books which are using these CLICK HERE links to artificially inflate their KU payouts. These titles were reported to Amazon by whole bevy of authors/readers from KBoards, as well as in separate efforts elsewhere, but Amazon has been incredibly slow to take down the offending titles.
Here is one example. Last week, this title still had one of these scammy CLICK HERE messages on the first page. It was reported to Amazon some time ago, and the title was eventually removed from sale a few days back. The buy button was then restored after the author presumably took remedial action as the CLICK HERE notice is now gone. However, Amazon didn’t bother checking the author’s other books – there are two more with identical CLICK HERE messages inside (example).
These aren’t isolated incidents – and Amazon’s laissez faire approach to the perpetrators is typical too.
Here’s another title doing the same, and, as a bonus, also breaching a bunch of other rules at the same time, like keyword stuffing in the title. This one had its buy button removed and a quality notice slapped on its product page in the last day or so – but Amazon dragged its feet, taking a week or two to act, a courtesy it doesn’t seem to extend to the innocent. And this particular author has three more titles on Amazon all pulling the exact same immediately obvious tricks under the exact same name (example). Clearly, Amazon hasn’t even bothered to spend five seconds looking at this author’s other titles.
I can understand Amazon’s need/desire to automate as much of its processes as possible, but when it comes to removing books from sale, especially when the system seems to generate so many false positives and completely miss the actual culprits, then it’s time for a serious rethink. At the very least, some human intervention should be required before a book is threatened with removal – let alone actually taken down and an author’s income (and reputation) put in jeopardy.
I’m all for erring on the side of safety, but Amazon seems to be in the curious position of erring on the side of safety with obvious scammers while immediately dropping the hammer on the innocent.
As I said up top, Amazon is usually very responsive to author concerns, particularly if it is something which impacts on customer experience. In 2013, I crowdsourced a list of problems and feature requests and gave it to Amazon at the London Book Fair.
Amazon’s response was fantastic, taking action on a number of fronts, and I was extremely impressed overall. However, one problem it skipped was the issue of Quality Notices. Here’s what I said then (remember, these were issues presented on behalf of a huge group of contributing authors):
Amazon occasionally sends out emails warning that books will be removed from sale because of potential copyright issues, breaches of the Terms of Service, formatting errors, or typos. But the automated system doesn’t work very well. […] The general issues with customer service (noted above) are compounding the problem. […] It’s particularly frustrating to see this crackdown on non-issues when egregious breaches of the Terms of Service seem to go unchecked for months.
Obviously, the situation has deteriorated. This needs to change.
What should you do if you have a rear TOC? Move it to the front.
At least, that’s what I’m doing. I don’t want to go through what Walter Jon Williams did. I have a big promo starting next week and can’t take the risk of getting caught up in something like this. (Note: if you are about to buy Digital, you will save a few dollars if you wait until March 19.)
So instead of working on my next book, I will be spending the day formatting and testing and re-uploading. I presume thousands of other authors will have to do the same thing.
Update March 12:
Turns out there is a lot more to this story, all of it worrying and none of it reflecting well on Amazon. I have a contact at KDP who I emailed two days ago and didn’t get any response. Which is poor, but exactly fits with how Amazon has handled this issue.
The problem is much more serious than outlined above. And Amazon is fully aware of what is happening and is doing very little about it. The only conclusion I can draw is that Amazon doesn’t care.
So here’s what I’ve been hearing over the last 24 hours: the scammer examples I linked to are actually quite tame. The serious guys aren’t just using TOCs to inflate their page reads, but, as I speculated in the post, links to the back of the book, footnotes, and all sorts of other wheezes (like filling books with page breaks, filling books with the same text in 10 different languages – done by Google Translate – and then having a link go to the English version at the back, etc. etc.).
In other words, cracking down on rear TOCs is completely pointless and is only causing the innocent to suffer. Good job Amazon!
And these scammers are far more successful than the examples linked to above. Many have been in receipt of All-Star Bonuses – taking that money from the authors who truly deserved it. Again, all of this has been reported to Amazon. Aside from not demanding the return of these fraudulently achieved bonuses and giving it to the authors who should have received them, Amazon is failing to sanction the culprits other than taking down the individual reported book – meaning the scammers are allowed to continue using these tricks in the rest of their books (and the most successful have giant catalogs), which don’t seem to be checked.
This is basic stuff. Amazon should be checking the rest of their books, banning repeat offenders, withholding royalties, and giving the bonuses to those who should have received them. But Amazon is simply not taking this seriously.
It gets worse.
The main guy at the centre of this has been printing money – getting up to a million page reads A DAY (from a screenshot he posted). He was named on a KBoards thread, and you can dig that info out yourself if you wish, and he also appears to be selling a turnkey scammer system for $47 a pop to internet marketer types who want to grab some of this “easy” KU cash – one of the reasons this has exploded lately. He also has a private Facebook group with over 1,000 members learning his tricks.
All of this was reported to Amazon publicly and privately weeks ago. Detailed information was sent to the jeff@ Amazon email address. But no action has been taken, aside from the piecemeal, half-hearted attempts to take down a book here and there. Meanwhile, these guys continue to rake it in – at everyone else’s expense.
This is simply not good enough, and we need to send that message very clearly to Amazon.
Update March 15:
Amazon made an official statement yesterday, after which I spoke to someone there.
I had, as you can imagine, a long list of questions. They will have to come back to me on most of those, but this is what I can share for now:
(a) Amazon didn’t confirm whether the TOC mess was related to its own efforts to stop scammers, but did admit that there were enforcement issues around TOCs and apologized for same. It will be reviewing those procedures to try and prevent a recurrence.
(b) Amazon will be trying to make it right with authors like Walter Jon Williams. It can’t discuss particulars but will begin reaching out to affected authors shortly.
(c) It sounds like the matter is being taken seriously and Amazon appreciates all the scammer reports (which you should continue making). It has its own processes for identifying this stuff, and has been investigating this stuff itself, but your reports also help. Amazon doesn’t publicize such efforts because it doesn’t want to broadcast scamming techniques before they can plug the holes, but they are attempting to deal with it.
That doesn’t cover everything. Far from it. But there were plenty of issues I raised that Amazon said they will be coming back to me on. Questions like:
1. When will Amazon have a system in place which can actually count which pages were read, rather than skipped?
2. When removing a scammer’s book, why does Amazon not look at their other titles?
3. Will Amazon be attempting to claw back bonuses paid to scammers, and will they be paying those bonuses to the authors which should have received them instead?
4. These scammers tend to be breaking all sorts of other rules too – it’s one easy way to identify them. Will Amazon have a more rigorous process to police such behaviors – e.g. title keyword stuffing – in the future?
5. Will Amazon be updating the Kindle Publishing Guidelines and other Help Pages with regard to TOCs? The statement yesterday said that people now don’t have to move their TOCs but there is conflicting (and ambiguous) information out there. Will this be clarified?
6. It hasn’t yet been established if any of the filler content these scammers are using was plagiarized, but that wouldn’t be a huge surprise given how little of that stuff the plagiarism tool actually seems to catch. Are there plans to improve same?
7. There are hundreds of millions of dollars which are going to be paid out by KU this year. Is it too much to ask that some resources go towards policing this stuff?
And there was lots more too – we had quite a long talk. That’s all I can remember off the top of my head. I’ll update further with any responses.
Update March 31:
I spoke with Amazon again at the beginning of last week. Sorry I couldn’t update this post sooner, but (a) I was busy and (b) I wanted to give Amazon some time to try make good on its pledges, and also not make any overly hasty conclusions about its response.
Amazon opened by saying that it was taking the matter very seriously, that it was a priority, and that it was devoting significant resources to the issue. But Amazon also said that the amount of scamming going on is actually relatively small.
I found the latter statement a little curious, but, before I get into that, here are responses to the questions posed above (and my thanks to Amazon for answering same):
- When will Amazon have a system in place which can actually count which pages were read, rather than skipped?
Amazon can’t disclose details for obvious reasons, but it is working on a more robust page counting system.
- When removing a scammer’s book, why does Amazon not look at their other titles?
Amazon does look at a scammer’s other titles when removing books and acts accordingly.
- Will Amazon be attempting to claw back bonuses paid to scammers, and will they be paying those bonuses to the authors which should have received them instead?
Amazon clearly stated that it will be clawing back All Star bonuses where scamming has been determined and it will also be awarding those bonuses to the authors who should have received them instead.
- These scammers tend to be breaking all sorts of other rules too – it’s one easy way to identify them. Will Amazon have a more rigorous process to police such behaviors – e.g. title keyword stuffing – in the future?
Amazon already has rigorous systems in place to check for abuses like this.
- Will Amazon be updating the Kindle Publishing Guidelines and other Help Pages with regard to TOCs? The statement yesterday said that people now don’t have to move their TOCs but there is conflicting (and ambiguous) information out there. Will this be clarified?
This should all be updated now and any ambiguity removed.
- It hasn’t yet been established if any of the filler content these scammers are using was plagiarized, but that wouldn’t be a huge surprise given how little of that stuff the plagiarism tool actually seems to catch. Are there plans to improve same?
Amazon is always making improvements to its tools and systems.
- There are hundreds of millions of dollars which are going to be paid out by KU this year. Is it too much to ask that some resources go towards policing this stuff?
Amazon is taking this matter seriously and significant resources are being devoted to this issue.
I wasn’t hugely satisfied with these responses, as you can imagine. Obviously, something doesn’t quite jive there. If the amount of scamming is small and Amazon is making this matter a priority and devoting significant resources to it, why are we seeing little improvement in the last three weeks?
Amazon has actually improved its response to direct reports of scamming – I think that’s fair to say. However, it also seems that it’s only titles which are directly reported to Amazon which are being taken down. Amazon doesn’t seem to be making a proactive attempt, independent of those reports, to identify scammers and remove those titles.
It’s not the job of readers/authors to police the Kindle Store. And if Amazon is paying us all collectively from a fixed pot than it has a duty to prevent fraudulent publishers from engaging in what can only be described as stealing from that pot. But Amazon isn’t just sleeping on the job, it’s leaving the doors unlocked and the windows open underneath a giant neon sign saying FREE MONEY.
Let’s be very clear about something: we aren’t taking about genuine authors here who are engaging in some extra-curricular shadiness. These scammers are not authors. They are internet marketers who outsource the “writing” of these books (or simply plagiarize and/or rework genuine books in many cases, I suspect). It would be a much trickier problem if we were talking about genuine authors engaging in shady behavior, but that’s not the case. No authors will complain if Amazon takes a hard line with this crowd.
The second and fourth answers are laughable, quite frankly – demonstrably false PR blandishments which show that Amazon is still hasn’t grasped the seriousness of this issue. But I wanted to wait and see if Amazon actually stepped up its response.
Even when Amazon does identify a title which has breached its guidelines, it still seems to be taking a kid glove approach with actual scammers. Often the titles are only down temporarily, and then re-uploaded with the same cover/title/author name and only one of the rule breaches addressed. For example, the scammer will often remove the more obnoxious CLICK HERE messaging, but keep the keyword stuffing aimed at flooding search results, and keep the filler content aimed at bloating their page count and payout, and then swap in more subtle inducements to click to the end. I’d have a little more sympathy for Amazon on this front if I didn’t predict exactly this in my original post.
The scammers have only focused on certain niches to date (mostly certain specific sub-categories of romance and self-help), so you may not be fully aware of how flooded the Kindle Store is with this crap.
Try searching the Kindle Store for “cowboy romance” or “victorian romance” or “seal romance” or “historical romance” or “regency romance” or “mail order romance” or “amish romance” or “shifter romance” and you will see exactly what I mean. The scammer books are usually immediately obvious from the title-keyword stuffing like “Romance: Regency Romance: [ACTUAL BOOK TITLE]” – in fact most of the scammer books take this exact form, right down to the position of the semi-colons in the title.
Which makes this stuff incredibly easy to find. If you want to.
A lot of these books are hovering around the 20,000 mark so maybe Amazon doesn’t think it’s important – but those borrows are taking money from the collective pool and the scammers are doing what scammers always do: they are working on volume. But even at rankings of 20,000 you can grab some decent visibility and squeeze genuine authors out of the sub-category charts. Some scammers are doing much better than that too – I’ve seen scammer titles hovering around the Top 500. And I’m sure there are many, many more titles at worse ranks, but I’m not digging particularly deeply here. Obviously Amazon has infinitely better tools for identifying this stuff. If it wants to.
Right now, it’s hard to avoid the impression that Amazon simply doesn’t care. I honestly don’t know how anyone could conclude otherwise.
