Self–Publishing – Love It or Leave It (Part 2)

A few days ago, I hit the ‘save and publish’ button on the Kindle Direct Publishing (KDP) website to launch Mandated Memoranda Publishing’s second book: Tragic Wonders – Stories, Poems, and Essays to Ponder. I found one glaring (to me) error in a chapter title very late in the quality control process. I left it.

This post is an update to a previous article on self–publishing. Several things are new with Kindle Previewer software and my process.

The Kindle Previewer (KP) version I used is 2.92. KDP no longer provides an emulated means to check your Kindle files targeted to iPhone/iPad. It generates a file with extension .azk. You’re supposed to sideload it to your Apple device to test it. I have no Apple device.

Additionally, KP 2.92 generates a fault when it produces the .azk file. Windows captures the fault this way:

Faulting Application Path: C:\..\Amazon\Kindle Previewer\lib\phantomjs_mobi82html.exe
Problem signature  
Problem Event Name: BEX
Application Name: phantomjs_mobi82html.exe
Application Version:
Application Timestamp: 4f7753dc
Fault Module Name: nvinit.dll_unloaded
Fault Module Version:
Fault Module Timestamp: 50ef1ca7
Exception Offset: 7520ce59
Exception Code: c0000005
Exception Data: 00000008
OS Version: 6.1.7601.
Locale ID: 1033
Additional Information 1: 0a9e
Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
Additional Information 3: 0a9e
Additional Information 4: 0a9e372d3b4ad19135b953a78882e789
Extra information about the problem  
Bucket ID: 4075841998

I sent a query to KDP about the fault and didn’t receive a reply once they figured I wasn’t asking for advice on my .azk file. This was a tad disturbing, to say the least. Then I went ahead and published Tragic Wonders. A friend with an iPhone purchased the book and said it worked fine. However, on 10 January 2014, I received a query from KDP about this issue. We’ll see if anything good results from the continued discussion.

Enough talk of Amazon’s problems. On to the new features of our process, some of which may be technical. We discovered and corrected some upsetting features imposed by Microsoft’s Notepad application. We used the text–align style attribute for chapter titles and table of contents (TOC) entries. We figured out some intricacies of nested .ncx file logical TOCs. We tried a different approach for page-breaks. Finally, we used Kindle Previewer to generate our last few .mobi files that we submitted to Amazon KDP.

We discovered through some disturbing errors in our .mobi files that Notepad was inserting non–printing characters. How could we tell if they were non–printing? It showed up in truncated logical TOC entries (generated by Kindlegen from .ncx entries) during the QA review. It also showed up in HTML TOC errors in the .mobi file text. Both errors occurred at the point where the lines in the files (.ncx and .htm) word wrapped.

It turns out this is well documented and has been an issue since Notepad first appeared. A workaround is: don’t narrow the Notepad application window so any TOC or logical TOC text word wraps. If you’ve done so and you have funny truncation error then you can retype those entries like I did (very tedious). A solution may be to use Notepad++ (but I haven’t, yet).

We used the text–align attribute to left justify chapter titles and TOC style entries. Amazon’s Kindle Publishing Guidelines only discourage specifying alignment for body text so it reflows freely.

We figured out some intricacies of nested .ncx file logical TOCs (and didn’t figure out other things). This is  genericized excerpt from our .ncx file:


I call your attention to the em space setting off Story Title 1 as an indent. This conveniently differentiated story titles from section titles. Turns out there is another way, but not all eBook distributors use that method.

Having had difficulty with page breaks in our first book Tiānmìng – Mandate of Heaven, we tried a different approach to page-breaks this time. Using <br> without <p> or other formatting to end chapters seems to work. This is an example of what we did:


We discovered that KDP was using a file structure to read and assemble their .mobi file (the one you can download from them) after you submit your files to KDP. I remember reading about that somewhere in the literature but now I had a concrete example in the zipped HTML files KDP produced for download. I recommend you use those HTML files for further development once most of your bugs are shaken out (i.e., after final edits are long done and most formatting issues have been retired).

You’ll need to package your files into separate folders labeled: image, xml, and html. The image folder contains your image files. The html folder contains your .htm file (if you use one .htm file like I have, up to now). And the xml folder contains your .ncx file. The .opf file should reside with these folders either on the desktop or in your working folder. The files that come from KDP expect this structure (you can see the changes from your original files in the downloaded ones when you compare them).

We used the HTML files KDP generated to finish our development after we went a few rounds with Amazon KDP’s Kindlegen to shake out bugs. This led us to use Kindle Previewer on our .opf file to produce a .mobi file. Kindle Previewer invokes Kindlegen and even produces the verbose error report we use to validate our .mobi files. We submitted the resulting .mobi file to KDP and did final quality assurance (QA) testing with the .mobi KDP produced from our submittal. I plan on structuring my files using this new approach in future books.

After I completed my book, I happened on these two eBooks on Kindle formatting and process. The first, The eBook Design and Development Guide [Kindle Edition], by Paul Salvette (Author), emphasizes formatting using cascading style sheets (CSS). It does a masterful job of explaining in simple terms how to use them. The process that is described bypasses the MS Word to filtered webpage step by copying and pasting the original manuscript into a text file and proceeding from there. I think this approach avoids some errors while enabling others.

The second book, EBook Formatting: KF8, Mobi & EPUB [Kindle Edition] by Matt Harrison (Author, Illustrator), says it is purely concerned with formatting. Heavily programming oriented, I expect it will provide clues for me to format my next book more professionally.

I’m not sure why Amazon’s KDP Systems Architect or chief designers can’t put out a series of books on Kindle book development akin to Microsoft’s series. I’d be first in line to buy the reference (if it’s $9.99 or less, of course).

Quo Vadis II

Tragic Wonders 1 by 1_6 quarter scaleWe’re almost finished with a round of editing for Tragic Wonders. In prior posts, you’ve seen some of the stories and essays that we dropped from TW because of thematic duplication. We’ve also separated out two poems that now provide a transition from the stories to the essays. The next step is copy editing. Seems the company is geared towards Microsoft Office production but, for the money, they’ll accommodate Amazon Kindle requirements. Whew! Actually, they’ve been encouraging, cooperative, and fast. We’ll see how much we can say about them at a later date.

At this point, we plan to get TW out before Christmas (Thanksgiving is the real goal, but…). To do that, I want to have at least two professional reviews in place before issuing the press release and posting the Kindle book to Amazon and registering with the LOC and Red City Reviews usually posts to Goodreads but Goodreads requires an ASIN, Amazon’s ISBN equivalent… You get the idea, the timing will be tricky. Also, we were planning on switching from PRWeb to PR Newswire. You know, the PR firm recently hacked for customer data.

We’ve had several favorable reviews on TW’s all important cover. Yes, it turns out we do judge a book by its cover (among other things, like favorable excerpts from perhaps unfavorable professional reviews).

Right now, we have Tiānmìng – Mandate of Heaven priced at 99 cents. We’re going to raise the price back to $2.99 when TW comes out at 99 cents. In other news, we’ve decided to postpone publication of A Digital Carol until next Christmas. Actually, we’ll get it out before Thanksgiving 2014 to pick up on the buying season. We’ve already got the cover concept roughed in.

Yup, we’ll get this marketing thing down, yet. Hopefully, before we’re out of funds.

We plan to start Who Shall be God early in 2014 while ADC is in editing (yes, book three, that became book four, is book three again). The WSBG themes are coming together. It turns out that the 38th parallel not only runs through the Korean DMZ but through Virginia and Maryland. Hmm, I wonder if that’s significant given what DC has been doing lately?

Book five is about China. We’re still waiting for Xi and company to declare their intent after the Third Plenum meeting of the Communist Party’s Central Committee. Other news of interest is that China has twice the shale gas reserves as the US does. That should swing the balance all other things being equal. We’ll see.

And finally, we’ve updated our links page.

Self–Publishing — Love It or Leave It

What a long, strange trip… Sing it with me. We here at Mandated Memoranda Publishing LLC have just given birth to: Tiānmìng – Mandate of Heaven. I had to check with Ninja to make sure I wasn’t exaggerating. We can’t provide a clear, error-proof method to do self-publishing, but we’ll tell you all we can remember (and we won’t charge a cent). Please forgive the length of the post. As I said, it was a long trip.

The following process assumes development on a Windows PC. It may work on a Mac but your mileage may vary. We read a lot of references. Most were $2.99, or less. Given their long-term control over the publishing formats, I cannot fathom folks reluctance to stick to Amazon’s preferred rules stated in their Kindle Publishing Guidelines, methods overview, one layer deeper guide, and examples (see right hand list). The publishing method we followed gives you almost total control over your book. The publishing method is “simple” assuming you are writing from a fairly well thought out story line (I use detailed graphics to develop the plot).

Develop your Cover (an important marketing tool) according to Amazon’s Publishing Guidelines (1.6 high to 1 wide ratio and greater than 1000 pixels wide). I use royalty free (one time charge for perpetual use) images from Getty Images. Getty requires attribution on your book’s copyrights page.

Type out your book in Microsoft Word. Early in the typing (first thing is best), modify Microsoft’s Normal style. Change the font to +body (Calibri, 11 point). Actually, you’ll be removing this later in the filtered HTML but the PDF for copyright submittal to the Library of Congress looks modern in this the format. As you modify the Normal style, select the paragraph attribute. In the paragraph dialog box, under ‘indentation’, set “special” to ‘first line’ and set the “by” value to 0.01” (the smallest inches value Word recognizes). Set the “spacing after” value to 6 pt. and “line spacing” to ‘single’. Find and set the “tabs” value to 0.01”, too. The 0.01″ settings influence something important called ‘text indent’ (see below). Leave the others at their default values (if you still had them at their original defaults in the first place).

Use user defined format styles based on your slightly modified Microsoft Normal style (undo what formatting you have if you have to and rebuild it based on Normal, it’s worth it). Define other styles based on the Normal style (for instance, those for Title and Subtitle, the *** symbol, page breaks, etc.). What you want is a well-controlled set of styles that you can modify globally solely by modifying the Normal style. A few styles must diverge from this approach. Use Microsoft’s Heading 1 style for chapter titles and Table of Contents (TOC) style for the table of contents.

Use the insert bookmarks dialog box in Microsoft Word to add two special bookmarks: toc and start. These are case-sensitive and define where the table of contents and beginning of your book are, respectively. I recommend that the toc bookmark go before the heading for the table of contents but after the previous page’s page break (you can adjust its placement in the filtered HTML if necessary). I strongly urge placing the start bookmark at the top of the title page.

Proof the manuscript thoroughly. Make sure to delete blank lines and spaces (you should use page breaks between chapters and the *** symbol between scenes in a chapter). Submit what you have to an editor. Pay one to tell you what your friends or family will not. A good developmental editor is worth their salt. Fold editor comments back into the primary copy. Proof again and, if possible, resubmit to the editor for a final look-see.

Then, and here’s the part we spun over for weeks, save the Microsoft Word file as filtered HTML. Make sure to apply the book’s title in the “title” box (where it says: add a title) on the “save as” dialog box before saving the file (it gets placed in the HTML and is used by Kindle devices). You’ll also use the original file to create a PDF for the Library of Congress Copyright process (you’ll imbed the Cover image in the original Word document for the PDF but not embed it for the filtered HTML). Your baseline has just diverged so make sure that you’ll make almost no changes to the text at this point. You’ll have to fold back any format changes to the PDF and master Word copies (what a pain that was).

We went back to square one several times. One of the go-rounds centered on updating the Microsoft Word TOC without first removing the old one (go to the reference tab and the TOC pull down menu for the remove TOC command). Not doing this added extraneous bookmarks (‘< a id=“_Tocnumbers” > < /a >’) in the HTML in front of headings (i.e., two sets of bookmarks, only one of which matched the TOC ‘href=#_Tocnumbers’ entries). When you open the filtered HTML, if you’ve made that mistake, you’ll see what I mean.

Open the filtered HTML file with Microsoft Notepad (or your favorite text editor that won’t add anything behind the scenes like Word does). Now remove all ‘font size=’ statements (you might leave some for Heading and Title styles but use relative sizes like 120%). Also remove all ‘font color=’ statements since the Kindle devices use their own defaults (and black font disappears on a black background selectable for Fire and PC readers). Now, your text will scale (more) properly on Paperwhite and Fire devices. The font will change on Paperwhite. And Fire and PC readers with black or sepia backgrounds will show all your text.

Change all ‘text indent=value’ to 0 from whatever Microsoft Word set them. This prevents the Kindle devices that naturally insert tabs at the beginning of paragraphs from doing so. Obviously, don’t do it if you like that formatting. I left most of the absolute margin specifications. I also left extraneous styles Microsoft added. If I were braver or surer of what I was doing, I’d have removed them (be careful, it’s easy to remove stuff you actually need).

I recommend formatting the page breaks separately with a slight bottom margin (0.01”). This prevents certain devices from ‘dangling a sentence’ when a reader lands on a chapter heading and pages back one page (I do that a lot). It does trade the aforementioned error for another. The page back command has to be performed twice to take. Not sure why that is but it only happens when you first land on a chapter heading. Such ‘path dependencies’ are likely database related, so go figure.

I also, foolishly, added language references all over the text. It helps the Microsoft Word spellchecker but is unnecessary (ever get an error that a foreign language dictionary wasn’t available, that’s why). I won’t even tell you how I did this so you won’t be tempted to try it. Hundreds of remnants are still in the filtered HTML to this day (‘< span > … < /span >’ items). However, I did neuter their effect by removing the ‘lang=en’ field.

Review your filtered HTML file in a web browser at this point. I found some extraneous HTML symbols that I failed to remove by doing this. It is much easier to check now than later in the process. I also spell checked a version (that I discarded) in Word for errors I might have introduced in the format editing process.

At this point you have a choice. You can upload your filtered HTML file to Amazon KDP for them to convert into a book. After they process your HTML, you download the files they generate and iteratively refine the HTML (I did not do this, exactly). However, do not submit your book for publication until it is finished; it will go live (more below). Or, you can construct two more files and use Amazon KDP’s Kindle generation program to build your book. You will make the same number of iterations to your book if you use the Kindlegen program, but, you may have more insight into the changes you make.

You will need two things from Amazon to continue. The ‘compiler’ or book generator: Kindlegen (I used PC version 2.8) and the test tool: Kindle Previewer (I used PC version 2.85). You can also get the latest versions here.

The two other files you’ll need are the Open Package File (or OPF) and the Navigation Control file for XML applications (or NCX) files. The OPF file defines to the Kindlegen program your books cover file, the metadata about your book, the text files, and how to find the ‘toc’ and ‘start’ bookmarks. The NCX file tells Kindlegen how to construct the pop-up table of contents folks see on their Kindle devices. It’s called a logical TOC. Amazon provides examples.

The example that stood out far above all others was the annotated Kindle Users Guide OPF (Guide.opf in the downloadable Guide example). Whoever did the annotation is our friend for life. The OPF file tells Kindlegen how to build your book (where your files are and how to process them). Populate the file (in Notepad or equivalent) and save as ‘all files’ (not the default ‘txt’ file). You will overwrite the existing OPF file, so archive it first. I had 28 different versions of the three files archived by the end of the process (no, I’m not proud of that, just really miffed).

There is a good example of an NCX file in the Guide example folder, too. With your HTML (or HTM), OPF, and NCX files you are ready to have Kindlegen build the book. Explicitly follow the instructions in the manual.html ‘read-me’ entry that comes in the folder with the Kindlegen.exe file. All this may sound like gobbledygook, but you can do it.

Just in case it hasn’t become obvious, you are, in essence, programming your book in a formatting computer language. You generate MOBI files (Mobipocket eBook file) by running Kindlegen on the OPF file. The MOBI that results is then opened in the Kindle Previewer to test it. Here’s a list of my test procedures that I ran in all 8 Kindle emulations and the PC viewer I had downloaded separately.

  • Open the book (tests where the beginning is sensed no matter what the beginning tab showed).
  • Use the pull down menus to select Cover, End, TOC, etc.
  • Use the pull down menus to confirm that the metadata (language, author, etc.) was recognized from the OPF file.
  • Page through from the Cover to the TOC.
  • Step through the TOC and return (using the Kindle Previewer return arrow) on each entry (tests landing and for any weird artifacts).
  • Step through the logical TOC the same way.
  • Finally, on the slowest setting, use auto-flip tool to review each page for formatting issues (I went backward and forward many times). I checked the images at this point after I had settled some issues early on (I forget what they were).

This does not include the many more times I reviewed the straight, filtered HTML in Internet Explorer and did spell checks on it (the HTML) in Microsoft Word as a kind of error checking (in the software sense). At no time did I save the filtered HTML from Word (so no new artifacts would arise).

Early on, I also verified all font sizes and types (using the Paperwhite emulation) and colors (using the Fire emulations and PC reader) were removed correctly. Needless to say, I must have read the text twenty times including once on my hardware Paperwhite.

Along the way I ran into issues. My Paperwhite gave me an error when I got to 1% of the end. It ‘erased’ the file from itself (it actually was wrapped in something called a .fused file). This occurred because of the non-breaking spaces (essentially, extra carriage-return line-feeds) at the end of the book. When I removed them and recompiled the MOBI file, my Paperwhite stopped choking on the book.

On closer examination, this effect showed up in the Paperwhite emulation when the auto-flip tool stopped at 99%. I did not buy a Fire and Kindle DX to test the book on hardware (we’re really not fixed to do so, at this time). However, I did ask a friend about his iPhone app. He had the older version and he mentioned the ‘dangling sentence’ issue described above. I fixed many other issues and generated at least 13 versions of MOBI files (more I think, but I lost count).

I urge you to use the keywords: toc and start, in the OPF and HTML files. I foolishly used the _Tocnumber reference for chapter one in my OPF file instead of the start keyword. Kindlegen gave me a pass but Amazon KDP processing did not. As they say, follow the directions. I made many other changes that I can no longer recount.

Finally (so I thought), I tested my ‘last’ MOBI file on all 9 emulated devices and my Paperwhite and it worked flawlessly. Each version had its beginning on the first chapter. I uploaded my Kindlegen’ed MOBI file and Amazon KDP processed it. The MOBI file I downloaded was bigger and opened to the page before the first chapter at the end of the internal table of content on Kindle Fires. It looked unprofessional. Turns out that is one way the Fires differ from the other Kindles, at least according to the KDP forums. Unfortunately, I published this version in frustration.

The best way to modify things is one change at a time; but the turnaround is so intensive that I did more. Bad mistake. I made changes, rebuilt the book, and uploaded it to Amazon KDP. The download from Amazon KDP had the beginning page issue resolved, sort of. I had changed the ‘beginning’ from the first chapter to the title page.

Now the Fire and PC versions landed on the Cover (which was okay), and the Paperwhite, Apple (at least in emulation), and older e-ink Kindles opened to the title page (same issue, just finessed). I spent time on the forums and gleaned some true facts (and lots of opinions) about this and other issues. Whether you’ll use the forums depends on your frustration tolerance level, I guess.

However, paging through the TOC brought up an artifact in the e-ink Kindle emulation (the $69 model) that should have remained hidden. All the other versions did not exhibit this bug; but I wasn’t going to completely test the new MOBI only to find further bugs. It turns out that I had encapsulated the entire HTML text after the TOC in a ‘< div > … < /div >’ pair (more than one change at a time). I removed it (Heaven know why, I don’t) and the artifact disappeared.

I uploaded a new MOBI generated with the corrected HTML, let Amazon KDP process it, and tested the downloaded MOBI version completely (see above). I remember doing this cycle 5 times. When I determined that it worked (the beginning issue was resolved), I re-published it (no new upload was necessary at this point). ‘Belt and suspenders’ that I am, I downloaded the sample when it came online hours later. I ordered a full copy from the sample and made sure it went to the end and opened where it was supposed to on my Paperwhite.

Turns out that if you delete the .sdr folder associated with a Kindle file it’s as if the file were brand new on the device (all reader entered bookmarks are lost, of course). You can delete the .sdr folder by attaching the device to your PC and deleting the folder in Windows. I did this with every MOBI version I tested. Don’t do this directly with a purchased book (you can reset the book from the ‘manage my Kindle’ page at Amazon).

I’m now a publisher. But I may never read Tiānmìng – Mandate of Heaven again. On to book two: Tragic Wonders. Come to think of it, this ordeal might actually qualify for the new book.