Good gracious – it’s GREP!

The other day I was asked if I could convert some short ebooks into PDFs using an existing print template set up in InDesign. Only thing was that the contents of the ebooks were edited sections of existing print books, and had never existed as Word files or InDesign files. So how to do it reasonably efficiently? To be absolutely sure that you have the right version of the text ideally you should work with the ebook files.

That’s fine – crack them open and you have HTML. That’s text, right? Well – yes, and in an ideal world InDesign could import HTML and use the HTML code to style your text. But this isn’t an ideal world – InDesign can’t import HTML yet. (I’m sure it’s only a matter of time, right, Adobe?)

But – back to the drawing board for the time being. So I copy and paste the text from my browser… That works, but – hey, hang on! – where’s my formatting? All those italics – gone. Oh lordy, am I going to have to go back over everything and replace the italics? Bolds? Headings!?

Back to the drawing board again. What if I copy-and-paste the HTML into InDesign? Yes but you still haven’t got any formatting? Ah, but you have got the codes for formatting.

You can see here, each paragraph is surrounded by a little bit of code and italics, bolds and headings, etc are surrounded by codes too. These codes work with the css files to style the text in your ebook or browser, and the great thing about this is that they won’t ever be wrong or typed incorrectly (so long as the original text is styled correctly of course). So you can do some find/change work using the code tags as a guide and you’ll soon have styled text without having to go through comparing both versions. Hurrah!

The find/change panel in InDesign is really powerful and I spend a lot of time using it when I’m typesetting. But to sort out this little problem more efficiently, it’s really useful to know a bit of grep. I knew some grep and sometimes use InDesign’s built-in grep queries, but I went back to the trusty Lynda video-training site and brushed up on it. And, wowzers, it really is like magic. (I’m nothing to do with Lynda.com, but I cannot recommend them highly enough – their courses are superb.)

What you need to know here is pretty simple stuff, actually, and is only scratching the surface of the capabilities of grep (and don’t even get me started on the possibilities of grep styles). If you’re ever setting long documents, or have to change from one format to another, a little bit of grep is the way to go.

Here I’m clearing out the paragraph tags and styling the body text at the same time – one click (do check your code is working first though!) and the body text is styled and the paragraph tags are gone. You’ll see that the paragraph tags are in the search field and inside them is (.*). This pretty much means find anything inside this text. Then in the replace field the $1 means put in anything you’ve found but only what you’ve found – not the paragraph tags (actually anything inside those parentheses you see around .*). And at the bottom of the find/change panel I’ve asked it to change the style to ‘text’.

Clearing out paragraph tags

You can use the same method to style headings, opening paragraphs, etc, too. Just substitute your paragraph tags for whatever else you have (H1, div, etc).

Here I’m styling italics with a italic character style and getting rid of the tags at the same time. Again, you can do this with your bold, underline etc – just change the search criteria.

You can also use ‘wild cards’ to clear out things like image tags that are slightly different throughout, so that you don’t have to search and delete manually through all the text.

Using wildcard codes to clear out unwanted text

You could also use grep to convert the image tags to placeholder boxes for the images if you needed too. Finally go through and clear out random div tags, etc. Then I’d do a final check for > and < which means you should pick up any remaining lurking code. Then you’re done. Ta dah! Styled text in just a few clicks.

Advertisement

Exporting fixed-format ePubs from InDesign

I WANT CANDY!! – Fixed-format EPub displayed on Readium

A couple of months ago Adobe launched InDesign CC 2014. Among its new features is the ability to export your documents as fixed-format ePub 3. This is really rather exciting. Prior to this update you either had to go to a specialist conversion company or pretty much hand code files. Either way it was a palaver and often an expensive palaver at that. I’ve been using a very good conversion script (ePub Crawler) for the past year (and will continue to use it for some projects), but this is something else! I’ve just got around to having a proper look at this new capability and I’m impressed. InDesign CC 2014 can now export valid ePub 3 fixed-format ebooks as quickly and easily as exporting a PDF. Barring a few minor issues and things-to-watch-out-for, the resulting file will be identical to the InDesign book file.

What is a fixed-format ebook?
Fixed-format ebooks reproduce what is on the printed page (like a PDF). The text is live and can be selected, searched, defined and read aloud (depending on ePub-reader software capabilities). The new InDesign export places all of the page elements exactly as they are in InDesign and each word is dealt with individually, so your line endings and spacing remain intact. (It also means that if you crack open the ePub files and look at the HTML it looks like the stuff of nightmares, but that’s another story.)

Currently Apple iBooks is the only platform that really has full support for all ePub 3 has to offer, however other retailers are following (I’ve had some success with opening the resulting files in Kobo apps, for example. They will also open in Google Chrome with the free Readium extension installed and I’m pretty sure they’d be fine on Google Play.) But I have spoken to some people who are still very dismissive about fixed-format ePub – there’s just not enough support on other platforms, why would you want to, isn’t it just easier to convert to PDF? Well, yes – but Apple don’t sell PDFs (nor do any other major ebook retailers to my knowledge) so if you want to sell a PDF book you’ll have limited places to sell it. And – did I mention? – if your book is set up correctly export is no more time-consuming than exporting to PDF. So you can still export to PDF if you want to and export a version you can at least sell on Apple iBooks for very little additional outlay – probably less than the cost of getting a proof copy printed.

Fixed-format conversion is most suitable for:
✔ Children’s picture books
✔ Photo books
✔ Highly designed and illustrated non fiction and reference
✔ ‘Coffee table’ books
✔ Cookery books (and other instructional books)
✔ Any book where it is important to keep the design and layout intact
✔ Prospectuses or catalogues

So who would benefit from this?
✔ Publishers and self-publishers wanting to release backlist titles quickly and cost effectively
✔ Publishers and self-publishers creating new titles in the above fields. If new titles are designed with an eye to exporting to ePub on completion, the convertion is very quick and pain free.
✔ Photographers and artists (Photo books, portfolios and ePub albums for wedding clients, for example)
✔ Bands and other performers (tour books, blogs, programmes, etc)
✔ Organisations and retailers who would benefit from conversion of prospectuses and catalogues, internal and external training materials, etc, to ePub to allow them to be read on a wider range of apps and devices

I’ve incorporated fixed-format ePub into the range of typesetting, layout, editorial and conversion services I offer. I don’t give flat-fee prices at the moment because every project is different.
I can:
✔ Convert your final InDesign files to fixed-format ePub more cheaply and quickly than I used to be able to offer. (If your files are suitable this takes the same time as saving to PDF.)
✔ Place video and audio prior to conversion
✔ Assess your files’ suitability and make font substitutions and layout tweaks prior to conversion
✔ Re-set and layout (from old backlist, etc)
✔ Set up new titles from scratch

Sample Books
I’ve set up a couple of 30-ish-page sample books. One is the full text of The Sleeping Beauty Picture Book, illustrated by Walter Crane and the other is a section of The Candy Maker’s Guide, a recipe book first published in 1896 by a manufacturer of confectionary and baking equipment. (Both public domain texts – the photos are mine.)

Here are some screen grabs from Sleeping Beauty:

Note the overlapping text on the headings. The text is still live.

InDesign layout sample – Sleeping Beauty

IBooks screen grab showing thumbnails

This one has a path around the image of the girl on the left. Every word is in exactly the same position as in the InDesign file. (You might have noticed the overflowing text box below. InDesign will not warn you of text overflow on export – it just assumes it’s what you want.)

InDesign layout sample – Sleeping Beauty

IPad screen grab showing thumbnails

Now here are some screen grabs from the sweets book:

Again we’ve got a clipping path around the sweet jar and I put a feather on it (the path is rough, I know – but the point is that it is exactly as it was.)

InDesign layout sample – sweets

IBooks screen grab

Here’s a spread showing a mixed single- and two-column layout and images with transparency.

Sweets spread example

You can set the ePub up so that it displays a table of contents as thumbnails like this, or if you set up a table-of-contents style, you can have it display a multi-level toc in words.

Thumbnail Table of Contents

This shows looking up ‘live’ words in the built-in dictionary.

IBooks showing definition of word

So what can’t you have?
✘ Hyperlinks in text or internal links
✘ Live text that is horizontally or vertically scaled, or kerned
✘ Live text with strokes, gradient fills, drop shadows etc.
✘ Live text on a path (text will flatten on conversion)
✘ Gradients (this can be overcome by flattening to jpg first)
✘ This conversion process is not suitable for adding audio with read-along text highlighting
✘ Postscript fonts

(Adobe are still working on enhancing the export, so some of these issues will be overcome soon.)

Backlist conversion and fonts
The major issue when converting InDesign documents is fonts. Many publishers are still using (or used to use) Postscript files. These won’t work in ebooks, so if your books contain Postscript fonts they will have to be replaced with Truetype or Opentype fonts. Many common fonts have Truetype or Opentype versions, but more obscure fonts might have to be replaced with similar fonts. It’s also important to make sure your fonts are licensed for ebook use. They often aren’t. Ideally new titles should be designed using correctly licensed Truetype or Opentype fonts.

Multimedia
You can also add video and audio if you so desire – obviously if you are converting a print book you might have to make some design tweaks to make room for the media. Video and audio can be placed directly to the InDesign file. You don’t have any choice over the video/audio controls’ appearance (but I believe you can edit the css if it really bothers you.) There is also support to link to Youtube etc if you don’t want to embed your media.

Here are a couple of screen grabs from a small test file I made.

How video looks on the page. You can’t customise the controls.

The audio controls can’t be customised.

A note about tracking

I was asked about tracking text. The screen grabs below show text tracked backwards and forwards.

InDesign showing text without tracking

InDesign showing tracked text in heading and bottom line.

EPub showing tracked text

Fixed-format epub with read-along audio

IPhone and iPad

I’ve now added read-along audio to the ‘list of things I can do’. Specifically fixed format ePub for IOS devices with an audio track that highlights each word.

I can’t show you a sample because I’ve been working on copyrighted material. Maybe one day I’ll get around to making something, but the first thing I learned about the process is that it’s time consuming – oh so very time consuming.

You will need:

• One fixed-format ePub
• One or more audio tracks saved in .m4a format
• A .smil file for every page that contains text

 

Method:

1. First you need to mark the start and end point of every word in the audio. (I split my audio into one file for each page –  you can use just one I think, but I haven’t tried it) Apple apparently recommends that you use a program called Audacity – which is handy because I already have it. But it is free anyway and easy to use. You will have to find the start and end point of each word by creating a selection, listening and fine-tuning it until it’s right. Then you press cmd b to create a label – you don’t have to write anything in the little box that comes up (I just did it to draw your attention to it).

When you have marked every word you can export the labels to a text file. When you open the text file all the start and end points of the label, and hence the word, are there in a list.

This will take some time. I’m sure some clever person has or will shortly automate this process, but until then there is nothing for it but to sit and listen … over and over again.

Marking the audio timings

 

2. Next you will need to create your .smil files. Below is a sample showing just three words. This is where you add your timings you made earlier (they don’t have to be in red though!).

<smil xmlns="http://www.w3.org/ns/SMIL" 
 xmlns:epub="http://www.idpf.org/2007/ops"
 version="3.0">
 <body>
<par id="par1">
<text src="6-page6.xhtml#W1"/>
<audio src="audio/audio3.m4a" clipBegin="0.000000s" clipEnd="0.632985s"/>
</par>
<par id="par2">
<text src="6-page6.xhtml#W2"/>
<audio src="audio/audio3.m4a" clipBegin="0.632985s" clipEnd="0.964874s"/>
</par>
<par id="par3">
<text src="6-page6.xhtml#W3"/>
<audio src="audio/audio3.m4a" clipBegin="0.964874s" clipEnd="1.214646s"/>
</par>
</body>
</smil>

3. Next you need to mark up your html. Each word is wrapped in a span with the id corresponding to the text source number in the .smil file. For this short example you’d end up with a paragraph that looked like this.

<p><span id="W1">The </span> <span id="W2">Cat</span> <span id="W3">sat</span></p>

 

4. Next you need to update your content.opf file.

Remember to list all your audio and .smil files in the manifest and give them a unique id.

Then in the entry for the html files with audio you must remember to add:

media-overlay=”ID of corresponding .smil file”. This is really important – if you don’t do this properly it won’t work.

 

5. Then go into the css file and add this.

}
 .-epub-media-overlay-active{
 color: red;

This is saying to highlight the words in red (but it can be any colour you like).

 

6. Finally zip your ePub file back up again and test it out. If you have got everything right you should see a speaker icon in the top right of the menu bar, and if you’ve got everything absolutely right you will be able to have your file read to you and turn the pages automatically or manually. (Have you got the volume turned up?)

So then you’ve got everything right  and you try to validate it. And find it won’t validate – yep that’s right – this is quite normal; the validator doesn’t accept media overlays or some such gubbins.

 

So there you have it. Not a quick snack!

 

I used Read Aloud ePub for iBooks by Liz Castro  as my guide. The section about using GREP to help in the markup was extremely useful.

What is an ePub file? Why won’t it validate when I make one in inDesign?

Yesterday an email dropped into my inbox and I found myself doing some impromptu ePub troubleshooting on an ePub that wouldn’t validate. The ePub had been exported through inDesign – I don’t know which version at the moment.

Now – inDesign is a great conversion tool – but unless you get everything set up right it’s unlikely that your ePub will validate without cracking it open and making some tweaks. In fact – it’s unlikely that it will validate straight from InDesign whatever you do. It’s just a question of how much you do in inDesign.

So I exported one of my test files with some deliberate errors to see what I landed up with. Oh dear.

Validation errors

A while back I wrote a post on what an e-book ‘file’ is. I mentioned that an ePub file isn’t a single file at all, but a zipped-up collection of files. Screen shot below.

unzipped ePub

Just to recap: the xhtml files are the content of the book. It’s a good idea to set them up as one file per chapter.

split screen

This is a split-screen view in Dreamweaver. The xhtml on left and how it should look on the right.

The toc.ncx file contains the table of contents.

Table of contents

The css file deals with how the text is styled. This can be a world of pain, but doesn’t tend to stop your file validating.

The content.opf file is the one you really want to check out if your file won’t validate.

Content.opf file

Notice that there are a whole load of metadata fields that begin with <dc:. You are likely to have validation problems if your title, date, language and identifier fields aren’t filled in. The title is the main one. E-readers often run the title across the top or bottom of the screen and it’s pulled in from here. InDesign should pull these in during the export – but only if it’s set up to. The title field (and some others) is dragged in from ‘File Info’ in inDesign – it’s NOT the file name. I know from working in print for ages that we rarely fill in the file info box for print, or if we’ve used files as templates they might contain old info. So that’s one to get into the habit of filling in.

The next section is the manifest. This is a packing list of everything that’s in the ePub folder. If anything is listed here it must be in the folder. Everything in the folder must be listed here. File names and file types must match, or your file may not validate.

The Spine lists the order that the files will appear. Again, if there are any mismatches your file might not validate.

Finally there is an optional section called the Guide. This isn’t created by inDesign at the moment. You will have to go in and create it manually. It seems to be more of an issue if you want to convert to Kindle format. It shows the Kindle software where the cover, table of contents and the start of the text is located. It won’t stop your ePub validating, but if you convert to Kindle format it may throw up an error if it’s not there.

content.opf file. Spine and guide

And finally don’t have spaces in file names – EVER. It’s a good idea to get into the habit of just using letters and numbers for any files you create. You’ll notice that the majority of warnings in the first image above are down to my inDesign file name containing spaces.

And remember – validation is only the beginning – you can have a file that validates perfectly, but it can still be riddled with formatting errors that will cause it to give poor user experience.

Five things I didn’t know about Photoshop last week

This weekend I studied a course on www.lynda.com called Photoshop for Designers: color by Nigel French. I love Lynda.com courses and can’t recommend them highly enough.

So five things I learned from the course, or came across as a result, that I didn’t have an inkling about already.

1. You can set up soft proofing so you see as colour-blind people would see. Apparently colour-blindness affects more people than you would think.

Soft proof. Colour in the navigator

This more useful for more graphic work, seeing where there is enough colour contrast, but interesting none the less.

2. Match colour. This is where you can map colours of one image on to another.

Before

After

Hey presto!

3. HUD colour picker. If you want to pick a colour as you go – ctrl alt  cmd click will bring up the colour picker. If you want to jump from one slider to the other press the space bar. Groovy!

4. Locking transparency. This is just a scribble, but if you lock a layer’s transparency, you can only effect pixels that aren’t transparent (so you can’t colour over the lines).

Locking transparency

5. Getting a hex number from the eyedropper tool.

Right click and the dropdown menu shows copy colour as hex number or html. These are numbers sampled from a rainbow gradient.

A rainbow of hex numbers

 

InDesign epub conversion. When is black not black pt 2

Last week I posted about an issue I’d been having with the css generated by InDesign cs6. When I tested files on the Kindle for iPad and then Kindle Fire, I found that my black text wouldn’t revert out in the night theme. (It’s fine in iBooks and Kobo)

After some investigation I discovered that InDesign seemed to have changed how it expressed black in the css.

css generated from InDesign cs5.5 and cs6

This is the css generated from the same document – the one on the left in cs5.5 on the right cs6. Notice the color value. I hadn’t changed anything in InDesign except open the file in cs6.

Colour swatch options when creating stylesheets

This is the colour menu in the paragraph styles dialogue. You have a choice of black and registration to create black. Black is the default (at least it is on my set up).

Css generated using ‘black’ and ‘registration’

And this is what I got when I set up one style using ‘registration’ black and one using ‘black’.

Interesting…

Now I know how to fix it, it’s not a problem, but it had me scratching my head for a bit.

 

Edit:

Below is a screenshot of the night theme on the kindle mac desktop app and the accompanying css. The text styled as ‘body’ is not showing up and has a color value of #1a1818 which is what InDesign has generated when I used the epub export dialogue. The other text has had the css edited in Dreamweaver to colour:inherit and no color value.

kindle for mac black screen and css