Workflow Series: Why I Type in Markdown

September 19, 2015

Like any good grad student, I have a tendency to procrastinate from time to time. A popular topic of distraction for those in academia seems to be looking for, testing, and fine tuning our workflow: how we manage our research, literature reviews, papers, odd-and-end writings, etc. Over the course of writing theses, course papers, grant proposals, conference presentations, and whatnot, I have certainly fallen prey to this form of procrastination.

In what I’m thinking will be a small series of entries, I will say a little bit about my various workflows and my (perhaps ad hoc) justification for using them. Hopefully this provides you with a little food for thought and a moment—or several moments—of procrastination, too.

The Limits of MS Word

Throughout my undergraduate days through the completion of my master’s thesis, I used what most students probably use for typing: Microsoft Word. My university (like most others probably) have a license agreement with Microsoft to provide its software to students very cheaply if not free.

It worked. I could get several-page papers written and edited without huge issues. As my academic career progressed, though, I ran into a number of limitations that made me seek out alternatives.

Black Text on White Screens—My Eyes!

As I needed to write more and more (and therefore spend more and more time on the computer), my eyes certainly felt it. The bright white background of MS Word and most word processors was too much. A white background and black text works lovely in print. The light reflected by the white paper is just the ambient light of your surroundings. Your eyes are already used to. On a computer screen, though, the white background is produced by turning on all the colors. The result: your eyes are being bombarded with all the photons of your computer’s backlight. Perhaps some people’s eyes can deal with that, but I soon found that long hours of it hurt.

Version Control

One of the joys of undergraduate studies is being able to write one or two drafts (well, let’s be realistic: one draft) before handing it in. When it comes to a larger thesis, dissertation, or article, though, you are going to have lots and lots of different drafts and versions.

For me, this meant folders filled with files like paper v1.docx, paper v2.docx, paper v2.someMoreChanges.docx, etc. I had no idea what had changed between versions, what the purpose of each version was supposed to be, or when certain changes had been made. It was a mess.

Organizing Thoughts

Gone also are the days where you write something once and only once and don’t need to do heavy-duty editing. Now I need to reuse bits and pieces between papers and move entire sections from one portion of the paper to another (or from one chapter to another chapter).

Granted, you can do that in Word. You can select a portion of text and drag it elsewhere or cut-and-paste. However, it’s not the easiest of processes. It just doesn’t seem made to handle reusing bits or rearranging easily.

While writing, I also often think of something I need to look up, some fact I should double check, or a random thought I would like to jot down for later reference. I don’t necessarily want this inner dialogue turned stream-of-consciousness notes to be read by advisors or peers. So, how do you make a comment in the middle of a paragraph within the text itself so it’s easily noticeable, without it being read when you send it out for comments from others? If you stick with just MS Word, I’m not sure there is a way.

An Alternative & Its Downsides

By the time I got to writing my dissertation proposal, I was looking for something else. A few friends and colleagues had recommended Scrivener. It certainly was a step up. I had more control over the writing environment and could easily organize and reorganize paragraphs, sections, etc. Because you compile at the very end and can select what to include or not in your compilation, you don’t really need to worry about version control. It’s also nice that you can organize things like photos, links, and stream of consciousness inner monologues easily without making them accessible to others.

I liked Scrivener well enough. Many of the features were overkill for me. And it’s nice that it’s cross-platform. There’s even a Linux version. However, all the time using it, I had a worry in the back of my mind: lock in. I didn’t want to get locked into always having to use Scrivener. I like to be able to (or at least know I will be able to) move my data from one program to another easily enough. Can you export (i.e., compile) your Scrivener project for another program? Yes, but some of your data gets lost in the process.

Moving from Programs to File Formats

It may seem that a benefit of MS Word is its seeming universality. Most word processors can read or import .docx files. Publishers often request submissions in MS Word. The Office Open XML (OOXML) or .docx file format is supposed to be an open standard. However, in practice it is anything but. In key ways, the .docx file format keeps people locked in. You can see more on this issue here.

Instead of asking what program should I use to type my papers, articles, theses, etc., a better question is: In what format should I store it? We’ve already seen how both MS Word and Scrivener work to keep users using only those solutions.

Perhaps a more open file format would be the actual standards-compliant Open Document Format (ODF). More and more European countries are mandating using ODF. However, not a lot of programs work easily with ODF besides LibreOffice. While LibreOffice is great and FLOSS (free/libre open source software), it suffers the very first problem I had with MS Word: the eye-numbing bright white background.

What’s the most open file format for text? Well, the good ol’ .txt file of course. It’s just a plain stream of text. There’s no special formatting to make you locked to a certain program. Pretty much every computing device from the 1970s mainframe to 2015’s Apple iPhone 6s can read a plain text file. And most likely all computing devices in my lifetime will continue to be able to read such files.

Of course, the downside of plain text is formatting. How can you format plain text? How can you make some text italics or mark something as a header or subheader? How do you include a title or footnotes? Plain text doesn’t have the ability to indicate such elements in a document.

The Solution: Markdown

Enter John Gruber’s Markdown. While originally created as an easy way to write HTML, Markdown has grown in popularity and use cases. The great thing about it is that’s easily human readable and machine readable. So, it’s easy to type and easy to convert to various formats, but also easy to read and write. For example, here’s what typing in Markdown looks like:

You can type in *italics* or make something **bold** easily. You can include a
link to [Google](http://www.google.com) or anythink else like so. It also
intelligently converts quotation marks, dashes---'single quotes' or "double
quotes"---and en-dashes, too. Good for when you have page numbers like 2--3.

When converted via a markdown parser like pandoc, the above paragraph renders like this:

You can type in italics or make something bold easily. You can include a link to Google or anythink else like so. It also intelligently converts quotation marks, dashes—‘single quotes’ or “double quotes”—and en-dashes, too. Good for when you have page numbers like 2–3.

Most importantly of all, Markdown is just a plain text file. Any program that can read text can read it. You can edit in any program like my favorite, vim, or you can use a simple text editor like gedit, kate, TextEditor, Notepad, etc. Or you can use LibreOffice or MS Office or some other word processor if you really want to.

So, how do you print or share it in the end? You’ll need to get it to others in some other format, probably MS Word or maybe a PDF. I use pandoc. It easily converts Markdown to MS Word, ODF, PDF (via LaTeX), and many, many other formats. It also extends the original Markdown specification, allowing for the easy addition of metadata (e.g., title, author, date, etc.) with which to construct a title page and formatting features like footnotes, lists, and tables. The default templates for Word and ODF format your headers, footnotes, and whatnot with pretty good defaults. But you can also make custom templates easily enough if you’d like. There’s also the supplementary tool pandoc-citeproc that will input citations and generate your bibliography all from your plain text source file. How awesome is that!?!

I find using Markdown as a format and not worrying about what program I use addresses the three main issues I had with using MS Word.

Any Editor = Any Color Scheme, Any Fonts

Separating the writing from the formatting means you can use anything to edit your original text. You use a bright color theme if you want like this in gedit:

Writing Markdown in gedit with Tango color theme

The color of the background and the formatting of the headers, footnotes, references, typefaces, etc. are all dealt with later. So, you can type using any font you want. (I prefer Mozilla’s Fira Mono.) If you prefer a dark background (like me) you can do that to. Here’s how vim looks when I’m writing:

Writing Markdown in vim

As you may have noticed, I’m writing this blog entry in Markdown, too. Hugo, the static site generator I use, has Markdown baked in, so it’s easy to format my text and add links, pictures, etc.

Great Version Control

Because Markdown is just plain text, you can use many different version control systems. The main ones are git and mercurial. These are really designed for collaborating on writing code and versioning the development of software. However, I find that they also work great for writing, too. You can create various branches, roll back to previous versions, and easily see what changes you’ve made to the documents throughout their history and why you made those changes.

You can also run tools like diff on your files to see what changes have been made between versions or to compare similar documents. I feel there’s just much more control over your text.

Many Ways to Organize

I must admit: Organization is one thing something like Scrivener does a much better job with. With a tool like that, it is easier to move various pieces and sections of your writing around. However, there are ways of doing it in Markdown.

One way is having different sections in different files. So, for instance, my dissertation is separated into different files for different chapters. This way I can just work on, edit, and distribute versions one chapter at a time. I have it set up so when I generate the final PDF version via LaTeX, I can specify to include only certain chapters.

Vim also gives you folding options. It’s a bit complicated, so I’ll get into it in more detail in a future post. Briefly, though, you can setup vim so that it is aware of section headers in your Markdown file. This will allow you to collapse sections—that is, it’ll hide the text in the section until you unfold it—and to quickly move entire sections around.

Finally, Markdown makes it easy to make comments to yourself. Because it was born out of easily writing HTML, Markdown allows you to type raw HTML if you’d like. This includes HTML comments. Anything you type that is enclosed between <!-- and --> is a comment in HTML. That means when you use pandoc to convert your file to MS Word, ODF, PDF, or something else, it won’t include those comments.

Some Downsides & Conclusion

So that’s briefly why I’ve gone to doing most of my writing in Markdown. I can use any editor, make it look however I want while typing without impacting the final formatting, and easily convert it to something else in the future if I want. I’m not bound to using any one program.

That said, there are a few downsides to using Markdown. One is receiving comments from people. Most people I need to send my stuff to want MS Word documents. That’s easy enough to provide thanks to pandoc. However, then they usually use the track changes and comments features in MS Word to make edits. That’s understandable: There’s really no standard way of commenting or editing Markdown.

Another possible downside is in separating the writing from the formatting. Yes, it’s nice that while you’re typing you can just specify what’s supposed to be in italics, what’s a header, what’s a footnote, etc. within the plain text without worrying about how it’s actually going to look in the end. However, a downside of that is also that you don’t know what it’s going to look like in the end. It’s sometimes hard to know how long the document is, how paragraphs are going to wrap, etc. So, when you’re converting to .docx or typesetting in PDF, it can take a lot of trial and error to get it to look how you want it to. And you can spend a fair amount of time fiddling with pandoc’s templates or various settings in LaTeX to get things to look just right. Some may find it easier to do this trial and error while they’re actually typing the document so they don’t have to worry about it later. For me, though, I like being able to just write without worrying how the final product is going to look until later.

In future posts in this series on workflow, I hope to go into a little more detail on the nitty gritty of this setup and what works for me.