Comparing doc tools: Stop doing the apples-to-oranges thing
The best way to assess documentation tools is to analyze the entire toolchain
If you've changed documentation tools, or even gone shopping for a new set of tools, you've probably noticed that it's difficult to compare, because it's never an apples-to-apples comparison between the old set and the new set.
When people first wade into the Sargasso Sea of tool choices, they ask questions like:
Is Markdown better than Framemaker?
Is Jekyll better and Flare?
Is Adobe Experience Manager a better approach than DITA with GitHub?
These questions are difficult to answer. In the first example, Markdown is a markup language and the other choice is a software package that can build output in multiple formats. If you choose Markdown you still need to choose a tool to convert your source files to the output format and assemble the final document. If you choose Framemaker, you live with certain restrictions; Framemaker doesn't fit easily into a docs-like-code type of workflow, because the source files are binary which isn't compatible with Git and other SCM systems. The latter examples are loaded with similar complications.
The best approach when considering new tools is to examine the entire toolchain that you would use with each tool being compared. You can only make a useful decision if you consider all the tools and resources in the process that you use to create source files and turn them into published documentation.
Toolchain layers
I describe components of the toolchain as layers, because they are they are like the layers in a network -- they can be defined separately in an abstract way, although in practice the layers might seem to overlap or merge because some tools perform tasks that belong on more than one layer.
The layers aren't ordered steps. They don't describe a linear process but rather describe categorical functions that most toolchains must have.
Here are the layers you typically need to think about when documenting a toolchain:
Deployment - Delivery of the final documentation to customers. After you have built a website or PDF, there must still be some kind of system to deliver them. Examples: Read the Docs, Netlify, Google Content Delivery Network.
Build - Source files become an ordered website or PDF book, including navigation or a table of contents. Examples: Pandoc, MKdoc, DITA-OT, Madcap Flare, Framemaker, Jekyll.
Transform - Source files start out in some markup language (Markdown, Dita, plain text), and become transformed into the final format, often HTML. Examples: Pandoc, MKdoc, DITA-OT, Madcap Flare, Framemaker, Jekyll.
Source control - The system that tracks and manages changes in the source code. Examples: Git, and Perforce.
Editing tools - Software that can be used to modify the source files. This can be a task-specific application or it could be a plain text editor. Examples: OxygenXML Editor, Atom, Framemaker.
Markup language - The machine-readable tags used to define elements in the source files. Examples: Markdown, DITA, XHTML, ReStructured Text, AsciiDoc.
Examples of toolchains and comparisons
Here's a toolchain table that describes one of my sites:
Note that Oxygen XML Editor is not only the editing tool, but also performs the transform and the build.
Since we're talking about toolchain comparisons, here's a table that weighs the current toolchain with a potential replacement.
This table shows that the difference is in the build and transform steps. The old method uses the Oxygen XML Editor, which can output a website at the push of a button. This is an example of one tool occupying several layers; Oxygen XML Editor is the editing tool and also performs the build and the transform.
The new method would make use of the DITA-OT executable to transform HTML pages but would require some additional scripting to stitch the output pages into a site.
(Incidentally, the benefit of using DITA-OT is that the entire process can be automated, which is not possible with the Oxygen XML Editor, because it violates the terms of the individual license. Oxygen XML Editor is still in the toolchain as the editing tool.)
Don't do the assessment yourself
The best approach is to look at the entire toolchain and involve as many other people as possible, even if it leads to tools that weren't your first choice.
To make an informed decision about tools, do all of the following.
Use a table like the examples above to document your current toolchain and the proposed replacement, or replacements.
If you haven't used a particular tool that you need to assess, find someone who has and ask them about how it is used so that you know your proposed toolchain is valid.
Seek feedback from anyone who would use the tools.
Check in with anyone who would have to maintain tools or deploy the documentation. Would your output format be satisfactory for them? They might need some sample output to decide.
Check in with any stakeholders who might have strong feelings about the documentation, including support, field engineers, product and program management. Would your new documentation have all the features they require? (They might have something to say about headings with embedded anchor links, search capabilities, or ease of navigation.)