
======= From Doconce to Other Formats =======
label{doconce2formats}

Transformation of a Doconce document `mydoc.do.txt` to various other
formats applies the script `doconce format`:
!bc sys
Terminal> doconce format format mydoc.do.txt
!ec
or just
!bc sys
Terminal> doconce format format mydoc
!ec
The `mako` or `preprocess` programs are always used to preprocess the
file first, and options to `mako` or `preprocess` can be added after the
filename. For example,
!bc sys
Terminal> doconce format latex mydoc -Dextra_sections -DVAR1=5     # preprocess
Terminal> doconce format latex yourdoc extra_sections=True VAR1=5  # mako
!ec
The variable `FORMAT` is always defined as the current format when
running `preprocess`. That is, in the last example, `FORMAT` is
defined as `latex`. Inside the Doconce document one can then perform
format specific actions through tests like `#if FORMAT == "latex"`.

Inline comments in the text are removed from the output by
!bc sys
Terminal> doconce format latex mydoc --skip_inline_comments
!ec
One can also remove all such comments from the original Doconce
file by running:
!bc
Terminal> doconce remove_inline_comments mydoc
!ec
This action is convenient when a Doconce document reaches its final form
and comments by different authors should be removed.


===== HTML =====

Making an HTML version of a Doconce file `mydoc.do.txt`
is performed by
!bc sys
Terminal> doconce format html mydoc
!ec
The resulting file `mydoc.html` can be loaded into any web browser for viewing.

===== Pandoc =====

Output in Pandoc's extended Markdown format results from
!bc sys
Terminal> doconce format pandoc mydoc
!ec
The name of the output file is `mydoc.mkd`.
From this format one can go to numerous other formats:
!bc sys
Terminal> pandoc -R -t mediawiki -o mydoc.mwk mydoc.mkd
!ec
Pandoc supports `latex`, `html`, `odt` (OpenOffice), `docx` (Microsoft
Word), `rtf`, `texinfo`, to mention some. The `-R` option makes
Pandoc pass raw HTML or LaTeX to the output format instead of ignoring it.
See the "Pandoc documentation": "http://johnmacfarlane.net/pandoc/README.html"
for the many features of the `pandoc` program.

Pandoc is useful to go from LaTeX mathematics to, e.g., HTML or MS Word.
There are two ways (experiment to find the best one for your document):
`doconce format pandoc` and then translating using `pandoc`, or
`doconce format latex`, and then going from LaTeX to the desired format
using `pandoc`.
Here is an example on the latter strategy:
!bc sys
Terminal> doconce format latex mydoc
Terminal> doconce ptex2tex mydoc
Terminal> pandoc -f latex -t docx -o mydoc.docx mydoc.tex
!ec
When we go through `pandoc`, only single equations or `align*`
environments are well understood.

Quite some `doconce replace` and `doconce subst` edits might be needed
on the `.mkd` or `.tex` files to successfully have mathematics that is
well translated to MS Word.  Also when going to reStructuredText using
Pandoc, it can be advantageous to go via LaTeX.

Here is an example where we take a Doconce snippet (without title, author,
and date), maybe with some unnumbered equations, and quickly generate
HTML with mathematics displayed my MathJax:
!bc sys
Terminal> doconce format pandoc mydoc
Terminal> pandoc -t html -o mydoc.html -s --mathjax mydoc.mkd
!ec
The `-s` option adds a proper header and footer to the `mydoc.html` file.
This recipe is a quick way of makeing HTML notes with (some) mathematics.


===== LaTeX =====

Making a LaTeX file `mydoc.tex` from `mydoc.do.txt` is done in two steps:
# Note: putting code blocks inside a list is not successful in many
# formats - the text may be messed up. A better choice is a paragraph
# environment, as used here.

__Step 1.__ Filter the doconce text to a pre-LaTeX form `mydoc.p.tex` for
     `ptex2tex`:
!bc sys
Terminal> doconce format latex mydoc
!ec
LaTeX-specific commands ("newcommands") in math formulas and similar
can be placed in files `newcommands.tex`, `newcommands_keep.tex`, or
`newcommands_replace.tex` (see Section ref{newcommands}).
If these files are present, they are included in the LaTeX document
so that your commands are defined.

__Step 2.__ Run `ptex2tex` (if you have it) to make a standard LaTeX file,
!bc sys
Terminal> ptex2tex mydoc
!ec
In case you do not have `ptex2tex`, you may run a (very) simplified version:
!bc sys
Terminal> doconce ptex2tex mydoc
!ec

Note that Doconce generates a `.p.tex` file with some preprocessor macros
that can be used to steer certain properties of the LaTeX document.
For example, to turn on the Helvetica font instead of the standard
Computer Modern font, run
!bc sys
Terminal> ptex2tex -DHELVETICA mydoc
!ec
The title, authors, and date are by default typeset in a non-standard
way to enable a nicer treatment of multiple authors having
institutions in common. However, the standard LaTeX "maketitle" heading
is also available through
!bc sys
Terminal> ptex2tex -DLATEX_HEADING=traditional mydoc
!ec
A separate titlepage can be generate by
!bc sys
Terminal> ptex2tex -DLATEX_HEADING=titlepage mydoc
!ec

The `ptex2tex` tool makes it possible to easily switch between many
different fancy formattings of computer or verbatim code in LaTeX
documents. After any `!bc` command in the Doconce source you can
insert verbatim block styles as defined in your `.ptex2tex.cfg`
file, e.g., `!bc cod` for a code snippet, where `cod` is set to
a certain environment in `.ptex2tex.cfg` (e.g., `CodeIntended`).
There are about 40 styles to choose from.

Also the `doconce ptex2tex` command supports preprocessor directives
for processing the `.p.tex` file. The command allows specifications
of code environments as well. Here is an example:
!bc sys
Terminal> doconce ptex2tex -DLATEX_HEADING=traditional -DMINTED \
          cycod=\begin{quote}\begin{python};\end{python}\end{quote} \
          fpro=minted fcod=minted sys=verbatim
!ec
Note that semicolon must be used to separate the begin and end
commands, unless only the environment name is given (such as
`verbatim` above, which implies `\begin{verbatim}` and `\end{verbatim}`).
The value `minted` can be used for code environments where the
language is specified, as in `fpro`, where `minted` implies
`\begin{python}{fortran}`. There is a similar support for `ans`:
`cppcod=ans` imples `\begin{c++}` and `\end{c++}` using the
`anslistings` package.

__Step 2b (optional).__ Edit the `mydoc.tex` file to your needs.
For example, you may want to substitute `section` by `section*` to
avoid numbering of sections, you may want to insert linebreaks
(and perhaps space) in the title, etc. This can be automatically
edited with the aid of the `doconce replace` and `doconce subst`
commands. The former works with substituting text directly, while the
latter performs substitutions using regular expressions.
Here are some examples:
!bc sys
Terminal> doconce replace 'section{' 'section*{' mydoc.tex
Terminal> doconce subst 'title\{(.+)Using (.+)\}' \
          'title{\g<1> \\\\ [1.5mm] Using \g<2>' mydoc.tex
!ec
A lot of tailored fixes to the LaTeX document can be done by
an appropriate set of text replacements and regular expression
substitutions. You are anyway encourged to make a script for
generating PDF from the LaTeX file.

__Step 3.__ Compile `mydoc.tex`
and create the PDF file:
!bc sys
Terminal> latex mydoc
Terminal> latex mydoc
Terminal> makeindex mydoc   # if index
Terminal> bibitem mydoc     # if bibliography
Terminal> latex mydoc
Terminal> dvipdf mydoc
!ec
If one wishes to use the `Minted_Python`, `Minted_Cpp`, etc.,
environments in `ptex2tex` for typesetting code (specified, e.g., in
the `*pro` and `*cod` environments in `.ptex2tex.cfg` or
`$HOME/.ptex2tex.cfg`), the `minted` LaTeX package is needed.  This
package is included by running `doconce format` with the `-DMINTED`
option:
!bc sys
Terminal> ptex2tex -DMINTED mydoc
!ec
In this case, `latex` must be run with the
`-shell-escape` option:
!bc sys
Terminal> latex -shell-escape mydoc
Terminal> latex -shell-escape mydoc
Terminal> makeindex mydoc   # if index
Terminal> bibitem mydoc     # if bibliography
Terminal> latex -shell-escape mydoc
Terminal> dvipdf mydoc
!ec


===== PDFLaTeX =====

Running `pdflatex` instead of `latex` follows almost the same steps,
but the start is
!bc sys
Terminal> doconce format latex mydoc
!ec
Then `ptex2tex` is run as explained above, and finally
!bc sys
Terminal> pdflatex -shell-escape mydoc
Terminal> makeindex mydoc   # if index
Terminal> bibitem mydoc     # if bibliography
Terminal> pdflatex -shell-escape mydoc
!ec

===== Plain ASCII Text =====

We can go from Doconce "back to" plain untagged text suitable for viewing
in terminal windows, inclusion in email text, or for insertion in
computer source code:
!bc sys
Terminal> doconce format plain mydoc.do.txt  # results in mydoc.txt
!ec

===== reStructuredText =====

Going from Doconce to reStructuredText gives a lot of possibilities to
go to other formats. First we filter the Doconce text to a
reStructuredText file `mydoc.rst`:
!bc sys
Terminal> doconce format rst mydoc.do.txt
!ec
We may now produce various other formats:
!bc sys
Terminal> rst2html.py  mydoc.rst > mydoc.html # html
Terminal> rst2latex.py mydoc.rst > mydoc.tex  # latex
Terminal> rst2xml.py   mydoc.rst > mydoc.xml  # XML
Terminal> rst2odt.py   mydoc.rst > mydoc.odt  # OpenOffice
!ec

The OpenOffice file `mydoc.odt` can be loaded into OpenOffice and
saved in, among other things, the RTF format or the Microsoft Word format.
However, it is more convenient to use the program `unovonv`
to convert between the many formats OpenOffice supports *on the command line*.
Run
!bc sys
Terminal> unoconv --show
!ec
to see all the formats that are supported.
For example, the following commands take
`mydoc.odt` to Microsoft Office Open XML format,
classic MS Word format, and PDF:
!bc sys
Terminal> unoconv -f ooxml mydoc.odt
Terminal> unoconv -f doc mydoc.odt
Terminal> unoconv -f pdf mydoc.odt
!ec

__Remark about Mathematical Typesetting.__
At the time of this writing, there is no easy way to go from Doconce
and LaTeX mathematics to reST and further to OpenOffice and the
"MS Word world". Mathematics is only fully supported by `latex` as
output and to a wide extent also supported by the `sphinx` output format.
Some links for going from LaTeX to Word are listed below.

 * URL:"http://ubuntuforums.org/showthread.php?t=1033441"
 * URL:"http://tug.org/utilities/texconv/textopc.html"
 * URL:"http://nileshbansal.blogspot.com/2007/12/latex-to-openofficeword.html"



===== Sphinx =====

Sphinx documents demand quite some steps in their creation. We have automated
most of the steps through the `doconce sphinx_dir` command:
!bc sys
Terminal> doconce sphinx_dir author="authors' names" \
          title="some title" version=1.0 dirname=sphinxdir \
          theme=mytheme file1 file2 file3 ...
!ec
The keywords `author`, `title`, and `version` are used in the headings
of the Sphinx document. By default, `version` is 1.0 and the script
will try to deduce authors and title from the doconce files `file1`,
`file2`, etc. that together represent the whole document. Note that
none of the individual Doconce files `file1`, `file2`, etc. should
include the rest as their union makes up the whole document.
The default value of `dirname` is `sphinx-rootdir`. The `theme`
keyword is used to set the theme for design of HTML output from
Sphinx (the default theme is `'default'`).

With a single-file document in `mydoc.do.txt` one often just runs
!bc sys
Terminal> doconce sphinx_dir mydoc
!ec
and then an appropriate Sphinx directory `sphinx-rootdir` is made with
relevant files.

The `doconce sphinx_dir` command generates a script
`automake-sphinx.py` for compiling the Sphinx document into an HTML
document.  One can either run `automake-sphinx.py` or perform the
steps in the script manually, possibly with necessary modifications.
You should at least read the script prior to executing it to have
some idea of what is done.

Te `doconce sphinx_dir` script copies directories named `figs` or `figures`
over to the Sphinx directory so that figures are accessible in the
Sphinx compilation.  If figures or movies are located in other
directories, `automake-sphinx.py` must be edited accordingly.
Links to local files (not `http:` or `file:` URLs) must be placed
in the `_static` subdirectory of the Sphinx directory. The
utility `doconce sphinxfix_localURLs` is run to check for local
links: for each such link, say `dir1/dir2/myfile.txt` it replaces
the link by `_static/myfile.txt` and copies `dir1/dir2/myfile.txt`
to a local `_static` directory (in the same directory as the
script is run). The user must copy all `_static/*` files to the
`_static` subdirectory of the Sphinx directory. Links to local
HTML files (say another Sphinx document) may present a problem if they link
to other files: all necessary files must be correctly copied to
the `_static` subdirectory of the Sphinx directory.
It may be wise to place relevant files in a `_static` directory
and link to these directly from the Doconce document - then links
to not need to be modified when creating  a Sphinx version of
the document.

Doconce comes with a collection of HTML themes for Sphinx documents.
These are packed out in the Sphinx directory, the `conf.py`
configuration file for Sphinx is edited accordingly, and a script
`make-themes.sh` can make HTML documents with one or more themes.
For example,
to realize the themes `fenics` and `pyramid`, one writes
!bc sys
Terminal> ./make-themes.sh fenics pyramid
!ec
The resulting directories with HTML documents are `_build/html_fenics`
and `_build/html_pyramid`, respectively. Without arguments,
`make-themes.sh` makes all available themes (!).

If it is not desirable to use the autogenerated scripts explained
above, here is the complete manual procedure of generating a
Sphinx document from a file `mydoc.do.txt`.

__Step 1.__ Translate Doconce into the Sphinx format:
!bc sys
Terminal> doconce format sphinx mydoc
!ec

__Step 2.__ Create a Sphinx root directory
either manually or by using the interactive `sphinx-quickstart`
program. Here is a scripted version of the steps with the latter:
!bc sys
mkdir sphinx-rootdir
sphinx-quickstart <<EOF
sphinx-rootdir
n
_
Name of My Sphinx Document
Author
version
version
.rst
index
n
y
n
n
n
n
y
n
n
y
y
y
EOF
!ec
The autogenerated `conf.py` file
may need some edits if you want to specific layout (Sphinx themes)
of HTML pages. The `doconce sphinx_dir` generator makes an extended `conv.py`
file where, among other things, several useful Sphinx extensions
are included.


__Step 3.__ Copy the `mydoc.rst` file to the Sphinx root directory:
!bc sys
Terminal> cp mydoc.rst sphinx-rootdir
!ec
If you have figures in your document, the relative paths to those will
be invalid when you work with `mydoc.rst` in the `sphinx-rootdir`
directory. Either edit `mydoc.rst` so that figure file paths are correct,
or simply copy your figure directories to `sphinx-rootdir`.
Links to local files in `mydoc.rst` must be modified to links to
files in the `_static` directory, see comment above.

__Step 4.__ Edit the generated `index.rst` file so that `mydoc.rst`
is included, i.e., add `mydoc` to the `toctree` section so that it becomes
!bc
.. toctree::
   :maxdepth: 2

   mydoc
!ec
(The spaces before `mydoc` are important!)

__Step 5.__ Generate, for instance, an HTML version of the Sphinx source:
!bc sys
make clean   # remove old versions
make html
!ec

Sphinx can generate a range of different formats:
standalone HTML, HTML in separate directories with `index.html` files,
a large single HTML file, JSON files, various help files (the qthelp, HTML,
and Devhelp projects), epub, LaTeX, PDF (via LaTeX), pure text, man pages,
and Texinfo files.

__Step 6.__ View the result:
!bc sys
Terminal> firefox _build/html/index.html
!ec

Note that verbatim code blocks can be typeset in a variety of ways
depending the argument that follows `!bc`: `cod` gives Python
(`code-block:: python` in Sphinx syntax) and `cppcod` gives C++, but
all such arguments can be customized both for Sphinx and LaTeX output.


===== Wiki Formats =====

There are many different wiki formats, but Doconce only supports three:
"Googlecode wiki": "http://code.google.com/p/support/wiki/WikiSyntax<Google Code>", , MediaWiki, and Creole Wiki. These formats are called
`gwiki`, `mwiki`, and `cwiki`, respectively.
Transformation from Doconce to these formats is done by
!bc sys
Terminal> doconce format gwiki mydoc.do.txt
Terminal> doconce format mwiki mydoc.do.txt
Terminal> doconce format cwiki mydoc.do.txt
!ec

The Googlecode wiki document, `mydoc.gwiki`, is most conveniently stored
in a directory which is a clone of the wiki part of the Googlecode project.
This is far easier than copying and pasting the entire text into the
wiki editor in a web browser.

When the Doconce file contains figures, each figure filename must in
the `.gwiki` file be replaced by a URL where the figure is
available. There are instructions in the file for doing this. Usually,
one performs this substitution automatically (see next section).

From the MediaWiki format one can go to other formats with aid
of "mwlib": "http://pediapress.com/code/". This means that one can
easily use Doconce to write "Wikibooks": "http://en.wikibooks.org"
and publish these in PDF and MediaWiki format.
At the same time, the book can also be published as a
standard LaTeX book or a Sphinx web document.


===== Tweaking the Doconce Output =====

Occasionally, one would like to tweak the output in a certain format
from Doconce. One example is figure filenames when transforming
Doconce to reStructuredText. Since Doconce does not know if the
`.rst` file is going to be filtered to LaTeX or HTML, it cannot know
if `.eps` or `.png` is the most appropriate image filename.
The solution is to use a text substitution command or code with, e.g., sed,
perl, python, or scitools subst, to automatically edit the output file
from Doconce. It is then wise to run Doconce and the editing commands
from a script to automate all steps in going from Doconce to the final
format(s). The `make.sh` files in `docs/manual` and `docs/tutorial`
constitute comprehensive examples on how such scripts can be made.
