Some Jupyter Notebook / nbconvert Housekeeping Hints

A few snippets and bits of pieces regarding housekeeping around Jupyter notebooks.

Clearing Output Cells

Via Matthias Bussonnier, this handy command for rendering a version of a notebook with a the output cells cleared:

jupyter nbconvert --to notebook --ClearOutputPreprocessor.enabled=True NOTEBOOK.ipynb

Adding --inplace will rewrite the notebook with cleared output cells.

Custom Templates

If you have a custom template or a custom config file in the current directory, you can invoke them using:

jupyter nbconvert --config=my_config.py NOTEBOOK.ipynb
jupyter nbconvert --template=my_template.tpl NOTEBOOK.ipynb

I found running the --log-level=DEBUG flag was also handy…

Via MinRk, additional paths can be set using c.TemplateExporter.template_path.append('/path/to/templates') though I’m not really sure where that setting needs to be applied. (Whilst I love the Jupyter project, I really struggle to keep track of where things are supposed to be located and which bits are working/don’t work anymore:-(

He also notes that [a]bsolute template paths will also work if you specify: c.TemplateExporter.template_path.append('/'), adding the comment that [a]bsolute paths for templates should probably work without modifying template_path, but they don’t right now.

It would be really handy if the ability to specify an absolute path in the command line setting did work out of the can…

Split a Long Notebook into Multiple Sub-Notebooks

The Jupyter notebooks allow you to split long cells into two cells using the cursor as a split point, but how about splitting a long notebook into multiple notebooks?

The following test script will split a notebook into sub-notebooks at an explicit split point – a markdown cell containing just the string SPLIT NOTEBOOK.

import IPython.nbformat as nb
import IPython.nbformat.v4.nbbase as nb4

mynb=nb.read('TEST_LONG_NOTEBOOK.ipynb',nb.NO_CONVERT)
#Partition a long notebook into subnotebooks at specificed split points
#Enter SPLIT NOTEBOOK on its own in a markdown cell to specify a split point
c=1
test=nb4.new_notebook()
for i in mynb['cells']:
    if (i['cell_type']=='markdown'):
        if ('SPLIT NOTEBOOK' in i['source']):
            nb.write(test,'subNotebook{}.ipynb'.format(c))
            c=c+1
            test=nb4.new_notebook()
        else:
            test.cells.append(nb4.new_markdown_cell(i['source']))
    elif (i['cell_type']=='code'):
        cc=nb4.new_code_cell(i['source'])
        for o in i['outputs']:
            cc['outputs'].append(o)
        test.cells.append(cc)
nb.write(test,'subNotebook{}.ipynb'.format(c))

I should probably tidy this up so that it reuses the original notebook name rather than the subNotebook stub. It might also be handy to turn this into a notebook extension that lets splits the current notebook into two relative to the current cursor location (e.g. all cells above the selected cell go in one sub-notebook, everything from the selected cell to the end of the notebook going into a second sub-notebook.

Another refinement might allow for the declaration of a comment set of cells to prefix each sub-notebook. By default, this could be the set of cells prior to the first split point. (Which is to say for N split points, there would be N, rather than N+1, sub-notebooks, the cells above the first split point appearing in each sub-notebook. The first sub-notebook would thus contain cells starting with the first cell after the first split point, prefixed by the cells appearing before the first split point; and the last sub-notebook would contain the cells starting with the first cell after the last split point, prefixed once again by the cells appearing before the first split point.

A second utility to merge, or concatenate, two or more notebooks when provided with their filenames might also be handy…

Anything Else?

So what other handy Jupyter notebook / nbconvert housekeeping hints and tricks am I missing?