Authoring Reusable Code in Python Files from Jupyter Notebooks Using %%writefile and %run magics or autoreloaded file modules

One of the charges often made against Jupyter notebooks is that you can often end up locking code up in a notebook in a less than portable way: the code runs fine in the notebook, but how do you use it elsewhere?

If you install the Python Jupytext package, you can actually author python files (files with a .py suffix) in a notebook environment, with code cells saved into the Python file as code and markdown saved into the file as comments.

But I realised there’s another IPython supported trick that preserves the notebook authoring and code use practice whilst at the same time making code available in a file that you can load into other notebooks. The trick relies on the pattern:

• save code in in a code cell to a file using the %%writefile FILENAME.py block cell magic; note that this saves but does not run the code in the code cell; by default, %%writefile will overwrite a pre-existing file of the same name;
• in the next cell, use the %run FILENAME.py to run the contents of the file; this has the same effect as running the unmagicked code cell.

The file is executed in an empty namespace containing shared global objects (such as previously imported modules) but not local, interactively set values. After execution, the IPython interactive namespace is updated with all variables defined by the program. Using the -i flag will load and run the script in the same interactive/IPython namespace as the unmagicked code cells.

This pattern allows you to save the code in one cell and then run it in the next. The contents of the saved file can also be run in another notebook using the %run / %run -i line magic.

If you want to add the content of additional code cells to a pre-existing file, you can use the -a (append) character with the block magic (for example, %%writefile -a FILENAME.py. Note that if you repeatedly run such a cell, with slightly updated code each time, for example, the new code will get appended to the file each time, so you may have multiple versions of the code saved to the same file.

Another way of accessing the contents of the saved file is just to treat it as a module. For example, from FILENAME import * or from FILENAME import MYFUNCTION. Note that trying to reimport a package of a function from a package will not generally work if that item has already been imported into a Python session, even if the underlying code has changed.

However, another magic can be used to force the reload of imported items whenever a code cell is run: autoreload. Load in the magic with %load_ext autoreload and then enable automatic reloads with %autoreload 2. If you now use the pattern of writing the contents of a code cell to a file, and then you run a cell to import the required function, each time you call the function, the latest version of the module, and any functions defined within it, will be read in from the file via the autoreload.