Coming back down to Earth

Jupyter has a very wide range of formats to export the content of a notebook; some of them are a graphical export (e.g. html or pdf – this requires a latex installation) or textual export (ascii, rtf), etc.

To start creating a python script from your notebook, open it into Jupyter, then

1. from the File menu select save and export notebook as
2. from the submenu select Executable script this will save a file in your Download directory named with the title of your notebook, but with .py extension

You can now open this file with your favorite code editor; here is what you will see:

• before every cell there will be a comment with the cell execution number
• every code cell will be copied in an individual block of commands
• markdown cells and raw cells will be presented as comments
• output will be removed

In the following sections I will list some cleanup actions which will help you transform this script into a more manageable piece of code

magic code is translated into the equivalent python call e.g.

?sum

becomes

get_ipython().run_line_magic('pinfo', 'sum')

most of the time you may want to get rid of all of this kind of code as some functionalities (e.g. accessing documentation) are intended only for interactive usage within a jupyter notebook.

Other functionalities (e.g. timing your cell execution) may be better managed with other libraries.

Jupyter conveniently shows pandas =DataFrame=s as tables; you may want to extract these results into files.

To access these tables from a script you can save them into files

the following example works for matplotlib and any library based on it, like seaborn.

import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
california = pd.read_csv("california_pb_2023.csv")

The simplest way to save an image is to

1. store the Axis object created into some variable
2. reach the Figure object from the .figure attribute
3. use the .savefig() method

ax = sns.histplot(california,x="Daily Mean Pb Concentration")
ax.figure.savefig("pb_2003.png")

A more complex sequence is required when working with multiple plots either stacked or overlapped.

In this case the pd.subplot() function creates multiple charts (axis) in a single figure

# this code is general: two charts in a single row
fig, (ax1, ax2) = plt.subplots(1, 2)
fig.suptitle('Horizontally stacked subplots')

# this code is specific for this function
from scipy import stats
stats.probplot(california["Daily Mean Pb Concentration"], plot=ax1)
stats.probplot(california["Daily Mean Pb Concentration"], plot=ax2,dist=stats.distributions.lognorm(s=1))

fig.savefig("probplots.png")

The following suggestions holds for any python script and are not strictly required for the execution.

• move all import statements at the beginning of the file
• organize the code in functions and classes; possibly add type annotations
• create a single entry point at the bottom of the code with the usual python if __name__ = “main”: main()=
• add command line options management using libraries like optparse (see here)
• separate data and configuration from code: libraries like toml (see here) can help reading configuration files
• transform absolute paths into relative paths
• consider using pyproject.html to collect dependencies and constraints
• consider using a linter (e.g. pylint or ruff) to evaluate code inconsistencies
• create unit tests to verify your functions individually; pytest helps in this task
• add documentation per each function or class as well as a module doc string
• use a code formatter to keep your style consistent (e.g. black)

I find it very convenient to have a shell script taking care of

• setting up the working directory of the process properly
• activate any virtual environment as needed
• fix the environment variables

e.g.

#!/bin/bash

# change the process directory to this one
cd $(dirname $0)

# activate a local virtual environment
source .venv/bin/activate

# set up some environment variables
export PYTHONPATH=$(pwd)

# launch the application
# forwards all command line arguments
python -m myapp $@