Tinkering a bit more with Github Actions, I’ve hacked together some sort of workflow for testing notebooks in a set of specified directories and then clearing the notebook output cells, zipping the notebooks into a release zip file, and then making the release zip file via a github release page.
The test and release is action is triggered by making a release with a body that contains a list of comma separate directory paths identifying the directories we want in the release. For example:
The following action is triggered by a release creation event:
name: example-release
on:
release:
types:
- created
workflow_dispatch
jobs:
release-demo:
runs-on: ubuntu-latest
container:
image: ouvocl/vce-tm351-monolith
env:
RELEASE_DIRS: ${{ github.event.release.body }}
steps:
- uses: actions/checkout@master
- name: Install nbval (TH edition) and workflow tools
run: |
python3 -m pip install --upgrade https://github.com//ouseful-PR/nbval/archive/table-test.zip
python3 -m pip install https://github.com/innovationOUtside/nb_workflow_tools/archive/master.zip
- name: Restart postgres
run: |
sudo service postgresql restart
- name: Start mongo
run: |
sudo mongod --fork --logpath /dev/stdout --dbpath ${MONGO_DB_PATH}
- name: Get directories
run: |
#IFS=$"\n" read -a file_paths <<< "${{ github.event.head_commit }}"
IFS="," read -a file_paths <<< "${{ github.event.release.body }}"
ls
# Test all directories
for file_path in "${file_paths[@]}"; do
pwd
py.test --nbval "$file_path" || continue
done
shell: bash
# For testing...
continue-on-error: true
- name: Create zipped files
run: |
IFS="," read -a file_paths <<< "${{ github.event.release.body }}"
for file_path in "${file_paths[@]}"; do
tm351zip -r clearOutput -a "$file_path" release.zip
done
echo "Release paths: $RELEASE_DIRS" > release-files.txt
echo "\n\nRelease zip contents:\n" >> release-files.txt
tm351zipview release.zip >> release-files.txt
shell: bash
- name: Create Release
id: create_release
uses: softprops/action-gh-release@v1
# The commit must be tagged for a release to happen
# Tags can be added via Github Desktop app
# https://docs.github.com/en/desktop/contributing-and-collaborating-using-github-desktop/managing-commits/managing-tags#creating-a-tag
with:
tag_name: ${{ github.ref }}-files
name: ${{ github.event.release.name }} files
#body: "Release files/directories: ${RELEASE_DIRS}"
body_path: release-files.txt
files: |
release.zip
It then runs the tests and then generates another release that includes the cleaned and zipped release files:
Ideally, we’d just add the zip file to the original release but I couldn’t spot a way to do that.
At the moment the action will publish the file release even if some notebook tests fail. A production action should fail if a test fails, or perhaps parse the release name and ignore the fails if the original release name contains a particular flag (for example, --force).
The idea of using the release form to create the release was to try to simplify the workflow and allow a release to be generated quite straightforwardly from a repository on the Github website.
OUseful.Info, the blog... 