More Automation Sketches – Creating Student Notebook Releases

Tinkering a bit more with Github Actions, I’ve hacked together some sort of workflow for testing notebooks in a set of specified directories and then clearing the notebook output cells, zipping the notebooks into a release zip file, and then making the release zip file via a github release page.

The test and release is action is triggered by making a release with a body that contains a list of comma separate directory paths identifying the directories we want in the release. For example:

The following action is triggered by a release creation event:

name: example-release
on:
  release:
    types:
      - created
  workflow_dispatch
        
jobs:
  release-demo:
    runs-on: ubuntu-latest
    container:
      image: ouvocl/vce-tm351-monolith
    env:
      RELEASE_DIRS: ${{ github.event.release.body }}
    steps:
    - uses: actions/checkout@master
    - name: Install nbval (TH edition) and workflow tools
      run: |
        python3 -m pip install --upgrade https://github.com//ouseful-PR/nbval/archive/table-test.zip
        python3 -m pip install https://github.com/innovationOUtside/nb_workflow_tools/archive/master.zip
    - name: Restart postgres
      run: |
        sudo service postgresql restart
    - name: Start mongo
      run: |
        sudo mongod --fork --logpath /dev/stdout --dbpath ${MONGO_DB_PATH}
    - name: Get directories
      run: |
        #IFS=$"\n" read -a file_paths <<< "${{ github.event.head_commit }}"
        IFS="," read -a file_paths <<< "${{ github.event.release.body }}"
        ls
        # Test all directories
        for file_path in "${file_paths[@]}"; do
          pwd
          py.test  --nbval "$file_path" || continue
        done
      shell: bash
      # For testing...
      continue-on-error: true
    - name: Create zipped files
      run: |
        IFS="," read -a file_paths <<< "${{ github.event.release.body }}"
        for file_path in "${file_paths[@]}"; do
          tm351zip -r clearOutput -a "$file_path" release.zip
        done
        echo "Release paths: $RELEASE_DIRS" > release-files.txt
        echo "\n\nRelease zip contents:\n" >> release-files.txt
        tm351zipview release.zip >> release-files.txt
      shell: bash
    - name: Create Release
      id: create_release
      uses: softprops/action-gh-release@v1
      # The commit must be tagged for a release to happen
      # Tags can be added via Github Desktop app
      # https://docs.github.com/en/desktop/contributing-and-collaborating-using-github-desktop/managing-commits/managing-tags#creating-a-tag
      with:
        tag_name: ${{ github.ref }}-files
        name: ${{ github.event.release.name }} files
        #body: "Release files/directories: ${RELEASE_DIRS}"
        body_path: release-files.txt
        files: |
          release.zip

It then runs the tests and then generates another release that includes the cleaned and zipped release files:

Ideally, we’d just add the zip file to the original release but I couldn’t spot a way to do that.

At the moment the action will publish the file release even if some notebook tests fail. A production action should fail if a test fails, or perhaps parse the release name and ignore the fails if the original release name contains a particular flag (for example, --force).

The idea of using the release form to create the release was to try to simplify the workflow and allow a release to be generated quite straightforwardly from a repository on the Github website.

Author: Tony Hirst

I'm a Senior Lecturer at The Open University, with an interest in #opendata policy and practice, as well as general web tinkering...

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: