More Automation Sketches – Creating Student Notebook Releases

Tinkering a bit more with Github Actions, I’ve hacked together some sort of workflow for testing notebooks in a set of specified directories and then clearing the notebook output cells, zipping the notebooks into a release zip file, and then making the release zip file via a github release page.

The test and release is action is triggered by making a release with a body that contains a list of comma separate directory paths identifying the directories we want in the release. For example:

The following action is triggered by a release creation event:

name: example-release
      - created
    runs-on: ubuntu-latest
      image: ouvocl/vce-tm351-monolith
      RELEASE_DIRS: ${{ github.event.release.body }}
    - uses: actions/checkout@master
    - name: Install nbval (TH edition) and workflow tools
      run: |
        python3 -m pip install --upgrade
        python3 -m pip install
    - name: Restart postgres
      run: |
        sudo service postgresql restart
    - name: Start mongo
      run: |
        sudo mongod --fork --logpath /dev/stdout --dbpath ${MONGO_DB_PATH}
    - name: Get directories
      run: |
        #IFS=$"\n" read -a file_paths <<< "${{ github.event.head_commit }}"
        IFS="," read -a file_paths <<< "${{ github.event.release.body }}"
        # Test all directories
        for file_path in "${file_paths[@]}"; do
          py.test  --nbval "$file_path" || continue
      shell: bash
      # For testing...
      continue-on-error: true
    - name: Create zipped files
      run: |
        IFS="," read -a file_paths <<< "${{ github.event.release.body }}"
        for file_path in "${file_paths[@]}"; do
          tm351zip -r clearOutput -a "$file_path"
        echo "Release paths: $RELEASE_DIRS" > release-files.txt
        echo "\n\nRelease zip contents:\n" >> release-files.txt
        tm351zipview >> release-files.txt
      shell: bash
    - name: Create Release
      id: create_release
      uses: softprops/action-gh-release@v1
      # The commit must be tagged for a release to happen
      # Tags can be added via Github Desktop app
        tag_name: ${{ github.ref }}-files
        name: ${{ }} files
        #body: "Release files/directories: ${RELEASE_DIRS}"
        body_path: release-files.txt
        files: |

It then runs the tests and then generates another release that includes the cleaned and zipped release files:

Ideally, we’d just add the zip file to the original release but I couldn’t spot a way to do that.

At the moment the action will publish the file release even if some notebook tests fail. A production action should fail if a test fails, or perhaps parse the release name and ignore the fails if the original release name contains a particular flag (for example, --force).

The idea of using the release form to create the release was to try to simplify the workflow and allow a release to be generated quite straightforwardly from a repository on the Github website.

Author: Tony Hirst

I'm a Senior Lecturer at The Open University, with an interest in #opendata policy and practice, as well as general web tinkering...

%d bloggers like this: