Live By Machine – CircleCI and Docker Hub AutoBuilds

Some time ago I put together a recipe for creating a simple data analysis workbench around the ergast F1 data using Chris Newell’s ergast API: Setting up a Containerised Desktop API server (MySQL + Apache / PHP 5) for the ergast Motor Racing Data API.

All the ingredients are in this repo.

The ergast Docker container is built using an automated Docker build whenever the Github repo is updated.

Following on from Simon Willison’s recipe for generating a commit log for San Francisco’s official list of trees, a scraper hosted on Github that does a daily scrape using CircleCI and then commits updates back to the repo, I’ve also set CircleCI to run against my repo using a daily cron job that copies the latest version of the ergast MySQL db file from the ergast website and then commits it to the repo.

To provide CircleCI with access to your Github account / organisations, go to your Github personal settings / Applications / OAuth applications.

The .cricleci/config.yml file is pretty much a straight rip-off of Simon’s:

version: 2
jobs:
  fetch_and_commit:
    docker:
      - image: circleci/python:3.6.4
    steps:
      - checkout
      - run:
          command: |
            cp ergastdb/data/f1db.sql.gz ergastdb/data/f1db-old.sql.gz
            curl -o ergastdb/data/f1db.sql.gz "http://ergast.com/downloads/f1db.sql.gz"
            git add ergastdb/data/f1db.sql.gz
            git config --global user.email "ergastbot@example.com"
            git config --global user.name "ergastbot"
            git commit -m "Daily update..." && \
              git push -q https://${GITHUB_PERSONAL_TOKEN}@github.com/psychemedia/ergast-f1-api.git master \
              || true
workflows:
  version: 2
  build:
    jobs:
      - fetch_and_commit
  nightly:
    triggers:
      - schedule:
          cron: "0 0 * * *"
          filters:
            branches:
              only:
                - master
    jobs:
      - fetch_and_commit

The Github Personal Access Token was set up just with permissions to access my public repos:

I then used the value of the token for the GITHUB_PERSONAL_TOKEN environmental variable in the appropriate CircleCI project:

What this means is that the Ergast API container should be regularly rebuilt, automatically, using a regularly updated copy of the ergast database.

Public CircleCI build logs can be found here:

https://circleci.com/api/v1.1/project/github/psychemedia/ergast-f1-api/

Add an optional build count (an integer) at the end of the URL for specific build details.