Technology Enhanced Learning (TEL) is “a thing” in the OU at the moment. I have no idea what folk (think they) mean by it.
Here’s what I mean by it, in the form of my own, ad hoc eTEL – emerging technology enhanced learning – mission statement.
What I aspire to is:
- explore how we might be able to use and repurpose emerging technology to support distance education;
- use the technology we teach our students about to deliver that teaching;
- use the technology we teach our students about to support that teaching;
- use the technology we teach our students about to produce the courses we are teaching;
- expose our students to emerging technologies that they can take and use in the outside world.
This obviously raises tensions, particularly where courses take two years to produce and then ideally (in the eyes of the organisation) remain unchanged for 5 years. The first step is risky, because it means trying new ways of doing things. The last step relates to my belief that universities should be helping push new ideas, technologies, techniques and processes out into society using our students as a vector.
A few weeks ago I spotted a review paper of “data wrangling” activities at the OU (Making sense of learner and learning Big Data: reviewing five years of Data Wrangling at the Open University UK). I saw it being linked to/promoted again today.
Apparently, “Data Wranglers [DWs] are a group of academics who analyse data about student learning and prepare reports with actionable recommendations based upon that data”. Also apparently, “[i]n practice” they also do “Big Data insights”. Or something. I’m not sure we have any “Big Data” do we? (Big data, meh.)
Furthermore, it seems that “Learning analytics are now increasingly taken into consideration at the OU when designing, writing and revising modules, and in the evaluation of specific teaching approaches and technologies”.
Looks around, confused…
…because something that I’ve been failing to understand for years and years and years and years is why no-one seems interested in taking the view that we are, in a lot of courses, delivering online content just like any other web publisher would, and as such we could be looking at ways of making our content “work better”, for some definition of “better”. Or even “work”.
In the learning analytics world, this possibly means building predictive models based on previous cohorts that show how students who dwelled this long on those content pages did well, while others who didn’t reveal that hidden answer of or visit that page, or who didn’t appear to visit any course pages, failed.
At this point, it’s probably worth mentioning that the OU, as a distance learning organisation, used to deliver course materials to students as print material, but increasingly we deliver material (that looks just like the print material) as HTML via a Moodle VLE. Each section of “as if” print material appears as a separate HTML page. (We also make PDFs available that students can download… It’d be interesting to know how many then print those PDF downloads out…)
It’s also worth mentioning that a lot of the teaching related activity pursued by the OU’s central academics relates to the production of course materials and assessment materials, which is to say, writing stuff, rather than delivery to students: when the course runs, it’s the moderators of online forums (which may include the occasional central academic) and the students’ personal tutors (Associate Lecturers, in OU parlance), who are the people who actually engage with students directly.
So to a large extent, once the stuff it’s written, that’s job done. Despite a laborious editing and publishing process to get the material onto the website, errors do slip through, and when spotted (often by pathfinder/vanguard students studying course material weeks ahead of the course schedule), corrected in another lengthy process (authors don’t have edit/write permissions on the course materials, and in some cases errors may be left uncorrected in situ with students expected to pick up the errata announcements via errata notices. Just like the print days…)
So what I keep on not understanding is why we don’t have someone paying attention to the course material as web content with a view to helping us better understand the obvious (because it’s nothing f****g difficult I want to learn from the pages), as I demoed nine years ago. For example:
- what’s the course dynamic in terms of content use (when are most students studying particular parts of the course)? – have we got the pacing about right?
- what’s the weekly rhythm of the course (what time of day are most students accessing the content pages?) – this could help forum moderators schedule their time;
- how much time are students spending, on average, in a particular study session, and does this vary (e.g. 1-2 hours on a weekday evening, 3-4 hours for daytime or weekday study, 45 mins over lunch periods), and so on; i.e. what user stories might we create *from the data*?
- how much time are students spending on particular pages? Are some pages just too long, or maybe have an idea or activity that is taking a lot of time to complete – or less time that we expect? Handy to know as a content designer (which is what course authors are). For the learning analytics surveillance freaks, can they spot students who spend more or less time than average on a particular page as a “likely fail” feature that they can celebrate?
- are those links to external resources clicked on? Ever?
- are the “optional activities” linked to on separate pages visited? Ever? Again, the learning analytics folk may be able to wet themselves finding correlation features on those pages, but I don’t really care about that. I just want to know, in the first instance, are the pages visited. Ever. (If they are, and it’s only a fraction of students who visit those pages/follow those links, then maybe it becomes useful to track the learning analytics stuff to see if we can figure what sort of student is making use of those resources. But rather than caring about a particular student, I’m more interested in getting a better user story dialled in that I can use to help as one more focal point to motivate content production in future courses.)
- are students using particular devices, or the same users using different sorts of devices at different times of time? With our insistence on still delivering software that needs to be installed on a traditional desktop computer, it would be useful to know if this can affect what a student might be able to study when based on device availability. And if it comes to trying to pitch particular computer requirements, it would be handy to know what the baseline is (which course webstats can provide an indicator of), and the extent to which this may vary across faculties or course levels.
Sometimes it can be comforting to see that your expectations about how the content would be used appear to be being met. Sometimes it can be revealing to find out that they’re not.
This is all basic stuff, and someone can probably have a fun time building some dashboards to report it. (Maybe there are some already, but no-one’s directed me to them despite my asking everyone I can think of.)
To reiterate on the why: I just want to be able to tell myself more informed stories about how the content appears to be being used en masse, and maybe also identifying different audience segments in the data (eg weekend studiers, weekday nighters, full-timers). Looking across courses (faculties, levels) it may be that we get different sorts of pattern / segmentation, which could be interesting from a user / user story informed content design perspective. It may well also prompt “learning analytics” discussions. (Writing this, I’ve come to realise I associate learning analytics with tracking back into individual data from “success” criteria such as assessment scores. For the content analysis, in the first instance, I’m just interested in how its generally being consumed. No individual data necessary. Once I’ve got broad usage pattern segments down, then maybe looking at performance level segments would be useful. But then, I’d rather just track the whole cohort score distribution to try and improve that.)
From looking at VLE pages, it looks as if there are Google Analytics and optimizely tracking scripts linked in the pages, although asking around I can’t find anyone who does anything with that data from the VLE pages. (Maybe the “DW”s do?) So I’m guessing the data is there?
PS One of the things I think optimizely may be used for is A/B testing by the Marketing folk on other bits of the website. Something I’ve pitched before is A/B testing on course materials (e.g. differently phrased or worked versions of the same activity).
This has generally been treated with disdain, but if it works for medical trials I don’t see why we can’t try it in education too. There is an argument here that we would need to track effect on attainment (the learning analytics thing), but I’m wary of the idea that changing a single page in several hundred could wildly affect attainment, unless it related to a particular key concept that the whole course hinged on. More realistically, if we see a page on average is taking students an hour to work through when we estimated it at 20 minutes, I’d be tempted to do A/B tests on it within a cohort. (Managing that if students chat about the topic in the common forums could represent a challenge!) The idea would be to see if we could improve the content performance more in line with expectations. As it is, the current approach would be to wait until the next presentation and give that whole cohort the new version. Which would of course be previously untested at scale. And may end up with students taking even longer to work through it.
Ever since I joined the OU, I’ve believed in trying to deliver distance education courses in an agile and responsive way, which is to say: making stuff up for students whilst the course is in presentation.
This is generally not done (by course/module teams at least) because the aim of most course/module teams is to prepare the course so thoroughly that it can “just” be presented to students.
I personally think we should try to improve the student experience of the course as it presents if we can by being responsive and reactive to student questions and issues.
So… TM351, the data management course that uses a VM, has started again, and issues / questions are already starting to hit the forums.
One of the questions – which I’d half noted but never really thought through in previous presentations (my not iterating/improving the course experience in, or between, previous presentations) – related to sharing Jupyter notebooks across different machines using Google Drive (equally, Dropbox or Microsoft OneDrive).
The VirtualBox VM we use is fired up using the vagrant provisioner. A Vagrantfile defines various configuration settings – which ports are exposed by the VM, for example. By default, the contents of the folder in which vagrant is started up in are shared into the VM. At the same time, vagrant creates a hidden
.vagrant folder that contains state relating to the instance of that VM.
The set up on a single machine is something like this:
If a student wants to work across several machines, they need to share their working course files (Jupyter notebooks, and so on) but not the VM machine state. Which is to say, they need a set up more like the following:
For students working across several machines, it thus makes sense to have all project files in one folder and a separate
.vagrant settings folder on each separate machine.
Checking the vagrant docs, it seems as if this is quite manageable using the synced folder configuration settings.
The default copies the current project folder (containing the vagrantfile and from which vagrant is rum), which I’m guessing is a setting something like:
config.vm.synced_folder "./", "/vagrant"
By explicitly setting this parameter, we can decide how we want the mapping to occur. For example:
config.vm.synced_folder "/PATH/ON/HOST", "/vagrant"
allows you to to specify the folder you want to share into the VM. Note that the
/PATH/ON/HOST folder needs to be created before trying to share it.
To put the new shared directory into effect, reload and reprovision the VM. For example:
vagrant reload --provision
Student notebooks located in the notebooks folder of that shared directory should now be available in the VM. Furthermore, if the shared folder is itself in a webshared folder (for example, a synced Dropbox, Google Drive or Microsoft OneDrive folder) it should be available wherever that folder is synched to.
For example, on a Mac (where
~ is an alias to my home directory), I can create a directory in my dropbox folder
~/Dropbox/TM351VMshare and then map this into the VM using by adding the following line to the Vagrantfile:
config.vm.synced_folder "~/Dropbox/TM351VMshare", "/vagrant"
Note the possibility of slight confusion – the shared folder will not now be the folder from which vagrant is run (unless the folder are running from is
Furthermore, the only thing that needs to be in the folder from which vagrant is run is the
Vagrantfile and the hidden
.vagrant folder that vagrant creates.
Fingers crossed this recipe works…;-)
config.vm.box = "ouseful/ou-robotics-test"
Now I’m thinking I should probably do the same for the TM351 VM, giving the hassle it seems to take trying to get the
.box file hosted for download on an OU URL…
When we put together the virtual machine for TM351, the data management and analysis course, we built a headless virtual machine that did not contain a graphical desktop, but instead ran a set of services that could be accessed within the machine at a machine level, and via a browser based UI at the user level.
Some applications, however, don’t expose an HTML based graphical user interface over http, instead they require access to a native windowing system.
One way round this is to run a system that can generate an HTML based UI within the VM and then expose that via a browser. For an example, see Accessing GUI Apps Via a Browser from a Container Using Guacamole.
Another approach is to expose an X11 window connection from the VM and connect to that on the host, displaying the windows natively on host as a result. See for example the Viewing Application UIs and Launching Applications from Shortcuts section of BYOA (Bring Your Own Application) – Running Containerised Applications on the Desktop.
The problem with the X11 approach is that is requires gubbins (technical term!) on the host to make it work. (I’d love to see a version of Kitematic extended not only to support docker-compose but also pre-packaged with something that could handle X11 connections…)
So another alternative is to create a virtual machine that does expose a desktop, and run the applications on that.
Here’s how I think the different approaches look:
As an example of the desktop VM idea, I’ve put together a build script for a virtual machine containing a Linux graphic desktop that runs the V-REP robot simulator. You can find it here: ou-robotics-vrep.
The script uses one Vagrant script to build the VM and another to launch it.
Along with the simulator, I packaged a Jupyter notebook server that can be used to create Python notebooks that can connect to the simulator and control the simulated robots running within it. These notebooks could be be viewed view a browser running on the virtual machine desktop, but instead I expose the notebook server so notebooks can be viewed in a browser on host.
The architecture thus looks something like this:
I’d never used Vagrant to build a Linux desktop box before, so here are a few things I learned about and observed along the way:
ubuntu-desktopnaively installs a whole range of applications as well. I wanted a minimal desktop that contained just the simulator application (though I also added in a terminal). For the minimal desktop,
apt-get install -y ubuntu-desktop --no-install-recommends;
- by default, Ubuntu requires a user to login (user: vagrant; password: vagrant). I wanted to have as simple an experience as possible so wanted to log the user in automatically. This could be achieved by adding the following to
[SeatDefaults] autologin-user=vagrant autologin-user-timeout=0 user-session=ubuntu greeter-session=unity-greeter
- a screensaver kept kicking in and kicking back to the login screen. I got round this by creating a desktop settings script (
#dock location gsettings set com.canonical.Unity.Launcher launcher-position Bottom #screensaver disable gsettings set org.gnome.desktop.screensaver lock-enabled false
and then pointing to that from a
desktop_settings.desktop file in the
/home/vagrant/.config/autostart/ directory (I set execute permissions set on the script and the
[Desktop Entry] Name=Apply Gnome Settings Exec=/opt/set-gnome-settings.sh Hidden=false NoDisplay=false X-GNOME-Autostart-enabled=true Type=Application
- because the point of the VM is largely to run the simulator, I thought I should autostart the simulator. This can be done with another
.desktopfile in the autostart directory:
[Desktop Entry] Name=V-REP Simulator Exec=/opt/V-REP_PRO_EDU_V3_4_0_Linux/vrep.sh Type=Application X-GNOME-Autostart-enabled=true
- the Jupyter notebook server is started as a service and reuses the installation I used for the TM351 VM;
- I thought I should also add a desktop shortcut to run the simulator, though I couldnlt find an icon to link to? Create an executable
run_vrep.desktopfile and place it on the desktop:
[Desktop Entry] Name=V-REP Simulator Comment=Run V-REP Simulator Exec=/opt/V-REP_PRO_EDU_V3_4_0_Linux/vrep.sh Icon= Terminal=false Type=Application
Her’s how it looks:
If you want to give it a try, comments on the build/install process would be much appreciated: ou-robotics-vrep.
I will also be posting a set of activities based on the RobotLab activities used in TM129 in the possibility that we start using V-REP on TM129. The activity notebooks will be posted in the repo and via the associated uncourse blog if you want to play along.
One issue I have noticed is that if I resize the VM window, V-REP crashes… I also can’t figure out how to open a V-REP scene file from script (issue) or how to connect using a VM hostname alias rather than IP address (issue).
One of the issues with distributing software to distance education students is ensuring that the software package they are trying to install hasn’t been corrupted ins some way during transport. For example, one of the ways we ship software to students is via USB memory stick. But in one course last year, it seems that some of the sticks were a bit dodgy, and the files wouldn’t install from them.
Which is where checksums come in.
If a student is having issues installing a piece of software, we can check the checksum of the distributed installer package to see if it matches the checksum of a pristine package. If it doesn’t, we know the problem is a corrupted installer package (rather than a problem downstream of that, for example).
So what is a checksum? Essentially, it’s a single number derived from all the bits in the file you’re generating the checksum for. If any bit in the file is changed, the checksum should too.
So here are a couple of ways of generating checksums…
Download the Windows fciv (Windows File Checksum Integrity Verifier) utility: https://support.microsoft.com/en-gb/kb/841290
Run a command of the form:
This will produce checksums using different coding mechanisms:
MAC / LINUX:
On a Mac, you can find the checksum using the following command:
openssl sha1 PATH/FILE_CHECK.SUFFIX
openssl md5 PATH/FILE_CHECK.SUFFIX
openssl md5 ~/USERRELATIVEPATH/FILE_CHECK.SUFFIX
If we distribute a copy of the checksum for installer packages, along with the installer packages, assuming that the checksum is not corrupted, a student can check that the installer package is not corrupted by generating the checksum for their installer package and comparing it to the distributed one.
When it comes to support in the event of a problem, and a call to the help desk, then the first question from support can be: “Have you checked the checksum of the original package?” (Or we can prompt students to do this themselves as part of self-service support…)
Even better if we shipped a simple one-click file integrity checking utility that:
1) runs a checksum test on itself to check that it’s working;
2) runs a checksum test on the distributed package(s) to check that they are not corrupted.
One of the issues I keep coming up against when trying to encourage folk to at least give Jupyter notebooks a try is that “it’s too X to install” (for negatively sentimented X…). One way round this is to use something like Binder, which allows you to create a Docker image from the contents of a public Github repository and run a container instance from that image on somebody else’s server, accessing the notebook running “in the cloud” via your own browser. A downside to this approach is that the public binder service can be a bit slow – and a bit flaky. And it probably doesn’t scale well if you’re running a class. (Institutions running their own elastic Binder service could work round this, of course…)
So here’s yet another possible way in for folk – O’Reilly Media’s LaunchBot (via O’Reilly Media CTO Andrew Odewahn’s draft article Computational Publishing with Jupyter) – though admittedly it also requires the possibly (probably?!) “too hard” requirement to install Docker as a pre-requisite. Which means it’s possibly doomed from the start in my institution…
Anyway, for anyone not in a Computing HE department, and who’s willing to install Docker first (it isn’t really hard at all unless your IT department has got their hands on your computer first: just download the appropriate Community Edition for your operating system (Mac, Windows, Linux) from here), the LaunchBot app provides a handy way to run Jupyter environments in a custom Docker container, “preloaded” with notebooks from a public Github repo.
By the by, if you’re on a Mac, you may be warned off installing LaunchBot.
Simply select the app in a Finder window, right click on it, and then select “Open” from the pop-up menu; agree that you do want to launch it, and it should run happily thereafter.
In a sense, LaunchBot is a bit like running a personal Binder instance: the LaunchBot app, which runs from the desktop but is accessed via browser, allows you to clone projects from public Github repos that contain a
Dockerfile, and perhaps a set of notebooks. (You can use this url as an example – it’s a set of notebooks from original OU/FutureLearn Learn to Code for Data Analysis course.) The project README is displayed as part of the project homepage, and an option given to view, edit and save the project
Dockerfile can then be used to launch an instance of a corresponding container using your own (locally running) Docker instance:
(A status bar in the top margin tracks progress as image layers are downloaded and
Dockerfile commands run.)
The base image specified in the
Dockerfile will be downloaded, the
Dockerfile run, and I’m guessing links to all services running on exposed ports are displayed:
In the free plan, you are limited to cloning from public repos, with no support for pushing changes back to a repo. In the $7 a month fee based plan, pulls from private repos and pushes to all repos are supported.
From a quick play, LaunchBot is perhaps easier to use than Kitematic. Whilst neither of them support launching linked containers using docker-compose, LaunchBot’s ability to clone a set of notebooks (and other files) from a repo, as well as the
Dockerfile, makes it more attractive for delivering content as well as the custom container runtime environment.
I had a quick look around to see if the source code for LaunchBot was around anywhere, but couldn’t find it offhand. The UI is a likely to be little bit scary for many people who don’t really get the arcana of git (which includes me! I prefer to use it via the web UI ;-) and could be simplified for users who are unlikely to want to push commits back. For example, students might only need to to pull a repo once from a remote master. On the other hand, it might be useful to support some simplified mechanism for pulling down updates (or restoring trashed notebooks?), with conflicts on the client side managed in a very sympathetic and hand-holdy way (perhaps by backing up any files that would otherwise be overwritten from the remote master?!). (At the risk of feature creeping LaunchBot, more git-comfortable users might find some way of integrating the
nbdime notebook diff-er useful?)
Being able to brand the LaunchPad UI would also be handy…
From a distributed authoring perspective, the git integration could be handy. As far as the TM351 module team experiments in coming up with our own distributed authoring processes go, the use of a private Github repo means we can’t use the LaunchBot approach, at least under the free plan. (From the odd scraps I’ve found around the OU’s new OpenCreate authoring system, it supposedly supports versioning in some way, so it’ll be interesting to see how that works…) The TM351 dockerised VM also makes use of multiple containers linked using docker-compose, so I guess to use it in a LaunchBot context I’d need to build a monolithic container that runs all the required services (Postgres and MongoDB, as well as Jupyter) in a single image (or may I could run docker-compose inside a Docker container?)
[See also Seven Ways of Running IPython / Jupyter Notebooks, which I guess I probably needs another update…]
PS this could also be a handy tool for supporting authoring and maintenance workflows around notebooks: nbval, a “py.test plugin to validate Jupyter notebooks”. I wonder if it’s happy to test the raising of particular errors/warnings, as well as cleanly executed code fragments? Certainly, one thing we’re lacking in our Jupyter notebook production route at the moment is a sensible way of testing notebooks. At the risk of complicating things further, I wonder how that might fit into something like LaunchBot?
PPS Binder also lacks support for docker-compose, though I have asked about this and there may be a way in if someone figures out a spawner that invokes docker-compose.