Fragment – ROER: Reproducible Open Educational Resources

Fragment, because I’m obviously not making sense with this to anyone…

In the words of David Wiley (@opencontent), in defining the “open” in open content and open educational resources [link], he identifies “the 5R activities” that are supported by open licensing:

  1. Retain – the right to make, own, and control copies of the content (e.g., download, duplicate, store, and manage)
  2. Reuse – the right to use the content in a wide range of ways (e.g., in a class, in a study group, on a website, in a video)
  3. Revise – the right to adapt, adjust, modify, or alter the content itself (e.g., translate the content into another language)
  4. Remix – the right to combine the original or revised content with other material to create something new (e.g., incorporate the content into a mashup)
  5. Redistribute – the right to share copies of the original content, your revisions, or your remixes with others (e.g., give a copy of the content to a friend)

Whilst the legal framework is the one that has to be in place for educational institutions as publishers of (third party) content, where there is particular emphasis on citing others and not reusing content in an unacknowledged way, I have always been more interested in the practice of reusing content, particular when that means reuse with modification.

Various others have suggested sixth Rs. For example, Chris Aldridge’s The Sixth “R” of Open Educational Resources identifies “Request update (or maybe pull Request, Recompile, or Report to keep it in the R family?)“, in the sense of keeping stuff in a repo so people can fork the report, update it and track changes (maybe Revisions is a better R for that?). Rather than be an R relating to a right you can assert, Revisions is more about the practice.

As a trawl through the history of this blog suggests (for example, Open Content Anecdotes from nigh on a decade ago), I’ve also been less interested in the legal framework around OERs than I am in the practical reuse (with modification) of particular (micro? atomic?) assets, such as diagrams, or problem sets. That is, the things you are more likely to spot as relevant, useful or interesting and weave into your own materials, or replace your own crappy diagrams.

To a large extent, the legal bit doesn’t stop me, particularly if no-one finds out. The blocker is in the practicalities associated with reversioning the physical asset, and making actual changes or modifications to it, that makes it appropriate for my course.

(You can always redraw diagrams, which can also help you get round copyright on non-openly licensed works, but that takes time, skill, and maybe a drawing package you don’t have access to.)

So the idea that I started trying to crystallise out almost a year ago now  — OERs in Practice: Re-use With Modification — is based around another R, a leading R, or a +1 R, which (as with Aldridge’s suggested R) is a practice based R: Reproducibility. (Hmm.. maybe this post should have been The 5+2Rs of Open Content?)

By their very nature, these resources are resources that include their own “source code” for creating assets, so that if you want to create a modified version of the asset, you can modify the source and then regenerate the asset. A trivial example is to use diagrams that are “written diagrams” – diagrams generated from textual, written descriptions or codified versions of them and rendered from the description by a particular chart generating tool (for example, Writing Diagrams, Writing Diagrams – Boxes and Arrows and Writing Diagrams (Incl. Mathematical Diagrams)).

As to why this is a fragment, I’m stopping here… discussion about reproducibility is elsewhere, and will be to follow too, along with why this approach is opens up new opportunities for educators as well as learners. For now, see these other fragments on that topic in date order.

[Update: via a comment, @opencontent reminds me that he also made the distinction between legal and practical issues, with practical concerns raised in the ALMS framework – I should have read on from the legal issues to the Poor Technical Choices Make Open Content Less Open section… See the comment thread to this post for more, as well as this related post from the previous time the ALMS model was raised to my attention: Open ALMS.  I also note this recent IRRODL paper on Defining OER-Enabled Pedagogy, which I need to read through…]

PS some more related fragments from a hastily written, unsuccessful internal Esteem Project bid:


One problem associated with producing rich educational materials is that inconsistencies can occur when cross referencing text with media assets such as charts, tables, diagrams and computer code produced via different production routes. The project will explore and demonstrate how emerging technologies and workflows developed to support reproducible work practices can be adopted for the development of reproducible educational resources, including but not limited to educational materials rich in mathematical content, scientific / engineering diagrams, exploratory and explanatory statistics, maps and geospatial analysis, music theory and analysis, interactive browser based activities, animations and dynamically created audio assets.

The aim is to demonstrate:

  • The range of assets that can be produced / directly authored by academics including static, animated and interactive elements such as print quality drawings, animated scientific diagrams, and interactive web activities and applications (eg interactive maps, 3D models, etc.)
  • The workflows associated with demonstrating the production, maintenance, and reuse with modification of the assets and works derived from them, including but not limited variations on a theme in the production of parameterised assessment materials
  • The potential for using reproducible materials to facilitate maintenance, reversioning / updating and reuse with modification of module materials The outputs will include:
    • A range of OU module and OpenLearn unit materials reworked using the proposed technologies and workflows
    • A library of reproducible educational resource templates for a range of topic areas capable of being reused with modification in order to produce a range of assets from a common template.

The project will demonstrate how freely available, open source technologies can be used to support the direct authoring of rich and interactive media assets in a reproducible way.


Reproducible research tools increasingly support the direct authoring of rich documents that blend text, data, code, code outputs, with media assets (audio, video, static and animated images, interactives) generated from text based computer scripts.

The project proposes the co-option of such tools for use as authoring tools for reproducible educational materials. A single “source document” can include text as well as scripts for generating tables and charts, for example, from data contained within the document itself, minimising the distance between the production of assets from the materials they are used in.

The resulting workflow supports consistency in production and maintenance as well as reuse with modification thereafter by allowing updates in situ that can be used to recreate modified assets (diagrams created dynamically using updated values, for example). Materials will also be modifiable by ALs for tutorial use.

Authoring tools also support the direct authoring and creation of interactive components based around templated third party widgets such as 3D molecule viewers, or interactive maps, allowing authors to directly author interactive components that can be embedded in online / browser accessed materials.

Examples of the sorts of assets I had in mind to rework can be found in several of the notebooks available here.

Author: Tony Hirst

I'm a lecturer at The Open University, with an interest in #opendata policy and practice, as well as general web tinkering...

2 thoughts on “Fragment – ROER: Reproducible Open Educational Resources”

  1. Tony, I always love reading what you write. I, too, am interested in the practical problems around reuse of open content. In the page you link two at the top of your post, if you scroll further down, you will find the ALMS framework. This is my framework for thinking about the practical problems associated with reuse. I maintain the two framework separately, one dedicated to legal issues and one to practical issues, in order to keep things tidy. I would love to hear your thoughts about the relationship between reproducibility and what I described in that second framework.

    1. HI David,
      Thanks for the comment; oops on me for not reading through – s’what makes me not a scholar, I guess ;-) Thanks for the link – it reminds me I did chase a link to ALMS before:

      That post is actually a complement to this one, and considers another important part of the jigsaw, specifically providing the environment required to transform “source” documents into “rendered” content. (A metaphor I started to ponder whilst walking the dog this afternoon was HTML: the web took off in part because people could take “rendered” pages – what you see in a web browser – and then “View Source” on them to see the raw HTML that actually defined the page. The raw HTML could be tweaked in a simple text editor and the re-rendered in the browser, giving you your own, modified page. That users could also make enough sense of the HTML to change the bits they wanted to change (the content bits, rather than the code bits) without needing to be programmers.
      In this post, and the notebooks linked to, I’m focussing on making the “HTML”/View source available; that previous post, looking at Binder, etc, is more about providing the “browser” that renders the source to the finished product.
      Azure notebooks, plus some boilerplate package install code cells in my notebooks, play the same role in providing a complete execution environment as does a configured Binder container.

      Thanks again for commenting by, it gets lonely here sometimes!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.