Way back when, when I first started blogging, I tried to push the idea of “live documents” that supported transclusion of content from elsewhere (e.g. Keeping Courses Current with Live Links; there was also a demo, but I think it’s rotted…?) A couple of days ago, Owen Stephens (re)introduced me to the notion of literate programming, “a methodology that combines a programming language with a documentation language”. The context was active reading of reactive documents, in which a reader interacts with a document that contains human readable paragraphs that describe some sort of mathematical or logical model which is embedded in the text as interactive, parameterised elements. (I can’t give a demo in this WordPress.com hosted blog because what I am allowed to do is really locked down… so to see what I’m talking about, check out the Explorable Explanations) example document… I can wait…)
Done that? Up to speed now?
My immediate impression was that it reminded me of the interactive, browser based programming style (e.g. Online Apps for Live Code Tutorials/Demos), in which learners can read and run, edit and run, and write and run, code examples in the browser (or more generally, in the context of an “electronic study guide” (eSG). It also brought to mind similarities with dexy.it and Sweave, a couple of (literate programming;-) frameworks that allow you to include programme code within a document and them execute it in order to produce an output that also appears in the document. (I remember one of the joys of course writing for an eSG is that you often have to hand over the text (including code and output in situ), a text file containing the code (for testing), and a text file containing the output. If (when) an error is found, version control across the various files can be come really problematic. Far easier if the document were to include code fragments that are then executed and used to produce the actual output that is in turn piped directy into the final document.) Wolfram’s Computable Document Format also comes to mind, as a document format that allows a reader to express executable mathematical statements, whether formally specified or, increasingly, using natural language.
So the document space I’m imagining here is one in which the document contains one or more components that are generated in response to some sort of request from an operational part of the document, or a part of the document that encodes some sort of performative action[?????], such as a search term that is used to trigger a search whose results are then included within the page, a piece of programme code that can be executed in order to generate an output, or a parameter for a model that can be run with the specified parameter value in order to produce an output that is rendered live within the document.
For example, this might include a ‘live’ document, that transcludes content from an external source:
A literate programme, that combines:
– some explanatory text;
– fragments of, or complete, programmes;
– the output of the programme.
Or a reactive document which contains:
– some explanatory text;
– parameterised programme code, or a parameterised mathematical or logical model; the code/model should also be executable, using parameter values specified by the reader;
– the output from executing the code or model.
(I guess a live document might be viewed as reactive in certain cases, for example, when a user specifies a search term or query that determines what content is pulled live into a document from an external source.)
There is something almost cell like going on here, in that part of the document contains the instructions that some document machinery can process in order to produce other parts of the document…
One obvious use case for living documents is in educational materials. For a long time now (even before the time of education CD-ROMs;-), eLearning materials have included interactive components. But these have often be external components that have been slotted in to the educational text, rather than being generated from the execution of a specified part of the the text. For example, many OU course materials include interactive self-assessment questions, or Flash based interactive exercises (hmm… I wonder when these are going to be rebranded as edu-apps and made available, for a fee, or via open license, in an OU edu-app market;-) [Note: the OU used to be a pretty significant educational software house in terms of output, with large numbers of highly skilled educational software developers who knew how to turn out software that worked in educational terms… but that was before the VLE came along…;-)]
Another use case is the area of data journalism. A criticism of many interactive visulisations produced to support news stories is that whilst they’re all very nice and shiny, they don’t actually work that well to communicate anything of substance at all (for example, see my comments on Michael Blastland’s talk at the OU Stats conference). Maybe a few well crafted reactive documents might start to address this balance, and engage at least part of the audience in a contextualised consideration of data (or model) based story…?
A third area I’d like to spend some time mulling over (maybe even in the context of Public Platforms…?) is policy development and public consultation, scoping out what may be possible and plausible if consultation documents were to propose particular models and then allow the engaged reader to explore the various parameter regimes associated with those models?
Hmmm…. maybe I need to start working on my resolutions for next year…?!
PS just in passing, as well as treating documents as living things, it can also be instructive to think of them as databases. This is a trivial mapping if the document has a regular tabluar structure, such as a spreadsheet sheet, or is otherwise formally structured (as for example in the case of an XML document, which typically describes some sort of hierarchical (document as data) structure) ,or even if it contains conventions in either style or content (for example, section headings being phrased in the form “Section NN: blah blah blah”; “Section NN: ” is a convention that can be used to identify the semantics of the text “blah blah blah” (in this case, as the text representing the header of section NN).
Several online consultation and review documents that engaged my interest were published recently, so I thought it might be useful to quickly compare how they’re presented and what they have to offer.
Public Data Corporation
Firstly, the Plans for the Public Data Corporation consultation. The consultation is presented as a WordPress blog (with some untidy default widgets left in the right hand sidebar) with a brief summary and list of ten (10) consultation questions listed on the front page, and then a separate page to solicit responses for each particular question:
The comments are captured using Disqus and a pre-moderation policy:
It is hard to see at a glance the extent to which people have engaged with the questions across the consultation. The premoderation policy means that there is a delay (and uncertainty) in publishing comments – so for example, the comments I posted on a Saturday morning (#bigsociety time?!;-) presumably won’t be released (if at all) until Monday morning at the earliest… meaning no on-site discussion in the comment thread over the weekend.
(See also SImon Dickson’s take on this consultation: Another Cabinet Office WP consultation.)
Where WordPress is used as a platform, single page RSS feeds and comment feeds per page are available, although it is up to the publisher to decide whether full or summary feeds are published for each page. The following Netvibes dashboard demonstrates an aggregation of single page and page level comment feeds for the PDC consultation:
This suggests that it may be possible to increase the surface area of a consultation using dashboard services, as well as developing dashboards to support the management and reactive moderation of a consultation.
Commons Committee Inquiry on Peer Review
The House of Commons Science and Technology Committee have just called for a new Inquiry into Peer Review.
In terms of online engagement, I guess this sets the minimum possible baseline?!
“Protection of Freedom Bill” Public Reading Stage
The Cabinet Office recently released a public reading stage for the Protection of Freedom Bill using a themed WordPress site. This site offers front page navigation with the number of public comments received through the platform to date identified for each page.
Comments are supported at a page level, with partial feeds supported at the page level (using ?feed=rss2&withoutcomments=1) along with full comment feeds.
Top level navigation across the document is preserved at the page level by means of the left-hand navigation sidebar.
Despite the legalistic nature of the Bill, paragraph level commenting is not directly supported.
(See also Simon Dickson’s response to this consultation: Can Cabinet Office’s WordPress-based commentable bills make a difference?.)
Department of Health Online Consultations
The Department of Health Online Consultations Hub provides a single home for current and recently closed consultations from the DoH. Consultations are split over several pages with clearly marked out text entry forms on at the bottom of pages where feedback is requested.(That is, page level structured commenting is supported.) By providing email credentials, users can obtain a link that allows them to return to their submission to the consultation at a later date.
Resource Discovery Taskforce Request for Comments on Metadata Guidelines on JISCPress
The JISC Resource Discovery Taskforce (RDTF) request for comments on UK Metadata Guidelines was published as a multipage document on JISCPress, a WordPress installation running the digress.it theme.
Front page sidebar navigation allows access to all areas of the document and summarises the number of comments per page. Mousing over a page link on the front page loads a preview of the page in the central pane. Following a link leads to a page with floating comment box that supports threaded commenting at the paragraph level:
Each paragraph is also given a unique URI allowing it to be uniquely referenced in posts on third party sites.
Along with comments by section, comments are viewable by commenter:
[Disclaimer: I was part of the project team that proposed JISCPress and the use of the digress.it WordPress plugin and am also a member of the RDTF technical advisory group associated with this RFC.]
Wordpress appears to be gaining traction as a consultation publishing platform, with either vanilla themes (e.g. Public Data Corporation proposal) or custom commentable document themes (JISC RDTF guidelines). WordPress native comments as well as third party commenting support using Disqus are demonstrated (it would be interesting to hear the rationale behind the choice of Disqus and an evaluation of how well it was deemed to have worked). Reactive and pre-moderation strategies are in evidence.
PS One more, that I should have included the first time round, on @lesteph’s ReadAndComment platform – LG Group Transparency Programme.
Whole document navigation is available from the front page as well as from the right hand sidebar on document pages (though it’s not clear if there would be a count of comments per page?) Comments are at page level via a WordPress comment entry form at the bottom of the page:
Steph hinted I won’t like the feeds… dare I look?!;-)
One of the things we didn’t put into the original JISCPress bid – though in hindsight we might have – was a use case for commentable documents in the context of government consultations soliciting formal responses from higher education institutions (for example, Universities UK: Review of External Examining Arrangements in the UK).
From a chat with Alison Nash in the OU’s recently reorganised Strategy Unit (I think?), it seems that candidate consultations are fielded by a member of that unit who then emails likely suspects (identified quite how, I’m not sure?) with either a link to, or copy of, the consultation document; (these are typically Word or PDF documents). As with many of the consultations we have looked at in the context of Write To Reply, the consultations typically have a set of questions associated with them that are distributed throughout the consultation document as a whole. Comments and responses to questions are then returned by email (I didn’t ask whether this is typically in the body of an email message, in a Word document, or as comments or highlighted changes on a copy of the orginal consultation), collated (again, I’m not sure how? One way would be to use a spreasheet, with rows for respondent and columns for each question (or vice versa)), and used to frame the institutional response. (I’m not sure if a draft of the institutional response was then circulated to the orginal commenters for final comment…?) The question that was then asked was: would a WriteToReply style approach be appropriate for managing returns of comments and answers to consultation questions in a rather more organised way than is currently the case?
(If anyone from the OU, or other HEIs who engage in producing formal instituional responses to consultations would like to provide further detail about the workflow for soliciting internal comments, producing drft and final versions of instituional responses, and then tracking the impact of comments made in the response, please post a comment to this post…)
Here are some thoughts/matters arising relating to how the WriteToReply/JISCPress/digress.it approach might apply:
– comments may need to be private; this could be achieved by hosting WordPress within the firewall, limiting who can view comments to members of the institution, or not making comments public (e.g. by moderating them, meaning that only the blog owner could see them). Limiting who can make comments can be achieved by requiring users to log in to the blog, and only providing certain users with log in accounts.
– it may not be appropriate to allowing commenting on all paragraphs, instead requiring users to only comment on actual questions. This might be achieved by disabling comments on all pages except a single summary page that contains one paragraph per question, maybe with links back to the actual posts that contain the question in context.
– if comments are solicited throughout the document, a dashboard tool such as Netvibes can be used to aggregate comments from different sections of the document; tools like Yahoo Pipes can also be used to aggregate comments from separate areas of the document and display them in a single view. Views over comments by individual commenters are also available and may be collated together on dashboard pages (for example, with separate pages aggregating comments from different sorts of commenter – e.g. allowing views over responses by Faculty, for example).
– once a formal response has been produced, it may be appropriate it post it on the consultation site to allow commenters to see how their comments were o weren’t integrated in to the official response (maybe leaving it open to them to submit a personal response to the consultation if they feel their views were not appropriately reflected, if at all. (The more I think about the process of these document based consultations, the more I feel a feedback loop is required that allows folk to see what sort of impact, if any, their comments may have had. I also briefly touched on this in On the Different Roles Documents and Comments May Take in a Commentable Document.) The consultation document site then becomes an important part of institutional memory, archiving as it does the original consultation, individual comments from members of the institution, and the institution’s formal response. It might also be the case that a draft of the institutional response is placed on the same site and comments on it solicited. (The site would then be hosting documents in two modes – the original consultation mode document, and then a draft mode document (again, this distinction appears in the Different Roles blog post.)
In many cases, it might be that the paragraph level commenting approach is not appropriate – unless comments are limited to just the consultation questions themselves, each as a separate commentable item. Where it is appropriate to isolate consultation questions from the surrounding text, a simple form may provide the best way of capturing comments.
In the OU, where I believe we are about to start rolling out Google Apps for Education to at least some of our students over the next month or two, it might be appropriate to look at using a Google form as platfrom for capturing comments. As well as satisfying the immediate goal (capture comments in a centralised way), this approach would also provide a legitimate and low risk use case for exloring how we might make use of the Google Apps environment as part of internal business processes.
The simplest case, then, would be for the internal staff member responsible for gathering comments to create a Google form. I don’t know if internal staff members have yet been issued with login details for how to access Google Apps on the open.ac.uk domain, but in the interim they can either create a personal Google account (or I could let them have an account on one of my Google apps domains!). Creating a form can be done either from the main docs menu, or within a Google spreadsheet (the posted form results are collated within a spreadsheet).
For most consultations based around a set of specific questions, the format of the form would look something like this:
That is, a copy and pasted copy of each consultation question (with minor tweaks so the question makes sense in a standalone questionnaire) as a separate form item, with a Paragraph text element for the response.
If additional commentary is required, the section head (which includes a description component) can be used to display it:
It might also be worth capturing “any other comments” in a final paragraph text comment at the end of the questionnaire.
Although the form, once published, would be open to anyone on the apps domain, (if they knew the URL), a further “security” measure would be to prompt the user for a consultation “pass phrase” emailed to them as part of the request for comments (“please enter this keyphrase when you complete the form so we can put your responses into the class of ‘high priority’ responses”;-) This might even be a required element.
Alternatively, a keyphrase element could be used to sort the responses in the results spreadsheet, or as suggested above in the context of digress.it, used to sort responses for example by Faculty, (Alternatively, an optional unique key code be be generated for each invited response to identify their responses. Or we could request an OU identifier, name, email address etc to track who made what comment (though these approaches are gameable and don’t necessarily imply that the person with a given identifier is the person who submitted the form…)). If users are logged in to the Google Apps environment, it may be that their identity is recorded anyway…? Hmm….
For just collecting responses, pretty much anyone could just set up the form and then email the link to the form to the potential commenters. With the availability of Google Apps script, and a little bit of developer time, it would also be possible to provide alerts to the internal consultation organiser whenever a form submission is made, provide automated collation of responses by question and pop these into a Google wordprocessor doc (I think…?!) and even manage a circulation list – so for example, a list of respondents could be created in a spreadsheet, used to mail out invitations for them to complete the form, and then track their response. In the advent of them not responding within a certain period, an automated reminder could be sent out. (I’m guessing it would take about a day to build and test such a workflow, which once created would be reusable.)
Another advantage of using the Google Apps approach would be that the response spreadsheet (or an automagically maintained Google wordprocessor doc version of it) could be shared to other members of the team providing the formal institutional response as an online shared document appearing in each individual’s Google docs “inbox”.
PS it seems that within a Google Apps for Edu environment, it may now also be possible for users to edit their form responses if they want to revise their answers…
PPS it’s also worth noting here a couple of practical considerations about how to write a consultation document bearing in mind that someone might put together a form to collate the responses. Firstly, the question should make sense as a standalone item (i.e. out of context) or very clearly identify what it is referring to rather than just “the above”. Secondly, if the questions are collated together in a single appendix, it makes it easier to just check off that each question has been included in the form. (It’s also handy as a one page item for someone who is putting together the response.) Links to the original context also help; (in a sense, this sort of Appendix is like “List of Tables” or “List of Figures” that acts as contents page for locating questions within the document). Reading over the questions in an Appendix will also make it obvious whether or not the question was written in such a way that it implicitly refers to content surrounding it in the original embedded context (“see the above” again…) Note that I’m not saying questions shouldn’t be embedded, just that when they are taken out of context, they still make sense and read well. In the example I give above about external examiners, the questions had to be tweaked so that they made sense as standalone items.
Chatting over possible use cases for digress.it in a meeting at UKOLN yesterday, it struck me that there are at least three different roles we might expect a commentable document to play in a open discussion context:
- draft document – in which comments are solicited on different parts of a document, with a view to producing a revised version of the document that takes into account the various comments made on the commentable version of it. For example, the publication of draft standards (e.g. British Standards – commentable drafts) or draft policy documents (e.g. Leicester University social media policy). Users may be able to see the consequences of comments by comparing final versions of the document with the orginal commentable version, and the comments associated with it.
- consultation document – in which issues are discussed and a series of consultation questions asked, often embedded within the various sections of the body of the document. For example, HEFCE REF Consultation. If a summary of responses is provided around the consultation, along with a review of what actions were taken that relate back to the consulation questions, commenters will be able to judge whether or not their comments appeared to influence the direction of post consultation outcomes.
- guidance document – in which comments may be round around guidance either requesting or providing clarification of particular points, or collecting examples of how others have practically implemented guidance. For example, COI Guidance on open source software. This sort of document can act as a hub for aggregating practical advice implementing guidance. In contrast to the previous two document/comments, the comments thermselves can become a means of sharing practical advice around the guidelines, and may effectively deliver practical guidance themselves. The outcomes of requests for clarification may also be trackable, if for example they result in revisions to the original guidance, or indeed if they result in a futher comment that clarifies the matter; (in this case, we might see clarifying comments as providing a similar role as do comments on a draft document?).
Combining elements of all three types listed above, we might also consider an amplification, or discussion, document, such as documents published in support of a meeting (“meetings without borders”, or “semi-permeable meetings”; for example Using WriteToReply to Publish Committee Papers. Is an Active Role for WTR in Meetings Also Possible?). [Added: I guess that educational materials might also be regarded as discussion documents?] Rather than being the focus of a conversation, these documents are part of an ongoing process, or conversation, where comments raised may either be seen to be a continuation of a discussion held in a meeting, or as part of a conversation that may be continued in a folow on meeting. Feedback to commenters about how comments are received may be realised through mentions to matters raised in comments appearing in the minutes of later meetings (which may even reference back to the orginal comment).
Looking at these various document types, it seems to me that it is possible for commenters to look for evidence in later/follow-on documents about the extent to which their comments may or may not have directly influenced the content of those follow-on documents, as well as providing opportunities for direct links back to comments that influenced later documents from those later documents themselves.
If a consultation platform can start to highlight the impact comments may have on practice or policy development through appropriate feedback, such as “follow-on feedback” (i.e. the demonstration of how a comment on one document influenced the content of another), then it feels right to me that it is more likely that people will start to see it as a tool that supports “real” involvement in a process?
PS Seems like I’m too late to add this distinction in as feedback to the COI draft guidance on commentable docs.
So it seems that the COI are taking comments on some draft guidance for commentable documents: TG134 Commentable documents – [draft guidance] (via Steph Gray/@lesteph, who first brought the idea of commentable consultation documents to my attention through his work at DIUS).
Getting on for 18 months or so ago, Joss Winn and I set up WriteToReply as a platform for experimenting with ideas around the republication of public documents in a commentable form on WordPress using the CommentPress theme. As you will know if you have visited WriteToReply, these theme atomises documents so that each paragraph has a separate URL, and each paragraph can be independently commented on. Since then, we have helped support the development of the WordPress theme, now versioned as digress.it and as maintained by Eddie Tejeda, and introduced the idea to JISC via the JISCPress Rapid Innovation project. Digress.it is also being used at De Montfort University to help amplify the outcomes of workshops and meetings as part of an internal project development process (Meeting/Workshop Amplification at DMU).
Although independently maintained (we set up Public Platforms Ltd, a not-for-profit company, limited by guarantee, to manage its interests), several government departments (DCMS, Cabinet Office) have approached us with requests to republish consultation documents or guidance for them, which we have been happy to do. We have also hosted essays and reports from a range of other organisations (for example, Cornerhouse, and UKOLN) and seen a range of JISC funded reports starting to appear on JISCPress.org. As a platform, WriteToReply has also been able to host republications of public documents on behalf of individuals who have felt that particular documents needed to be made available in form where they could be easily discussed online. We have also taken it onto ourselves to republish documents that we felt should be made available in a commentable form, or that we believed would provide a good case study for how commentable documents in general, and the atomisable, commentable documents that we can republish using digress.it in particular, might be used to widen a discussion around a particular topic; (so for example, we have explored publishing committee related papers from the Public Sector Transparency Board).
With the COI looking to produce guidance on commentable documents as a way of “giv[ing] people the opportunity to comment publicly online about ideas, proposals and plans”, I’d like to think we’ve played a contributory role, through WriteToReply, in developing ideas around how commentable documents can play a a role in document based public consultation and policy development and will be able to continue to contribute to this approach.
As for my thoughts on the draft guidance, which covers engendering trust, managing commentable documents and technical considerations? I’ll probably be posting those on the TG134 Commentable documents – [draft guidance] website directly ;-)
PS see also another draft guidance note – TG135 Underlying data publication [draft guidance] on the publication of data by government departments and agencies.
How many times have you been to a meeting or a workshop within your institution where group discussions result in flip charts and posters that are used as part of a “reporting back” activity, and then are taken away at the end of the day for who knows what reason?
Way back when, in a real-time computing course I think, I was introduced to the notion of an “atomic transaction”. As Wikipedia succinctly puts it: “atomicity: a property of database transactions which are guaranteed to either completely occur, or have no effects.” Now I’m not saying that meetings completely occur and have no effects, but many of them do seem to be atomic in that what happens in the meeting stays in the meeting, to paraphrase another well known saying…
In a handful of recent posts, I’ve started thinking about how we can soften the boundaries of meetings so that they can become part of a wider – and ongoing – “conversation”, rather than being activities that are located in a very specific time and place (e.g. Amplified Meetings and Participatory Deliberation…, Using WriteToReply to Publish Committee Papers and Backchannel Side Effects – Personal Meeting Notes).
That is, there are now weveral ways where we can widen the availability of papers and discussions both in terms of time (extending the period of time over which participants can draw on and contribute back to meeting resources) and reach (i.e. making it possible for me people to contribute).
Examples of how we might do this include:
– annotating documents using commenting platforms such as WriteToReply and JISCPress;
– capturing backchannel comments and interlacing them with meeting reports or using them as video or audio captions.
Anyway, earlier today I spotted a great example of the use of a commenting platform to extend the life of a workshop via a tweet from @josswinn pointing to a new site at DMU – First meeting.
This document summarises the outcomes from discussions in the first DUALL engagement meeting on July 1st 2010 and offers a set of recommendations for the design of an ICT reporting tool. It is not detailed set of minutes but rather aims to present the broad overview of discussion. The full presentation from the meeting is available below. There was an extremely good representation from both the IESD and the Faculty of Technology. For the group discussion it was decided to break into two groups, based on departmental basis so as to allow for discussion on the detailed requirement of each area to be sub-metered.
This document has been published so that you can comment on the outcome of the meeting in detail. Each paragraph can be directly responded to and threaded discussions can occur around each paragraph. To leave a comment, simply click on the speech bubble next to the paragraph.
A few things to note:
– the document is published using the digress.it theme on a local installation of WordPress at DMU;
– the document is published on the public web – although it could equally have been published behind the DMU authentication layer (i.e. “on the intranet”);
– the documents are viewable by, and commentable on, by anyone (I think? But I think it’s also the case that comments could be limited to people who log on the blog, e.g. using DMU credentials or single sign-on… so I think that comments could be restricted to DMU folk if required?)
– this opening up of discussion particularly around the IT area should be heartwarming for Brian Kelly at least, who’s been trying to get institutional web managers to share via web team blogs (e.g. Revisiting Web Team Blogs); maybe they should also be sharing policy discussions…?!
– exploring the use of new ICT systems to discuss ICT is a Good Thing and an Appropriate Thing. For example, on WriteToReply, the Cabinet Office have been keen to publish several of their documents (e.g. Government ICT Strategy, Government Open Source Action Plan).
If any other institutions have started exploring the use of the digress.it theme and the WriteToReply approach to document publishing, please add a link below :-)
Via a tweet from Joss earlier today, I came across this post on How to read David Willetts’ speech to the Royal Institution in which William Cullerne Brown interlaces his own comments (shown in red) at the appropriate points within the speech transcript:
I’ve recently been experimenting with various “single page” commentable documents – such as speeches – in my doodlings area of WriteToReply (most recently posting VInce Cable’s “Higher Education” speech in a commentable form), so when I saw the above approach it got me wondering: could we use WriteToReply to achieve a similar effect?
The digress.it them already provides:
– a paragraph level RSS feed out from a particular web page, where each feed item carries the URL of the original paragraph;
– a feed of comments an individual commenter (commentator?) has made across all the posts on a particular blog, where each feed item contains a guid that refers to the paragraph that was being commented on.
So, whilst I don’t have time at the moment to post a working demo, it should be able to combine these two feeds to provide an inline commented view of a document, such as the one shown above, that represent a version of the (single page) doc with inline annotations from a single commentator.
This approach could also be used to provide an “expert commentary” around a document, e.g. in an educational setting?