Some time ago I came across ordo, “a lightweight feedback tool for Jupyter”; here’s a quick initial review of what we can do with it (Binderised demo)…
Installing and enabling the extension gives you a couple of toolbar buttons:
The tick is “Feedback Mode” for running cells and evaluating the output, the pencil is “Edit Mode” for creating/editing feedback messages.
The README.ipynb demo notebook has some feedback cells already set up. For example, the first cell tests a simple sum. In “Feedback mode”, if you get an incorrect answer, you are alerted to the fact with an error message, which can either be the default message or a custom one assigned to that cell.
Clicking the eye reveals the answer; when you get the answer right, you are awarded with confirmatory feedback, again, either as a default message or as a custom message defined for that cell.
In the edit mode, you can click in a code cell and raise some value setting controls for the cell:
If you click the Make Solution button, the current cell output is set as the desired output.
Alternatively, you can explicitly set the desired solution, as well as custom success/failure messages on each cell:
Making a custom solution allows you to specify different sorts of output cell types… I think using Make Solution is probably easier!
Note that if you do opt to explicitly define a solution, any previous solution will not be displayed.
However, you can see the desired output in the corresponding cell metadata field:
The same is true if you add custom success or failure messages:
As before, the cell metadata does reveal what the current value is if the default feedback message has been changed.
For example, if we assign the following success feedback message to a cell:
the cell metadata updated with the non-default value:
Ordo looks like a really handy tool for baking explicit answers for cell based tests into notebook metadata. As such, it could be good as a quick way of implementing formative feedback into teaching notebooks, as long as the users have the ordo extension installed and enabled in the notebook server they are accessing the notebooks from.
If you can’t explicitly declare the exact answer you’re expecting as the cell output, it isn’t much use though…
PS A couple of other comments…
An ordo annotated notebook degrades gracefully in the sense that if the extension is not annotated, no bad things happen, you just don’t get the test run and the feedback displayed.
The ability to close the alert messages is neat. Peaking at the code:
it looks like we can just add:
<button class="close" type="button" data-dismiss="alert">×</button>
into the div and that gives us a button we can use to collapse the alert box.
[I note WordPress is still crap at handling taglike text… WTF do I have to do to get it to display properly?]
The alert-dismissible class attribute does not seem to be required.
This dismissible behaviour could be used when using alert boxes in eg nbgrader test generated feedback because it could be used to provide messages for markers that they could collapse… hmm… would that definitely delete it from the feedback document? I suppose if we added the alert-dismissible class attribute, we could also filter such divs out in an nbgrader feedback generator processor?
OUseful.Info, the blog... 