VS Code as an Integrated, Extensible Authoring Environment for Rich Media Asset Creation

One of the things I’m a bit conflicted about in the context of asset creation tools is the extent to which they should be scripted. The examples I started pulling together in my Subject Matter Notebooks demos (an effort which has stalled again…:-( are essentially scripted: the asset generating examples are generated from script. But there are other ways of creating assets within a Jupyter environment, using interactive editors such as the draw.io editor:

One of the concerns I have with tools like this in the JupyterLab context is that you can start to step away from the narrated description of how an asset was created. You can see this even more clearly in things like the plotly chart editor:

Yes, you can use this tool to create a diagram or source file from which you can generate a png or interactive chart asset, for example. And yes, you can edit that source file via the UI. But the linear, literate, narrated construction of the asset is lost.

But that’s a debate for another post… This post is about VS Code.

Just like a Jupyter user interface environment can be used as a rich (generative) interactive asset authoring and display environment (for example, OpenJALE, my Open Jupyter Authoring and Learning Environment demo), so too can VS Code. And with a much larger developer community, VS Code currently provides support for a wider range of tools than the Jupyter ecosystem, and competition between them that drives further improvement.

So here’s a quick review of just some of the tools that VS Code already integrates via extensions that support the authoring of rich media (I’ll review various extensions that support teaching computing related subjects in another post).

As we’ve mentioned draw.io already, in the context of its availability as JupyterLab extension, let’s start there: the hediet.vscode-drawio extension provides .drawio extension senstivity in the form of an embedded drawio provided in much the same way as the JupyterLab extension. I donlt know if the JupyterLab extension works in a collaborative mode, but the VS Code extension claims to, via VS Code Liveshare. Another handy feature is the ability to edit the diagram and its XML source code side by side: edit the diagram and the XML updates, edit the XML and the diagram updates (does this approach also work in JupyterLab? I guess it should if the diagram is reactive to changes in the source file?)

When it comes to editing markdown files, a markdown side-by-side previewer is offered natively, with support for things like embedded LaTeX maths expressions provided by simple extensions such as goessner.mdmath, or with SMILES chemical molecule rendering support too, mathpix.vscode-mathpix-markdown, or as part of more comprehensive extensions such as shd101wyy.markdown-preview-enhanced. The latter (docs) adds support for maths equations, PDF generation, scripted diagram generation (including flowchart.js, Mermaid, PlantUML, Graphviz, Vega/Vega-Lite, Ditaa (ASCII art)). The extension can also execute code and render code outputs into the document (cf. Rmd in RStudio). The shd101wyy.markdown-preview-enhanced extension really is quite incredible; I encourage you to check it out..

Other extensions that provide additional support to the markdown editing include spellchecking (for example, swyphcosmo.spellchecker), linting (for example, DavidAnson.vscode-markdownlint), pasting images into a markdown document from the clipboard (for example, telesoho.vscode-markdown-paste-image) and standalone diagram generating extensions (for example, gebv.pikchr (pikchr syntax editing and previews; render server required) or qiqiworld.vscode-markdown-tinymind (mindmaps); SimonSiefke.svg-preview for SVG previews; vstirbu.vscode-mermaid-preview for mermaid.js diagrams; several previewers for Graphviz/dot language, including tintinweb.graphviz-interactive-preview; the markdown-preview-enhanced extensions seems to cover several of these). I haven’t found a tikz renderer/previewer extension yet although there are lots of LaTeX related extensions, including the James-Yu.latex-workshop extension, which I notice supports some tikz snippets so maybe it does do tikz diagram rendering more genrally too.

Support is also available (currently in the VS Code insiders preview edition) for Jupytext (donjayamanne.vscode-jupytext), which will provide VS Code notebook style editing of markdown documents.

Musical scores / music typesetting in VS Code is also available using extensions such as lhl2617.VSLilyPond (a LilyPond wrapper) and ABC notation (softaware.abc-music).

I haven’t spotted any extensions yet to support the creation and editing of electronic circuit diagram schematics.

3D model previews are available using extensions such as slevesque.vscode-3dviewer or michead.vscode-mesh-viewer. (In terms of image viewers, I haven’t found a FITS image viewer extension yet, or extensions that provide lookup and previewing lidar or satellite imagery? Maybe the astronomers et al. don’t do VS Code?)

Developer RandomFractalsInc publishes an interesting set of extensions that support interactive exploration of data and maps: the RandomFractalsInc.vscode-data-preview extension provides exploratory charting, the RandomFractalsInc.vscode-vega-viewer and RandomFractalsInc.vscode-chartjs extensions provide previews of Vega and Chart.js charts respectively, and the RandomFractalsInc.geo-data-viewer provides support for rendering a range of geo data formats including GeoJSON, TopoJSON, KML, GPX, and shapefiles. Other map preview extensions include jumpinjackie.vscode-map-preview. (I haven’t spotted a 3D map/elevation raster previewer yet.)