Where Documentation, Cloud-hosted Interactive Tutorials and Continuous Integration Testing Intersect¶
Description
As part of an open-source science and engineering project spread between national laboratories and universities, we have developed an approach that embraces the idea of “document driven development”, by making all our documentation runnable within our continuous integration test suite. Most notably, for our tutorials and examples, we reuse a set of Jupyter Notebooks in three ways: as the base for “live” tutorials and demos, as static documentation disseminated on web pages (and other generated forms), and as integration tests that we run in our continuous integration system. While the idea of runnable examples is not new, their full realization as both a document that can be used to guide a room full of workshop attendees through a tutorial, and as a detailed test of the software, is not common in open scientific computing. But, fundamentally, this is not a difficult task and should arguably become a standard approach, regardless of the specific technologies used. We have found that this approach has helped maximize the limited software engineering resources and combined the skillsets of scientist/programmers and programmer/software-engineer/devops in the same project.
This talk will show-and-tell how the documentation for our project grew from the usual Sphinx-generated RTD system, to combining documentation with code examples, largely via Jupyter Notebooks, to integrating those examples into our CI test system, to using those regressed Notebooks as live interactive documented tutorials for a conference-size room full of interested parties.
From there, the talk will turn to the future and discuss the challenges we face of ensuring quality in both the documentation and the correctness of the code, as we look to scale our approach beyond a few core developers.
The examples, tools and technologies for this talk were developed by the Institute for the Design of Advanced Energy Systems (IDAES; website: ideas.org), a Department of Energy funded project that is developing a Python-based framework for the design and optimization of innovative steady state and dynamic processes.
- Conference: Write the Docs Portland
- Year: 2020
