This article discusses data science notebooks, a popular document format for publishing code, results, and more. Find out more below!
Join the DZone community and get the full member experience. Data science notebooks, a popular document format used for publishing code, results, and explanations in readable and executable form, broke new ground by combining an ongoing narrative with interactive elements and displays. The result was a new way to capture and transfer knowledge about the process of discovering insights. By studying why data science notebooks have worked so well, we can understand more about related areas with similar characteristics, such as Technology Operations (TechOps). At first glance, many of the attributes of data science notebooks also apply to TechOps. However, the data scientist and TechOps cohort have different objectives. A data scientist is interested in variable results based on changing elements within queries. A TechOps team responsible for complex operational systems looks for variables and patterns, seeks to understand the root cause, and takes corrective action. Data science notebooks are conducive to instruction and are easy to change. However, in a production operations setting, things need to be repeatable rather than variable. To align with the different user needs in TechOps, the notebook concept evolved into runbooks. Notebooks allow users to create and share documents that combine live code, equations, rich text, visualizations, narrative text, images, videos, plots, widgets, and graphical user interfaces into a single document. Since the first notebook over 30 years ago, interfaces have grown exponentially, enabling use cases from data cleaning and transformation to numerical simulation, statistical modeling, and data visualization. The way notebooks have been used can be adapted to fit the complex operations environments TechOps teams work within. The blending of a narrative with live code and results of queries and analytics create a living document that can reflect what is happening in real-time and create an archive. Many of the same elements of the data science notebook pattern can be used to record ad hoc activity and preserve knowledge. The beauty of notebooks lies in the integrated platform of live code, narrative, and everything else needed to demonstrate results or findings. This is an especially powerful pattern that also applies to modern TechOps practices in which a runbook or incident response can span many different systems. In TechOps, incorporating a rich, interactive narrative increases the likelihood that the process can be duplicated rather than created from scratch. It also allows operations teams to search for possible root causes, sifting through a rich set of information, using combinations of “trigger symptoms” that precipitated an event. The notebooks used in data science and the runbooks used in TechOps share the same set of principles that are expressed differently in the implementations. Centralization and access to different resources: Hosted in a browser (rather than on a local computer), notebooks enable data scientists to access data from anywhere and do more complex processing.
