Introducing people to a new concept can be tough – especially when it includes complex algorithms and code-based logic. At PMG, we pride ourselves on engineering new solutions that empower media teams to make faster, smarter decisions but explaining the tech behind these solutions can be difficult.
Some of our solutions are easily explained, with each input having a predictable and straightforward output. But as our tools and people continue to develop, more in-depth and robust education around these tools was needed. This was when we turned to Jupyter notebooks.
How We Use Jupyter Notebooks
Jupyter notebooks are typically used for data visualization, data cleansing, and simple script building, however, we have recently started to stretch Juptyer’s boundaries to create an interactive learning platform.
By using Jupyter within Google’s Colaboratory tool, we create simple UIs that allow users to test tools and algorithms with real data in a playground environment that easily visualizes the changes that would be made in a real-world situation. These UIs walk the users through a step-by-step explanation of why the tool made the decision that it did in any given situation which makes the user more comfortable with real changes that will eventually be made to live media.
The Good
Creating user inputs in a Jupyter Notebook is almost effortless because it utilizes a minimally modified version of the code that powers the actual tool. Simply adding a tag to a variable name describing that variable type immediately populates an input field. Input types range from simple numbers or strings but can get as complex as drop-down menus and sliding scales. To ensure the process runs smoothly, the Colaboratory tool will verify the format of the user inputted data in real-time before the user even attempts to run the test.
Connecting to other Google products such as Bigquery, Google Cloud Storage, and Google Sheets is also extremely easy with Colaboratory’s built-in OAuth process. The user is able to authenticate with their Google account and easily access the notebook and any related data.
You can create amazing, descriptive visuals with a few lines of code within Colaboratory as well. Colaboratory supports almost any Python module, meaning you can utilize packages such as matplotlib to create visualizations such as line graphs, histograms, and bar charts that all react to the data the user has provided within the notebook UI.
Finally, and most importantly, using a notebook is simple and quick for the user. There is no setup needed and the user doesn’t even need to know the term ‘Juptyer notebook.’ And with the help of Google and Colaboratory, sending someone a custom visualization UI is as simple as sending a URL.
The Bad
To keep our secret sauce a secret, we host all of our code in private Github repos, which unfortunately means we can’t pull code straight into Colaboratory. Luckily, there aren’t many Colaboratory-specific Python caveats, so we can usually copy-paste code, but this can still cause headaches around versioning and copy errors.
Although Colaboratory projects are stored in Google’s cloud, there can only be one user in a notebook at a time; otherwise, the notebook will continuously overwrite different users’ inputs. Because of this, we have multiple copies of the same Colaboratory project circulating so that more than one person can utilize the same tool’s Colaboratory doc at once. This, combined with the fact that we can’t sync code from Github, means that if one bit of logic is changed in the production tool, you have to hunt down all the Colaboratory documents using that logic and manually keep them in parity.
Finally, while Colaboratory is great for user-facing inputs, rebuilding an algorithm in the tool can get messy quickly. The coding UI isn’t the most user-friendly, and debugging can be a pain, making Colaboratory less than fun to code in.
All that being said, we have seen great results with this type of UI-based learning system to better communicate our ideas and solutions to the rest of the company. By having a playground to get their hands dirty and test their own real-life examples, users have been able to clearly explain to other employees and clients how complex scripts and solutions work.