from matplotlib import pyplot as plt
import numpy as np
import pandas as pd
import geopandas as gpd
np.random.seed(42)

Lecture 13B: From Notebooks to the Web: Github Pages, Web Servers, and Dash

December 1, 2020

Housekeeping

• Assignment #7 (including final project proposal) due on Thursday
• This is the last pre-recorded lecture! We'll have synchronous Zoom lectures this Thursday (12/3) and next Tuesday (12/8)

Web-based Data Viz

We'll discuss ways to translate our analysis results to the Web in a meaningful way. We'll cover three methods to do so, each with their own pros and cons:

Last lecture

1. Embedding interactive charts on the Web via Github Pages

This lecture

1. Creating Web apps & dashboards using the Python library Dash

Next week

1. Creating Web apps & dashboards using the Python library Panel

Quick recap:

Last time, we covered:

1. Quick review of how to saving interactive charts so they can be embedded on the web
2. Using Github Pages to set up a blog-like website to embed data visualizations

Part 1: Embedding Interactive Charts on the Web

We reviewed examples of how to save interactive charts using the following packages:

• Altair
• Hvplot
• Folium

Part 2: Template Github Pages with Examples

• I've created a template repository that you can use to quickly get up and running with Github Pages
• Available at: https://github.com/MUSA-550-Fall-2020/github-pages-starter
• The rendered site is available at: https://musa-550-fall-2020.github.io/github-pages-starter

Where we left off: Setup your own Github Pages project page

Steps:

• Follow the above instructions to set up your own project website
• Create an interactive Altair chart for the Gapminder health/income dataset
• Create a new post by adding a new post (with the proper file name convention) to the _posts folder
• Embed the Altair chart in your new post

Notes

• As a reminder, the gapminder dataset is available from the built-in vega_datasets package
• We want to plot a scatter chart with the income on the x-axis and the life expectancy on the y-axisExercise:

On to new content...Dashboards with Dash

Part 3: Web Servers

A few quick basics before we move on to interactive Web apps with Dash...

Some (oversimplified) basics

• The browser needs a file --> it requests it via the HTTP protocol
• The web server receives the request, and if the requested file exists, sends it back to the browser using HTTP

Two main types:

Static and dynamic

Static Web Servers

• Serves hosted HTML files to the browser "as-is"
• Response from the server is always the same — static
• Example: Github Pages

Dynamic Web Servers

• Capable of serving dynamic content to browsers
• Extra software that executes server-side before sending the response back to the browser
• Common example: querying a database server
  • browser sends request for data with specific query parameters
  • server responds dynamically, sending the requested data

For more information: Mozilla documentation

CARTO: a dynamic web server

Anatomy of a request

• Base URL: shown in blue
• ?: separates the base URL from the query parameters
• Query parameters: tells the web server how to respond to the request
• &: separator between the query parameters

Flask: a micro web framework in Python

A very lightweight package for framework for dynamic web apps

Flask documentation

Example 1: your first Flask app

• Store the code for our app in a ".py" file:
• Create a "local" web server by executing our application code locally
• This allows us to build and test our app before deploying the code to a remote web server

Running Python scripts from the command line

To start up our Flask app, we'll need to execute the Python script from the command line. We have two options for that...

Option A: Launch a terminal in Jupyer notebook

Steps:

From within the Jupyter Notebook Dashboard:

• Navigate to the "hello-flask" folder
• Click on the "hello.py" file to launch the file editor
• Launch a new terminal window by clicking on "New -> Terminal"

Flask code: hello.py

Important: this won't work on Binder, it needs to be running on your local machine

Option B: Use the Terminal (MacOS) or Anaconda Prompt (Windows)

• Open up the Terminal or Anaconda Prompt app
• Similar to how we've been launching Jupyter notebooks frim in the course
• Navigate to the folder with this week's code and follow the steps below

To start a local server

Run the following commands from within the Terminal window in Jupyter notebook:

cd hello-flask
python hello.py

The "python" command

• This will execute the *.py file
• Only the code below if __name__ == '__main__': gets executed

StackOverflow answer about the if-name-main syntax in Python.

Note

There are sometimes Anaconda-related issues where "python" will point to the wrong version of Python. You can double check by running:

which python

from the Jupyter Terminal interface. This should print out a path name that includes the name of our Python environment (musa-550-fall-2020)

Take a look at hello.py

View your application!

Navigate in the browser to: http://0.0.0.0:5000

You should see the text: "Hello, World!"

Important: Windows Users

You will likely need to navigate to localhost:5000 instead of http://0.0.0.0:5000

Note: the web address http://0.0.0.0 is an alias for "localhost" — this means the web server is running locally on your computer.

Editing your app with a running server

Try editing the hello.py file in your editor, saving, and re-loading the application page. You should see the app update to reflect the changes!

Example 2: using templates

• Flask can load and render HTML files when specific routes are requested by the browser
• Templates should be stored in a templates/ directory
• The hello-template.py app loads the templates/hello.html when the /hello/ route is requested by the browser

Flask code: hello-template.py

From the terminal window

python hello-template.py

The main page ("/") renders the same thing as the previous app ("Hello, World!").

Navigate to the "/hello/" route and you should see:

Example 3: setting up an API with request parameters

hello-api.py: a more sophisticated example using request parameters

Run:

python hello-api.py

Flask code: hello-api.py

Navigate to the "/shootings" route

This uses default request parameters: days = 90 and fatal = 0

Now pass "days" and "fatal" parameters

This returns the number of fatal/nonfatal shootings in the past X "days"

Getting closer to a more realistic use case...

Steps:

1. Get request parameters from the browser (input by the user)
2. Query CARTO database for data based on that input
3. Perform an operation on that data (in this case, count number of shootings)
4. Return computed result back to the browser

Note: If you receive a "OSError: Address already in use"

• If you encounter the "Address already in use" error, you are trying to run multiple Flask servers on the same port
• The port is the number after the address; e.g., for http://0.0.0.0:5000/, the port is 5000
• The easiest solution: run Ctrl-C to end all of the Flask servers but one, or change the port=5000 keyword when initialize the app in the Python script

Now, that we have the basics on to a more advanced library...

Part 4: Dash

The result of Python programmers asking the question: can we build a dashboard just in Python?

The answer is yes...for the most part.

You still need to use some CSS styling and know about different HTML elements. But everything is coded in Python.

Dash

• Built on Flask, similar web framework setup
• Construct Python wrappers of common HTML elements, like "div", "p", etc.
• Allows you to define layout of page purely in Python
• Use Python functions to control how browser interacts with the app — just like Flask

Benefits

• Everything can be done in Python — no more HTML templates
• Use Python to define how user input elements (e.g., sliders) interact with server — no more custom Javascript
• Very sleek, beautiful widgets and interactive components built in to the library
• Can handle Markdown too
• Good tutorials and user guide — https://dash.plot.ly/

Downsides

• Built to be used with the Plotly visualization library
• Private company that has open-sourced their interactive visualization library
• A bit more work to get non-Plotly visualizations working — need to use our IFrame trick

The general workflow

In your Python Dash app, there are two main steps:

1. Define the HTML layout, e.g., "div" elements, slider elements, etc.
2. Define functions that take inputs from the interactive widgets from Step 1. and return the output for other HTML elements.
   • When the user changes a slider, dropdown, etc, the function will run, and return the updated output

Dash HTML components = "the app layout"

• Wrappers for all of the common HTML elements
• Stored in the dash_html_components module

See: https://dash.plot.ly/dash-html-components

# The standard import
import dash_html_components as html

Dash HTML Example

html.Div([
    html.H1('Hello Dash'),
    html.Div([
        html.P('Dash converts Python classes into HTML'),
        html.P('This conversion happens behind the scenes by Dash's JavaScript front-end')
    ])
])

This gets converted automatically to the following HTML in your web app:

<div>
    <h1>Hello Dash</h1>
    <div>
        <p>Dash converts Python classes into HTML</p>
        <p>This conversion happens behind the scenes by Dash's JavaScript front-end</p>
    </div>
</div>

Dash core components = "widgets"

• A set of fancy widgets already built in!
• Stored in the dash_core_components module

See: https://dash.plot.ly/dash-core-components

# The standard import for these components
import dash_core_components as dcc

Note: Markdown is also supported!

Use the dcc.Markdown() object to automatically convert Markdown into HTML in your web apps!

Reference: HTML and CSS tutorials

The core components and HTML components in Dash remove most of the direct HTML/CSS, but still good to know some of the basics:

• HTML Tutorial
• CSS Tutorial

Useful to keep these references handy if you are unsure about syntax for HTML and/or CSS

The Dash Getting Started Guide

https://dash.plot.ly/getting-started

Combining Dash with Altair and Folium

• You are welcome to incorporate Plotly visualizations directly into your dashboard, but it can get complicated quickly
• I am not a huge fan of Plotly's API and documentation
• We will focus on a few examples showing Altair + Dash and Folium + Dash

Key: any valid ".html" block can be embedded within an "IFrame" element

Two key elements of Dash apps

• You must define the "app.layout" — this is your HTML layout
• You must define your callback functions and the input/output that they take.
  • You mark certain functions as callbacks so Dash knows they interact with your HTML code.

Steps:

1. Define a IFrame() element as part of the layout
2. Given some user input, generate our Altair charts and save them as HTML
3. Assign the chart HTML to the IFrame on our page

Let's see some Dash + Altair examples...

Two examples in the "dash-altair/" folder...

Visualizing the "penguins" dataset

Just like with Flask, we can run from the Jupyter terminal:

python dash_altair_penguins.py

Dash app

Translating the shootings app to Dash

We can run from the Jupyter terminal:

python dash_altair_shootings.py

Dash app

More information: Dash user guide

• https://dash.plot.ly/getting-started
• Very good series of chapters illustrating different aspects of Dash
• Relies on Plotly visualizations but idea is the same for IFrame elements