Software/Jupyter – vis
wiki:Software/Jupyter

Jupyter

Jupyter for JSC is in alpha phase


The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, machine learning and much more.




Project Jupyter created JupyterHub to support many users. The Hub can offer notebook servers to a class of students, a corporate data science workgroup, a scientific research project, or a high performance computing group. With JupyterHub a multi-user Hub which spawns, manages, and proxies multiple instances of the single-user Jupyter notebook (IPython notebook) server can be created.





Where are the sources?

How does it work?

JupyterHub

JupyterHub is a multi-user server that manages and proxies multiple instances of the single-user Jupyter notebook server.

There are three basic processes involved:

  • multi-user Hub (Python/Tornado)
  • configurable http proxy (node-http-proxy)
  • multiple single-user IPython notebook servers (Python/IPython/Tornado)

The proxy is the only process that listens on a public interface. The Hub sits behind the proxy at /hub. Single-user servers sit behind the proxy at /user/[username].

more: https://jupyterhub.readthedocs.io/en/0.7.2/howitworks.html

Installation

JupyterHub is started from eg. /usr/local/bin/jupyterhub with

#!/usr/bin/python3

from jupyterhub.app import main
main()
  • Hence, python files are installed in default location for python packages
    eg. /usr/local/lib/python3.5/dist-packages/jupyterhub
  • HTML/JS/CSS files are installed in
    eg. /usr/local/share/jupyter/hub
    • static
      • components (external components downloaded on install of JupyterHub in setup.py)
      • ...
    • templates (for pythen template engine Jinja2)
Callpath

Websites

The websites of JupyterHub are stored in share/jupyter/hub

  • They are build with the template engine Jinja2 for Python for a dynamic generation of the html pages (a detailed description of the API can be found here).
  • Components like buttons are used from bootstrap. It gets included via stylesheets here.
    They are downloaded on installation of JupyterHub by setup.py to share/jupyter/hub/components/
  • home.html
    • provides "block main" and "block script" required by page.html
    • distinguages between "user.running" and "not user.running"
    • uses button class btn-danger and btn-success from bootstrap buttons for stop- and start-server button
    • stop-server button
      • home.js implements function if stop-button is clicked
        • for communication back to the server functions from the JupyterHubAPI in jhapi.js are called
          • jhapi.js uses jquery.js for ajax-functionality (send/recive commands between client/server)
            • values are send back to client with an an asynchronous HTTP (Ajax) request using jquery.ajax() here
              => this triggers a handler of the tornado web server (eg. stop_server sends command "DELETE" to url users/<username>/server, which triggers function UserServerAPIHandler::delete)
    • start-server button
      • clicking the start-server button opens the webpage "http://<jupyterhub-ip>/hub/spawn", which is generated by Jinja2 from the template spawn.html
        • this page includes a <form> element, which includes the generated part "spawner_options_form":
           <form enctype="multipart/form-data" id="spawn_form" action="{{url}}" method="post" role="form">
                {{spawner_options_form | safe}}
                <br>
                <input type="submit" value="Spawn" class="btn btn-jupyter">
          </form>
          
        • "spawner_options_form" is defined in spawner.py in function options_form()
Modifications

JupyterHub supports userdefined notebook spawners (examples).

At JSC our spawners for each partition of the supercomputer adds the functionality to start a jupyter notebook by submitting a SLURM job on the system.

These additional parameters must be set by the user before he/she starts the server, which are

  • number of nodes
  • run time

This is the minimum number of parameters, but more might be required in future. In the following the bits and pieces are shown that result in the modification, which adds this possibility to JupyterHub:

HTML design

JupyterHub´s design is based on bootstrap.

On the right you see a possible implementation of an interface for additional parameter of our custom spawner.
This prototype implementation can be downloaded here



















any feedback welcomed - h.zilken@…, j.goebbert@…

Last modified 2 years ago Last modified on 10/16/17 17:38:53

Attachments (6)