Version 25 (modified by 7 years ago) ( diff ) | ,
---|
Jupyter
Jupyter for JSC is in alpha phase
The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, machine learning and much more.
Project Jupyter created JupyterHub to support many users.
The Hub can offer notebook servers to a class of students, a corporate data science workgroup, a scientific research project, or a high performance computing group.
With JupyterHub a multi-user Hub which spawns, manages, and proxies multiple instances of the single-user Jupyter notebook (IPython notebook) server can be created.
Where are the sources?
- Jupyter: https://github.com/jupyter/jupyter
- Jupyter Notebook: https://github.com/jupyter/notebook
- Jupyter Console: https://github.com/jupyter/jupyter_console
- Jupyter NBViewer: https://github.com/jupyter/nbviewer
- Jupyter Kernel
- Jupyter IPyKernel: https://github.com/ipython/ipykernel
- Jupyter IPyWidgets: https://github.com/jupyter-widgets/ipywidgets
- other
- Tornado Web Server: https://github.com/tornadoweb/tornado
- Jupyter Notebook: https://github.com/jupyter/notebook
- JupyterLab: https://github.com/jupyterlab/jupyterlab
- JupyterHub: https://github.com/jupyterhub/jupyterhub
- Jupyter Http Proxy: https://github.com/jupyterhub/configurable-http-proxy
How does it work?
JupyterHub
JupyterHub is a multi-user server that manages and proxies multiple instances of the single-user Jupyter notebook server.
There are three basic processes involved:
- multi-user Hub (Python/Tornado)
- configurable http proxy (node-http-proxy)
- multiple single-user IPython notebook servers (Python/IPython/Tornado)
The proxy is the only process that listens on a public interface. The Hub sits behind the proxy at /hub. Single-user servers sit behind the proxy at /user/[username].
more: https://jupyterhub.readthedocs.io/en/0.7.2/howitworks.html
Installation
JupyterHub is started from eg. /usr/local/bin/jupyterhub with
#!/usr/bin/python3 from jupyterhub.app import main main()
- Hence, python files are installed in default location for python packages
eg. /usr/local/lib/python3.5/dist-packages/jupyterhub - HTML/JS/CSS files are installed in
eg. /usr/local/share/jupyter/hub- static
- components (external components downloaded on install of JupyterHub in setup.py)
- ...
- templates (for pythen template engine Jinja2)
- static
Callpath
- start the Tornado event loop app.py:1638
- initialize JupyterHub and all its services app.py:1365
- init handlers (connect urls with functions) app.py:802
- add handlers handlers
- add apihandlers apihandlers
- init webserver with handlers app.py:1284
- init Jinja2 template engine app.py:1291
- static_path points to HTML/JS/JINJA2 files webfiles
- init handlers (connect urls with functions) app.py:802
- start JupyterHub and all its services app.py:1517
- start webserver app.py:1532
- start http proxy app.py:1542
- start services app.py:1551
Websites
The websites of JupyterHub are stored in share/jupyter/hub
- They are build with the template engine Jinja2 for Python for a dynamic generation of the html pages (a detailed description of the API can be found here).
- Components like buttons are used from bootstrap. It gets included via stylesheets here.
- home.html
- provides "block main" and "block script" required by page.html
- distinguages between "user.running" and "not user.running"
- uses button class btn-danger and btn-success from bootstrap buttons
- "block script" includes JavaScript file home.js
- implements function if stop-button is clicked
- for communication back to the server functions from the JupyterHubAPI in jhapi.js are called
- jhapi.js uses jquery.js for ajax-functionality (send/recive commands between client/server)
- values are send back to client with an an asynchronous HTTP (Ajax) request using jquery.ajax() here
=> this triggers a handler of the tornado web server (eg. start_server sends command "POST" to url users/<username>/server, which triggers function UserServerAPIHandler::post)
- values are send back to client with an an asynchronous HTTP (Ajax) request using jquery.ajax() here
- jhapi.js uses jquery.js for ajax-functionality (send/recive commands between client/server)
- for communication back to the server functions from the JupyterHubAPI in jhapi.js are called
- implements function if stop-button is clicked
Modifications
JupyterHub supports userdefined notebook spawners. At JSC our spawners for each partition of the supercomputer adds the functionality to start a jupyter notebook by submitting a SLURM job on the system.
These additional parameters must be set by the user before he/she starts the server, which are
- number of nodes
- run time
This is the minimum number of parameters, but more might be required in future. In the following the bits and pieces are shown that result in the modification, which adds this possibility to JupyterHub:
HTML design
JupyterHub´s design is based on bootstrap.
On the right you see a possible implementation of an interface for additional parameter of our custom spawner.
any feedback welcomed - h.zilken@…, j.goebbert@…
Attachments (6)
- jupyterpreview.png (73.4 KB ) - added by 7 years ago.
- jupyterhub-logo.png (12.7 KB ) - added by 7 years ago.
- jupyter-logo.png (88.4 KB ) - added by 7 years ago.
- JupyterHub-Spawner-GUI.png (13.0 KB ) - added by 7 years ago.
- JupyterHub-Spawner.zip (2.5 MB ) - added by 7 years ago.
- JupyterHub-Structure.png (87.4 KB ) - added by 6 years ago.