Documentation
Waiide
WAIIDE JupyterHub Integration Flow

WAIIDE JupyterHub Integration Flow

Calliope Integration: This component is integrated into the Calliope AI platform. Some features and configurations may differ from the upstream project.

This document describes the complete flow of how WAIIDE integrates with JupyterHub, from user login to WAIIDE access.

Integration Overview 

  graph TD
    A[User Login] --> B[JupyterHub Spawner]
    B --> C{Environment Detection}
    C -->|JupyterHub Detected| D[JupyterHub Mode]
    C -->|No JupyterHub| E[Standalone Mode]
    
    D --> F[Start WAIIDE:8081]
    D --> G[Start jupyterhub-singleuser:8080]
    G --> H[jupyter-server-proxy]
    H --> I[Route /proxy/8081/ to WAIIDE]
    
    E --> J[Start API Server:8080]
    J --> K[Auto-start WAIIDE:8081]
    J --> L[Proxy all non-API to WAIIDE]
    
    I --> M[User accesses WAIIDE]
    L --> M

Detailed Flow - JupyterHub Mode

1. User Authentication 

User → JupyterHub Login → OAuth2 Authentication → User Dashboard

2. Container Spawn 

# JupyterHub calls DockerSpawner
spawner.start() → Docker API → Create Container → Set Environment:
  - JUPYTERHUB_SERVICE_PREFIX=/user/alice/
  - JUPYTERHUB_USER=alice
  - JUPYTERHUB_API_TOKEN=<token>
  - JUPYTERHUB_API_URL=http://hub:8081/hub/api

3. Container Startup 

# Container entrypoint flow
1. Start as root (UID 0)
2. Detect $JUPYTERHUB_USER → Set USER_HOME=/home/alice
3. Create directories:
   /home/alice/workspace
   /home/alice/.jupyter
   /home/alice/.WAIIDE-server
4. Fix ownership: chown -R 1000:100 /home/alice
5. Drop privileges: exec su calliope (UID 1000)

4. Service Detection 

# Environment check in entrypoint
if [ -n "$JUPYTERHUB_SERVICE_PREFIX" ] || 
   [ -n "$JUPYTERHUB_USER" ] || 
   [ -n "$JUPYTERHUB_API_TOKEN" ]; then
    # → JupyterHub Mode
else
    # → Standalone Mode
fi

5. Component Startup (JupyterHub Mode)

a. WAIIDE Server (Port 8071) 

node /opt/calliope/WAIIDE-server/out/server-main.js \
    --host 127.0.0.1 \
    --port 8071 \
    --without-connection-token \
    /home/alice/workspace &

b. jupyterhub-singleuser (Port 8070) 

jupyterhub-singleuser \
    --port=8070 \
    --ip=0.0.0.0 \
    --ServerApp.default_url="/ide" \
    --ServerApp.base_url="/user/alice/" &

6. Request Routing

User Request Flow: 

Browser: https://hub.example.com/user/alice/
    ↓
JupyterHub Proxy (nginx/traefik)
    ↓
Route to container IP:8070
    ↓
jupyterhub-singleuser receives request
    ↓
jupyter-server-proxy intercepts /ide/*
    ↓
Strip prefix: /user/alice/ide/file.js → /file.js
    ↓
Forward to localhost:8071 (WAIIDE Server)
    ↓
Response flows back (no rewriting needed)

OAuth Integration for Named Servers

Problem with Named Servers

When using named servers (e.g., /user/alice/waiide/), JupyterHub sets incorrect OAuth scopes:

# Incorrect (missing server name):
JUPYTERHUB_OAUTH_ACCESS_SCOPES=["access:servers!server=alice/", ...]

# Correct:
JUPYTERHUB_OAUTH_ACCESS_SCOPES=["access:servers!server=alice/waiide", ...]

Container-Side Fixes

  1. oauth_named_server_fix.py - Intercepts and fixes OAuth redirect URLs
  2. jupyter_scope_fix.py - Patches scope validation, adds missing scope
  3. PermissiveIdentityProvider - Allows any authenticated JupyterHub user

Fix Flow: 

# 1. OAuth redirect interception
/user/alice/waiide/hub/api/oauth2/authorize → /hub/api/oauth2/authorize

# 2. Scope patching
if server_name == "waiide":
    add_scope("access:servers!server=alice/waiide")

# 3. Permissive validation
if has_jupyterhub_token and current_user:
    return authenticated = True

Standalone Mode Flow

1. Container Startup

Same permission flow, but no JupyterHub environment detected.

2. API Server Startup 

# api_server.py starts on port 8070
server = HTTPServer(('0.0.0.0', 8070), WaiideAPIHandler)

# Auto-starts WAIIDE if not running
if not check_vscode_health():
    start_vscode_if_needed()

3. Request Handling 

# API endpoints
/api → handle_api()
/api/services → handle_api_services()
/api/* → handle_api_basic()

# Everything else proxied to WAIIDE
/* → proxy_to_vscode()

4. URL Rewriting (if prefix exists)

When JUPYTERHUB_SERVICE_PREFIX is set in standalone mode:

# Request path stripping
/user/alice/workspace/file.js → /workspace/file.js

# Response content rewriting
<script src="/static/app.js"> → <script src="/user/alice/static/app.js">

WebSocket Handling

Both modes support WebSocket connections:

1. Client: Upgrade: websocket
2. Proxy detects upgrade request
3. Establish backend connection to WAIIDE
4. Bidirectional forwarding between client ↔ WAIIDE

Key Differences Between Modes

FeatureJupyterHub ModeStandalone Mode
Main Port Servicejupyterhub-singleuserAPI Server
AuthenticationJupyterHub OAuthNone
URL RewritingBy jupyter-server-proxyBy API server
Health ChecksJupyterHub + APIAPI only
Service DiscoveryVia JupyterHubVia API
Default URL/ide/

Security Flow

JupyterHub Mode

  1. User authenticates with hub
  2. Hub provides OAuth token
  3. Container validates token with hub API
  4. All requests require valid JupyterHub session

Standalone Mode

  1. No authentication
  2. Direct access to WAIIDE
  3. Suitable for development only

Troubleshooting Integration

Check Mode Detection 

docker exec <container> sh -c 'env | grep JUPYTER'

Verify Service Status 

# Check both services
docker exec <container> ps aux | grep -E "(server-main|jupyterhub-singleuser)"

Test Proxy Path 

# Should return WAIIDE HTML
curl -H "Authorization: token <jupyter-token>" \
  http://container:8070/ide/
WAIIDE JupyterHub Integration - Hub Side ImplementationWAIIDE JupyterHub Integration Summary