WAIIDE System Architecture

Calliope Integration: This component is integrated into the Calliope AI platform. Some features and configurations may differ from the upstream project.

Overview

WAIIDE (Web AI IDE) is a containerized WAIIDE Server designed for seamless JupyterHub integration. It operates in two distinct modes based on environment detection, providing flexibility for both JupyterHub-managed and standalone deployments.

Architecture Diagram

┌─────────────────────────────────────────────────────────────────┐
│                  WAIIDE Container                                │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  ┌─────────────────────────────────────────────────────────┐    │
│  │                 Entrypoint Script                        │    │
│  │  (entrypoint-jupyterhub.sh)                            │    │
│  │                                                         │    │
│  │  - Detects JupyterHub environment variables            │    │
│  │  - Fixes permissions (root → UID 1000)                 │    │
│  │  - Starts appropriate services                         │    │
│  └─────────────────────────┬───────────────────────────────┘    │
│                            │                                     │
│  ┌─────────────────────────┴───────────────────────────────┐    │
│  │                                                         │    │
│  │  JupyterHub Mode                  Standalone Mode      │    │
│  │  (JUPYTERHUB_SERVICE_PREFIX set)  (No JupyterHub env)  │    │
│  │                                                         │    │
│  │  ┌─────────────────────┐         ┌──────────────────┐  │    │
│  │  │ jupyterhub-         │         │ API Server       │  │    │
│  │  │ singleuser          │         │ (api_server.py)  │  │    │
│  │  │ Port: 8080          │         │ Port: 8080       │  │    │
│  │  └──────────┬──────────┘         └────────┬─────────┘  │    │
│  │             │                              │            │    │
│  │             ↓                              ↓            │    │
│  │  ┌─────────────────────┐         ┌──────────────────┐  │    │
│  │  │ jupyter-server-     │         │ URL Rewriting &  │  │    │
│  │  │ proxy               │         │ Proxying         │  │    │
│  │  └──────────┬──────────┘         └────────┬─────────┘  │    │
│  │             │                              │            │    │
│  │             └──────────────┬───────────────┘            │    │
│  │                            ↓                            │    │
│  │                 ┌─────────────────────┐                 │    │
│  │                 │   WAIIDE Server    │                 │    │
│  │                 │   Port: 8081        │                 │    │
│  │                 │   (localhost only)  │                 │    │
│  │                 └─────────────────────┘                 │    │
│  └─────────────────────────────────────────────────────────┘    │
│                                                                  │
│  Exposed Ports: 8080, 8081                                      │
│  Primary Port: 8080                                             │
└──────────────────────────────────────────────────────────────────┘

Operating Modes

1. JupyterHub Mode

Activated when any of these environment variables are detected:

• JUPYTERHUB_SERVICE_PREFIX
• JUPYTERHUB_USER
• JUPYTERHUB_API_TOKEN

Components Started:

1. WAIIDE Server (port 8081)

   • Bound to localhost only
   • No authentication (relies on JupyterHub)
   • Workspace: /home/{JUPYTERHUB_USER}/workspace

2. jupyterhub-singleuser (port 8080)

   • Main service endpoint
   • Handles OAuth authentication with hub
   • Includes jupyter-server-proxy

3. jupyter-server-proxy

   • Routes /proxy/8081/ → localhost:8081
   • Automatic URL stripping
   • No manual URL rewriting needed

2. Standalone Mode

Activated when no JupyterHub environment is detected.

Components Started:

1. API Server (port 8080)

   • Custom Python HTTP server
   • Provides JupyterHub-compatible API endpoints
   • Auto-starts WAIIDE Server if needed
   • Handles URL rewriting for JupyterHub prefixes

2. WAIIDE Server (port 8081)

   • Started automatically by API server
   • Same configuration as JupyterHub mode

Key Components

Entrypoint Script (entrypoint-jupyterhub.sh)

Responsibilities:

• Environment detection
• Permission management (root → UID 1000)
• Home directory setup
• Service orchestration
• Signal handling for clean shutdown

Permission Flow:

1. Container starts as root
2. Creates/fixes ownership of directories
3. Drops privileges to UID 1000 (user: calliope)
4. Starts services as non-root user

API Server (api_server.py)

Features:

• HTTP request handling with WebSocket support
• URL prefix stripping/adding for JupyterHub
• Content rewriting (HTML/CSS/JS)
• Health check endpoints
• WAIIDE Server lifecycle management

API Endpoints:

• GET /api - Service information and status
• GET /api/services - Service discovery
• GET /api/* - Basic JupyterHub compatibility
• All other paths → Proxied to WAIIDE Server

OAuth Integration

Named Server Fixes:

1. oauth_named_server_fix.py - Corrects OAuth redirect URLs
2. jupyter_scope_fix.py - Patches scope validation
3. jupyter_server_config.py - Custom Jupyter configuration

Data Flow

Request Flow in JupyterHub Mode

User Browser → JupyterHub Proxy → Container (8080) → jupyterhub-singleuser 
    → jupyter-server-proxy → WAIIDE Server (8081)

Request Flow in Standalone Mode

User Browser → Container (8080) → API Server → WAIIDE Server (8081)

URL Rewriting

JupyterHub Mode:

• Input: /user/username/proxy/8081/path/to/resource
• jupyter-server-proxy strips to: /path/to/resource
• WAIIDE receives clean paths

Standalone Mode with Prefix:

• Input: /user/username/waiide/path/to/resource
• API server strips to: /path/to/resource
• Response content rewritten to include prefix

File System Layout

/home/{username}/
├── workspace/                 # User's WAIIDE workspace
│   ├── .WAIIDE/              # WAIIDE settings
│   ├── README.md             # Welcome file
│   └── assets/               # CalliopeAI assets
├── .WAIIDE-server/           # WAIIDE Server data
│   ├── data/                 # User data
│   │   └── Machine/          # Machine-specific settings
│   └── extensions/           # Installed extensions
├── .jupyter/                 # Jupyter configuration
│   └── jupyter_server_config.py
└── .local/                   # User-specific data
    └── share/
        └── jupyter/
            └── runtime/      # Jupyter runtime files

Network Architecture

Port Binding

• 8080: Main service (0.0.0.0:8080) - External
• 8081: WAIIDE Server (127.0.0.1:8081) - Internal only

Security

• WAIIDE Server only accessible via localhost
• Authentication handled by JupyterHub or API server
• No direct external access to WAIIDE

Container Lifecycle

Startup Sequence

1. Container Start (as root)

   • Fix permissions on home directory
   • Create required directories
   • Drop to UID 1000

2. Environment Detection

   • Check for JupyterHub variables
   • Select operating mode

3. Service Startup

   • Start WAIIDE Server (background)
   • Start main service (JupyterHub or API)
   • Wait for services to be ready

4. Ready State

   • Health endpoints available
   • WAIIDE accessible via proxy

Shutdown Sequence

1. Signal Received (SIGTERM/SIGINT)
2. Graceful Shutdown
   • Stop main service
   • Stop WAIIDE Server
   • Clean exit

Performance Considerations

• WAIIDE Server startup: ~10-15 seconds
• First request may be slow (cold start)
• WebSocket connections maintained for real-time features
• Content rewriting adds minimal overhead (<5ms)

Security Model

1. Authentication: Handled by JupyterHub OAuth
2. Authorization: Token validation via hub API
3. Isolation: User processes run as UID 1000
4. Network: WAIIDE only on localhost
5. Permissions: No sudo access in container