JupyterHub WAIIDE Spawning Issues - Fixed

Calliope Integration: This component is integrated into the Calliope AI platform. Some features and configurations may differ from the upstream project.

Issues Identified

Based on the spawning logs, we identified several issues causing “weird” behavior during WAIIDE container startup in JupyterHub:

1. Configuration Warnings ⚠️

Problem: Deprecated Jupyter Server configuration options causing warnings:

• Config option 'extra_handlers' not recognized by 'ServerApp'
• Config option 'port_retries' not recognized by 'ServerProxy'

Impact: These warnings clutter logs and indicate deprecated configurations that may cause compatibility issues.

Fixed:

• Removed c.ServerProxy.port_retries = 10 from jupyter_server_config.py
• Removed problematic extra_handlers configuration that was interfering with normal API routes

2. Extension Loading Errors ❌

Problem: Missing package.json files for extensions:

[Error]: Unable to read file '/opt/calliope/.WAIIDE-server/extensions/calliope-WAIIDE-agent/package.json'
[Error]: Unable to read file '/opt/calliope/.WAIIDE-server/extensions/markdown-language-features/package.json'
[Error]: Unable to read file '/opt/calliope/.WAIIDE-server/extensions/merge-conflict/package.json'
[Error]: Unable to read file '/opt/calliope/.WAIIDE-server/extensions/microsoft-authentication/package.json'
[Error]: Unable to read file '/opt/calliope/.WAIIDE-server/extensions/typescript-language-features/package.json'

Impact: Extensions fail to load properly, causing degraded WAIIDE functionality.

Fixed:

• Enhanced extension copying process with integrity verification
• Added fallback recovery for missing package.json files
• Improved validation during the setup process

3. Repeated 404 API Errors 🔄

Problem: Continuous 404 errors every 10 seconds:

[W 2025-08-04 22:09:42.798 ServerApp] 404 GET /api (@127.0.0.1) 12.80ms

Impact: Log spam and potential performance impact from failed API calls.

Fixed:

• Removed the problematic extra_handlers configuration that was intercepting all routes
• Now relies on jupyter-server-proxy to properly handle WAIIDE routing at /ide

4. Marketplace Extensions Not Showing Up 🛒

Problem: Extensions marketplace not accessible, with WAIIDE Server ignoring marketplace configuration:

Ignoring option 'extensions-gallery': not supported for server.

Impact: Users cannot install extensions from the WAIIDE marketplace.

Fixed:

• Removed ineffective --extensions-gallery command line parameters
• Added marketplace setup script execution during startup
• Configured marketplace through product.json file (the correct method for WAIIDE Server)
• Extensions marketplace now properly accessible in the WAIIDE UI

Fixes Applied

WAIIDE Changes

1. scripts/jupyter_server_config.py:

   • Removed deprecated c.ServerProxy.port_retries configuration
   • Removed problematic extra_handlers that interfered with Jupyter API routes
   • Simplified routing to rely on jupyter-server-proxy

2. scripts/entrypoint-jupyterhub.sh:

   • Enhanced extension copying with integrity verification
   • Added package.json recovery mechanism
   • Improved validation during extension setup

3. Created scripts/fix-jupyterhub-spawning.sh:

   • Comprehensive fix script for existing deployments
   • Extension integrity checker and fixer
   • Marketplace configuration setup
   • Health check script creation
   • Permission fixing utilities

New Health Check

Created /opt/calliope/healthcheck.sh to monitor:

• WAIIDE server process status
• JupyterHub server status (when applicable)
• Extension directory integrity

Testing the Fixes

Option 1: Deploy New Image

1. Rebuild the Docker image with the updated WAIIDE
2. Deploy to your ECS/container environment
3. Spawn a new WAIIDE instance

Option 2: Apply Fixes to Running Container

1. SSH/exec into a running container
2. Run the fix script:

   /opt/calliope/scripts/fix-jupyterhub-spawning.sh
3. Restart the container services

Verification

After applying fixes, you should see:

• ✅ No more configuration warnings in startup logs
• ✅ No more extension package.json errors
• ✅ No more repeated 404 /api errors
• ✅ Marketplace extensions accessible in WAIIDE
• ✅ Clean, faster startup process
• ✅ All extensions loading properly

Example Clean Startup Log

After fixes, startup should look like:

🏠 WAIIDE server will use: /opt/calliope
🔐 Detected JupyterHub user: lmata
📁 User workspace: /home/lmata/workspace
✅ Running as non-root user (UID: 1000)
📍 Using explicit PORT: 8070
🚀 Running under JupyterHub
🔗 JupyterHub mode: Using jupyter-server-proxy for WAIIDE integration
🚀 Starting WAIIDE Server on port 8071 (internal only)
🛒 Configuring marketplace access...
Setting up marketplace configuration...
Product configuration updated successfully!
Marketplace setup complete!
✅ All extensions loaded successfully
📦 Starting jupyterhub-singleuser with jupyter-server-proxy on port 8070
✅ All WAIIDE components are ready!

JupyterHub Side Recommendations

While the WAIIDE side fixes address the main issues, consider these JupyterHub configuration improvements:

1. Health Check Configuration: Use the new health check script
2. Resource Limits: Ensure adequate memory for extension loading
3. Startup Timeout: Allow sufficient time for extension setup (30-60 seconds)
4. Logging Level: Set to INFO or WARNING to reduce log noise

Summary

These fixes address the core issues causing “weird” spawning behavior:

• Cleaner logs with no more configuration warnings
• Faster startup with proper extension loading
• Working marketplace enabling extension installation
• Better reliability with integrity checks
• Easier debugging with health check utilities

The spawning process should now be much cleaner and more reliable, with full marketplace functionality.