Refactor documentation structure, add SUMMARY.md sidebar, and automated consistency checks

README.md

@@ -61,7 +61,7 @@ soundtouch-cli --host 192.168.1.100 volume set --level 50
 soundtouch-cli --host 192.168.1.100 preset list
 ```
 
-For full CLI documentation, see [docs/CLI-REFERENCE.md](docs/CLI-REFERENCE.md).
+For full CLI documentation, see [docs/guides/CLI-REFERENCE.md](docs/guides/CLI-REFERENCE.md).
 
 ### SoundTouch Service (Cloud Shutdown Protection)
 
@@ -82,11 +82,11 @@ soundtouch-service
 ```
 Open `http://localhost:8000` in your browser to manage your devices. Documentation is also available directly through the web interface.
 
-For a comprehensive guide on transitioning your system, see the [Bose Cloud Shutdown: Survival Guide](docs/CLOUD-SHUTDOWN-GUIDE.md).
+For a comprehensive guide on transitioning your system, see the [Bose Cloud Shutdown: Survival Guide](docs/guides/SURVIVAL-GUIDE.md).
 
-Detailed service configuration and Docker instructions can be found in [docs/SOUNDTOUCH-SERVICE.md](docs/SOUNDTOUCH-SERVICE.md).
+Detailed service configuration and Docker instructions can be found in [docs/guides/SOUNDTOUCH-SERVICE.md](docs/guides/SOUNDTOUCH-SERVICE.md).
 
-For professional migration tips and safety measures, see the [Migration & Safety Guide](docs/MIGRATION-SAFETY-GUIDE.md).
+For professional migration tips and safety measures, see the [Migration & Safety Guide](docs/guides/MIGRATION-SAFETY.md).
 
 ### Library Usage
 

docs/SUMMARY.md

@@ -0,0 +1,54 @@
+# Table of Contents
+
+## User Guides
+* [Cloud Shutdown Survival Guide](guides/SURVIVAL-GUIDE.md)
+* [Migration & Safety Guide](guides/MIGRATION-SAFETY.md)
+* [CLI Reference](guides/CLI-REFERENCE.md)
+* [Getting Started](guides/GETTING-STARTED.md)
+* [SoundTouch Service](guides/SOUNDTOUCH-SERVICE.md)
+* [Initial Device Setup](guides/DEVICE-INITIAL-SETUP.md)
+* [HTTPS Setup](guides/HTTPS-SETUP.md)
+* [Deployment](guides/DEPLOYMENT.md)
+* [Troubleshooting](guides/TROUBLESHOOTING.md)
+
+## Technical Reference
+* [API Cookbook](reference/API-COOKBOOK.md)
+* [API Endpoints](reference/API-ENDPOINTS.md)
+* [System Endpoints](reference/SYSTEM-ENDPOINTS.md)
+* [Speaker Endpoint](reference/SPEAKER-ENDPOINT.md)
+* [WebSocket Events](reference/WEBSOCKET-EVENTS.md)
+* [Discovery](reference/DISCOVERY.md)
+* [Zone Management](reference/ZONE-MANAGEMENT.md)
+* [Preset Management](reference/PRESET-MANAGEMENT.md)
+* [Source Selection](reference/SOURCE-SELECTION.md)
+* [Volume Controls](reference/VOLUME-CONTROLS.md)
+* [Bass Controls](reference/BASS-CONTROLS.md)
+* [Key Controls](reference/KEY-CONTROLS.md)
+* [Feature Mapping](reference/FEATURE-MAPPING.md)
+
+## Analysis & Research
+* [API Coverage Analysis](analysis/API-COVERAGE.md)
+* [Supported URLs](analysis/SUPPORTED-URLS.md)
+* [Upstream URLs](analysis/UPSTREAM-URLS.md)
+* [Anonymization Summary](analysis/ANONYMIZATION-SUMMARY.md)
+* [Device Redirect Methods](analysis/DEVICE-REDIRECT-METHODS.md)
+* [Wiki API Comparison](analysis/WIKI-COMPARISON.md)
+
+## Appendix (Other Documents)
+* [API Navigation Reference](API-NAVIGATION-REFERENCE.md)
+* [Claude Instructions](CLAUDE.md)
+* [Content Selection Implementation](CONTENT-SELECTION-IMPLEMENTATION.md)
+* [Device Customization Setup](DEVICE-CUSTOMIZATION-SETUP.md)
+* [Device Logging](DEVICE-LOGGING.md)
+* [Feature History](FEATURE_HISTORY.md)
+* [Host/Port Parsing](HOST-PORT-PARSING.md)
+* [Manual Network Discovery](MANUAL-NETWORK-DISCOVERY.md)
+* [Navigation Guide](NAVIGATION-GUIDE.md)
+* [Official API Verification](OFFICIAL-API-VERIFICATION.md)
+* [Preset Quickstart](PRESET-QUICKSTART.md)
+* [Project Patterns](PROJECT-PATTERNS.md)
+* [Service Availability Implementation](SERVICE-AVAILABILITY-IMPLEMENTATION.md)
+* [SoundTouch Service Announcement](SOUNDTOUCH-SERVICE-ANNOUNCEMENT.md)
+* [Undocumented Community Features](UNDOCUMENTED-COMMUNITY-FEATURES.md)
+* [Unimplemented Endpoints](UNIMPLEMENTED-ENDPOINTS.md)
+* [Preset Store](preset-store.md)

docs/guides/MIGRATION-SAFETY.md

@@ -19,11 +19,14 @@ Before you proceed with the actual migration, follow these steps:
     - Insert the USB stick into the SoundTouch speaker's **SERVICE** port.
     - Reboot the speaker (unplug and replug).
     - The speaker will now allow SSH connections as `root` with no password.
+    - **Verify**: Run `ssh -oHostKeyAlgorithms=+ssh-rsa root@<SPEAKER-IP>` to confirm access. (Note: older devices may require enabling `ssh-rsa` support).
 2.  **Network Isolation (Optional but Recommended)**: Ensure the device is on a stable wired connection if possible, or a dedicated 2.4GHz SSID to avoid drops during SSH operations.
 3.  **Initial Discovery & Sync**: 
     - Run `soundtouch-cli discover devices` to ensure connectivity.
     - Use the Web UI or CLI to "Sync" the device. This will automatically backup your presets and system configuration files to your local server.
-4.  **Validate SSH Access**: Confirm the device responds to SSH without a password (the service handles this, but a manual check with `ssh root@<IP>` is a good sanity check).
+4.  **Validate SSH Access**: Confirm the device responds to SSH without a password. 
+    - In the Web UI **Migration** tab, select your speaker and verify that the "SSH Connection" status shows ✅ Success.
+    - This toolkit automatically handles the necessary SSH parameters (ciphers and key exchanges) required by older Bose firmware.
 5.  **Use XML Migration First**: The `XML` migration method is less invasive than the `Hosts` method. It only changes the application config and doesn't require modifying the system's DNS/CA trust store if you don't need full HTTPS interception initially.
 6.  **Monitor Logs**: Run the `soundtouch-service` with `DEBUG` or `INFO` logging to see the step-by-step progress of the migration.
 

docs/guides/SURVIVAL-GUIDE.md

@@ -40,11 +40,15 @@ Open your web browser and navigate to the service's web interface:
 To migrate your speakers, the service needs SSH access. You can enable it by:
 1. Creating an empty file named `remote_services` on a USB stick.
 2. Inserting the USB stick into the SoundTouch speaker's service port.
-3. Rebooting the speaker.
+3. Rebooting the speaker (unplug/replug).
+
+**Verify SSH Access:**
+- Confirm the device responds to SSH without a password: `ssh -oHostKeyAlgorithms=+ssh-rsa root@<IP>`
+- Or use the **Migration** tab in the Web UI to see if the device shows a "✅ Success" status for SSH.
 Once enabled, you can log in as `root` (no password).
 
 #### 4. Setup Through the Web UI
-The web interface handles the entire process in a guided flow. Before proceeding, we strongly recommend reviewing the [Migration & Safety Guide](MIGRATION-SAFETY-GUIDE.md).
+The web interface handles the entire process in a guided flow. Before proceeding, we strongly recommend reviewing the [Migration & Safety Guide](MIGRATION-SAFETY.md).
 
 *   **Step 1: Settings**: Configure your server's IP or domain. This ensures the speakers know where to find the local services.
 *   **Step 2: Devices**: The service automatically scans for SoundTouch devices on your network. If a device is not found, you can manually add its IP address.

pkg/service/handlers/docs_consistency_test.go

@@ -0,0 +1,71 @@
+package handlers
+
+import (
+	"io/fs"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+func TestDocsConsistency(t *testing.T) {
+	// Root of the project relative to this test file
+	// The test runs in the directory of the package
+	projectRoot := "../../.."
+	docsDir := filepath.Join(projectRoot, "docs")
+	summaryPath := filepath.Join(docsDir, "SUMMARY.md")
+
+	summaryContent, err := os.ReadFile(summaryPath)
+	if err != nil {
+		t.Fatalf("Failed to read SUMMARY.md: %v", err)
+	}
+
+	summaryText := string(summaryContent)
+
+	// List of directories to check
+	dirsToCheck := []string{".", "guides", "reference", "analysis"}
+
+	for _, dir := range dirsToCheck {
+		dirPath := filepath.Join(docsDir, dir)
+		err := filepath.WalkDir(dirPath, func(path string, d fs.DirEntry, err error) error {
+			if err != nil {
+				return err
+			}
+			if d.IsDir() {
+				// Don't recurse into subdirectories if we are checking the root,
+				// as they are handled separately or ignored (like archive)
+				if dir == "." && path != dirPath {
+					return filepath.SkipDir
+				}
+				return nil
+			}
+			if !strings.HasSuffix(d.Name(), ".md") {
+				return nil
+			}
+
+			// Skip SUMMARY.md itself
+			if d.Name() == "SUMMARY.md" {
+				return nil
+			}
+
+			// Get relative path from docs/
+			relPath, err := filepath.Rel(docsDir, path)
+			if err != nil {
+				return err
+			}
+
+			// Check if this file is linked in SUMMARY.md
+			// We look for [Label](relPath)
+			linkPattern := "(" + relPath + ")"
+			if !strings.Contains(summaryText, linkPattern) {
+				t.Errorf("Documentation file %s is not linked in docs/SUMMARY.md", relPath)
+			}
+
+			return nil
+		})
+
+		if err != nil {
+			t.Errorf("Error walking directory %s: %v", dir, err)
+		}
+	}
+}

pkg/service/handlers/handlers_docs.go

@@ -15,7 +15,7 @@ func (s *Server) HandleDocs(w http.ResponseWriter, r *http.Request) {
 	path := strings.TrimPrefix(r.URL.Path, "/docs")
 	path = strings.TrimPrefix(path, "/")
 	if path == "" {
-		path = "CLOUD-SHUTDOWN-GUIDE.md"
+		path = "guides/SURVIVAL-GUIDE.md"
 	}
 
 	// Ensure we only serve files from the docs directory
@@ -31,10 +31,26 @@ func (s *Server) HandleDocs(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 
+	// Load sidebar (SUMMARY.md)
+	summaryContent, _ := os.ReadFile(filepath.Join("docs", "SUMMARY.md"))
+	sidebar := ""
+	if len(summaryContent) > 0 {
+		// Render summary to HTML
+		sidebar = string(blackfriday.Run(summaryContent))
+		// Adjust links in sidebar to be relative to /docs/
+		sidebar = strings.ReplaceAll(sidebar, "href=\"guides/", "href=\"/docs/guides/")
+		sidebar = strings.ReplaceAll(sidebar, "href=\"reference/", "href=\"/docs/reference/")
+		sidebar = strings.ReplaceAll(sidebar, "href=\"analysis/", "href=\"/docs/analysis/")
+		// Fix relative links that don't have a directory prefix (root docs)
+		// We look for href="filename.md" and replace with href="/docs/filename.md"
+		// This avoids manual listing of every file.
+		sidebar = s.fixSidebarLinks(sidebar)
+	}
+
 	// Render markdown to HTML
 	output := blackfriday.Run(content)
 
-	// Wrap in a simple HTML template
+	// Wrap in a documentation template with sidebar
 	w.Header().Set("Content-Type", "text/html")
 	fmt.Fprintf(w, `<!DOCTYPE html>
 <html>
@@ -44,21 +60,65 @@ func (s *Server) HandleDocs(w http.ResponseWriter, r *http.Request) {
     <link rel="icon" href="/media/favicon-braille.svg" type="image/svg+xml">
     <link rel="stylesheet" href="/web/css/style.css">
     <style>
-        body { max-width: 800px; margin: 40px auto; padding: 0 20px; line-height: 1.6; color: #333; }
+        body { margin: 0; padding: 0; display: flex; font-family: sans-serif; height: 100vh; overflow: hidden; }
+        .sidebar { width: 300px; background: #f8f9fa; border-right: 1px solid #dee2e6; padding: 20px; overflow-y: auto; flex-shrink: 0; }
+        .content-area { flex-grow: 1; overflow-y: auto; padding: 40px; }
+        .markdown-body { max-width: 800px; margin: 0 auto; line-height: 1.6; color: #333; }
         h1, h2, h3 { color: #2196F3; }
         pre { background: #f4f4f4; padding: 15px; border-radius: 5px; overflow-x: auto; }
         code { font-family: monospace; background: #eee; padding: 2px 4px; border-radius: 3px; }
         pre code { background: none; padding: 0; }
         a { color: #2196F3; text-decoration: none; }
         a:hover { text-decoration: underline; }
-        .back-link { margin-bottom: 20px; display: block; }
+        .back-link { margin-bottom: 20px; display: block; font-weight: bold; }
+        .sidebar h2 { font-size: 1.1em; margin-top: 20px; color: #666; text-transform: uppercase; letter-spacing: 1px; }
+        .sidebar ul { list-style: none; padding: 0; }
+        .sidebar li { margin-bottom: 8px; }
+        .sidebar a { color: #444; font-size: 0.95em; }
+        .sidebar a:hover { color: #2196F3; }
     </style>
 </head>
 <body>
-    <a href="/" class="back-link">&larr; Back to Toolkit</a>
-    <div class="markdown-body">
+    <div class="sidebar">
+        <a href="/" class="back-link">&larr; Back to Toolkit</a>
         %s
     </div>
+    <div class="content-area">
+        <div class="markdown-body">
+            %s
+        </div>
+   	</div>
 </body>
-</html>`, path, output)
+</html>`, path, sidebar, output)
+}
+
+// fixSidebarLinks ensures that relative links in the SUMMARY.md (sidebar)
+// are correctly prefixed with /docs/ for the web UI.
+func (s *Server) fixSidebarLinks(sidebar string) string {
+	// Root links like [Label](file.md) become href="file.md"
+	// We want href="/docs/file.md", but only if it doesn't already start with /docs/
+	// and isn't an external link.
+	// Since blackfriday renders [Label](file.md) as <a href="file.md">
+
+	// A simple but effective way is to use a regex or just check for common patterns.
+	// We already handled subdirectories. Now we handle files in the root of docs/
+
+	// We'll look for href="filename.md" where filename doesn't contain a slash
+	// and isn't already prefixed.
+
+	// Since we know our doc files always end in .md, we can look for that.
+	lines := strings.Split(sidebar, "\n")
+	for i, line := range lines {
+		if strings.Contains(line, "href=\"") && !strings.Contains(line, "href=\"/docs/") && !strings.Contains(line, "://") {
+			// Extract filename
+			start := strings.Index(line, "href=\"") + 6
+			end := strings.Index(line[start:], "\"") + start
+			filename := line[start:end]
+
+			if strings.HasSuffix(filename, ".md") && !strings.Contains(filename, "/") {
+				lines[i] = strings.ReplaceAll(line, "href=\""+filename+"\"", "href=\"/docs/"+filename+"\"")
+			}
+		}
+	}
+	return strings.Join(lines, "\n")
 }

pkg/service/handlers/web/index.html

@@ -27,10 +27,15 @@
             <div class="info-box prerequisite-box">
                 <strong>🔌 Prerequisite: Enable SSH</strong><br>
                 Migration requires SSH access. To enable it:
-                <ol style="margin-top: 5px; margin-bottom: 0;">
+                <ol style="margin-top: 5px; margin-bottom: 5px;">
                     <li>Create an empty file named <code>remote_services</code> on a USB stick.</li>
                     <li>Insert it into the speaker's <strong>SERVICE</strong> port and reboot the speaker.</li>
                 </ol>
+                <strong>Verify connection:</strong>
+                <ul style="margin-top: 5px; margin-bottom: 0; padding-left: 20px;">
+                    <li>Use the <strong>Migration</strong> tab to select your device and verify that <em>SSH Connection</em> shows ✅ Success.</li>
+                    <li>Or manually: <code>ssh -oHostKeyAlgorithms=+ssh-rsa root@&lt;SPEAKER-IP&gt;</code> (no password).</li>
+                </ul>
             </div>
             <ol class="guide-steps">
                 <li>
@@ -55,14 +60,14 @@
 
             <div class="info-box safety-box">
                 <strong>⚠️ Safety First:</strong> Before starting any migration, please read our
-                <a href="/docs/MIGRATION-SAFETY-GUIDE.md" target="_blank">Professional Migration & Safety Guide</a>.
+                <a href="/docs/guides/MIGRATION-SAFETY.md" target="_blank">Professional Migration & Safety Guide</a>.
                 The toolkit automatically creates backups, but understanding the process is key to a smooth transition.
             </div>
 
             <h3>Useful Links</h3>
             <ul>
-                <li><a href="/docs/CLOUD-SHUTDOWN-GUIDE.md" target="_blank">Cloud Shutdown Survival Guide</a></li>
-                <li><a href="/docs/CLI-REFERENCE.md" target="_blank">CLI Reference</a></li>
+                <li><a href="/docs/guides/SURVIVAL-GUIDE.md" target="_blank">Cloud Shutdown Survival Guide</a></li>
+                <li><a href="/docs/guides/CLI-REFERENCE.md" target="_blank">CLI Reference</a></li>
             </ul>
         </div>