mirror of
https://github.com/nubenetes/awesome-kubernetes.git
synced 2026-07-13 18:30:44 +00:00
feat(ai): implement Special Assets architecture, Zero-to-Hero learning paths, and AI dimension renaming
This commit is contained in:
Nubenetes Bot
12
GEMINI.md
12
GEMINI.md
@@ -69,6 +69,15 @@ This file contains the accumulated instructions and long-term vision for the aut
|
||||
- **Scoring & Ranking**: Prioritize models using the established 2026 hierarchy (Generation 3.x > 2.x > 1.x; Pro > Flash).
|
||||
- **Resilient Fallback**: Automatically transition between models and API keys upon encountering 404 (Unsupported) or 429 (Rate Limit) errors.
|
||||
|
||||
27. **Special Assets Management (V1 & V2)**: High-value files defined in [`data/special_assets.yaml`](data/special_assets.yaml) require specialized handling:
|
||||
- **High-Precision Reorganization (V1)**: These files MUST use nested semantic grouping (## and ###) to organize links without ever deleting technically valid content.
|
||||
- **Exhaustive Inclusion (V2)**: Unlike standard categories, V2 pages for Special Assets MUST include 100% of the ALIVE links from V1.
|
||||
- **AI Curation Discovery**: The discovery engine MUST actively search for new high-quality curation sources (e.g., "Awesome" repos) and suggest them for inclusion in `curation_sources.yaml`.
|
||||
28. **Zero-to-Hero V2 Architecture**: The V2 Portal MUST be structured as a learning journey:
|
||||
- **Complexity Hierarchy**: Resources MUST be grouped by level: Fundamentals -> Intermediate -> Advanced -> Architect.
|
||||
- **Strategic Dimensions**: The "AI and Artificial Intelligence" dimension is the primary entry point for agentic innovation. Dimension naming MUST prioritize industry-standard terms over internal terminology.
|
||||
- **Clickable Navigational Maps**: Every V2 page MUST include a Table of Contents (TOC) with nested anchors for all complexity levels.
|
||||
|
||||
## 🛠️ Structural Evolution & Navigation
|
||||
...
|
||||
* **No Link Limits**: There are NO hard limits on the number of links per page or per section (##/###). Nubenetes is built to host thousands of references.
|
||||
@@ -193,3 +202,6 @@ The bot must rotate between profiles to avoid detection:
|
||||
- **Dynamic Discovery**: Agents MUST utilize the dynamic discovery engine to automatically adopt the newest Gemini models and rotate keys upon reaching quotas.
|
||||
- **Engineering Blog Discovery**: Integrated RSS/Atom ingestion into the curation engine to source high-depth architectural content directly from top-tier technical companies.
|
||||
|
||||
- **AI and Artificial Intelligence Dimension**: Renamed from "Intelligent Control Plane" for better industry alignment.
|
||||
- **Zero-to-Hero Grouping**: Implemented complexity-based levels (Fundamentals to Architect) for high-density learning paths.
|
||||
- **Special Assets Logic**: Integrated data/special_assets.yaml to ensure exhaustive preservation of critical lists (Introduction, YAML, Awesome repos).
|
||||
|
||||
29
README.md
29
README.md
@@ -51,6 +51,9 @@
|
||||
10. [10. Branching Strategy and Lifecycle](#10-branching-strategy-and-lifecycle)
|
||||
11. [11. Contributing to the Archive](#11-contributing-to-the-archive)
|
||||
12. [12. Developer Experience and VSCode Setup](#12-developer-experience-and-vscode-setup)
|
||||
14. [14. Special Assets and Learning Paths](#14-special-assets-and-learning-paths)
|
||||
* [14.1. Special Assets Management](#141-special-assets-management)
|
||||
* [14.2. Zero-to-Hero Learning Architecture](#142-zero-to-hero-learning-architecture)
|
||||
* [12.1. Extension Recommendations](#121-extension-recommendations)
|
||||
* [12.2. Recommended settings.json](#122-recommended-settingsjson)
|
||||
13. [13. Repository Inventory and Configuration](#13-repository-inventory-and-configuration)
|
||||
@@ -435,6 +438,8 @@ graph TD
|
||||
### 7.6. Strategic Benefits
|
||||
- **Technical Immutability (V1)**: AI agents are strictly forbidden from overwriting human-curated titles, manual 🌟 stars, or additional descriptive comments in the V1 archive, ensuring the bot respects and preserves manual engineering effort.
|
||||
- **Self-Healing Infrastructure**: The engine automatically detects and rescues broken links (e.g., GitHub `master` -> `main` branch migration) and identifies parked/expired domains that bypass standard health checks.
|
||||
- **Zero-to-Hero Learning Paths**: V2 resources are systematically grouped by complexity level (Fundamentals, Intermediate, Advanced, Architect), transforming the portal into a structured educational journey for Cloud Native engineering.
|
||||
- **Special Assets Preservation**: High-value documents (Introduction, YAML, Awesome repos) undergo high-precision semantic grouping in V1 and exhaustive inclusion in V2 to ensure 100% technical preservation.
|
||||
- **Linguistic Diversity and Global Access**: AI agents automatically detect the source language. **V1 Archive** preserves descriptions in the resource's native language (e.g., Spanish) to respect original context, while the **V2 Portal** provides professional English summaries and explicit language tagging for global accessibility.
|
||||
- **Rich Metadata Enrichment**: For YouTube videos and technical blogs, the system automatically extracts **Authors**, **Duration**, and **Reading Times**, providing high-density context in the V2 Elite portal.
|
||||
- **Safety Guard Build Validation**: Before any Pull Request is created, a dedicated safety engine validates Markdown syntax, Mermaid diagrams, and runs a test MkDocs build to ensure 100% site stability.
|
||||
@@ -465,7 +470,7 @@ The heart of the new Nubenetes is a suite of AI Agents that operate on our `deve
|
||||
* **Data & Infra:** `@Databricks`, `@ApacheSpark`, `@snowflakedb`, `@HashiCorp`, `@PulumiCorp`, `@ArgoProj`, `@fluxcd`.
|
||||
2. **V2VisionEngine (`src/v2_optimizer.py`)**:
|
||||
- **Elite Selection:** Scans the massive V1 archive to select the "Elite" top-tier resources.
|
||||
- **2026 Taxonomy:** Reorganizes the content into high-density dimensions (e.g., "Intelligent Control Plane") using **relevance-first sorting**.
|
||||
- **2026 Taxonomy:** Reorganizes the content into high-density dimensions (e.g., "AI and Artificial Intelligence") using **relevance-first sorting**.
|
||||
- **MVQ Hardening:** Automatically identifies stale repositories (>4 years without activity) to exclude them from the Elite portal.
|
||||
3. **IntelligentHealthChecker (`src/intelligent_health_checker.py`)**:
|
||||
- **Resilience:** Performs asynchronous health checks with 3x retry and identity rotation.
|
||||
@@ -660,3 +665,25 @@ To maintain transparency and ease of navigation, all key configuration, database
|
||||
<center>
|
||||
Give us a 🌟 on GitHub if you like this archive!
|
||||
</center>
|
||||
|
||||
---
|
||||
|
||||
## 💎 Special Assets & Learning Paths
|
||||
Nubenetes prioritizes high-value technical documents through a specialized preservation and educational architecture.
|
||||
|
||||
### 📚 Special Assets Management
|
||||
Certain files are designated as **Special Assets** (defined in [`data/special_assets.yaml`](data/special_assets.yaml)) due to their foundational importance. These include:
|
||||
- **Introduction & Fundamentals**: Specialized grouping to ensure a perfect "Day 0" experience.
|
||||
- **YAML & JSON Ecosystem**: Exhaustive technical references for configuration languages.
|
||||
- **Awesome Repositories**: Preserved curation lists that act as gateways to specialized sub-ecosystems.
|
||||
|
||||
**Rules of Engagement:**
|
||||
1. **Exhaustive V2 Inclusion**: 100% of ALIVE links from these V1 files are migrated to the V2 Elite portal, bypassing standard impact filters.
|
||||
2. **High-Precision Grouping**: AI agents use nested hierarchies (Sections & Subsections) to organize these files without losing any technically valid reference.
|
||||
|
||||
### 🎓 Zero-to-Hero Learning Architecture
|
||||
The V2 Portal is structured as an educational journey rather than a flat list. Resources are programmatically classified into four expertise tiers:
|
||||
- **Fundamentals**: Core concepts and "Getting Started" material.
|
||||
- **Intermediate**: Practical implementations and standard tooling.
|
||||
- **Advanced**: Performance optimization and complex technical internals.
|
||||
- **Architect**: System design, trade-offs, and long-term strategic direction.
|
||||
|
||||
28
data/special_assets.yaml
Normal file
28
data/special_assets.yaml
Normal file
@@ -0,0 +1,28 @@
|
||||
# Nubenetes Special Assets Configuration
|
||||
# Defines files that require prioritized preservation in V1 and exhaustive inclusion in V2.
|
||||
|
||||
special_assets:
|
||||
- file: "other-awesome-lists.md"
|
||||
v1_rule: "Group by sub-topic (e.g., AI, K8s, Programming). Do not truncate."
|
||||
v2_rule: "Exhaustive: Include 100% of ALIVE links from V1. Maintain sub-topic grouping."
|
||||
keywords: ["awesome", "list", "curation"]
|
||||
|
||||
- file: "yaml.md"
|
||||
v1_rule: "Ensure distinct sections for YAML and JSON technical resources."
|
||||
v2_rule: "Exhaustive: Include all valid YAML/JSON tools and specs."
|
||||
keywords: ["yaml", "json", "schema"]
|
||||
|
||||
- file: "introduction.md"
|
||||
v1_rule: "Semantic clustering: Group by 'Fundamentals', 'Getting Started', and 'Advanced Concepts'."
|
||||
v2_rule: "Elite curated view: Highlight only high-impact fundamental resources."
|
||||
keywords: ["introduction", "getting started", "fundamentals"]
|
||||
|
||||
# Navigation Renaming
|
||||
dimension_renaming:
|
||||
"Intelligent Control Plane": "AI and Artificial Intelligence"
|
||||
|
||||
# Advanced Classification Rules for V2
|
||||
v2_learning_paths:
|
||||
enabled: true
|
||||
levels: ["Fundamentals", "Intermediate", "Advanced", "Expert/Architect"]
|
||||
structure: "Zero to Hero"
|
||||
@@ -310,13 +310,37 @@ class AgenticCurator:
|
||||
|
||||
async def suggest_reorganization(self):
|
||||
log_event("[*] Starting Internal Reorganization Audit...", section_break=True)
|
||||
# Load Special Assets config
|
||||
special_rules = {}
|
||||
if os.path.exists("data/special_assets.yaml"):
|
||||
try:
|
||||
with open("data/special_assets.yaml", "r") as f:
|
||||
special_rules = yaml.safe_load(f).get("special_assets", [])
|
||||
except: pass
|
||||
special_files = {sa["file"]: sa for sa in special_rules}
|
||||
|
||||
for file in os.listdir(self.docs_dir):
|
||||
if not file.endswith(".md") or file == "index.md": continue
|
||||
path = os.path.join(self.docs_dir, file)
|
||||
with open(path, "r") as f: content = f.read()
|
||||
if len(re.findall(r"^\s*-\s*\[", content, re.MULTILINE)) > 25:
|
||||
log_event(f" [!] REORGANIZING: {file}")
|
||||
prompt = f"Reorganize '{file}' into logical sections (##). English headers only. Content:\n{content[:4000]}"
|
||||
|
||||
is_special = file in special_files
|
||||
link_count = len(re.findall(r"^\s*-\s*\[", content, re.MULTILINE))
|
||||
|
||||
# Reorganize if special OR if flat and large
|
||||
if is_special or (link_count > 25 and len(re.findall(r"^## ", content, re.M)) < 2):
|
||||
log_event(f" [!] REORGANIZING: {file} ({'Special' if is_special else 'Standard'})")
|
||||
|
||||
depth_instruction = (
|
||||
"SOPHISTICATED HIERARCHY: Create nested sections (##) and subsections (###). "
|
||||
"Group links by technical theme. Maintain all existing links. No deletions."
|
||||
if is_special else "Group into logical sections (##)."
|
||||
)
|
||||
|
||||
prompt = (
|
||||
f"You act as a Technical Content Architect. Reorganize the file '{file}' based on this rule: {depth_instruction}\n"
|
||||
f"IMPORTANT: DO NOT DELETE any valid link. English headers only. Content:\n{content[:5000]}"
|
||||
)
|
||||
try:
|
||||
reorganized = await call_gemini_with_retry(prompt, response_format="text", prefer_flash=True)
|
||||
if len(reorganized) > len(content) * 0.7:
|
||||
|
||||
@@ -10,7 +10,9 @@ async def fetch_github_trending_cloud_native() -> list[dict]:
|
||||
"topic:kubernetes+stars:>1000",
|
||||
"topic:mcp-server+stars:>0",
|
||||
"topic:model-context-protocol+stars:>0",
|
||||
"topic:ai-agents+stars:>50"
|
||||
"topic:ai-agents+stars:>50",
|
||||
"awesome+stars:>1000",
|
||||
"topic:generative-ai+stars:>500"
|
||||
]
|
||||
all_repos = []
|
||||
headers = {'Accept': 'application/vnd.github.v3+json'}
|
||||
@@ -59,6 +61,8 @@ async def discover_trending_assets() -> list[dict]:
|
||||
|
||||
if "mcp" in desc_lower or "context-protocol" in desc_lower or "mcp" in name_lower:
|
||||
category = "ai-agents-mcp"
|
||||
elif "awesome" in name_lower:
|
||||
category = "other-awesome-lists"
|
||||
elif "ai" in desc_lower or "agent" in desc_lower:
|
||||
category = "ai"
|
||||
elif "security" in desc_lower:
|
||||
|
||||
@@ -17,9 +17,12 @@ STRUCTURE_MAP_PATH = "data/structure_map.yaml"
|
||||
|
||||
class V2VisionEngine:
|
||||
def __init__(self):
|
||||
# Load Special Assets & Rules
|
||||
self.special_assets_rules = self._load_special_assets()
|
||||
|
||||
# 100% Comprehensive 2026 Taxonomy
|
||||
self.dimensions = {
|
||||
"Intelligent Control Plane": ["ai", "ai-agents-mcp", "chatgpt", "mlops"],
|
||||
"AI and Artificial Intelligence": ["ai", "ai-agents-mcp", "chatgpt", "mlops"],
|
||||
"Architectural Foundations": ["introduction", "faq", "kubernetes", "linux", "git", "cloud-arch-diagrams", "matrix-table", "other-awesome-lists", "about"],
|
||||
"Platform & Site Reliability": ["sre", "devops", "developerportals", "scaffolding", "finops", "chaos-engineering", "performance-testing-with-jenkins-and-jmeter", "project-management-methodology", "project-management-tools", "qa", "test-automation-frameworks", "testops"],
|
||||
"Hardened Infrastructure": ["iac", "terraform", "pulumi", "crossplane", "ansible", "securityascode", "kubernetes-security", "aws-security", "oauth", "devsecops", "kustomize", "liquibase", "chef"],
|
||||
@@ -33,31 +36,33 @@ class V2VisionEngine:
|
||||
}
|
||||
|
||||
self.library_criteria = (
|
||||
"You are a Technical Librarian in 2026. Your mission is to build a high-density, professional reference library.\n"
|
||||
"You are a Senior Technical Librarian and Architect in 2026. Your mission is to build a high-density, professional reference library.\n"
|
||||
"PHASE 1: TECHNICAL PRESERVATION (HIGH INCLUSIVITY)\n"
|
||||
"- KEEP >90% of technical resources.\n"
|
||||
"PHASE 2: SOPHISTICATED SYNTHESIS & DATING\n"
|
||||
"- Extract precise PUBLICATION DATE (YYYY-MM-DD or YYYY): Look for dates in the URL, Twitter/X post dates, or text context. Return 'N/A' if truly unknown.\n"
|
||||
"- Detect source content LANGUAGE (e.g., 'English', 'Spanish', 'French').\n"
|
||||
"- Identify RESOURCE_TYPE: (Blog, Repository, Video, Tool, Documentation, Guide, Case Study).\n"
|
||||
"- Assign COMPLEXITY: (Beginner, Intermediate, Advanced, Architect).\n"
|
||||
"- Assign QUALITY level (0-5 stars):\n"
|
||||
" * 0 stars: Good technical resource (Baseline).\n"
|
||||
" * 1 star (🌟): High-quality technical guide or tool.\n"
|
||||
" * 2 stars (🌟🌟): Exceptional, enterprise-grade resource.\n"
|
||||
" * 3 stars (🌟🌟🌟): Elite Gem. Recommended for all architects.\n"
|
||||
" * 4 stars (🌟🌟🌟🌟): Masterclass content or Essential Industry Tool.\n"
|
||||
" * 5 stars (🌟🌟🌟🌟🌟): Legendary Resource (e.g., K8s Official Docs, Foundations like Prometheus/Envoy).\n"
|
||||
"- Assign a MATURITY TAG based on content type/status.\n"
|
||||
"PHASE 3: MANDATORY DESCRIPTIONS (V1 PRIORITY)\n"
|
||||
"- If 'Current Desc' is already provided and descriptive, DO NOT CHANGE IT.\n"
|
||||
"- If 'Current Desc' is empty, too short, or non-descriptive, generate a professional 1-2 sentence summary.\n"
|
||||
"- Extract precise PUBLICATION DATE (YYYY-MM-DD or YYYY).\n"
|
||||
"- Detect source content LANGUAGE.\n"
|
||||
"- Identify RESOURCE_TYPE and complexity LEVEL.\n"
|
||||
"PHASE 3: ZERO-TO-HERO CLASSIFICATION\n"
|
||||
"- Categorize into: 'Fundamentals', 'Intermediate', 'Advanced', or 'Architect' level.\n"
|
||||
"- For special curation lists (e.g. Awesome repos), identify the primary curation topic.\n"
|
||||
"PHASE 4: MANDATORY DESCRIPTIONS (V1 PRIORITY)\n"
|
||||
"- If 'Current Desc' is empty or too short, generate a professional 1-2 sentence summary.\n"
|
||||
"- Style: Technical, neutral, and informative. Language: English only.\n"
|
||||
)
|
||||
self.inventory = self._load_inventory()
|
||||
self.structure_map = self._load_structure_map()
|
||||
self.maturity_audit = []
|
||||
|
||||
def _load_special_assets(self) -> Dict:
|
||||
path = "data/special_assets.yaml"
|
||||
if os.path.exists(path):
|
||||
try:
|
||||
with open(path, "r") as f:
|
||||
return yaml.safe_load(f) or {}
|
||||
except: return {}
|
||||
return {}
|
||||
|
||||
def _load_inventory(self) -> Dict:
|
||||
if os.path.exists(INVENTORY_PATH):
|
||||
try:
|
||||
@@ -257,28 +262,29 @@ class V2VisionEngine:
|
||||
to_evaluate = []
|
||||
force_eval = os.getenv("FORCE_EVAL", "false").lower() == "true"
|
||||
|
||||
# We want to re-evaluate the tags and years, so we will bypass cache for tagging logic,
|
||||
# but use cache for AI stars if available to save cost.
|
||||
# Load Special Assets for 100% Inclusion
|
||||
special_files = [sa["file"] for sa in self.special_assets_rules.get("special_assets", [])]
|
||||
|
||||
for l in links:
|
||||
url = l["url"]
|
||||
# To allow the new logic to apply to cached items, we re-process GitHub links
|
||||
# and re-apply the tag logic even if it's in the cache.
|
||||
item = l.copy()
|
||||
if not force_eval and url in self.inventory and "stars" in self.inventory[normalize_url(url)]:
|
||||
item.update(self.inventory[normalize_url(url)])
|
||||
# If cache has a generated description and item is missing one, use it
|
||||
if "ai_summary" in self.inventory[normalize_url(url)] and not item["description"]:
|
||||
item["description"] = self.inventory[normalize_url(url)]["ai_summary"]
|
||||
norm_url = normalize_url(url)
|
||||
|
||||
# --- DATABASE-FIRST: Try to reuse cached evaluations ---
|
||||
if not force_eval and norm_url in self.inventory and "stars" in self.inventory[norm_url]:
|
||||
item.update(self.inventory[norm_url])
|
||||
if "ai_summary" in self.inventory[norm_url] and not item["description"]:
|
||||
item["description"] = self.inventory[norm_url]["ai_summary"]
|
||||
|
||||
# --- TRACK MATURITY CHANGES ---
|
||||
old_tag = self.inventory.get(normalize_url(url), {}).get("tag")
|
||||
old_tag = self.inventory.get(norm_url, {}).get("tag")
|
||||
|
||||
# Re-evaluate if description is still missing even after cache check
|
||||
if not item.get("description"):
|
||||
# Special Assets: If description is missing, we MUST evaluate but we NEVER drop
|
||||
if not item.get("description") or norm_url not in self.inventory:
|
||||
to_evaluate.append(item)
|
||||
continue
|
||||
|
||||
# Re-apply GitHub metadata and mature tagging for cached items
|
||||
# Update GitHub metadata for cached items
|
||||
if "github.com" in url:
|
||||
gh_meta = await self._fetch_github_metadata(url)
|
||||
item.update(gh_meta)
|
||||
@@ -290,7 +296,8 @@ class V2VisionEngine:
|
||||
# Audit Check
|
||||
if old_tag and old_tag != item["tag"]:
|
||||
self.maturity_audit.append({
|
||||
"url": url, "title": item["title"], "type": "Promotion" if "STANDARD" in item["tag"] or "STABLE" in item["tag"] else "Reclassification",
|
||||
"url": url, "title": item["title"],
|
||||
"type": "Promotion" if "STANDARD" in item["tag"] or "STABLE" in item["tag"] else "Reclassification",
|
||||
"old": old_tag, "new": item["tag"]
|
||||
})
|
||||
|
||||
@@ -298,17 +305,17 @@ class V2VisionEngine:
|
||||
|
||||
if not to_evaluate: return refined
|
||||
|
||||
# Batch Evaluation with Zero-to-Hero Leveling
|
||||
BATCH_SIZE = 50
|
||||
for i in range(0, len(to_evaluate), BATCH_SIZE):
|
||||
batch = to_evaluate[i:i+BATCH_SIZE]
|
||||
batch_num = i//BATCH_SIZE + 1
|
||||
log_event(f" [>] Processing Batch {batch_num} with AI (Mandatory Descriptions)...")
|
||||
log_event(f" [>] Processing Batch {batch_num} with AI (Zero-to-Hero Architecture)...")
|
||||
|
||||
prompt = (
|
||||
f"{self.library_criteria}\n"
|
||||
"UNIVERSAL ENGLISH CURATION: ALL output 'summary' fields MUST be in ENGLISH. If source is non-English (e.g. Spanish), TRANSLATE to professional English.\n"
|
||||
"Respond ONLY with a JSON object: {\"results\": [{\"idx\": int, \"year\": \"YYYY\", \"stars\": 0-5, \"is_video\": bool, \"tag\": \"[TAG]\", \"summary\": \"1-2 sentences description\", \"language\": \"...\", \"type\": \"...\", \"level\": \"...\"}, ...]}\n\n"
|
||||
"LINKS:\n" + "\n".join([f"{idx}. {l['title']} ({l['url']}) - Current Desc: {l['description'][:50]}" for idx, l in enumerate(batch)])
|
||||
"Respond ONLY with a JSON object: {\"results\": [{\"idx\": int, \"year\": \"YYYY\", \"stars\": 0-5, \"is_video\": bool, \"tag\": \"[TAG]\", \"summary\": \"1-2 sentences description\", \"language\": \"...\", \"type\": \"...\", \"level\": \"Fundamentals|Intermediate|Advanced|Architect\"}, ...]}\n\n"
|
||||
"LINKS:\n" + "\n".join([f"{idx}. {l['title']} ({l['url']}) - Desc: {l['description'][:60]}" for idx, l in enumerate(batch)])
|
||||
)
|
||||
|
||||
try:
|
||||
@@ -323,6 +330,9 @@ class V2VisionEngine:
|
||||
norm_url = normalize_url(item["url"])
|
||||
old_tag = self.inventory.get(norm_url, {}).get("tag")
|
||||
|
||||
# SPECIAL ASSET BYPASS: If file is special, force 5 stars or preservation
|
||||
is_special = item["original_file"] in special_files
|
||||
|
||||
eval_data = {
|
||||
"year": str(res.get("year", "N/A")),
|
||||
"stars": min(max(int(res.get("stars", 0)), 0), 5),
|
||||
@@ -333,11 +343,11 @@ class V2VisionEngine:
|
||||
"resource_type": res.get("type", "Reference"),
|
||||
"complexity": res.get("level", "Intermediate")
|
||||
}
|
||||
|
||||
item.update(eval_data)
|
||||
if not item["description"] and item["ai_summary"]:
|
||||
item["description"] = item["ai_summary"]
|
||||
|
||||
# GitHub overrides
|
||||
|
||||
if "github.com" in item["url"]:
|
||||
gh_meta = await self._fetch_github_metadata(item["url"])
|
||||
item.update(gh_meta)
|
||||
@@ -346,7 +356,7 @@ class V2VisionEngine:
|
||||
|
||||
item["tag"] = self._calculate_tag(item)
|
||||
|
||||
# Audit Check for AI re-evaluation
|
||||
# Audit Check
|
||||
if old_tag and old_tag != item["tag"]:
|
||||
self.maturity_audit.append({
|
||||
"url": item["url"], "title": item["title"], "type": "AI Reclassification",
|
||||
@@ -354,22 +364,22 @@ class V2VisionEngine:
|
||||
})
|
||||
|
||||
refined.append(item)
|
||||
|
||||
# Update inventory correctly
|
||||
# Update inventory
|
||||
self.inventory[norm_url] = {
|
||||
"title": item["title"], "year": item["year"], "stars": item["stars"],
|
||||
"is_video": item["is_video"], "ai_summary": item["ai_summary"],
|
||||
"language": item["language"], "resource_type": item["resource_type"],
|
||||
"complexity": item["complexity"], "tag": item["tag"], "status": "online"
|
||||
"complexity": item["complexity"], "tag": item["tag"], "status": "online",
|
||||
"original_file": item["original_file"]
|
||||
}
|
||||
if "gh_stars" in item: self.inventory[norm_url]["gh_stars"] = item["gh_stars"]
|
||||
if "gh_updated" in item: self.inventory[norm_url]["gh_updated"] = item["gh_updated"]
|
||||
except: continue
|
||||
except:
|
||||
except Exception as e:
|
||||
log_event(f" [!] AI Error in batch: {e}")
|
||||
for l in batch:
|
||||
item = l.copy()
|
||||
item["year"], item["stars"], item["is_video"] = "N/A", 0, "youtube" in l["url"]
|
||||
item["tag"] = self._calculate_tag(item)
|
||||
item["year"], item["stars"], item["tag"] = "N/A", 0, "[COMMUNITY-TOOL]"
|
||||
refined.append(item)
|
||||
await asyncio.sleep(0.3)
|
||||
return refined
|
||||
@@ -424,36 +434,51 @@ class V2VisionEngine:
|
||||
except: pass
|
||||
return {}
|
||||
|
||||
async def _rebuild_structure(self, inventory: List[Dict]) -> Dict[str, Dict]:
|
||||
async def _rebuild_structure(self, library_inventory: List[Dict]) -> Dict[str, Dict]:
|
||||
special_files = [sa["file"] for sa in self.special_assets_rules.get("special_assets", [])]
|
||||
v2_structure = {dim: {"summary": "", "categories": {}} for dim in self.dimensions.keys()}
|
||||
file_to_dim = {}
|
||||
for dim, files in self.dimensions.items():
|
||||
for f in files: file_to_dim[f + ".md"] = dim
|
||||
|
||||
for item in inventory:
|
||||
dim = file_to_dim.get(item["original_file"], "Architectural Foundations")
|
||||
cat_name = item["original_file"].replace(".md", "").capitalize()
|
||||
if cat_name not in v2_structure[dim]["categories"]:
|
||||
v2_structure[dim]["categories"][cat_name] = []
|
||||
v2_structure[dim]["categories"][cat_name].append(item)
|
||||
|
||||
for dim in v2_structure.keys():
|
||||
if not v2_structure[dim]["categories"]: continue
|
||||
for cat in v2_structure[dim]["categories"]:
|
||||
# Sort by: 1. Stars (DESC), 2. Year (DESC, N/A at the end)
|
||||
v2_structure[dim]["categories"][cat].sort(
|
||||
key=lambda x: (
|
||||
-x.get("stars", 1),
|
||||
-(int(x["year"]) if x.get("year", "").isdigit() else 0)
|
||||
)
|
||||
)
|
||||
for item in library_inventory:
|
||||
orig_file = item.get("original_file", "unknown.md")
|
||||
dim = file_to_dim.get(orig_file, "Architectural Foundations")
|
||||
cat_name = orig_file.replace(".md", "").replace("-", " ").title()
|
||||
is_special = orig_file in special_files
|
||||
|
||||
prompt = f"Write a professional 2026 executive summary for '{dim}'. Focus on high-density value. 1 sentence only."
|
||||
try:
|
||||
v2_structure[dim]["summary"] = await call_gemini_with_retry(prompt, response_format="text", prefer_flash=True)
|
||||
except:
|
||||
v2_structure[dim]["summary"] = f"Impact-driven reference library for {dim}."
|
||||
|
||||
# Filtering: Keep if stars >= 3 OR if it's a Special Asset
|
||||
if not is_special and item.get("stars", 0) < 3:
|
||||
continue
|
||||
|
||||
if cat_name not in v2_structure[dim]["categories"]:
|
||||
v2_structure[dim]["categories"][cat_name] = {
|
||||
"Fundamentals": [], "Intermediate": [], "Advanced": [], "Architect": []
|
||||
}
|
||||
|
||||
level = item.get("complexity", "Intermediate")
|
||||
if level not in v2_structure[dim]["categories"][cat_name]: level = "Intermediate"
|
||||
v2_structure[dim]["categories"][cat_name][level].append(item)
|
||||
|
||||
for dim in v2_structure:
|
||||
for cat in list(v2_structure[dim]["categories"].keys()):
|
||||
has_content = False
|
||||
for level in v2_structure[dim]["categories"][cat]:
|
||||
if v2_structure[dim]["categories"][cat][level]:
|
||||
has_content = True
|
||||
v2_structure[dim]["categories"][cat][level].sort(
|
||||
key=lambda x: (-x.get("stars", 1), -(int(x["year"]) if str(x.get("year", "")).isdigit() else 0))
|
||||
)
|
||||
if not has_content:
|
||||
del v2_structure[dim]["categories"][cat]
|
||||
else:
|
||||
# Maintain Executive Summary for Dimension
|
||||
prompt = f"Write a professional 2026 executive summary for '{dim}'. Focus on high-density value. 1 sentence only."
|
||||
try:
|
||||
v2_structure[dim]["summary"] = await call_gemini_with_retry(prompt, response_format="text", prefer_flash=True)
|
||||
except:
|
||||
v2_structure[dim]["summary"] = f"Impact-driven reference library for {dim}."
|
||||
|
||||
return v2_structure
|
||||
|
||||
async def _write_premium_files(self, data: Dict[str, Dict], mosaic_html: str, videos_html: str):
|
||||
@@ -563,64 +588,80 @@ class V2VisionEngine:
|
||||
slug = dim.lower().replace(" ", "-").replace("&", "and").replace("(", "").replace(")", "").replace(" ", "-")
|
||||
md = f"# {dim}\n\n"
|
||||
md += f"!!! info \"Architectural Context\"\n {content['summary']}\n\n"
|
||||
for cat, links in content["categories"].items():
|
||||
md += f"## {cat}\n"
|
||||
for l in links:
|
||||
year, stars_val = l.get("year", "N/A"), l.get("stars", 0)
|
||||
stars = ("🌟" * stars_val) if stars_val > 0 else ""
|
||||
tag = l.get("tag", "[ENTERPRISE-STABLE]")
|
||||
|
||||
# Determine color mapping for new tags
|
||||
if "STANDARD" in tag or "FOUNDATIONAL" in tag: color = "success"
|
||||
elif "EMERGING" in tag: color = "warning"
|
||||
elif "LEGACY" in tag: color = "critical"
|
||||
elif "STABLE" in tag: color = "info"
|
||||
else: color = "primary"
|
||||
|
||||
title_clean = l['title'].replace("==", "")
|
||||
if stars_val >= 3 or "STANDARD" in tag:
|
||||
title_display = f"**=={title_clean}==**"
|
||||
elif stars_val == 2:
|
||||
title_display = f"**{title_clean}**"
|
||||
else:
|
||||
title_display = title_clean
|
||||
|
||||
year_prefix = f"**({year})** " if year and year != "N/A" else ""
|
||||
|
||||
gh_info = f" <span class='md-tag md-tag--info'>⭐ {l['gh_stars']}</span>" if "gh_stars" in l else ""
|
||||
icon = " 🎥" if l.get("is_video") else ""
|
||||
|
||||
# Language Tagging
|
||||
lang = l.get("language", "English")
|
||||
lang_tag = ""
|
||||
if lang.lower() != "english":
|
||||
lang_tag = f" <span class='md-tag md-tag--warning'>[{lang.upper()} CONTENT]</span>"
|
||||
|
||||
# Complexity Tagging
|
||||
level = l.get("complexity", "Intermediate")
|
||||
level_tag = ""
|
||||
if level.lower() in ["architect", "advanced"]:
|
||||
level_tag = f" <span class='md-tag md-tag--critical'>[{level.upper()} LEVEL]</span>"
|
||||
|
||||
# Resource Type Tagging
|
||||
res_type = l.get("resource_type", "Reference")
|
||||
type_tag = ""
|
||||
if res_type.lower() in ["case study", "guide", "documentation"]:
|
||||
type_tag = f" <span class='md-tag md-tag--primary'>[{res_type.upper()}]</span>"
|
||||
|
||||
# --- Table of Contents for the Page ---
|
||||
md += "## Table of Contents\n"
|
||||
for cat in content["categories"].keys():
|
||||
cat_slug = cat.lower().replace(" ", "-")
|
||||
md += f"- [{cat}](#{cat_slug})\n"
|
||||
for level, level_links in content["categories"][cat].items():
|
||||
if level_links:
|
||||
level_slug = f"{cat_slug}-{level.lower()}"
|
||||
md += f" - [{level}](#{level_slug})\n"
|
||||
md += "\n---\n\n"
|
||||
|
||||
# Rich Metadata Tags (Author, Duration, RT)
|
||||
rich_tags = ""
|
||||
if l.get("author"): rich_tags += f" <small>by **{l['author']}**</small>"
|
||||
if l.get("duration"): rich_tags += f" <span class='md-tag md-tag--info'>⏱️ {l['duration']}</span>"
|
||||
if l.get("reading_time"): rich_tags += f" <span class='md-tag md-tag--info'>📖 {l['reading_time']}</span>"
|
||||
|
||||
md += f" - {year_prefix}[{title_display}]({l['url']}){icon}{gh_info}{lang_tag}{level_tag}{type_tag}{rich_tags} {stars} <span class='md-tag md-tag--{color}'>{tag}</span>\n"
|
||||
if l['description']:
|
||||
desc = l['description']
|
||||
if "\n" in desc:
|
||||
md += "\n" + "\n".join([" " + line for line in desc.split("\n")]) + "\n\n"
|
||||
for cat, levels in content["categories"].items():
|
||||
cat_slug = cat.lower().replace(" ", "-")
|
||||
md += f"## {cat}\n\n"
|
||||
for level, links in levels.items():
|
||||
if not links: continue
|
||||
level_slug = f"{cat_slug}-{level.lower()}"
|
||||
md += f"### {cat} - {level}\n"
|
||||
for l in links:
|
||||
year, stars_val = l.get("year", "N/A"), l.get("stars", 0)
|
||||
stars = ("🌟" * stars_val) if stars_val > 0 else ""
|
||||
tag = l.get("tag", "[ENTERPRISE-STABLE]")
|
||||
|
||||
# Determine color mapping
|
||||
if "STANDARD" in tag or "FOUNDATIONAL" in tag: color = "success"
|
||||
elif "EMERGING" in tag: color = "warning"
|
||||
elif "LEGACY" in tag: color = "critical"
|
||||
elif "STABLE" in tag: color = "info"
|
||||
else: color = "primary"
|
||||
|
||||
title_clean = l['title'].replace("==", "")
|
||||
if stars_val >= 3 or "STANDARD" in tag:
|
||||
title_display = f"**=={title_clean}==**"
|
||||
elif stars_val == 2:
|
||||
title_display = f"**{title_clean}**"
|
||||
else:
|
||||
md += f" {desc}\n"
|
||||
title_display = title_clean
|
||||
|
||||
year_prefix = f"**({year})** " if year and year != "N/A" else ""
|
||||
gh_info = f" <span class='md-tag md-tag--info'>⭐ {l['gh_stars']}</span>" if "gh_stars" in l else ""
|
||||
icon = " 🎥" if l.get("is_video") else ""
|
||||
|
||||
# Language Tagging
|
||||
lang = l.get("language", "English")
|
||||
lang_tag = ""
|
||||
if lang.lower() != "english":
|
||||
lang_tag = f" <span class='md-tag md-tag--warning'>[{lang.upper()} CONTENT]</span>"
|
||||
|
||||
# Complexity Tagging
|
||||
l_val = l.get("complexity", "Intermediate")
|
||||
level_tag = ""
|
||||
if l_val.lower() in ["architect", "advanced"]:
|
||||
level_tag = f" <span class='md-tag md-tag--critical'>[{l_val.upper()} LEVEL]</span>"
|
||||
|
||||
# Resource Type Tagging
|
||||
res_type = l.get("resource_type", "Reference")
|
||||
type_tag = ""
|
||||
if res_type.lower() in ["case study", "guide", "documentation"]:
|
||||
type_tag = f" <span class='md-tag md-tag--primary'>[{res_type.upper()}]</span>"
|
||||
|
||||
# Rich Metadata
|
||||
rich_tags = ""
|
||||
if l.get("author"): rich_tags += f" <small>by **{l['author']}**</small>"
|
||||
if l.get("duration"): rich_tags += f" <span class='md-tag md-tag--info'>⏱️ {l['duration']}</span>"
|
||||
if l.get("reading_time"): rich_tags += f" <span class='md-tag md-tag--info'>📖 {l['reading_time']}</span>"
|
||||
|
||||
md += f" - {year_prefix}[{title_display}]({l['url']}){icon}{gh_info}{lang_tag}{level_tag}{type_tag}{rich_tags} {stars} <span class='md-tag md-tag--{color}'>{tag}</span>\n"
|
||||
if l['description']:
|
||||
desc = l['description']
|
||||
if "\n" in desc:
|
||||
md += "\n" + "\n".join([" " + line for line in desc.split("\n")]) + "\n\n"
|
||||
else:
|
||||
md += f" {desc}\n"
|
||||
md += "\n"
|
||||
with open(os.path.join(V2_DIR, f"{slug}.md"), "w") as f: f.write(md)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user