Khada

Daily Log - December 4, 2025 ​

Summary ​

Comprehensive documentation review and update, with special focus on the extractor component. Fixed incorrect references, added detailed extractor documentation, and expanded best practices and common pitfalls with extractor-specific guidance.

Work Completed ​

1. Fixed Architecture Documentation ​

Problem: architecture.md had incorrect and incomplete information about the extractor:

  • Referenced packages/extractor/ but didn't explain the full structure
  • Missing details about WAD downloading, hash tables, and path resolution
  • Data flow diagram was too simplified

Solution:

  1. Expanded extractor section with complete component breakdown:
    • LeagueConvert.CommandLine/ - CLI interface and WAD resolution
    • WadDownloader.cs - CDTB-based WAD downloading
    • LeaguePathResolver.cs - Auto-detection and path resolution
    • HashTables.cs - Hash table management
  2. Added detailed data flow with extractor workflow steps
  3. Explained WAD file format, hash tables, and downloading process
  4. Documented League installation auto-detection

Files Changed:

  • packages/docs/architecture.md

2. Created Comprehensive Extractor Documentation ​

Problem: No dedicated documentation for the extractor component, making it difficult for developers to understand:

  • How WAD downloading works
  • What hash tables are and why they're needed
  • How to use the extractor CLI
  • Troubleshooting common issues

Solution: Created new extractor.md with:

  1. Overview: What the extractor does and its role in the pipeline
  2. Installation: Prerequisites, building from source, pre-built binaries
  3. Usage: Complete command reference with examples:
    • convert-wad command with all options
    • convert-all command for batch processing
    • Filtering by skin IDs
    • PBE vs live server selection
  4. Architecture: Detailed explanations of:
    • WAD file format (XXHash64, compression types)
    • Hash tables (Game and BinHashes)
    • WAD downloading via CDTB
    • League installation detection
    • Skin extraction process
  5. Key Components: Documentation of:
    • StringWad.cs - WAD parsing with hash resolution
    • WadDownloader.cs - CDTB integration
    • LeaguePathResolver.cs - Path resolution
    • HashTables.cs - Hash table management
  6. Integration: How extractor fits into Khada pipeline
  7. Troubleshooting: Solutions for common issues:
    • CDTB not installed
    • Hash tables not loading
    • WAD not found
    • Python path issues
    • Cache problems

Files Changed:

  • packages/docs/extractor.md (new file)

3. Expanded Best Practices ​

Problem: best-practices.md only covered material/shader practices, missing extractor-specific guidance.

Solution: Added extractor-specific best practices:

  1. Hash Table Management:
    • Always handle loading failures gracefully
    • Don't assume hash tables are always available
  2. WAD Downloading:
    • Clear PBE cache to ensure fresh downloads
    • Don't use stale PBE cache
  3. Path Resolution:
    • Support both file paths and champion names
    • Don't assume input is always a file path
  4. Error Handling:
    • Provide clear error messages with context
    • Don't use generic error messages

Files Changed:

  • packages/docs/best-practices.md

4. Expanded Common Pitfalls ​

Problem: common-pitfalls.md only covered material/shader pitfalls, missing extractor-specific mistakes.

Solution: Added 4 new extractor-specific pitfalls:

  1. Stale PBE Cache (Pitfall #7):
    • Problem: Using cached WAD files for PBE without clearing
    • Solution: Always clear PBE cache before downloading
    • Files affected: WadDownloader.cs
  2. Missing Hash Tables (Pitfall #8):
    • Problem: Not handling hash table loading failures
    • Solution: Check if hash tables are loaded before use
    • Files affected: HashTables.cs, StringWad.cs
  3. Incorrect Path Resolution (Pitfall #9):
    • Problem: Assuming input is always a file path
    • Solution: Resolve champion names to WAD paths
    • Files affected: Program.cs, LeaguePathResolver.cs
  4. Python Path Detection Issues (Pitfall #10):
    • Problem: Hardcoding Python path or not handling multiple versions
    • Solution: Try specific Python versions first, then fallback
    • Files affected: WadDownloader.cs

Files Changed:

  • packages/docs/common-pitfalls.md

5. Updated Documentation Index ​

Problem: README.md didn't reference the new extractor documentation.

Solution: Updated README to:

  1. Add extractor.md to the structure list
  2. Add "Quick Reference" section for navigation:
    • For Material/Shader Work
    • For Extractor Work
    • For System Understanding

Files Changed:

  • packages/docs/README.md

Key Learnings ​

1. Extractor Architecture Understanding ​

Lesson: The extractor is a complex component with multiple responsibilities:

  • WAD file parsing (using OfficialLeagueToolkit)
  • Hash table management (Game and BinHashes)
  • WAD downloading via CDTB (Python integration)
  • League installation auto-detection
  • Path resolution (files vs champion names)

Rule: Always consider all these aspects when working with the extractor.

2. Hash Tables Are Critical ​

Lesson: Hash tables are required for:

  • Resolving hashed file paths in WAD files (XXHash64)
  • Resolving property names in binary files (SDBM)
  • Without them, the extractor cannot function

Rule: Always ensure hash tables are loaded before WAD parsing. Provide fallback mechanisms.

3. PBE Cache Management ​

Lesson: PBE (Public Beta Environment) changes frequently. Cached WADs may be outdated.

Rule: Always clear PBE cache before downloading. For live servers, cache is acceptable.

4. Input Flexibility ​

Lesson: Users may provide either:

  • File paths: /path/to/Zoe.wad.client
  • Champion names: Zoe (expects auto-resolution)

Rule: Always check input type and resolve champion names to WAD paths before use.

5. Python Path Detection ​

Lesson: Different systems have Python in different locations. CDTB may be installed for specific Python versions.

Rule: Try specific Python versions first (3.10, 3.11, 3.12), then fallback to generic python3.

Code Patterns Documented ​

Hash Table Loading with Fallbacks ​

csharp
if (!await HashTables.TryLoadLatest(logger))
{
    logger.Warning("Failed to load latest hash tables, trying cached...");
    if (!await HashTables.TryLoadExisting(path, logger))
    {
        logger.Fatal("No hash tables available");
        return false;
    }
}

PBE Cache Clearing ​

csharp
if (patchline.Equals("pbe", StringComparison.OrdinalIgnoreCase))
{
    if (File.Exists(cachedPath))
    {
        logger.Information("Clearing cached PBE WAD: {Path}", cachedPath);
        File.Delete(cachedPath);
    }
}

Path Resolution ​

csharp
if (LeaguePathResolver.IsFilePath(input))
{
    resolvedPath = input;
}
else
{
    resolvedPath = await LeaguePathResolver.ResolveInputAsync(
        input, dataFinalFolder, patchline, allowDownload, logger);
    if (resolvedPath == null)
        return; // Error already logged
}

Python Path Detection ​

csharp
var paths = new[]
{
    "/opt/homebrew/bin/python3.10",  // Specific version
    "/opt/homebrew/bin/python3.11",
    "/opt/homebrew/bin/python3.12",
    "/opt/homebrew/bin/python3",     // Generic fallback
    "python3"
};

Documentation Structure ​

Files Created ​

  • packages/docs/extractor.md - Comprehensive extractor documentation

Files Updated ​

  • packages/docs/architecture.md - Fixed and expanded extractor section
  • packages/docs/best-practices.md - Added extractor-specific practices
  • packages/docs/common-pitfalls.md - Added 4 extractor-specific pitfalls
  • packages/docs/README.md - Updated index with extractor reference

Testing Performed ​

Documentation Review ​

  • ✅ All extractor components documented
  • ✅ All command options explained
  • ✅ Troubleshooting section complete
  • ✅ Code examples provided
  • ✅ Cross-references to other docs

Verification ​

  • ✅ YAML syntax validated for all markdown files
  • ✅ No linter errors
  • ✅ Links and references checked

Notes for Future AIs ​

  1. Always read extractor.md before working on extractor-related code
  2. Check common-pitfalls.md for extractor-specific mistakes (pitfalls #7-10)
  3. Review best-practices.md for extractor-specific patterns
  4. Understand hash tables - they're critical for WAD parsing
  5. Handle PBE cache - always clear before downloading
  6. Support flexible input - both file paths and champion names
  7. Test Python path detection - try specific versions first

Built for engineers and AI assistants working on Khada.

© 2025–2026 Khada