mirror of
https://github.com/opentibiabr/canary
synced 2026-08-16 06:26:09 -04:00
de9d7880d9
Add a deterministic profile-aware gameplay content auditor This commit adds a deterministic profile-aware static auditor for Canary gameplay content. The auditor models the repository using the same mutually exclusive datapack profiles used at runtime. The canary profile is built from data, plus the data canary. The otservbr global profile is built from data plus data otservbr global. This keeps definitions from alternative datapacks isolated and prevents one profile from incorrectly satisfying or conflicting with another profile. This is a tooling-only change. It does not modify server runtime behavior. The new auditor is available as a Python CLI through python m tools.canary_audit. It extracts typed gameplay definitions, references, registrations, and unresolved expressions from Lua, XML, and appearance data. The covered areas include items, monsters, NPCs storages actions, movement, events, weapons, and spells. The tool also extracts authoritative item IDs from the data appearances.dat file. This extraction follows the runtime requirements for protobuf ID fields, flags, and the uint16 item ID domain, instead of treating every numeric field as an item definition. Lua analysis is intentionally conservative. The scanner ignores comments and string contents, tracks lexical scope and constructor ownership, and leaves dynamic expressions unresolved instead of guessing. This makes unresolved dynamic coverage visible without fabricating false missing references. XML validation was added for item overrides, creature references, storage ranges, overlapping ranges, and out-of-range values. The XML parser disables external entities and fails closed if the parser cannot enforce that safety setting. The auditor generates deterministic schema-validated artifacts for CI and local review. Generated artifacts • project-index.json • symbol-registry.json • reference-report.json • summary.md Semantic fingerprints are stable and narrowly scoped. Finding fingerprints includes occurrence multiplicity, so a waiver for one known legacy occurrence cannot hide a newly duplicated occurrence later. Baseline waivers are supported for existing known issues. The current shared items.xml contains four reversed item ranges. These produce eight profile-specific error findings and are recorded with individual fingerprints and engineering reasons. The waivers keep the initial CI gate usable while ensuring that any new error-level finding still blocks CI. Main changes included in this commit • Adds the tools.canary_audit Python CLI • Adds profile-aware scanning for the canary datapack profile • Adds profile-aware scanning for the otservbr global datapack profile • Keeps gameplay profiles isolated during extraction and rule evaluation • Adds typed definitions for items, monsters, NPCs, storages, actions, movement, events, weapons, and spells • Adds typed references for gameplay content relationships • Adds typed registrations for action IDs, unique IDs, movement, events, weapons, spells, and related content • Tracks unresolved dynamic Lua expressions instead of guessing their values • Extracts authoritative item IDs from data items appearances.dat • Parses protobuf fields for ID and flags according to runtime requirements • Enforces the uint16 item ID domain • Adds conservative Lua analysis that ignores comments and string contents • Tracks lexical scope during Lua extraction • Tracks constructor ownership during Lua extraction • Adds XML validation for item overrides • Adds XML validation for creature references • Adds XML validation for storage ranges • Adds XML validation for overlapping ranges • Adds XML validation for out-of-range values • Adds deterministic JSON and Markdown audit artifacts • Adds schema validation for generated artifacts • Adds stable semantic fingerprints • Adds narrowly scoped baseline waivers • Adds occurrence multiplicity to fingerprints • Adds a read-only GitHub Actions workflow for audit validation • Adds CI schema validation • Adds CI test execution • Adds CI scans for both supported profiles • Publishes the audit summary in CI • Uploads generated audit artifacts in CI • Adds repository documentation for architecture commands extension points, exit codes, and coverage boundaries Correctness and safety improvements • Keeps canary and otservbr global datapack definitions isolated • Prevents alternative datapack content from satisfying the wrong profile • Prevents alternative datapack content from conflicting with the wrong profile • Uses full protobuf field parsing for appearance data • Avoids treating unrelated numeric fields as item definitions • Disables XML external entities • Fails closed when XML entity hardening cannot be enforced • Confines discovery paths to the repository • Confines output paths to the repository • Rejects symlink escapes • Writes output artifacts atomically per file • Applies limits to Lua tokens • Applies per-file fact limits • Applies total fact limits • Applies diagnostic limits • Applies finding limits • Uses indexed interval lookups for range validation • Uses bounded finding evidence • Uses heap-based overlap sweeps • Uses precomputed Lua delimiter pairs to avoid quadratic behavior Coverage boundaries Binary OTBM maps are not parsed by this auditor. Action ID and unique ID selectors are indexed as registrations, but the auditor does not claim that matching map selectors exist. Dynamic Lua expressions remain visible as unresolved coverage. They are not fabricated as missing references. Validation performed • python m unittest discover s tools canary_audit tests t . p test_*.py v • python m tools.canary_audit validate-schemas • python m tools.canary_audit scan profile all output-dir artifacts canary-audit • python m tools.canary_audit validate input-dir artifacts canary-audit Validation results • 60 unit and integration tests passed • Full two-profile scan completed with exit code 0 • 39311 facts were produced • 2194 findings were produced • 8 legacy errors were waived with exact fingerprints • 0 operational diagnostics • 0 new blocking findings • Generated JSON artifacts passed schema validation No C++ build was run because this change only adds Python tooling documentation, configuration schemas, and CI support. Overall, this commit adds a deterministic profile-aware gameplay content auditor that can be run locally and in CI. It improves repository visibility for unresolved references, duplicate definitions, duplicate registrations, invalid ranges, and profile-specific content issues while keeping runtime behavior unchanged.
7.1 KiB
7.1 KiB
Canary Documentation
This directory is the main documentation hub for the Canary repository.
Documentation Index
This documentation is organized around four primary guides, with specialized references in the surrounding subdirectories:
| Document | Purpose |
|---|---|
README.md |
Project overview and documentation index |
architecture.md |
System design, components and technical architecture |
development.md |
Development environment, coding standards and contribution workflow |
operations.md |
Deployment, monitoring, security, backups and production operations |
systems/content-reference-auditor.md |
Profile-aware gameplay content and identifier auditing |
What is Canary?
Canary is a modern MMORPG server engine that evolved from the OTServBR ecosystem. The project aims to provide a clean, maintainable, and extensible codebase that supports both custom game projects and OpenTibia-based servers. The repository includes support for multiple datapacks, Lua scripting, database persistence, a Docker quickstart for local testing, automated testing and observability tooling.
Key characteristics:
- C++20 core engine
- Lua gameplay scripting
- MariaDB/MySQL persistence
- Docker quickstart support for local testing and LAN demos
- Automated testing
- Metrics and observability
- Optional OpenTelemetry metrics integration
- Multiple datapack support
- Modern build tooling using CMake and vcpkg
Documentation Structure
docs/
├── README.md
├── architecture.md
├── development.md
├── operations.md
├── building/
├── docker/
├── lua-api/
└── systems/
Architecture Guide
The Architecture Guide explains how Canary is built internally.
Topics include:
- High-level system architecture
- Core engine design
- Networking layer
- Protocol handling
- Creature system
- Combat engine
- Map management
- Lua integration
- Database architecture
- Event system
- Scheduler and dispatcher
- Metrics and observability
Read the Architecture Guide.
Recommended for:
- Engine contributors
- System designers
- Technical reviewers
- Advanced server developers
Development Guide
The Development Guide explains how to build, test and contribute to Canary.
Topics include:
- Development environment setup
- Docker development workflow
- Native compilation
- CMake usage
- vcpkg dependency management
- Lua development
- C++ coding standards
- Testing practices
- Pull request workflow
- Debugging techniques
Read the Development Guide.
Recommended for:
- Contributors
- Developers
- Lua scripters
- Engine maintainers
Operations Guide
The Operations Guide focuses on operating Canary safely, including native production deployments and the limits of the local Docker quickstart.
Topics include:
- Infrastructure planning
- Deployment models
- Docker quickstart operations
- Configuration management
- Security practices
- Monitoring and alerting
- Backup strategies
- Disaster recovery
- Upgrade procedures
- Incident response
Read the Operations Guide.
Recommended for:
- Server administrators
- DevOps engineers
- Infrastructure teams
- Community operators
Repository Overview
canary/
│
├── src/ # C++ server source code
├── data/ # Core shared server resources
├── data-canary/ # Minimal Canary datapack
├── data-otservbr-global/ # Global datapack
├── docs/ # Project documentation
├── tests/ # Automated tests
├── docker/ # Local Docker quickstart assets
├── metrics/ # Observability and metrics
│
├── schema.sql # Database schema
├── config.lua.dist # Configuration template
├── CMakeLists.txt # Build configuration
├── CMakePresets.json # Build presets
└── vcpkg.json # Dependency manifest
The project supports both the lightweight data-canary datapack and the larger data-otservbr-global datapack, allowing operators to choose between a minimal engine-focused setup and a more complete game experience.
Typical Lifecycle
The documentation follows the same lifecycle as a Canary deployment:
Architecture
↓
Development
↓
Testing
↓
Deployment
↓
Operations
↓
Monitoring
↓
Maintenance
Technology Stack
Core Technologies
- C++20
- Lua
- MariaDB / MySQL
- CMake
- vcpkg
Infrastructure
- Docker Compose quickstart
- GitHub Actions
- OpenTelemetry-based metrics
- Prometheus and Grafana examples
Supported Platforms
- Linux
- Windows
- macOS
Deployment Options
Canary can be run using:
Docker
Recommended for:
- Local development
- Testing
- LAN demos
The repository Docker Compose stack is a quickstart. It uses the published Canary runtime image, builds the MyAAC quickstart image, and should not be used as a production deployment with default settings. See docker/DOCKER.md for the complete quickstart contract.
Native Installation
Recommended for:
- Production environments
- Custom infrastructure
- Advanced monitoring setups
The repository provides Docker quickstart assets, CMake build presets and operational guidance for these approaches.
Contributing
Contributors are encouraged to:
- Report bugs
- Improve documentation
- Write tests
- Submit pull requests
- Improve Lua systems
- Enhance engine functionality
Before contributing:
- Read
architecture.md - Read
development.md - Follow coding standards
- Run tests locally
- Update documentation when necessary
Additional Resources
Audience Guide
| Role | Recommended Reading |
|---|---|
| New Contributor | README → Development |
| Lua Developer | README → Development |
| Engine Developer | README → Architecture → Development |
| DevOps Engineer | README → Operations |
| Server Administrator | README → Operations |
| Technical Lead | README → Architecture → Operations |
Conclusion
The Canary documentation is organized around three core perspectives:
- Architecture — how the system works
- Development — how the system is built
- Operations — how the system is run
Together, these documents provide a complete reference for understanding, extending, deploying and operating Canary in both development and production environments.