1 commit

Author	SHA1	Message	Date
Eduardo Dantas	ec41dfca1c	build: share content-addressed vcpkg dependencies across CMake and MSBuild (#4079) ... Share content addressed vcpkg dependencies across CMake and MSBuild This commit adds a schema v3 content addressed vcpkg dependency pool that can be shared safely across compatible Git worktrees and independent repositories. The goal is to avoid duplicating expanded vcpkg dependency installations when different checkouts use the same dependency contract, while keeping build outputs isolated per worktree. The shared cache only reuses dependency roots when the full dependency fingerprint proves that the inputs are identical. This avoids the unsafe model of sharing one mutable vcpkg installed directory across unrelated builds. Manifests registries overlays triplets features install options compiler identity SDK toolset vcpkg revision and cache implementation are all part of the validated contract. The shared cache remains opt in. CI clean clones containers and normal local builds continue using local dependency roots unless CANARY_SHARED_CACHE_ROOT is explicitly configured. Main behavior added • Adds a schema v3 shared vcpkg dependency pool • Shares vcpkg downloads across compatible repositories • Shares ABI addressed vcpkg binary packages • Shares expanded vcpkg installed roots only when the dependency fingerprint matches • Shares vcpkg buildtrees and packages roots only when the dependency fingerprint matches • Keeps CMake and Ninja build outputs local to each worktree • Keeps MSBuild outputs local to each worktree • Keeps objects PCH PDB files generated sources intermediate files and executables local • Allows compatible CMake and MSBuild consumers to converge on the same dependency tree • Gives incompatible configurations separate dependency pools • Falls back to build local vcpkg roots when the shared cache is unavailable or cannot be verified Fingerprint model The dependency fingerprint is build system neutral. It covers the dependency inputs that decide whether an expanded vcpkg dependency tree can be reused safely. The dependency fingerprint includes the manifest registry contents ordered overlays target triplet host triplet manifest features install options compiler identity toolset SDK vcpkg executable toolchain revision dependency tools and the normalized cache module implementation. The consumer fingerprint is separate. It adds the build system specific consumer identity such as the CMake generator and tool path or the MSBuild configuration platform toolset and SDK. This lets CMake and MSBuild share the same dependency tree when their dependency inputs match while still validating their own consumer contract independently. CMake integration The CMake integration now includes the shared cache resolver before project initialization. The resolver selects fingerprinted vcpkg roots only after validating the full dependency contract. CMake presets no longer hardcode VCPKG INSTALLED DIR paths. Instead the shared cache module can resolve the installed root when the shared cache is enabled and verified. When it is not enabled the build continues with local vcpkg storage. CMake preset updates also add clean packages and clean buildtrees install options so transient vcpkg data does not remain in local preset directories unnecessarily. CMake changes included • Includes cmake SharedBuildCache cmake from the top level CMakeLists • Removes hardcoded VCPKG INSTALLED DIR values from presets • Lets the shared cache resolver choose the dependency root when safe • Keeps local fallback behavior available • Adds clean packages after build and clean buildtrees after build to vcpkg install options • Requires fresh CMake configuration when dependency paths change • Keeps build outputs inside the normal preset build directories Visual Studio and MSBuild integration This commit adds an opt in Visual Studio Solution bridge. The bridge generates an ignored machine local SharedVcpkgCache props file under vcproj and validates the complete contract before vcpkg restore. The generated props file records the dependency fingerprint consumer fingerprint cache root expected manifest root triplets configuration toolset SDK install options and resolved installed root. Before MSBuild restores dependencies or prepares the build, SharedVcpkgCache targets revalidate the generated contract. If the evaluated MSBuild values no longer match the generated contract, the build fails closed and asks the developer to regenerate the props file. Visual Studio changes included • Adds vcproj SharedVcpkgCache targets • Adds support for generated vcproj .canary shared cache SharedVcpkgCache props • Adds shared cache validation before vcpkg manifest install • Validates manifest root target triplet host triplet link configuration toolset SDK and install options • Pins the Visual Studio instance used by the vcpkg dependency contract • Pins the CMake tool used to resolve the dependency contract • Uses fingerprint specific installed buildtrees and packages roots • Keeps local fallback through vcproj local .vcpkg buildtrees and .vcpkg packages when shared cache is inactive • Keeps Solution build outputs local • Adds generated shared cache files to gitignore while keeping the tracked target file PowerShell tooling Two PowerShell helpers were added for setup auditing cleanup and Solution contract generation. Both require PowerShell 7 2 and run with strict validation. The main setup helper configures the shared cache root registers repository families manages relevant user environment values normalizes global sccache base directories configures vcpkg downloads and binary cache locations and can generate the Visual Studio Solution bridge file. The Solution helper evaluates MSBuild properties resolves the shared dependency contract through the CMake resolver writes the generated props file and supports audit only and validate only modes. Tooling added • tools configure shared build cache ps1 • tools configure shared solution cache ps1 • Shared cache setup mode • Audit only validation mode • Safe repository registration • Safe repository unregister support • Global sccache path normalization • Guarded transient vcpkg cleanup • Guarded fingerprint specific cleanup • Visual Studio Solution props generation • Visual Studio Solution contract validation • Visual Studio Solution contract auditing Safety model The shared cache fails closed whenever identity or filesystem safety cannot be proven. Mutable shared roots are only used on a verified local fixed filesystem and are isolated by fingerprint. Safety rules included • Requires a verified local fixed filesystem for mutable shared pools • Rejects unsafe shared cache locations • Keeps the shared cache outside every registered checkout • Rejects ambiguous identity • Rejects stale or mismatched generated contracts • Rejects unsafe cleanup targets • Uses operation locks around shared cache mutations • Uses vcpkg locks around dependency roots • Preserves existing authenticated VCPKG BINARY SOURCES values • Does not print or persist authenticated binary source values • Fails closed when repositories configure trees generated contracts paths identities or active processes cannot be validated • Keeps opt out and unavailable cache paths build local Docker and baseline updates Container dependency installation now receives the full manifest dependency configuration instead of only vcpkg json. Docker builds copy vcpkg configuration json the local vcpkg registry and triplets into the manifest context before installing dependencies. The vcpkg baseline update workflow now updates both vcpkg json and vcpkg configuration json atomically. It checks both the builtin baseline and the default registry baseline, updates both when needed, and reports the previous builtin and registry baseline values separately in the generated PR text. Docker and workflow changes included • Updates Dockerfile arm to copy vcpkg configuration json vcpkg registry and triplets • Updates Dockerfile dev to copy vcpkg configuration json vcpkg registry and triplets • Updates Dockerfile x86 to copy vcpkg configuration json vcpkg registry and triplets • Preserves preprovisioned Docker manifest installs • Updates the vcpkg baseline workflow to read both baseline declarations • Updates the vcpkg baseline workflow to modify both files in the same PR • Updates generated baseline PR comments to show previous builtin and registry baselines Protobuf and generated source handling The Visual Studio project also improves protobuf generation while adding the shared dependency bridge. Protobuf changes included • Uses a normalized absolute protobuf source root • Uses the host triplet protoc path • Adds incremental inputs and outputs for ProtoCompile • Adds generated protobuf compile items only when protobuf generation is enabled • Excludes generated protobuf compile items when protobuf support is disabled Documentation and repository updates • Adds shared build cache documentation • Documents setup and migration flow • Documents CMake integration • Documents Visual Studio integration • Documents cross repository and cross worktree compatibility • Documents concurrency and locking behavior • Documents fallback behavior • Documents auditing recovery and cleanup procedures • Updates README with shared build cache guidance • Updates gitignore for generated local shared cache files Validation performed • cmake fresh preset windows release • Full CMake Windows Release build • No op CMake rebuild after the full build • Full MSBuild Release x64 build • Incremental MSBuild Release x64 rebuild with no compilation or link • Full MSBuild Debug x64 build using its isolated dependency fingerprint • Incremental MSBuild Debug x64 rebuild • Verified that CMake Release and Solution Release resolve the same dependency fingerprint • pwsh File tools configure shared build cache ps1 AuditOnly • PowerShell parser validation for both helpers • XML parsing • JSON parsing • cmake list presets • actionlint • git diff check Overall this commit adds an opt in shared vcpkg dependency cache for Canary. It lets compatible CMake and MSBuild builds reuse expensive dependency artifacts across worktrees and repositories while keeping build outputs local and failing safely back to local roots whenever the shared dependency contract cannot be fully validated.	2026-08-13 18:50:05 -03:00