Compare commits
1 commit
main
...
refactor/p
| Author | SHA1 | Date | |
|---|---|---|---|
|
bcf19526e9 |
20 changed files with 1783 additions and 1 deletions
267
LANCommander.Documentation/SDK/PackFormat.md
Normal file
267
LANCommander.Documentation/SDK/PackFormat.md
Normal file
|
|
@ -0,0 +1,267 @@
|
|||
# LANCommander Pack Format (`.lcp`)
|
||||
|
||||
## Overview
|
||||
|
||||
The LANCommander Pack (LCP) format is a custom binary container for distributing game files, patches, and metadata. It replaces the previous LCX format (YAML metadata + ZIP archive) with a purpose-built binary format that supports:
|
||||
|
||||
- **Pack identity and versioning** -- Each pack carries a unique ID and a freeform version string, enabling version tracking across game releases and patches.
|
||||
- **Patch lineage** -- Patch packs reference their parent pack and the version they are based on, forming an ordered chain from base game through successive patches.
|
||||
- **Per-entry integrity** -- Every file entry includes a CRC32 checksum. Section-level checksums cover the data and directory regions independently.
|
||||
- **Per-entry compression** -- Each entry specifies its own compression method (None, Deflate, or ZStd), allowing already-compressed files (e.g., game archives) to skip double-compression.
|
||||
- **Differential patching** -- Entries carry an operation field (Create, Modify, Delete), enabling patch packs that contain only what changed between versions.
|
||||
- **Chunked transport** -- Large packs can be split into fixed-size chunks for transport, then transparently reassembled for extraction.
|
||||
- **Forward-only extraction** -- The entry data region is designed for sequential reads. Extraction does not require seeking, enabling streaming from network sources.
|
||||
- **Random-access directory** -- An optional directory section at the end of the file provides offset-based access to individual entries without scanning the data region.
|
||||
- **Internal manifest** -- Rich metadata (game title, description, scripts, media references, dependencies) is stored as a manifest file *inside* the pack body, not in the binary header. The header stays lean and fixed-size for fast identification and validation.
|
||||
|
||||
---
|
||||
|
||||
## File Extension and Magic
|
||||
|
||||
| Item | Value |
|
||||
|---|---|
|
||||
| Extension | `.lcp` |
|
||||
| Magic bytes | `LCPK` (4 bytes, ASCII) |
|
||||
| Chunk magic | `LCPC` (4 bytes, ASCII) |
|
||||
|
||||
---
|
||||
|
||||
## Binary Layout
|
||||
|
||||
A pack file consists of four sections laid out sequentially:
|
||||
|
||||
```
|
||||
+---------------------+
|
||||
| Header | 112 bytes, fixed size
|
||||
+---------------------+
|
||||
| Entry Headers | Variable, one per file
|
||||
| + File Data |
|
||||
| (repeated) |
|
||||
+---------------------+
|
||||
| Directory Section | Optional, one entry per file
|
||||
+---------------------+
|
||||
| Footer | 28 bytes, fixed size
|
||||
+---------------------+
|
||||
```
|
||||
|
||||
All multi-byte integers are little-endian. Strings are UTF-8 unless otherwise noted.
|
||||
|
||||
---
|
||||
|
||||
## Header (112 bytes)
|
||||
|
||||
The header is fixed-size, enabling quick reads for identification, validation, and routing without parsing the pack body.
|
||||
|
||||
| Offset | Size | Field | Description |
|
||||
|--------|------|-------|-------------|
|
||||
| 0 | 4 | Magic | `LCPK` (ASCII). Identifies the file as a LANCommander pack. |
|
||||
| 4 | 2 | FormatVersion | Pack format version. Current value: `2`. Used for forward compatibility; readers should reject versions they do not understand. |
|
||||
| 6 | 2 | Flags | Bit field. See [Flags](#flags). |
|
||||
| 8 | 8 | EntryCount | Number of file entries in the data section (`uint64`). |
|
||||
| 16 | 16 | PackId | GUID identifying this specific pack. Generated when the pack is created. Used to establish relationships between game packs and patch packs. |
|
||||
| 32 | 16 | ParentPackId | GUID of the parent pack. For base game packs, this is `Guid.Empty` (all zeros). For patch packs, this references the base game pack's `PackId`, creating an identity link between related packs. |
|
||||
| 48 | 32 | PackVersion | Freeform version string for this pack, UTF-8, null-padded to 32 bytes. Maximum 32 bytes of UTF-8 content. Not restricted to semantic versioning -- any format is valid (e.g., `1.0.0`, `Build 12345`, `2025.01.15`, `Gold`). |
|
||||
| 80 | 32 | ParentVersion | Version string of the pack that this one follows, UTF-8, null-padded to 32 bytes. For base game packs, this is empty (all zeros). For patch packs, this is the version of the pack that must be installed before this patch can be applied -- either the base game version or the previous patch version. Forms a version chain for ordering. |
|
||||
|
||||
### Flags
|
||||
|
||||
| Bit | Name | Description |
|
||||
|-----|------|-------------|
|
||||
| 0 | HasDirectory | When set, the pack contains a directory section before the footer. Enables random-access reads. |
|
||||
| 1-15 | Reserved | Must be zero. |
|
||||
|
||||
---
|
||||
|
||||
## Entry Header (variable size)
|
||||
|
||||
Each file in the pack is preceded by an entry header. Entry headers are written sequentially, interleaved with their file data.
|
||||
|
||||
| Offset | Size | Field | Description |
|
||||
|--------|------|-------|-------------|
|
||||
| 0 | 4 | PathLength | Length of the `Path` field in bytes (`uint32`). |
|
||||
| 4 | var | Path | Relative file path, UTF-8. Uses `/` as the separator regardless of platform. |
|
||||
| 4+n | 1 | Operation | Entry operation. See [Operations](#operations). |
|
||||
| 5+n | 1 | Compression | Compression method. See [Compression](#compression). |
|
||||
| 6+n | 4 | Attributes | File attributes (`uint32`). Platform-specific file attribute flags. |
|
||||
| 10+n | 8 | Timestamp | Last write time in UTC ticks (`int64`). |
|
||||
| 18+n | 8 | UncompressedSize | Original file size in bytes (`uint64`). |
|
||||
| 26+n | 8 | CompressedSize | Size of the data following this header (`uint64`). Equal to `UncompressedSize` when `Compression` is `None`. |
|
||||
| 34+n | 4 | Checksum | CRC32 of the uncompressed file data (`uint32`). |
|
||||
|
||||
Immediately following the entry header is the file data (`CompressedSize` bytes). For `Delete` operations, `CompressedSize` is 0 and no data follows.
|
||||
|
||||
### Operations
|
||||
|
||||
| Value | Name | Description |
|
||||
|-------|------|-------------|
|
||||
| 0 | Create | New file. Used in base game packs and when a patch adds a file. |
|
||||
| 1 | Modify | Modified file. Used in patch packs when a file changed between versions. |
|
||||
| 2 | Delete | Deleted file. Header-only, no data follows. Used in patch packs when a file was removed. |
|
||||
|
||||
### Compression
|
||||
|
||||
| Value | Name | Description |
|
||||
|-------|------|-------------|
|
||||
| 0 | None | Uncompressed. Data is stored as-is. |
|
||||
| 1 | Deflate | DEFLATE compression. |
|
||||
| 2 | ZStd | Zstandard compression. |
|
||||
|
||||
Compression is per-entry, allowing mixed strategies within a single pack. Files that are already compressed (e.g., `.zip`, `.pak`, `.mp3`) should use `None` to avoid wasted CPU on negligible size reduction.
|
||||
|
||||
---
|
||||
|
||||
## Directory Section (optional, variable size)
|
||||
|
||||
When the `HasDirectory` flag is set, the directory section provides an index of all entries with their offsets into the data section. This enables random access to individual files without scanning the entire pack.
|
||||
|
||||
Each directory entry:
|
||||
|
||||
| Offset | Size | Field | Description |
|
||||
|--------|------|-------|-------------|
|
||||
| 0 | 4 | PathLength | Length of the `Path` field in bytes (`uint32`). |
|
||||
| 4 | var | Path | Relative file path, UTF-8. |
|
||||
| 4+n | 1 | Operation | Entry operation (mirrors the entry header). |
|
||||
| 5+n | 8 | Offset | Byte offset from the start of the file to the entry header (`uint64`). |
|
||||
| 13+n | 8 | UncompressedSize | Original file size (`uint64`). |
|
||||
| 21+n | 8 | CompressedSize | Compressed data size (`uint64`). |
|
||||
| 29+n | 4 | Checksum | CRC32 of the uncompressed file data (`uint32`). |
|
||||
|
||||
---
|
||||
|
||||
## Footer (28 bytes)
|
||||
|
||||
The footer is at the very end of the file. Readers locate it by seeking to `EOF - 28`.
|
||||
|
||||
| Offset | Size | Field | Description |
|
||||
|--------|------|-------|-------------|
|
||||
| 0 | 8 | DirectoryOffset | Byte offset from the start of the file to the directory section (`uint64`). Zero if no directory. |
|
||||
| 8 | 8 | EntryCount | Number of entries (`uint64`). Mirrors the header for validation. |
|
||||
| 16 | 4 | DataChecksum | CRC32 of the entire data section (all entry headers + file data, from byte 112 to directory start) (`uint32`). |
|
||||
| 20 | 4 | DirectoryChecksum | CRC32 of the directory section (`uint32`). Zero if no directory. |
|
||||
| 24 | 4 | Magic | `LCPK` (ASCII). Allows readers to confirm they found a valid footer. |
|
||||
|
||||
---
|
||||
|
||||
## Chunk Format
|
||||
|
||||
Packs larger than a configurable threshold (default: 4 GB - 1 byte) are split into numbered chunks for transport. The first chunk contains the pack data starting from byte 0 (including the pack header). Subsequent chunks are prefixed with a chunk header.
|
||||
|
||||
### Chunk file naming
|
||||
|
||||
| Chunk | Filename |
|
||||
|-------|----------|
|
||||
| 0 | `{name}.lcp` |
|
||||
| 1 | `{name}.lcp.001` |
|
||||
| 2 | `{name}.lcp.002` |
|
||||
| ... | ... |
|
||||
|
||||
### Chunk Header (16 bytes)
|
||||
|
||||
Present on chunks 1+ only. Chunk 0 has no chunk header -- it starts directly with the pack header.
|
||||
|
||||
| Offset | Size | Field | Description |
|
||||
|--------|------|-------|-------------|
|
||||
| 0 | 4 | ChunkMagic | `LCPC` (ASCII). |
|
||||
| 4 | 4 | ChunkIndex | Zero-based index of this chunk (`uint32`). |
|
||||
| 8 | 4 | TotalChunks | Total number of chunks (`uint32`). |
|
||||
| 12 | 4 | ParentCrc | CRC32 of the pack header (first 112 bytes of chunk 0) (`uint32`). Used to verify that all chunks belong to the same pack. |
|
||||
|
||||
To read a chunked pack, concatenate all chunks in order, skipping the 16-byte chunk header on chunks 1+. The result is a standard pack stream.
|
||||
|
||||
---
|
||||
|
||||
## Integrity Model
|
||||
|
||||
Integrity verification operates at three levels:
|
||||
|
||||
1. **Per-entry checksum** -- Each entry header contains a CRC32 of its uncompressed file data. Verified during extraction by computing CRC32 on the fly and comparing.
|
||||
|
||||
2. **Data section checksum** -- The footer's `DataChecksum` covers all bytes between the end of the header (byte 112) and the start of the directory section (or footer, if no directory). Verified by reading the region and computing CRC32.
|
||||
|
||||
3. **Directory section checksum** -- The footer's `DirectoryChecksum` covers all bytes in the directory section. Verified independently.
|
||||
|
||||
4. **Chunk binding** -- Each chunk header stores `ParentCrc`, a CRC32 of the pack header bytes. Verifies that chunks belong together without needing to read the full pack.
|
||||
|
||||
---
|
||||
|
||||
## Pack Relationships and Versioning
|
||||
|
||||
### Identity
|
||||
|
||||
Every pack has a `PackId` (GUID), generated at creation time. This uniquely identifies the pack.
|
||||
|
||||
### Parent link
|
||||
|
||||
Patch packs set `ParentPackId` to the base game pack's `PackId`. This creates a type-agnostic relationship: the system can find all patches for a game by matching `ParentPackId`. Base game packs set `ParentPackId` to `Guid.Empty`.
|
||||
|
||||
### Version chain
|
||||
|
||||
Versions are freeform strings (up to 32 bytes UTF-8). The `ParentVersion` field on a patch names the exact version it was built against.
|
||||
|
||||
Example chain:
|
||||
|
||||
```
|
||||
Base game pack:
|
||||
PackId: A1B2C3D4-...
|
||||
ParentPackId: 00000000-...
|
||||
PackVersion: "1.0.0"
|
||||
ParentVersion: ""
|
||||
|
||||
Patch 1:
|
||||
PackId: E5F6A7B8-...
|
||||
ParentPackId: A1B2C3D4-...
|
||||
PackVersion: "1.1.0"
|
||||
ParentVersion: "1.0.0"
|
||||
|
||||
Patch 2:
|
||||
PackId: C9D0E1F2-...
|
||||
ParentPackId: A1B2C3D4-...
|
||||
PackVersion: "1.2.0"
|
||||
ParentVersion: "1.1.0"
|
||||
```
|
||||
|
||||
This allows the system to:
|
||||
- Determine patch order by following the `ParentVersion` chain.
|
||||
- Validate that a patch is applicable to the currently installed version.
|
||||
- Identify all packs in a game's lineage via `ParentPackId`.
|
||||
|
||||
---
|
||||
|
||||
## Differential Patching
|
||||
|
||||
Patch packs are created by diffing two pack manifests (directory sections):
|
||||
|
||||
1. **Read directories** of the old and new packs.
|
||||
2. **Compare entries** by path:
|
||||
- Present in new but not old: `Create`
|
||||
- Present in both but checksums differ: `Modify`
|
||||
- Present in old but not new: `Delete`
|
||||
3. **Build the patch pack** containing only the changed entries. `Create` and `Modify` entries carry their full file data from the new pack. `Delete` entries are header-only (no data).
|
||||
|
||||
Applying a patch is identical to normal extraction -- the entry `Operation` field drives behavior. `Create` and `Modify` write files, `Delete` removes them. The extraction engine requires no special patch logic.
|
||||
|
||||
---
|
||||
|
||||
## Extraction Behavior
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| VerifyChecksums | `true` | Verify per-entry CRC32 during extraction. |
|
||||
| OverwriteExisting | `true` | Overwrite files that already exist on disk. |
|
||||
| PreserveTimestamps | `true` | Restore original last-write timestamps after extraction. |
|
||||
| SkipUnchangedFiles | `true` | Compare CRC32 of existing local files against the entry checksum. Skip extraction if they match. |
|
||||
|
||||
When `SkipUnchangedFiles` is enabled, the extractor computes CRC32 of the local file before reading the entry data. If checksums match, the entry data is skipped in the stream (seeked or drained), avoiding unnecessary I/O.
|
||||
|
||||
---
|
||||
|
||||
## Replacing LCX
|
||||
|
||||
The LCX format bundled a YAML metadata file with game archives, media, and scripts into a ZIP file. The pack format replaces this by:
|
||||
|
||||
1. Storing the metadata as a **manifest file inside the pack body** -- just another entry alongside game files. The manifest format (YAML or otherwise) is independent of the binary container format.
|
||||
2. Storing game archives, media, scripts, and all other files as **pack entries** with individual checksums and optional compression.
|
||||
3. Adding **version tracking** and **patch lineage** directly in the header, which LCX had no concept of.
|
||||
4. Providing **integrity verification** at multiple levels, which ZIP's per-entry CRC32 only partially covered.
|
||||
|
||||
The pack header does not contain metadata fields like game title, description, or dependency lists. These belong in the manifest file inside the pack, keeping the binary format stable as metadata requirements evolve.
|
||||
|
|
@ -54,6 +54,7 @@ public static class IServiceCollectionExtensions
|
|||
|
||||
services.AddSingleton<MigrationHistoryService>();
|
||||
services.AddSingleton<MigrationService>();
|
||||
services.AddSingleton<PackService>();
|
||||
|
||||
services.TryAddSingleton<IChatClient, ChatClient>();
|
||||
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ using System.Threading.Tasks;
|
|||
|
||||
namespace LANCommander.SDK
|
||||
{
|
||||
internal class ExtractionResult
|
||||
public class ExtractionResult
|
||||
{
|
||||
public bool Success { get; set; }
|
||||
public bool Canceled { get; set; }
|
||||
|
|
|
|||
|
|
@ -39,6 +39,7 @@
|
|||
|
||||
<ItemGroup>
|
||||
<InternalsVisibleTo Include="LANCommander.Server.Tests" />
|
||||
<InternalsVisibleTo Include="LANCommander.SDK.Tests" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
|
|
|
|||
11
LANCommander.SDK/Models/Pack/PackChunkHeader.cs
Normal file
11
LANCommander.SDK/Models/Pack/PackChunkHeader.cs
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
namespace LANCommander.SDK.Models.Pack;
|
||||
|
||||
public class PackChunkHeader
|
||||
{
|
||||
public const string ChunkMagic = "LCPC";
|
||||
public const int ChunkHeaderSize = 16;
|
||||
|
||||
public uint ChunkIndex { get; set; }
|
||||
public uint TotalChunks { get; set; }
|
||||
public uint ParentCrc { get; set; }
|
||||
}
|
||||
8
LANCommander.SDK/Models/Pack/PackCompression.cs
Normal file
8
LANCommander.SDK/Models/Pack/PackCompression.cs
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
namespace LANCommander.SDK.Models.Pack;
|
||||
|
||||
public enum PackCompression : byte
|
||||
{
|
||||
None = 0,
|
||||
Deflate = 1,
|
||||
ZStd = 2,
|
||||
}
|
||||
11
LANCommander.SDK/Models/Pack/PackDirectoryEntry.cs
Normal file
11
LANCommander.SDK/Models/Pack/PackDirectoryEntry.cs
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
namespace LANCommander.SDK.Models.Pack;
|
||||
|
||||
public class PackDirectoryEntry
|
||||
{
|
||||
public string Path { get; set; } = string.Empty;
|
||||
public PackEntryOperation Operation { get; set; } = PackEntryOperation.Create;
|
||||
public ulong Offset { get; set; }
|
||||
public ulong UncompressedSize { get; set; }
|
||||
public ulong CompressedSize { get; set; }
|
||||
public uint Checksum { get; set; }
|
||||
}
|
||||
13
LANCommander.SDK/Models/Pack/PackEntryHeader.cs
Normal file
13
LANCommander.SDK/Models/Pack/PackEntryHeader.cs
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
namespace LANCommander.SDK.Models.Pack;
|
||||
|
||||
public class PackEntryHeader
|
||||
{
|
||||
public string Path { get; set; } = string.Empty;
|
||||
public PackEntryOperation Operation { get; set; } = PackEntryOperation.Create;
|
||||
public PackCompression Compression { get; set; } = PackCompression.None;
|
||||
public uint Attributes { get; set; }
|
||||
public long Timestamp { get; set; }
|
||||
public ulong UncompressedSize { get; set; }
|
||||
public ulong CompressedSize { get; set; }
|
||||
public uint Checksum { get; set; }
|
||||
}
|
||||
8
LANCommander.SDK/Models/Pack/PackEntryOperation.cs
Normal file
8
LANCommander.SDK/Models/Pack/PackEntryOperation.cs
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
namespace LANCommander.SDK.Models.Pack;
|
||||
|
||||
public enum PackEntryOperation : byte
|
||||
{
|
||||
Create = 0,
|
||||
Modify = 1,
|
||||
Delete = 2,
|
||||
}
|
||||
9
LANCommander.SDK/Models/Pack/PackExtractionOptions.cs
Normal file
9
LANCommander.SDK/Models/Pack/PackExtractionOptions.cs
Normal file
|
|
@ -0,0 +1,9 @@
|
|||
namespace LANCommander.SDK.Models.Pack;
|
||||
|
||||
public class PackExtractionOptions
|
||||
{
|
||||
public bool VerifyChecksums { get; set; } = true;
|
||||
public bool OverwriteExisting { get; set; } = true;
|
||||
public bool PreserveTimestamps { get; set; } = true;
|
||||
public bool SkipUnchangedFiles { get; set; } = true;
|
||||
}
|
||||
10
LANCommander.SDK/Models/Pack/PackFlags.cs
Normal file
10
LANCommander.SDK/Models/Pack/PackFlags.cs
Normal file
|
|
@ -0,0 +1,10 @@
|
|||
using System;
|
||||
|
||||
namespace LANCommander.SDK.Models.Pack;
|
||||
|
||||
[Flags]
|
||||
public enum PackFlags : ushort
|
||||
{
|
||||
None = 0,
|
||||
HasDirectory = 1 << 0,
|
||||
}
|
||||
11
LANCommander.SDK/Models/Pack/PackFooter.cs
Normal file
11
LANCommander.SDK/Models/Pack/PackFooter.cs
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
namespace LANCommander.SDK.Models.Pack;
|
||||
|
||||
public class PackFooter
|
||||
{
|
||||
public const int FooterSize = 28;
|
||||
|
||||
public ulong DirectoryOffset { get; set; }
|
||||
public ulong EntryCount { get; set; }
|
||||
public uint DataChecksum { get; set; }
|
||||
public uint DirectoryChecksum { get; set; }
|
||||
}
|
||||
19
LANCommander.SDK/Models/Pack/PackHeader.cs
Normal file
19
LANCommander.SDK/Models/Pack/PackHeader.cs
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
using System;
|
||||
|
||||
namespace LANCommander.SDK.Models.Pack;
|
||||
|
||||
public class PackHeader
|
||||
{
|
||||
public const string Magic = "LCPK";
|
||||
public const int MagicSize = 4;
|
||||
public const int VersionFieldSize = 32;
|
||||
public const int HeaderSize = 112; // Magic(4) + Version(2) + Flags(2) + EntryCount(8) + PackId(16) + ParentPackId(16) + PackVersion(32) + ParentVersion(32)
|
||||
|
||||
public ushort Version { get; set; } = 2;
|
||||
public PackFlags Flags { get; set; } = PackFlags.None;
|
||||
public ulong EntryCount { get; set; }
|
||||
public Guid PackId { get; set; } = Guid.Empty;
|
||||
public Guid ParentPackId { get; set; } = Guid.Empty;
|
||||
public string PackVersion { get; set; } = string.Empty;
|
||||
public string ParentVersion { get; set; } = string.Empty;
|
||||
}
|
||||
10
LANCommander.SDK/Models/Pack/PackManifest.cs
Normal file
10
LANCommander.SDK/Models/Pack/PackManifest.cs
Normal file
|
|
@ -0,0 +1,10 @@
|
|||
using System.Collections.Generic;
|
||||
|
||||
namespace LANCommander.SDK.Models.Pack;
|
||||
|
||||
public class PackManifest
|
||||
{
|
||||
public PackHeader Header { get; set; } = new();
|
||||
public List<PackDirectoryEntry> Entries { get; set; } = [];
|
||||
public PackFooter Footer { get; set; } = new();
|
||||
}
|
||||
13
LANCommander.SDK/Models/Pack/PackOptions.cs
Normal file
13
LANCommander.SDK/Models/Pack/PackOptions.cs
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
using System;
|
||||
|
||||
namespace LANCommander.SDK.Models.Pack;
|
||||
|
||||
public class PackOptions
|
||||
{
|
||||
public PackCompression Compression { get; set; } = PackCompression.None;
|
||||
public bool WriteDirectory { get; set; } = true;
|
||||
public Guid PackId { get; set; } = Guid.NewGuid();
|
||||
public Guid ParentPackId { get; set; } = Guid.Empty;
|
||||
public string PackVersion { get; set; } = string.Empty;
|
||||
public string ParentVersion { get; set; } = string.Empty;
|
||||
}
|
||||
16
LANCommander.SDK/Models/Pack/PackVerificationResult.cs
Normal file
16
LANCommander.SDK/Models/Pack/PackVerificationResult.cs
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
using System.Collections.Generic;
|
||||
|
||||
namespace LANCommander.SDK.Models.Pack;
|
||||
|
||||
public class PackVerificationResult
|
||||
{
|
||||
public bool IsValid { get; set; }
|
||||
public List<PackVerificationFailure> Failures { get; set; } = [];
|
||||
}
|
||||
|
||||
public class PackVerificationFailure
|
||||
{
|
||||
public string Path { get; set; } = string.Empty;
|
||||
public uint ExpectedChecksum { get; set; }
|
||||
public uint ActualChecksum { get; set; }
|
||||
}
|
||||
182
LANCommander.SDK/Services/ConcatenatingStream.cs
Normal file
182
LANCommander.SDK/Services/ConcatenatingStream.cs
Normal file
|
|
@ -0,0 +1,182 @@
|
|||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace LANCommander.SDK.Services;
|
||||
|
||||
/// <summary>
|
||||
/// A read-only stream that presents an ordered list of files as a single continuous stream.
|
||||
/// Used to reassemble chunked pack files transparently.
|
||||
/// </summary>
|
||||
public class ConcatenatingStream : Stream
|
||||
{
|
||||
private readonly IReadOnlyList<string> _filePaths;
|
||||
private readonly IReadOnlyList<long> _skipBytes;
|
||||
private int _currentIndex;
|
||||
private Stream? _currentStream;
|
||||
private long _position;
|
||||
private long _length;
|
||||
|
||||
/// <param name="filePaths">Ordered list of file paths to concatenate.</param>
|
||||
/// <param name="skipBytes">
|
||||
/// Number of bytes to skip at the start of each file (e.g., chunk header size).
|
||||
/// Must have the same count as filePaths. Use 0 for no skip.
|
||||
/// </param>
|
||||
public ConcatenatingStream(IReadOnlyList<string> filePaths, IReadOnlyList<long> skipBytes)
|
||||
{
|
||||
if (filePaths.Count != skipBytes.Count)
|
||||
throw new ArgumentException("filePaths and skipBytes must have the same count.");
|
||||
|
||||
_filePaths = filePaths;
|
||||
_skipBytes = skipBytes;
|
||||
_currentIndex = 0;
|
||||
|
||||
_length = 0;
|
||||
for (int i = 0; i < filePaths.Count; i++)
|
||||
{
|
||||
var fileInfo = new FileInfo(filePaths[i]);
|
||||
_length += fileInfo.Length - skipBytes[i];
|
||||
}
|
||||
}
|
||||
|
||||
public override bool CanRead => true;
|
||||
public override bool CanSeek => false;
|
||||
public override bool CanWrite => false;
|
||||
public override long Length => _length;
|
||||
|
||||
public override long Position
|
||||
{
|
||||
get => _position;
|
||||
set => throw new NotSupportedException("ConcatenatingStream does not support seeking.");
|
||||
}
|
||||
|
||||
public override int Read(byte[] buffer, int offset, int count)
|
||||
{
|
||||
var totalRead = 0;
|
||||
|
||||
while (totalRead < count)
|
||||
{
|
||||
if (_currentStream == null)
|
||||
{
|
||||
if (!OpenNextStream())
|
||||
break;
|
||||
}
|
||||
|
||||
var bytesRead = _currentStream!.Read(buffer, offset + totalRead, count - totalRead);
|
||||
|
||||
if (bytesRead == 0)
|
||||
{
|
||||
_currentStream.Dispose();
|
||||
_currentStream = null;
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
totalRead += bytesRead;
|
||||
_position += bytesRead;
|
||||
}
|
||||
|
||||
return totalRead;
|
||||
}
|
||||
|
||||
public override async Task<int> ReadAsync(byte[] buffer, int offset, int count, CancellationToken cancellationToken)
|
||||
{
|
||||
var totalRead = 0;
|
||||
|
||||
while (totalRead < count)
|
||||
{
|
||||
cancellationToken.ThrowIfCancellationRequested();
|
||||
|
||||
if (_currentStream == null)
|
||||
{
|
||||
if (!OpenNextStream())
|
||||
break;
|
||||
}
|
||||
|
||||
var bytesRead = await _currentStream!.ReadAsync(buffer.AsMemory(offset + totalRead, count - totalRead), cancellationToken);
|
||||
|
||||
if (bytesRead == 0)
|
||||
{
|
||||
_currentStream.Dispose();
|
||||
_currentStream = null;
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
totalRead += bytesRead;
|
||||
_position += bytesRead;
|
||||
}
|
||||
|
||||
return totalRead;
|
||||
}
|
||||
|
||||
public override async ValueTask<int> ReadAsync(Memory<byte> buffer, CancellationToken cancellationToken = default)
|
||||
{
|
||||
var totalRead = 0;
|
||||
|
||||
while (totalRead < buffer.Length)
|
||||
{
|
||||
cancellationToken.ThrowIfCancellationRequested();
|
||||
|
||||
if (_currentStream == null)
|
||||
{
|
||||
if (!OpenNextStream())
|
||||
break;
|
||||
}
|
||||
|
||||
var bytesRead = await _currentStream!.ReadAsync(buffer.Slice(totalRead), cancellationToken);
|
||||
|
||||
if (bytesRead == 0)
|
||||
{
|
||||
_currentStream.Dispose();
|
||||
_currentStream = null;
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
totalRead += bytesRead;
|
||||
_position += bytesRead;
|
||||
}
|
||||
|
||||
return totalRead;
|
||||
}
|
||||
|
||||
private bool OpenNextStream()
|
||||
{
|
||||
if (_currentIndex >= _filePaths.Count)
|
||||
return false;
|
||||
|
||||
_currentStream = new FileStream(_filePaths[_currentIndex], FileMode.Open, FileAccess.Read, FileShare.Read);
|
||||
|
||||
var skip = _skipBytes[_currentIndex];
|
||||
if (skip > 0)
|
||||
_currentStream.Seek(skip, SeekOrigin.Begin);
|
||||
|
||||
_currentIndex++;
|
||||
return true;
|
||||
}
|
||||
|
||||
public override void Flush() { }
|
||||
|
||||
public override long Seek(long offset, SeekOrigin origin)
|
||||
=> throw new NotSupportedException("ConcatenatingStream does not support seeking.");
|
||||
|
||||
public override void SetLength(long value)
|
||||
=> throw new NotSupportedException("ConcatenatingStream is read-only.");
|
||||
|
||||
public override void Write(byte[] buffer, int offset, int count)
|
||||
=> throw new NotSupportedException("ConcatenatingStream is read-only.");
|
||||
|
||||
protected override void Dispose(bool disposing)
|
||||
{
|
||||
if (disposing)
|
||||
{
|
||||
_currentStream?.Dispose();
|
||||
_currentStream = null;
|
||||
}
|
||||
|
||||
base.Dispose(disposing);
|
||||
}
|
||||
}
|
||||
229
LANCommander.SDK/Services/PackBinaryReader.cs
Normal file
229
LANCommander.SDK/Services/PackBinaryReader.cs
Normal file
|
|
@ -0,0 +1,229 @@
|
|||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using System.Text;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using Force.Crc32;
|
||||
using LANCommander.SDK.Models.Pack;
|
||||
|
||||
namespace LANCommander.SDK.Services;
|
||||
|
||||
internal static class PackBinaryReader
|
||||
{
|
||||
public static async Task<PackHeader> ReadHeaderAsync(Stream stream, CancellationToken ct = default)
|
||||
{
|
||||
var buffer = new byte[PackHeader.HeaderSize];
|
||||
await ReadExactAsync(stream, buffer, ct);
|
||||
|
||||
var magic = Encoding.ASCII.GetString(buffer, 0, PackHeader.MagicSize);
|
||||
|
||||
if (magic != PackHeader.Magic)
|
||||
throw new InvalidDataException($"Invalid pack magic: expected '{PackHeader.Magic}', got '{magic}'");
|
||||
|
||||
return new PackHeader
|
||||
{
|
||||
Version = ReadUInt16(buffer, 4),
|
||||
Flags = (PackFlags)ReadUInt16(buffer, 6),
|
||||
EntryCount = ReadUInt64(buffer, 8),
|
||||
PackId = ReadGuid(buffer, 16),
|
||||
ParentPackId = ReadGuid(buffer, 32),
|
||||
PackVersion = ReadVersionString(buffer, 48),
|
||||
ParentVersion = ReadVersionString(buffer, 80),
|
||||
};
|
||||
}
|
||||
|
||||
public static async Task<PackEntryHeader> ReadEntryHeaderAsync(Stream stream, CancellationToken ct = default)
|
||||
{
|
||||
var pathLenBuf = new byte[4];
|
||||
await ReadExactAsync(stream, pathLenBuf, ct);
|
||||
var pathLength = ReadUInt32(pathLenBuf, 0);
|
||||
|
||||
var pathBuf = new byte[pathLength];
|
||||
await ReadExactAsync(stream, pathBuf, ct);
|
||||
var path = Encoding.UTF8.GetString(pathBuf);
|
||||
|
||||
// Operation (1) + Compression (1) + Attributes (4) + Timestamp (8) +
|
||||
// UncompressedSize (8) + CompressedSize (8) + Checksum (4) = 34
|
||||
var fixedBuf = new byte[34];
|
||||
await ReadExactAsync(stream, fixedBuf, ct);
|
||||
|
||||
return new PackEntryHeader
|
||||
{
|
||||
Path = path,
|
||||
Operation = (PackEntryOperation)fixedBuf[0],
|
||||
Compression = (PackCompression)fixedBuf[1],
|
||||
Attributes = ReadUInt32(fixedBuf, 2),
|
||||
Timestamp = ReadInt64(fixedBuf, 6),
|
||||
UncompressedSize = ReadUInt64(fixedBuf, 14),
|
||||
CompressedSize = ReadUInt64(fixedBuf, 22),
|
||||
Checksum = ReadUInt32(fixedBuf, 30),
|
||||
};
|
||||
}
|
||||
|
||||
public static async Task<PackDirectoryEntry> ReadDirectoryEntryAsync(Stream stream, CancellationToken ct = default)
|
||||
{
|
||||
var pathLenBuf = new byte[4];
|
||||
await ReadExactAsync(stream, pathLenBuf, ct);
|
||||
var pathLength = ReadUInt32(pathLenBuf, 0);
|
||||
|
||||
var pathBuf = new byte[pathLength];
|
||||
await ReadExactAsync(stream, pathBuf, ct);
|
||||
var path = Encoding.UTF8.GetString(pathBuf);
|
||||
|
||||
// Operation (1) + Offset (8) + UncompressedSize (8) + CompressedSize (8) + Checksum (4) = 29
|
||||
var fixedBuf = new byte[29];
|
||||
await ReadExactAsync(stream, fixedBuf, ct);
|
||||
|
||||
return new PackDirectoryEntry
|
||||
{
|
||||
Path = path,
|
||||
Operation = (PackEntryOperation)fixedBuf[0],
|
||||
Offset = ReadUInt64(fixedBuf, 1),
|
||||
UncompressedSize = ReadUInt64(fixedBuf, 9),
|
||||
CompressedSize = ReadUInt64(fixedBuf, 17),
|
||||
Checksum = ReadUInt32(fixedBuf, 25),
|
||||
};
|
||||
}
|
||||
|
||||
public static async Task<PackFooter> ReadFooterAsync(Stream stream, CancellationToken ct = default)
|
||||
{
|
||||
stream.Seek(-PackFooter.FooterSize, SeekOrigin.End);
|
||||
|
||||
var buffer = new byte[PackFooter.FooterSize];
|
||||
await ReadExactAsync(stream, buffer, ct);
|
||||
|
||||
var magic = Encoding.ASCII.GetString(buffer, 24, PackHeader.MagicSize);
|
||||
|
||||
if (magic != PackHeader.Magic)
|
||||
throw new InvalidDataException($"Invalid footer magic: expected '{PackHeader.Magic}', got '{magic}'");
|
||||
|
||||
return new PackFooter
|
||||
{
|
||||
DirectoryOffset = ReadUInt64(buffer, 0),
|
||||
EntryCount = ReadUInt64(buffer, 8),
|
||||
DataChecksum = ReadUInt32(buffer, 16),
|
||||
DirectoryChecksum = ReadUInt32(buffer, 20),
|
||||
};
|
||||
}
|
||||
|
||||
public static async Task<List<PackDirectoryEntry>> ReadDirectoryEntriesAsync(Stream stream, PackFooter footer, CancellationToken ct = default)
|
||||
{
|
||||
stream.Seek((long)footer.DirectoryOffset, SeekOrigin.Begin);
|
||||
|
||||
var entries = new List<PackDirectoryEntry>();
|
||||
|
||||
for (ulong i = 0; i < footer.EntryCount; i++)
|
||||
{
|
||||
ct.ThrowIfCancellationRequested();
|
||||
entries.Add(await ReadDirectoryEntryAsync(stream, ct));
|
||||
}
|
||||
|
||||
return entries;
|
||||
}
|
||||
|
||||
public static async Task<PackChunkHeader> ReadChunkHeaderAsync(Stream stream, CancellationToken ct = default)
|
||||
{
|
||||
var buffer = new byte[PackChunkHeader.ChunkHeaderSize];
|
||||
await ReadExactAsync(stream, buffer, ct);
|
||||
|
||||
var magic = Encoding.ASCII.GetString(buffer, 0, 4);
|
||||
|
||||
if (magic != PackChunkHeader.ChunkMagic)
|
||||
throw new InvalidDataException($"Invalid chunk magic: expected '{PackChunkHeader.ChunkMagic}', got '{magic}'");
|
||||
|
||||
return new PackChunkHeader
|
||||
{
|
||||
ChunkIndex = ReadUInt32(buffer, 4),
|
||||
TotalChunks = ReadUInt32(buffer, 8),
|
||||
ParentCrc = ReadUInt32(buffer, 12),
|
||||
};
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Computes the CRC32 of a region of the stream between two positions.
|
||||
/// The stream position is restored after computation.
|
||||
/// </summary>
|
||||
public static async Task<uint> ComputeStreamCrc32Async(Stream stream, long start, long end, CancellationToken ct = default)
|
||||
{
|
||||
var originalPosition = stream.Position;
|
||||
stream.Seek(start, SeekOrigin.Begin);
|
||||
|
||||
uint crc = 0;
|
||||
var buffer = new byte[65536];
|
||||
var remaining = end - start;
|
||||
|
||||
while (remaining > 0)
|
||||
{
|
||||
ct.ThrowIfCancellationRequested();
|
||||
var toRead = (int)Math.Min(remaining, buffer.Length);
|
||||
var bytesRead = await stream.ReadAsync(buffer.AsMemory(0, toRead), ct);
|
||||
|
||||
if (bytesRead == 0)
|
||||
break;
|
||||
|
||||
crc = Crc32Algorithm.Append(crc, buffer, 0, bytesRead);
|
||||
remaining -= bytesRead;
|
||||
}
|
||||
|
||||
stream.Seek(originalPosition, SeekOrigin.Begin);
|
||||
return crc;
|
||||
}
|
||||
|
||||
private static async Task ReadExactAsync(Stream stream, byte[] buffer, CancellationToken ct)
|
||||
{
|
||||
var totalRead = 0;
|
||||
|
||||
while (totalRead < buffer.Length)
|
||||
{
|
||||
var bytesRead = await stream.ReadAsync(buffer.AsMemory(totalRead), ct);
|
||||
|
||||
if (bytesRead == 0)
|
||||
throw new EndOfStreamException($"Unexpected end of stream. Expected {buffer.Length} bytes, got {totalRead}.");
|
||||
|
||||
totalRead += bytesRead;
|
||||
}
|
||||
}
|
||||
|
||||
private static Guid ReadGuid(byte[] buffer, int offset)
|
||||
{
|
||||
var bytes = new byte[16];
|
||||
Array.Copy(buffer, offset, bytes, 0, 16);
|
||||
return new Guid(bytes);
|
||||
}
|
||||
|
||||
private static string ReadVersionString(byte[] buffer, int offset)
|
||||
{
|
||||
// Find the end of the string (first null byte or field boundary)
|
||||
var length = 0;
|
||||
for (int i = 0; i < PackHeader.VersionFieldSize; i++)
|
||||
{
|
||||
if (buffer[offset + i] == 0)
|
||||
break;
|
||||
length++;
|
||||
}
|
||||
return Encoding.UTF8.GetString(buffer, offset, length);
|
||||
}
|
||||
|
||||
private static ushort ReadUInt16(byte[] buffer, int offset)
|
||||
=> (ushort)(buffer[offset] | (buffer[offset + 1] << 8));
|
||||
|
||||
private static uint ReadUInt32(byte[] buffer, int offset)
|
||||
=> (uint)(buffer[offset]
|
||||
| (buffer[offset + 1] << 8)
|
||||
| (buffer[offset + 2] << 16)
|
||||
| (buffer[offset + 3] << 24));
|
||||
|
||||
private static ulong ReadUInt64(byte[] buffer, int offset)
|
||||
=> (ulong)buffer[offset]
|
||||
| ((ulong)buffer[offset + 1] << 8)
|
||||
| ((ulong)buffer[offset + 2] << 16)
|
||||
| ((ulong)buffer[offset + 3] << 24)
|
||||
| ((ulong)buffer[offset + 4] << 32)
|
||||
| ((ulong)buffer[offset + 5] << 40)
|
||||
| ((ulong)buffer[offset + 6] << 48)
|
||||
| ((ulong)buffer[offset + 7] << 56);
|
||||
|
||||
private static long ReadInt64(byte[] buffer, int offset)
|
||||
=> (long)ReadUInt64(buffer, offset);
|
||||
}
|
||||
168
LANCommander.SDK/Services/PackBinaryWriter.cs
Normal file
168
LANCommander.SDK/Services/PackBinaryWriter.cs
Normal file
|
|
@ -0,0 +1,168 @@
|
|||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using System.Text;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using Force.Crc32;
|
||||
using LANCommander.SDK.Models.Pack;
|
||||
|
||||
namespace LANCommander.SDK.Services;
|
||||
|
||||
internal static class PackBinaryWriter
|
||||
{
|
||||
public static async Task WriteHeaderAsync(Stream stream, PackHeader header, CancellationToken ct = default)
|
||||
{
|
||||
var buffer = new byte[PackHeader.HeaderSize];
|
||||
var magic = Encoding.ASCII.GetBytes(PackHeader.Magic);
|
||||
|
||||
Array.Copy(magic, 0, buffer, 0, PackHeader.MagicSize);
|
||||
WriteUInt16(buffer, 4, header.Version);
|
||||
WriteUInt16(buffer, 6, (ushort)header.Flags);
|
||||
WriteUInt64(buffer, 8, header.EntryCount);
|
||||
WriteGuid(buffer, 16, header.PackId);
|
||||
WriteGuid(buffer, 32, header.ParentPackId);
|
||||
WriteVersionString(buffer, 48, header.PackVersion);
|
||||
WriteVersionString(buffer, 80, header.ParentVersion);
|
||||
|
||||
await stream.WriteAsync(buffer, ct);
|
||||
}
|
||||
|
||||
public static async Task WriteEntryHeaderAsync(Stream stream, PackEntryHeader entry, CancellationToken ct = default)
|
||||
{
|
||||
var pathBytes = Encoding.UTF8.GetBytes(entry.Path);
|
||||
|
||||
// PathLength (4) + Path (variable) + Operation (1) + Compression (1) +
|
||||
// Attributes (4) + Timestamp (8) + UncompressedSize (8) + CompressedSize (8) + Checksum (4)
|
||||
var fixedSize = 4 + pathBytes.Length + 1 + 1 + 4 + 8 + 8 + 8 + 4;
|
||||
var buffer = new byte[fixedSize];
|
||||
var offset = 0;
|
||||
|
||||
WriteUInt32(buffer, offset, (uint)pathBytes.Length); offset += 4;
|
||||
Array.Copy(pathBytes, 0, buffer, offset, pathBytes.Length); offset += pathBytes.Length;
|
||||
buffer[offset++] = (byte)entry.Operation;
|
||||
buffer[offset++] = (byte)entry.Compression;
|
||||
WriteUInt32(buffer, offset, entry.Attributes); offset += 4;
|
||||
WriteInt64(buffer, offset, entry.Timestamp); offset += 8;
|
||||
WriteUInt64(buffer, offset, entry.UncompressedSize); offset += 8;
|
||||
WriteUInt64(buffer, offset, entry.CompressedSize); offset += 8;
|
||||
WriteUInt32(buffer, offset, entry.Checksum);
|
||||
|
||||
await stream.WriteAsync(buffer, ct);
|
||||
}
|
||||
|
||||
public static async Task<uint> WriteDirectoryAsync(Stream stream, List<PackDirectoryEntry> entries, CancellationToken ct = default)
|
||||
{
|
||||
uint crc = 0;
|
||||
|
||||
foreach (var entry in entries)
|
||||
{
|
||||
var pathBytes = Encoding.UTF8.GetBytes(entry.Path);
|
||||
var entrySize = 4 + pathBytes.Length + 1 + 8 + 8 + 8 + 4;
|
||||
var buffer = new byte[entrySize];
|
||||
var offset = 0;
|
||||
|
||||
WriteUInt32(buffer, offset, (uint)pathBytes.Length); offset += 4;
|
||||
Array.Copy(pathBytes, 0, buffer, offset, pathBytes.Length); offset += pathBytes.Length;
|
||||
buffer[offset++] = (byte)entry.Operation;
|
||||
WriteUInt64(buffer, offset, entry.Offset); offset += 8;
|
||||
WriteUInt64(buffer, offset, entry.UncompressedSize); offset += 8;
|
||||
WriteUInt64(buffer, offset, entry.CompressedSize); offset += 8;
|
||||
WriteUInt32(buffer, offset, entry.Checksum);
|
||||
|
||||
crc = Crc32Algorithm.Append(crc, buffer);
|
||||
await stream.WriteAsync(buffer, ct);
|
||||
}
|
||||
|
||||
return crc;
|
||||
}
|
||||
|
||||
public static async Task WriteFooterAsync(Stream stream, PackFooter footer, CancellationToken ct = default)
|
||||
{
|
||||
var buffer = new byte[PackFooter.FooterSize];
|
||||
var magic = Encoding.ASCII.GetBytes(PackHeader.Magic);
|
||||
|
||||
WriteUInt64(buffer, 0, footer.DirectoryOffset);
|
||||
WriteUInt64(buffer, 8, footer.EntryCount);
|
||||
WriteUInt32(buffer, 16, footer.DataChecksum);
|
||||
WriteUInt32(buffer, 20, footer.DirectoryChecksum);
|
||||
Array.Copy(magic, 0, buffer, 24, PackHeader.MagicSize);
|
||||
|
||||
await stream.WriteAsync(buffer, ct);
|
||||
}
|
||||
|
||||
public static async Task WriteChunkHeaderAsync(Stream stream, PackChunkHeader chunk, CancellationToken ct = default)
|
||||
{
|
||||
var buffer = new byte[PackChunkHeader.ChunkHeaderSize];
|
||||
var magic = Encoding.ASCII.GetBytes(PackChunkHeader.ChunkMagic);
|
||||
|
||||
Array.Copy(magic, 0, buffer, 0, 4);
|
||||
WriteUInt32(buffer, 4, chunk.ChunkIndex);
|
||||
WriteUInt32(buffer, 8, chunk.TotalChunks);
|
||||
WriteUInt32(buffer, 12, chunk.ParentCrc);
|
||||
|
||||
await stream.WriteAsync(buffer, ct);
|
||||
}
|
||||
|
||||
public static uint ComputeHeaderCrc(PackHeader header)
|
||||
{
|
||||
var buffer = new byte[PackHeader.HeaderSize];
|
||||
var magic = Encoding.ASCII.GetBytes(PackHeader.Magic);
|
||||
|
||||
Array.Copy(magic, 0, buffer, 0, PackHeader.MagicSize);
|
||||
WriteUInt16(buffer, 4, header.Version);
|
||||
WriteUInt16(buffer, 6, (ushort)header.Flags);
|
||||
WriteUInt64(buffer, 8, header.EntryCount);
|
||||
WriteGuid(buffer, 16, header.PackId);
|
||||
WriteGuid(buffer, 32, header.ParentPackId);
|
||||
WriteVersionString(buffer, 48, header.PackVersion);
|
||||
WriteVersionString(buffer, 80, header.ParentVersion);
|
||||
|
||||
return Crc32Algorithm.Compute(buffer);
|
||||
}
|
||||
|
||||
private static void WriteGuid(byte[] buffer, int offset, Guid value)
|
||||
{
|
||||
var bytes = value.ToByteArray();
|
||||
Array.Copy(bytes, 0, buffer, offset, 16);
|
||||
}
|
||||
|
||||
private static void WriteVersionString(byte[] buffer, int offset, string value)
|
||||
{
|
||||
var bytes = Encoding.UTF8.GetBytes(value ?? string.Empty);
|
||||
var length = Math.Min(bytes.Length, PackHeader.VersionFieldSize);
|
||||
Array.Copy(bytes, 0, buffer, offset, length);
|
||||
// Zero-fill remainder
|
||||
for (int i = length; i < PackHeader.VersionFieldSize; i++)
|
||||
buffer[offset + i] = 0;
|
||||
}
|
||||
|
||||
private static void WriteUInt16(byte[] buffer, int offset, ushort value)
|
||||
{
|
||||
buffer[offset] = (byte)value;
|
||||
buffer[offset + 1] = (byte)(value >> 8);
|
||||
}
|
||||
|
||||
private static void WriteUInt32(byte[] buffer, int offset, uint value)
|
||||
{
|
||||
buffer[offset] = (byte)value;
|
||||
buffer[offset + 1] = (byte)(value >> 8);
|
||||
buffer[offset + 2] = (byte)(value >> 16);
|
||||
buffer[offset + 3] = (byte)(value >> 24);
|
||||
}
|
||||
|
||||
private static void WriteUInt64(byte[] buffer, int offset, ulong value)
|
||||
{
|
||||
buffer[offset] = (byte)value;
|
||||
buffer[offset + 1] = (byte)(value >> 8);
|
||||
buffer[offset + 2] = (byte)(value >> 16);
|
||||
buffer[offset + 3] = (byte)(value >> 24);
|
||||
buffer[offset + 4] = (byte)(value >> 32);
|
||||
buffer[offset + 5] = (byte)(value >> 40);
|
||||
buffer[offset + 6] = (byte)(value >> 48);
|
||||
buffer[offset + 7] = (byte)(value >> 56);
|
||||
}
|
||||
|
||||
private static void WriteInt64(byte[] buffer, int offset, long value)
|
||||
=> WriteUInt64(buffer, offset, (ulong)value);
|
||||
}
|
||||
795
LANCommander.SDK/Services/PackService.cs
Normal file
795
LANCommander.SDK/Services/PackService.cs
Normal file
|
|
@ -0,0 +1,795 @@
|
|||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using Force.Crc32;
|
||||
using LANCommander.SDK.Models.Pack;
|
||||
using Microsoft.Extensions.Logging;
|
||||
|
||||
namespace LANCommander.SDK.Services;
|
||||
|
||||
public class PackService(ILogger<PackService> logger)
|
||||
{
|
||||
private const int BufferSize = 65536;
|
||||
|
||||
public delegate void OnProgressDelegate(long position, long length);
|
||||
|
||||
public event OnProgressDelegate? OnProgress;
|
||||
public event Action<PackEntryHeader>? OnEntryStarted;
|
||||
public event Action<PackEntryHeader>? OnEntryCompleted;
|
||||
|
||||
#region Packing
|
||||
|
||||
/// <summary>
|
||||
/// Packs a source directory into a pack stream. All files are written as Create operations.
|
||||
/// </summary>
|
||||
public async Task PackAsync(string sourceDirectory, Stream output, PackOptions options, CancellationToken ct = default)
|
||||
{
|
||||
var files = Directory.GetFiles(sourceDirectory, "*", SearchOption.AllDirectories);
|
||||
var basePath = sourceDirectory.TrimEnd(Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar);
|
||||
|
||||
var header = new PackHeader
|
||||
{
|
||||
Version = 2,
|
||||
Flags = options.WriteDirectory ? PackFlags.HasDirectory : PackFlags.None,
|
||||
EntryCount = (ulong)files.Length,
|
||||
PackId = options.PackId,
|
||||
ParentPackId = options.ParentPackId,
|
||||
PackVersion = options.PackVersion,
|
||||
ParentVersion = options.ParentVersion,
|
||||
};
|
||||
|
||||
await PackBinaryWriter.WriteHeaderAsync(output, header, ct);
|
||||
|
||||
var directoryEntries = new List<PackDirectoryEntry>(files.Length);
|
||||
uint dataCrc = 0;
|
||||
long totalBytes = files.Sum(f => new FileInfo(f).Length);
|
||||
long bytesWritten = 0;
|
||||
|
||||
foreach (var filePath in files)
|
||||
{
|
||||
ct.ThrowIfCancellationRequested();
|
||||
|
||||
var relativePath = Path.GetRelativePath(basePath, filePath).Replace('\\', '/');
|
||||
var fileInfo = new FileInfo(filePath);
|
||||
var entryOffset = (ulong)output.Position;
|
||||
|
||||
var entryHeader = new PackEntryHeader
|
||||
{
|
||||
Path = relativePath,
|
||||
Operation = PackEntryOperation.Create,
|
||||
Compression = PackCompression.None,
|
||||
Attributes = (uint)fileInfo.Attributes,
|
||||
Timestamp = fileInfo.LastWriteTimeUtc.Ticks,
|
||||
UncompressedSize = (ulong)fileInfo.Length,
|
||||
CompressedSize = (ulong)fileInfo.Length,
|
||||
};
|
||||
|
||||
// Compute CRC32 of the file
|
||||
entryHeader.Checksum = await ComputeFileCrc32Async(filePath, ct);
|
||||
|
||||
OnEntryStarted?.Invoke(entryHeader);
|
||||
|
||||
// Write entry header, tracking data CRC
|
||||
var headerStartPos = output.Position;
|
||||
await PackBinaryWriter.WriteEntryHeaderAsync(output, entryHeader, ct);
|
||||
var headerBytes = await ReadStreamRangeAsync(output, headerStartPos, output.Position, ct);
|
||||
dataCrc = Crc32Algorithm.Append(dataCrc, headerBytes);
|
||||
|
||||
// Write file data
|
||||
await using (var fileStream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.Read))
|
||||
{
|
||||
var buffer = new byte[BufferSize];
|
||||
int bytesRead;
|
||||
|
||||
while ((bytesRead = await fileStream.ReadAsync(buffer, ct)) > 0)
|
||||
{
|
||||
ct.ThrowIfCancellationRequested();
|
||||
await output.WriteAsync(buffer.AsMemory(0, bytesRead), ct);
|
||||
dataCrc = Crc32Algorithm.Append(dataCrc, buffer, 0, bytesRead);
|
||||
bytesWritten += bytesRead;
|
||||
OnProgress?.Invoke(bytesWritten, totalBytes);
|
||||
}
|
||||
}
|
||||
|
||||
OnEntryCompleted?.Invoke(entryHeader);
|
||||
|
||||
directoryEntries.Add(new PackDirectoryEntry
|
||||
{
|
||||
Path = relativePath,
|
||||
Operation = PackEntryOperation.Create,
|
||||
Offset = entryOffset,
|
||||
UncompressedSize = entryHeader.UncompressedSize,
|
||||
CompressedSize = entryHeader.CompressedSize,
|
||||
Checksum = entryHeader.Checksum,
|
||||
});
|
||||
}
|
||||
|
||||
// Write directory and footer
|
||||
var footer = new PackFooter
|
||||
{
|
||||
EntryCount = (ulong)files.Length,
|
||||
DataChecksum = dataCrc,
|
||||
};
|
||||
|
||||
if (options.WriteDirectory)
|
||||
{
|
||||
footer.DirectoryOffset = (ulong)output.Position;
|
||||
footer.DirectoryChecksum = await PackBinaryWriter.WriteDirectoryAsync(output, directoryEntries, ct);
|
||||
}
|
||||
|
||||
await PackBinaryWriter.WriteFooterAsync(output, footer, ct);
|
||||
|
||||
logger.LogInformation("Packed {Count} files from {Directory}", files.Length, sourceDirectory);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Packs a source directory into chunked files. Returns the list of chunk file paths.
|
||||
/// </summary>
|
||||
public async Task<IReadOnlyList<string>> PackChunkedAsync(
|
||||
string sourceDirectory,
|
||||
string outputDirectory,
|
||||
string baseName,
|
||||
PackOptions options,
|
||||
long maxChunkSize = 4L * 1024 * 1024 * 1024 - 1,
|
||||
CancellationToken ct = default)
|
||||
{
|
||||
// First, pack into a single temp file
|
||||
var tempPath = Path.Combine(outputDirectory, $"{baseName}.lcp.tmp");
|
||||
await using (var tempStream = new FileStream(tempPath, FileMode.Create, FileAccess.ReadWrite, FileShare.None))
|
||||
{
|
||||
await PackAsync(sourceDirectory, tempStream, options, ct);
|
||||
}
|
||||
|
||||
var tempFileInfo = new FileInfo(tempPath);
|
||||
|
||||
// If it fits in one chunk, just rename
|
||||
if (tempFileInfo.Length <= maxChunkSize)
|
||||
{
|
||||
var finalPath = Path.Combine(outputDirectory, $"{baseName}.lcp");
|
||||
|
||||
if (File.Exists(finalPath))
|
||||
File.Delete(finalPath);
|
||||
|
||||
File.Move(tempPath, finalPath);
|
||||
return [finalPath];
|
||||
}
|
||||
|
||||
// Split into chunks
|
||||
var chunkPaths = new List<string>();
|
||||
|
||||
await using var sourceStream = new FileStream(tempPath, FileMode.Open, FileAccess.Read, FileShare.Read);
|
||||
|
||||
var headerBytes = new byte[PackHeader.HeaderSize];
|
||||
await sourceStream.ReadAsync(headerBytes.AsMemory(), ct);
|
||||
sourceStream.Seek(0, SeekOrigin.Begin);
|
||||
|
||||
var parentCrc = Crc32Algorithm.Compute(headerBytes);
|
||||
var chunkIndex = 0u;
|
||||
var totalChunks = (uint)Math.Ceiling((double)tempFileInfo.Length / maxChunkSize);
|
||||
|
||||
while (sourceStream.Position < sourceStream.Length)
|
||||
{
|
||||
ct.ThrowIfCancellationRequested();
|
||||
|
||||
var chunkFileName = chunkIndex == 0
|
||||
? $"{baseName}.lcp"
|
||||
: $"{baseName}.lcp.{chunkIndex:D3}";
|
||||
var chunkPath = Path.Combine(outputDirectory, chunkFileName);
|
||||
chunkPaths.Add(chunkPath);
|
||||
|
||||
await using var chunkStream = new FileStream(chunkPath, FileMode.Create, FileAccess.Write, FileShare.None);
|
||||
|
||||
long chunkBytesAvailable = maxChunkSize;
|
||||
|
||||
// Write chunk header for non-first chunks
|
||||
if (chunkIndex > 0)
|
||||
{
|
||||
var chunkHeader = new PackChunkHeader
|
||||
{
|
||||
ChunkIndex = chunkIndex,
|
||||
TotalChunks = totalChunks,
|
||||
ParentCrc = parentCrc,
|
||||
};
|
||||
|
||||
await PackBinaryWriter.WriteChunkHeaderAsync(chunkStream, chunkHeader, ct);
|
||||
chunkBytesAvailable -= PackChunkHeader.ChunkHeaderSize;
|
||||
}
|
||||
|
||||
// Copy data from source to chunk
|
||||
var buffer = new byte[BufferSize];
|
||||
|
||||
while (chunkBytesAvailable > 0 && sourceStream.Position < sourceStream.Length)
|
||||
{
|
||||
var toRead = (int)Math.Min(chunkBytesAvailable, buffer.Length);
|
||||
var bytesRead = await sourceStream.ReadAsync(buffer.AsMemory(0, toRead), ct);
|
||||
|
||||
if (bytesRead == 0)
|
||||
break;
|
||||
|
||||
await chunkStream.WriteAsync(buffer.AsMemory(0, bytesRead), ct);
|
||||
chunkBytesAvailable -= bytesRead;
|
||||
}
|
||||
|
||||
chunkIndex++;
|
||||
}
|
||||
|
||||
// Clean up temp file
|
||||
File.Delete(tempPath);
|
||||
|
||||
logger.LogInformation("Split pack into {Count} chunks in {Directory}", chunkPaths.Count, outputDirectory);
|
||||
return chunkPaths;
|
||||
}
|
||||
|
||||
#endregion
|
||||
|
||||
#region Unpacking
|
||||
|
||||
/// <summary>
|
||||
/// Unpacks a pack stream to a destination directory. Handles Create, Modify, and Delete operations.
|
||||
/// Works on forward-only (non-seekable) streams.
|
||||
/// </summary>
|
||||
public async Task<ExtractionResult> UnpackAsync(
|
||||
Stream input,
|
||||
string destinationDirectory,
|
||||
PackExtractionOptions options,
|
||||
CancellationToken ct = default)
|
||||
{
|
||||
var result = new ExtractionResult
|
||||
{
|
||||
Directory = destinationDirectory,
|
||||
};
|
||||
|
||||
try
|
||||
{
|
||||
Directory.CreateDirectory(destinationDirectory);
|
||||
|
||||
var header = await PackBinaryReader.ReadHeaderAsync(input, ct);
|
||||
long totalBytesRead = PackHeader.HeaderSize;
|
||||
|
||||
for (ulong i = 0; i < header.EntryCount; i++)
|
||||
{
|
||||
ct.ThrowIfCancellationRequested();
|
||||
|
||||
var entryHeader = await PackBinaryReader.ReadEntryHeaderAsync(input, ct);
|
||||
OnEntryStarted?.Invoke(entryHeader);
|
||||
|
||||
if (entryHeader.Operation == PackEntryOperation.Delete)
|
||||
{
|
||||
var deletePath = Path.Combine(destinationDirectory, entryHeader.Path.Replace('/', Path.DirectorySeparatorChar));
|
||||
|
||||
if (File.Exists(deletePath))
|
||||
{
|
||||
File.Delete(deletePath);
|
||||
logger.LogDebug("Deleted {Path}", entryHeader.Path);
|
||||
}
|
||||
|
||||
OnEntryCompleted?.Invoke(entryHeader);
|
||||
continue;
|
||||
}
|
||||
|
||||
var localPath = Path.Combine(destinationDirectory, entryHeader.Path.Replace('/', Path.DirectorySeparatorChar));
|
||||
var localDir = Path.GetDirectoryName(localPath);
|
||||
|
||||
if (localDir != null)
|
||||
Directory.CreateDirectory(localDir);
|
||||
|
||||
// Skip unchanged files if option is set
|
||||
if (options.SkipUnchangedFiles && File.Exists(localPath))
|
||||
{
|
||||
var localCrc = await ComputeFileCrc32Async(localPath, ct);
|
||||
|
||||
if (localCrc == entryHeader.Checksum)
|
||||
{
|
||||
// Skip the data in the stream
|
||||
await SkipBytesAsync(input, (long)entryHeader.CompressedSize, ct);
|
||||
totalBytesRead += (long)entryHeader.CompressedSize;
|
||||
|
||||
result.Files.Add(new ExtractionResult.FileEntry
|
||||
{
|
||||
EntryPath = entryHeader.Path,
|
||||
LocalPath = localPath,
|
||||
});
|
||||
|
||||
logger.LogDebug("Skipped unchanged file {Path}", entryHeader.Path);
|
||||
OnEntryCompleted?.Invoke(entryHeader);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
// Extract file
|
||||
uint crc = 0;
|
||||
|
||||
await using (var fileStream = new FileStream(localPath, FileMode.Create, FileAccess.Write, FileShare.None))
|
||||
{
|
||||
var buffer = new byte[BufferSize];
|
||||
var remaining = (long)entryHeader.CompressedSize;
|
||||
|
||||
while (remaining > 0)
|
||||
{
|
||||
ct.ThrowIfCancellationRequested();
|
||||
var toRead = (int)Math.Min(remaining, buffer.Length);
|
||||
var bytesRead = await input.ReadAsync(buffer.AsMemory(0, toRead), ct);
|
||||
|
||||
if (bytesRead == 0)
|
||||
throw new EndOfStreamException($"Unexpected end of stream while extracting {entryHeader.Path}");
|
||||
|
||||
await fileStream.WriteAsync(buffer.AsMemory(0, bytesRead), ct);
|
||||
crc = Crc32Algorithm.Append(crc, buffer, 0, bytesRead);
|
||||
remaining -= bytesRead;
|
||||
totalBytesRead += bytesRead;
|
||||
OnProgress?.Invoke(totalBytesRead, input.CanSeek ? input.Length : 0);
|
||||
}
|
||||
}
|
||||
|
||||
// Verify checksum
|
||||
if (options.VerifyChecksums && crc != entryHeader.Checksum)
|
||||
{
|
||||
logger.LogWarning("Checksum mismatch for {Path}: expected {Expected:X8}, got {Actual:X8}",
|
||||
entryHeader.Path, entryHeader.Checksum, crc);
|
||||
}
|
||||
|
||||
// Preserve timestamps
|
||||
if (options.PreserveTimestamps && entryHeader.Timestamp != 0)
|
||||
{
|
||||
var timestamp = new DateTime(entryHeader.Timestamp, DateTimeKind.Utc);
|
||||
File.SetLastWriteTimeUtc(localPath, timestamp);
|
||||
}
|
||||
|
||||
result.Files.Add(new ExtractionResult.FileEntry
|
||||
{
|
||||
EntryPath = entryHeader.Path,
|
||||
LocalPath = localPath,
|
||||
});
|
||||
|
||||
logger.LogDebug("Extracted {Path} ({Size} bytes)", entryHeader.Path, entryHeader.UncompressedSize);
|
||||
OnEntryCompleted?.Invoke(entryHeader);
|
||||
}
|
||||
|
||||
result.Success = true;
|
||||
}
|
||||
catch (OperationCanceledException)
|
||||
{
|
||||
result.Canceled = true;
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
logger.LogError(ex, "Failed to unpack to {Directory}", destinationDirectory);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Unpacks chunked pack files to a destination directory.
|
||||
/// </summary>
|
||||
public async Task<ExtractionResult> UnpackChunkedAsync(
|
||||
IReadOnlyList<string> chunkPaths,
|
||||
string destinationDirectory,
|
||||
PackExtractionOptions options,
|
||||
CancellationToken ct = default)
|
||||
{
|
||||
// First chunk has no extra header; subsequent chunks have a ChunkHeaderSize to skip
|
||||
var skipBytes = new long[chunkPaths.Count];
|
||||
|
||||
for (int i = 1; i < chunkPaths.Count; i++)
|
||||
skipBytes[i] = PackChunkHeader.ChunkHeaderSize;
|
||||
|
||||
await using var stream = new ConcatenatingStream(chunkPaths, skipBytes);
|
||||
return await UnpackAsync(stream, destinationDirectory, options, ct);
|
||||
}
|
||||
|
||||
#endregion
|
||||
|
||||
#region Directory Browsing
|
||||
|
||||
/// <summary>
|
||||
/// Reads the directory and footer from a seekable stream. Returns the full pack manifest.
|
||||
/// </summary>
|
||||
public async Task<PackManifest> ReadDirectoryAsync(Stream seekableStream, CancellationToken ct = default)
|
||||
{
|
||||
if (!seekableStream.CanSeek)
|
||||
throw new NotSupportedException("ReadDirectoryAsync requires a seekable stream.");
|
||||
|
||||
var footer = await PackBinaryReader.ReadFooterAsync(seekableStream, ct);
|
||||
|
||||
// Read header
|
||||
seekableStream.Seek(0, SeekOrigin.Begin);
|
||||
var header = await PackBinaryReader.ReadHeaderAsync(seekableStream, ct);
|
||||
|
||||
var entries = new List<PackDirectoryEntry>();
|
||||
|
||||
if (footer.DirectoryOffset > 0)
|
||||
entries = await PackBinaryReader.ReadDirectoryEntriesAsync(seekableStream, footer, ct);
|
||||
|
||||
return new PackManifest
|
||||
{
|
||||
Header = header,
|
||||
Entries = entries,
|
||||
Footer = footer,
|
||||
};
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Reads the directory from a pack file path.
|
||||
/// </summary>
|
||||
public async Task<PackManifest> ReadDirectoryAsync(string packFilePath, CancellationToken ct = default)
|
||||
{
|
||||
await using var stream = new FileStream(packFilePath, FileMode.Open, FileAccess.Read, FileShare.Read);
|
||||
return await ReadDirectoryAsync(stream, ct);
|
||||
}
|
||||
|
||||
#endregion
|
||||
|
||||
#region Verification
|
||||
|
||||
/// <summary>
|
||||
/// Verifies all entry checksums by reading the pack sequentially.
|
||||
/// Requires a seekable stream.
|
||||
/// </summary>
|
||||
public async Task<PackVerificationResult> VerifyAsync(Stream seekableStream, CancellationToken ct = default)
|
||||
{
|
||||
if (!seekableStream.CanSeek)
|
||||
throw new NotSupportedException("VerifyAsync requires a seekable stream.");
|
||||
|
||||
var result = new PackVerificationResult { IsValid = true };
|
||||
|
||||
seekableStream.Seek(0, SeekOrigin.Begin);
|
||||
var header = await PackBinaryReader.ReadHeaderAsync(seekableStream, ct);
|
||||
|
||||
for (ulong i = 0; i < header.EntryCount; i++)
|
||||
{
|
||||
ct.ThrowIfCancellationRequested();
|
||||
|
||||
var entryHeader = await PackBinaryReader.ReadEntryHeaderAsync(seekableStream, ct);
|
||||
|
||||
if (entryHeader.Operation == PackEntryOperation.Delete)
|
||||
continue;
|
||||
|
||||
uint crc = 0;
|
||||
var buffer = new byte[BufferSize];
|
||||
var remaining = (long)entryHeader.CompressedSize;
|
||||
|
||||
while (remaining > 0)
|
||||
{
|
||||
var toRead = (int)Math.Min(remaining, buffer.Length);
|
||||
var bytesRead = await seekableStream.ReadAsync(buffer.AsMemory(0, toRead), ct);
|
||||
|
||||
if (bytesRead == 0)
|
||||
break;
|
||||
|
||||
crc = Crc32Algorithm.Append(crc, buffer, 0, bytesRead);
|
||||
remaining -= bytesRead;
|
||||
}
|
||||
|
||||
if (crc != entryHeader.Checksum)
|
||||
{
|
||||
result.IsValid = false;
|
||||
result.Failures.Add(new PackVerificationFailure
|
||||
{
|
||||
Path = entryHeader.Path,
|
||||
ExpectedChecksum = entryHeader.Checksum,
|
||||
ActualChecksum = crc,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Verify data section checksum against footer
|
||||
var footer = await PackBinaryReader.ReadFooterAsync(seekableStream, ct);
|
||||
var dataEnd = footer.DirectoryOffset > 0 ? (long)footer.DirectoryOffset : seekableStream.Length - PackFooter.FooterSize;
|
||||
var dataCrc = await PackBinaryReader.ComputeStreamCrc32Async(seekableStream, PackHeader.HeaderSize, dataEnd, ct);
|
||||
|
||||
if (dataCrc != footer.DataChecksum)
|
||||
{
|
||||
result.IsValid = false;
|
||||
logger.LogWarning("Data section checksum mismatch: expected {Expected:X8}, got {Actual:X8}",
|
||||
footer.DataChecksum, dataCrc);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
#endregion
|
||||
|
||||
#region Diffing and Patching
|
||||
|
||||
/// <summary>
|
||||
/// Compares two pack manifests and returns a manifest representing the diff.
|
||||
/// The returned manifest contains entries with appropriate operations (Create, Modify, Delete).
|
||||
/// </summary>
|
||||
public PackManifest ComputeDiff(PackManifest oldPack, PackManifest newPack)
|
||||
{
|
||||
var oldEntries = oldPack.Entries.ToDictionary(e => e.Path, StringComparer.OrdinalIgnoreCase);
|
||||
var newEntries = newPack.Entries.ToDictionary(e => e.Path, StringComparer.OrdinalIgnoreCase);
|
||||
|
||||
var diffEntries = new List<PackDirectoryEntry>();
|
||||
|
||||
// Find added and modified entries
|
||||
foreach (var newEntry in newPack.Entries)
|
||||
{
|
||||
if (!oldEntries.TryGetValue(newEntry.Path, out var oldEntry))
|
||||
{
|
||||
// New file
|
||||
diffEntries.Add(new PackDirectoryEntry
|
||||
{
|
||||
Path = newEntry.Path,
|
||||
Operation = PackEntryOperation.Create,
|
||||
Offset = newEntry.Offset,
|
||||
UncompressedSize = newEntry.UncompressedSize,
|
||||
CompressedSize = newEntry.CompressedSize,
|
||||
Checksum = newEntry.Checksum,
|
||||
});
|
||||
}
|
||||
else if (oldEntry.Checksum != newEntry.Checksum)
|
||||
{
|
||||
// Modified file
|
||||
diffEntries.Add(new PackDirectoryEntry
|
||||
{
|
||||
Path = newEntry.Path,
|
||||
Operation = PackEntryOperation.Modify,
|
||||
Offset = newEntry.Offset,
|
||||
UncompressedSize = newEntry.UncompressedSize,
|
||||
CompressedSize = newEntry.CompressedSize,
|
||||
Checksum = newEntry.Checksum,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Find deleted entries
|
||||
foreach (var oldEntry in oldPack.Entries)
|
||||
{
|
||||
if (!newEntries.ContainsKey(oldEntry.Path))
|
||||
{
|
||||
diffEntries.Add(new PackDirectoryEntry
|
||||
{
|
||||
Path = oldEntry.Path,
|
||||
Operation = PackEntryOperation.Delete,
|
||||
Offset = 0,
|
||||
UncompressedSize = 0,
|
||||
CompressedSize = 0,
|
||||
Checksum = 0,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return new PackManifest
|
||||
{
|
||||
Header = new PackHeader
|
||||
{
|
||||
Version = 2,
|
||||
Flags = PackFlags.HasDirectory,
|
||||
EntryCount = (ulong)diffEntries.Count,
|
||||
},
|
||||
Entries = diffEntries,
|
||||
Footer = new PackFooter
|
||||
{
|
||||
EntryCount = (ulong)diffEntries.Count,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Creates a patch pack from a diff manifest and the new pack stream.
|
||||
/// Seeks into the new pack to extract only changed/added entries, and writes
|
||||
/// Delete entries as header-only (no data).
|
||||
/// </summary>
|
||||
public async Task CreatePatchAsync(
|
||||
PackManifest diff,
|
||||
Stream newPackStream,
|
||||
Stream patchOutput,
|
||||
PackOptions options,
|
||||
CancellationToken ct = default)
|
||||
{
|
||||
if (!newPackStream.CanSeek)
|
||||
throw new NotSupportedException("CreatePatchAsync requires a seekable new pack stream.");
|
||||
|
||||
var header = new PackHeader
|
||||
{
|
||||
Version = 2,
|
||||
Flags = options.WriteDirectory ? PackFlags.HasDirectory : PackFlags.None,
|
||||
EntryCount = (ulong)diff.Entries.Count,
|
||||
PackId = options.PackId,
|
||||
ParentPackId = options.ParentPackId,
|
||||
PackVersion = options.PackVersion,
|
||||
ParentVersion = options.ParentVersion,
|
||||
};
|
||||
|
||||
await PackBinaryWriter.WriteHeaderAsync(patchOutput, header, ct);
|
||||
|
||||
var patchDirectoryEntries = new List<PackDirectoryEntry>();
|
||||
uint dataCrc = 0;
|
||||
|
||||
foreach (var entry in diff.Entries)
|
||||
{
|
||||
ct.ThrowIfCancellationRequested();
|
||||
|
||||
var patchOffset = (ulong)patchOutput.Position;
|
||||
|
||||
if (entry.Operation == PackEntryOperation.Delete)
|
||||
{
|
||||
var deleteHeader = new PackEntryHeader
|
||||
{
|
||||
Path = entry.Path,
|
||||
Operation = PackEntryOperation.Delete,
|
||||
Compression = PackCompression.None,
|
||||
Attributes = 0,
|
||||
Timestamp = 0,
|
||||
UncompressedSize = 0,
|
||||
CompressedSize = 0,
|
||||
Checksum = 0,
|
||||
};
|
||||
|
||||
var headerStartPos = patchOutput.Position;
|
||||
await PackBinaryWriter.WriteEntryHeaderAsync(patchOutput, deleteHeader, ct);
|
||||
var headerBytes = await ReadStreamRangeAsync(patchOutput, headerStartPos, patchOutput.Position, ct);
|
||||
dataCrc = Crc32Algorithm.Append(dataCrc, headerBytes);
|
||||
|
||||
patchDirectoryEntries.Add(new PackDirectoryEntry
|
||||
{
|
||||
Path = entry.Path,
|
||||
Operation = PackEntryOperation.Delete,
|
||||
Offset = patchOffset,
|
||||
});
|
||||
}
|
||||
else
|
||||
{
|
||||
// Seek to the entry in the new pack and copy its header + data
|
||||
newPackStream.Seek((long)entry.Offset, SeekOrigin.Begin);
|
||||
var sourceEntry = await PackBinaryReader.ReadEntryHeaderAsync(newPackStream, ct);
|
||||
|
||||
var entryHeader = new PackEntryHeader
|
||||
{
|
||||
Path = sourceEntry.Path,
|
||||
Operation = entry.Operation,
|
||||
Compression = sourceEntry.Compression,
|
||||
Attributes = sourceEntry.Attributes,
|
||||
Timestamp = sourceEntry.Timestamp,
|
||||
UncompressedSize = sourceEntry.UncompressedSize,
|
||||
CompressedSize = sourceEntry.CompressedSize,
|
||||
Checksum = sourceEntry.Checksum,
|
||||
};
|
||||
|
||||
OnEntryStarted?.Invoke(entryHeader);
|
||||
|
||||
var headerStartPos = patchOutput.Position;
|
||||
await PackBinaryWriter.WriteEntryHeaderAsync(patchOutput, entryHeader, ct);
|
||||
var headerBytes = await ReadStreamRangeAsync(patchOutput, headerStartPos, patchOutput.Position, ct);
|
||||
dataCrc = Crc32Algorithm.Append(dataCrc, headerBytes);
|
||||
|
||||
// Copy file data
|
||||
var buffer = new byte[BufferSize];
|
||||
var remaining = (long)sourceEntry.CompressedSize;
|
||||
|
||||
while (remaining > 0)
|
||||
{
|
||||
var toRead = (int)Math.Min(remaining, buffer.Length);
|
||||
var bytesRead = await newPackStream.ReadAsync(buffer.AsMemory(0, toRead), ct);
|
||||
|
||||
if (bytesRead == 0)
|
||||
throw new EndOfStreamException($"Unexpected end of stream while reading {sourceEntry.Path}");
|
||||
|
||||
await patchOutput.WriteAsync(buffer.AsMemory(0, bytesRead), ct);
|
||||
dataCrc = Crc32Algorithm.Append(dataCrc, buffer, 0, bytesRead);
|
||||
remaining -= bytesRead;
|
||||
}
|
||||
|
||||
OnEntryCompleted?.Invoke(entryHeader);
|
||||
|
||||
patchDirectoryEntries.Add(new PackDirectoryEntry
|
||||
{
|
||||
Path = entry.Path,
|
||||
Operation = entry.Operation,
|
||||
Offset = patchOffset,
|
||||
UncompressedSize = sourceEntry.UncompressedSize,
|
||||
CompressedSize = sourceEntry.CompressedSize,
|
||||
Checksum = sourceEntry.Checksum,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
var footer = new PackFooter
|
||||
{
|
||||
EntryCount = (ulong)diff.Entries.Count,
|
||||
DataChecksum = dataCrc,
|
||||
};
|
||||
|
||||
if (options.WriteDirectory)
|
||||
{
|
||||
footer.DirectoryOffset = (ulong)patchOutput.Position;
|
||||
footer.DirectoryChecksum = await PackBinaryWriter.WriteDirectoryAsync(patchOutput, patchDirectoryEntries, ct);
|
||||
}
|
||||
|
||||
await PackBinaryWriter.WriteFooterAsync(patchOutput, footer, ct);
|
||||
|
||||
logger.LogInformation("Created patch with {Count} entries", diff.Entries.Count);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Applies a patch pack to an existing installation directory.
|
||||
/// This is identical to UnpackAsync since the entry operations drive the behavior.
|
||||
/// </summary>
|
||||
public async Task<ExtractionResult> ApplyPatchAsync(
|
||||
Stream patchStream,
|
||||
string installDirectory,
|
||||
PackExtractionOptions options,
|
||||
CancellationToken ct = default)
|
||||
{
|
||||
return await UnpackAsync(patchStream, installDirectory, options, ct);
|
||||
}
|
||||
|
||||
#endregion
|
||||
|
||||
#region Helpers
|
||||
|
||||
private static async Task<uint> ComputeFileCrc32Async(string filePath, CancellationToken ct = default)
|
||||
{
|
||||
uint crc = 0;
|
||||
var buffer = new byte[BufferSize];
|
||||
|
||||
await using var stream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.Read);
|
||||
|
||||
int bytesRead;
|
||||
while ((bytesRead = await stream.ReadAsync(buffer, ct)) > 0)
|
||||
{
|
||||
crc = Crc32Algorithm.Append(crc, buffer, 0, bytesRead);
|
||||
}
|
||||
|
||||
return crc;
|
||||
}
|
||||
|
||||
private static async Task SkipBytesAsync(Stream stream, long count, CancellationToken ct = default)
|
||||
{
|
||||
if (stream.CanSeek)
|
||||
{
|
||||
stream.Seek(count, SeekOrigin.Current);
|
||||
return;
|
||||
}
|
||||
|
||||
var buffer = new byte[BufferSize];
|
||||
var remaining = count;
|
||||
|
||||
while (remaining > 0)
|
||||
{
|
||||
var toRead = (int)Math.Min(remaining, buffer.Length);
|
||||
var bytesRead = await stream.ReadAsync(buffer.AsMemory(0, toRead), ct);
|
||||
|
||||
if (bytesRead == 0)
|
||||
break;
|
||||
|
||||
remaining -= bytesRead;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Reads a range of bytes from a seekable stream. Used to compute CRC of written data.
|
||||
/// </summary>
|
||||
private static async Task<byte[]> ReadStreamRangeAsync(Stream stream, long start, long end, CancellationToken ct = default)
|
||||
{
|
||||
if (!stream.CanSeek)
|
||||
return [];
|
||||
|
||||
var currentPos = stream.Position;
|
||||
stream.Seek(start, SeekOrigin.Begin);
|
||||
|
||||
var length = (int)(end - start);
|
||||
var buffer = new byte[length];
|
||||
var totalRead = 0;
|
||||
|
||||
while (totalRead < length)
|
||||
{
|
||||
var bytesRead = await stream.ReadAsync(buffer.AsMemory(totalRead), ct);
|
||||
|
||||
if (bytesRead == 0)
|
||||
break;
|
||||
|
||||
totalRead += bytesRead;
|
||||
}
|
||||
|
||||
stream.Seek(currentPos, SeekOrigin.Begin);
|
||||
return buffer;
|
||||
}
|
||||
|
||||
#endregion
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue