feat: add Cyclopedia protobuf export pipeline (#160)

Add editor support for exporting Cyclopedia and CipSoft protobuf assets from the currently loaded map.

Main changes:
- Add File > Client Assets actions for Static House Data export, Cyclopedia minimap and satellite export, and restoring timestamped client asset backups.
- Keep generic minimap and tileset exports under File > Export.
- Export staticdata, staticmapdata, and mapdata protobuf files with content-hashed filenames.
- Update catalog-content.json so generated assets are picked up by the client.
- Merge compatible CipSoft templates when they match the loaded map.
- Fall back to generated data for custom maps instead of failing the export.
- Support filtered house exports.
- Report warnings for missing or failed static map entries.
- Emit aligned MINIMAP and sprite-based SATELLITE assets for the documented scales, including the 1/16 layer.
- Keep Cyclopedia export progress in the status bar instead of using a blocking modal progress dialog.

Performance:
- Precompute floor and chunk plans for global map exports.
- Write BMP output directly.
- Reuse minimap renders for matching satellite assets.
- Parallelize asset hash and LZMA encoding through a bounded worker queue.
- Remove modal progress overhead from the export path.
- Revert the tile-grid satellite optimization after profiling showed it shifted cost without improving total export time.

Documentation:
- Document the export contract in docs/static-data.md.
- Add docs/cyclopedia-export-roadmap.md with deferred optimization candidates and follow-up notes.

Scope:
- Focus this change on the Cyclopedia/staticdata protobuf export path and the export-specific performance work needed for practical global map exports.
- Move unrelated runtime, performance, and UI experiments out of this branch.

Validation:
- Ran static inspection and git diff checks during the export changes.
- Reviewed Visual Studio CPU profiles after each global map export performance pass.
- Verified hash and LZMA work moved into bounded async workers.
- Verified the restore action is labeled as restore and grouped under File > Client Assets, not File > Export.

This adds the editor-side pipeline needed to generate client Cyclopedia assets from a loaded map while keeping the export flow recoverable, documented, and practical for large maps.
This commit is contained in:
Eduardo Dantas 2026-05-28 08:35:21 -03:00 committed by GitHub
parent ce8d5093ee
commit d78a2546c7
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
21 changed files with 6238 additions and 15 deletions

8
.gitignore vendored
View file

@ -17,6 +17,14 @@ data/user
world
source/lua/FastNoiseLite.h
# SQLite runtime files
*.db-shm
*.db-wal
*.db-journal
*.sqlite-shm
*.sqlite-wal
*.sqlite-journal
# Visual Studio
*.MAP
*.user

View file

@ -36,6 +36,17 @@
- Prefer forward declarations in headers when possible. Do not move heavy includes into public headers unless the type definition is required there.
- For hot-path or broad runtime changes, account for compile-time impact: do not scatter expensive includes across many translation units without updating the precompiled header strategy.
## Cyclopedia Export Contract Gate
- Before changing Cyclopedia static data, `staticmapdata`, `mapdata`, minimap/satellite asset generation, catalog backup/restore behavior, or the related protobuf schemas, read `docs/static-data.md`.
- Preserve the CipSoft-compatible contract unless the user explicitly asks to change it:
- hash-named `staticdata`, `staticmapdata`, and `map` files must match their content and stay referenced by `catalog-content.json`.
- `staticmapdata` house previews must preserve compatible template `origin`, `dimensions`, linear `skip + 1` semantics, and item draw order.
- `mapdata` must preserve compatible template `SUBAREA` assets and emit aligned `MINIMAP` and `SATELLITE` assets for the documented scales, including the `1/16` layer.
- Surface View must use `SATELLITE` assets rendered from real sprite ids, including ground, border, and item sprites in draw order; minimap colors are fallback only.
- Do not replace sprite ids with GL texture/atlas ids in export paths. `GameSprite::getSpriteID(...)` is the asset identity path; `GameSprite::getHardwareID(...)` is for rendering textures.
- If a Cyclopedia export contract change is intentional, update `docs/static-data.md` in the same change and include a non-build validation rationale. Do not rely on visual assumptions alone.
## PR Communication Policy
- Do not post any PR comments/reviews automatically.

View file

@ -7,6 +7,7 @@ You can find the project for hosting your own server at [Canary](https://github.
Getting Started
=========
* [Wiki](https://github.com/opentibiabr/remeres-map-editor/wiki).
* [Cyclopedia export documentation](docs/static-data.md).
I want to contribute
====================

View file

@ -19,6 +19,12 @@
<item name="$Export Minimap..." action="EXPORT_MINIMAP" help="Export minimap to an image file."/>
<item name="$Export Tilesets..." action="EXPORT_TILESETS" help="Export tilesets to an xml file."/>
</menu>
<menu name="$Client Assets">
<item name="Export $Static House Data..." action="EXPORT_STATIC_HOUSE_DATA" help="Export staticdata and staticmapdata protobuf assets."/>
<item name="Export $Cyclopedia Minimap/Satellite..." action="EXPORT_CYCLOPEDIA_MAP" help="Export cyclopedia mapdata, minimap, and satellite assets."/>
<separator/>
<item name="$Restore Client Assets Backup..." action="REVERT_CYCLOPEDIA_ASSETS" help="Restore a timestamped client assets backup snapshot."/>
</menu>
<menu name="$Reload">
<item name="$Reload" hotkey="F5" action="RELOAD_DATA" help="Reloads all data files."/>
</menu>

View file

@ -0,0 +1,109 @@
# Cyclopedia Export Performance Roadmap
This document records follow-up optimization candidates for the Cyclopedia map export after the initial static data compatibility and export performance work.
The current implementation intentionally keeps the exported data contract stable. Follow-up work must preserve the rules documented in [static-data.md](static-data.md), especially the sprite-based `SATELLITE` assets, aligned `MINIMAP`/`SATELLITE` layers, hash-named files, and deterministic `mapdata` references.
## Completed in This PR
- Removed the modal export progress dialog from the Cyclopedia map export path and kept progress reporting in the status bar, so the editor remains usable while the export runs.
- Replaced repeated full-map floor/chunk scans with a precomputed Cyclopedia floor plan.
- Reduced BMP encoding overhead by writing directly into a pre-sized BMP buffer.
- Reused rendered minimap chunks when building the matching satellite chunk for the same area.
- Added a bounded asynchronous asset encoding pipeline so BMP hashing and CIP LZMA compression can overlap with render work.
- Kept async workers isolated from `Map`, `wxImage`, protobuf objects, and UI state. Workers only receive owned BMP byte buffers, preserving thread-safety and deterministic asset insertion order.
## Deferred Optimization Candidates
### 1. Satellite Sprite Sampling Cache
Profile data after the current work shows `buildCyclopediaSatelliteChunk`, `getSampledSpriteForSpriteId`, `getTinySpriteForSpriteId`, sprite sheet lookup, and sprite image loading as the next major export costs.
Potential approach:
- cache resolved sprite sample metadata more aggressively inside the export run;
- avoid repeated `GameSprite::getDrawOffset()` and pattern/subtype resolution when the same item sprite is sampled many times;
- keep cache keys based on exported sprite identity from `GameSprite::getSpriteID(...)`, not GL texture or atlas ids.
Why it was deferred:
- this path is part of the visual compatibility contract for `SATELLITE` assets;
- caching the wrong identity or pattern can produce visually plausible but incorrect CipSoft-like output;
- the change needs focused visual validation against real exported assets.
### 2. Satellite Render Loop Restructure
The satellite renderer still composes sprite samples in the main export flow. The inner loop draws ground, border, and item sprites in tile draw order while also sampling neighboring tile footprints.
Potential approach:
- precompute per-tile draw item lists for a chunk;
- precompute source tile positions needed by the 2x2 sampling window;
- reduce repeated vector clearing, item filtering, and per-source position construction inside the hot loop.
Why it was deferred:
- a simple tile pointer grid was profiled and did not improve the total export cost enough to justify the added code and temporary memory;
- a larger loop rewrite has a higher regression risk because it touches draw order and footprint sampling;
- the current implementation is slower, but its behavior is easier to reason about and matches the documented contract.
### 3. Dedicated Export Snapshot for Parallel Rendering
The current async work avoids reading live editor structures from worker threads. Rendering still uses the live `Map` and sprite APIs on the main export path.
Potential approach:
- build a compact, immutable export snapshot for the needed floors/chunks;
- include tile minimap colors, ordered exported sprite ids, pattern data, and sprite sample metadata;
- render independent chunks from that snapshot on worker threads.
Why it was deferred:
- this is a larger architectural change, not a narrow optimization;
- it must define memory bounds for global maps to avoid trading CPU time for excessive RAM use;
- it must prove thread-safety around sprite image access or copy the required sprite data into the snapshot;
- it requires broader validation because it changes where export data is read from, even if the output format remains the same.
### 4. Compression Tuning and Asset Deduplication
Hashing and LZMA compression remain visible in profiles even after being moved to bounded worker tasks.
Potential approach:
- evaluate whether repeated empty or near-empty chunks can be detected earlier;
- deduplicate identical encoded assets only if `mapdata` references remain correct;
- benchmark compression parameters against client compatibility and file size.
Why it was deferred:
- the current CIP LZMA container path is compatible and deterministic;
- changing compression behavior may affect client loading assumptions or output size expectations;
- asset deduplication could complicate backup/restore behavior and `mapdata` reference validation.
### 5. Backup and Filesystem Cost
Profiles still show smaller costs in backup and file creation paths.
Potential approach:
- reduce redundant directory creation checks;
- batch filesystem existence checks where possible;
- avoid moving unchanged generated assets when content hashes already match.
Why it was deferred:
- this is no longer the primary bottleneck after the UI and encoding changes;
- backup/restore behavior is safety-critical for client assets;
- correctness and recoverability are more important than small wins in this area.
## Validation Expectations for Future Work
Future performance work should include:
- `git diff --check` and focused static inspection when no build is requested;
- profile comparison on a global map export before and after the change;
- visual validation of `SATELLITE` Surface View output, especially ground, border, item draw order, and multi-tile sprite footprints;
- validation that `MINIMAP` and `SATELLITE` assets are still emitted for all documented scales, including `1/16`;
- validation that generated asset filenames still embed hashes matching their content.
Avoid accepting an optimization based only on code intuition. The tile-grid experiment showed that a change can reduce one visible function cost while adding equivalent overhead elsewhere.

297
docs/static-data.md Normal file
View file

@ -0,0 +1,297 @@
# Cyclopedia Static and Map Data Export
This document explains how this RME fork exports Cyclopedia house data, map data, minimap assets, and satellite assets.
For follow-up performance work that intentionally stays outside the compatibility contract, see [cyclopedia-export-roadmap.md](cyclopedia-export-roadmap.md).
Scope:
- `staticdata-<sha256>.dat`
- `staticmapdata-<sha256>.dat`
- `map-<sha256>.dat`
- `minimap-<scale>-<chunk_x>-<chunk_y>-<floor>-<sha256>.bmp.lzma`
- `satellite-<scale>-<chunk_x>-<chunk_y>-<floor>-<sha256>.bmp.lzma`
- integration through `catalog-content.json`
Main code references:
- `source/protobuf/staticdata.proto`
- `source/protobuf/staticmapdata.proto`
- `source/protobuf/mapdata.proto`
- `source/iomap_otbm.cpp`
- `IOMapOTBM::saveStaticData(...)`
- `IOMapOTBM::saveCyclopediaMapData(...)`
- `IOMapOTBM::serializeCyclopediaMapData(...)`
- `mergeStaticDataTemplate(...)`
- `buildStaticMapHouseTemplate(...)`
- `buildStaticMapHousePreviewData(...)`
- `buildStaticMapHouseTemplates(...)`
## 1. Overview
The Cyclopedia export is split into two user-facing flows:
1. Static house export writes `staticdata` and `staticmapdata`.
2. Cyclopedia map export writes `mapdata` and its referenced minimap/satellite assets.
In the client, Cyclopedia combines:
- static house data (`staticdata` + `staticmapdata`)
- map data and map assets (`mapdata` + `minimap`/`satellite`)
- dynamic state from the server, such as house auctions, status, and bids
## 2. Files and Catalog
Data files are written under the selected assets directory and referenced by `catalog-content.json`.
Catalog entries managed by the exporter:
- `type: "staticdata"` points to `staticdata-<sha256>.dat`.
- `type: "staticmapdata"` points to `staticmapdata-<sha256>.dat`.
- `type: "map"` points to `map-<sha256>.dat`.
Minimap and satellite files are not listed directly in `catalog-content.json`. They are referenced by `MapData.mapassets` inside `map-<sha256>.dat`.
Important rules:
- the hash in every hash-named data filename must match the file content.
- template files are rejected when their filename hash does not match their content.
- the default template lookup names are `staticdata.dat`, `staticmapdata.dat`, and `map.dat`, but final written files use hash-named filenames.
- existing replaced files are moved to a timestamped snapshot under `bkps` before the new file is written.
- backup snapshots are never overwritten; if two exports happen in the same second, a numeric suffix is added to the snapshot directory.
- when `mapdata` is replaced, the previous `mapdata` file and its referenced minimap/satellite assets are moved to the export snapshot.
- template `SUBAREA` assets are preserved in place because merged `mapdata` keeps referencing them.
- if a preserved `SUBAREA` asset is missing from the output assets directory, the exporter restores it from the newest matching backup snapshot or from the source client assets before writing the new `mapdata`.
- before updating `catalog-content.json`, every asset referenced by the final `mapdata` must exist in the output assets directory.
## 3. Protobuf Structure
## 3.1 staticdata
Contains per-house metadata:
- house id
- name
- city
- rent
- beds
- square meters
- flags
- anchor position
Primary uses:
- filling the Cyclopedia house list
- resolving static metadata by `houseId`
- serving as the base for id remapping during template merge
## 3.2 staticmapdata
Legacy CIP-compatible house preview format:
- `house_id`
- `data.origin (pos_x, pos_y, pos_z)`
- `data.dimensions (pos_x=width, pos_y=height, pos_z=floors)`
- `data.preview.layer.tile[]`
- `item[].value` (item client ids)
- `skip`
## 3.3 mapdata
Map data contains Cyclopedia map bounds and asset descriptors:
- `topleftedge`
- `bottomrightedge`
- `mapassets[]`
- `type` (`SUBAREA`, `MINIMAP`, or `SATELLITE`)
- `topleft`
- `filename`
- `widthsquare`
- `heightsquare`
- `scale`
The exporter scans floors `0..7` and emits minimap and satellite chunks for each floor that contains visible tile data. `SUBAREA` entries come from the template mapdata and are kept for client compatibility.
Current emitted scales:
- `1/64` with `1024` square chunks and `0.5` pixels per square.
- `1/32` with `512` square chunks and `1.0` pixel per square.
- `1/16` with `256` square chunks and `2.0` pixels per square.
The same scale set is emitted for both minimap and satellite assets.
## 4. CIP Serialization Semantics
`staticmapdata` tiles are serialized linearly inside a `width * height * floors` volume.
Decoding:
1. `linearIndex` starts at `0`.
2. each serialized entry represents the current cell.
3. next index is `linearIndex = linearIndex + skip + 1`.
Index-to-local-coordinate mapping:
- `floorArea = width * height`
- `floor = linearIndex / floorArea`
- `planeIndex = linearIndex % floorArea`
- `x = planeIndex / height`
- `y = planeIndex % height`
Absolute world coordinate:
- `worldX = origin.x + x`
- `worldY = origin.y + y`
- `worldZ = origin.z + floor`
## 5. Current RME Export Flow
Static house export:
1. load `staticdata` and `staticmapdata` templates when they are available.
2. serialize current map house data.
3. merge generated `staticdata` with the template only when at least one current-map house matches it.
4. build house preview templates from template `staticmapdata` only for compatible template exports.
5. export each house preview while preserving template framing when available, otherwise use dynamic map framing.
6. write hash-named final files and update `catalog-content.json`.
Cyclopedia map export:
1. load the existing `map` template when available.
2. scan map bounds across floors `0..7`.
3. render minimap chunks from tile minimap colors.
4. render satellite chunks for Surface View from the actual tile sprite stack, with minimap colors used only as fallback terrain.
5. write each chunk as BMP bytes inside the CIP LZMA asset container.
6. merge compatible template `mapdata` fields.
7. write hash-named `map-<sha256>.dat`, write referenced assets, and update `catalog-content.json`.
Compatibility details currently applied:
- no per-tile house/context mask is serialized in the compatibility export.
- when a valid house template is present, template framing (`origin/dimensions/skip`) is preserved.
- template item payload can be preserved to keep visual parity with the CIP client.
- dynamic fallback is used for custom maps or houses without a matching template entry.
- minimap sea/background pixels use the Cip minimap water color `(51, 102, 153)`.
- minimap downscaling uses weighted averaging to avoid checker/pixel aliasing.
- minimap upscaling stays nearest-neighbor to keep tile-aligned pixels.
- Surface View must be emitted as `SATELLITE` assets; minimap colors are only used as a fallback behind sprite pixels.
- Surface View draws the ground, border, and item sprites in tile draw order so the exported map stays close to CipSoft's surface view.
- Surface View sprites are composed on a transparent intermediate image; satellite sea color is only applied to pixels that remain empty on the ground layer.
- downscaling preserves alpha during sampling, then exported Surface View pixels that contain sprite data are resolved as opaque so the client does not tint them with the sea/background color.
## 6. How the Client Receives and Renders
Logical flow:
1. client loads `staticdata`, `staticmapdata`, and `map` from assets.
2. `mapdata` points the client to minimap and satellite chunk filenames.
3. server sends dynamic house list/state using house ids.
4. client matches dynamic `houseId` with static metadata/preview.
5. house preview is decoded via `origin + dimensions + tile/skip/item`.
6. map chunks are rendered from `MapData.mapassets` using `topleft`, `widthsquare`, `heightsquare`, and `scale`; `MINIMAP` feeds Map View and `SATELLITE` feeds Surface View.
7. external house context (`outside`/blur background) is rendered in a separate pass, aligned to the same preview frame.
Key point:
- outside blur/context depends on correct house framing.
- if `origin/dimensions` diverge from the expected CIP frame, context appears stretched, shifted, or out-of-scale.
- if minimap chunks are downscaled by nearest-neighbor sampling, large areas can look checker-patterned or overly pixelated.
## 7. Typical Visual Regression Causes
House preview symptom:
- house tiles look mostly correct, but outside blur/context has wrong scale/alignment.
Typical causes:
- exported `staticmapdata` has `origin/dimensions` different from the CIP template.
- template file was loaded from a hash-mismatched asset.
Map asset symptom:
- exported minimap appears checker-patterned or too pixelated.
Typical causes:
- downscaled minimap chunks used point sampling instead of averaging.
- the `1/16` layer was not emitted, forcing the client to magnify a lower-resolution layer.
Current exporter protections:
- validates embedded hash in data filenames.
- rejects invalid templates.
- tries a hash-valid sibling template when available.
- falls back to current-map dynamic export when no compatible template is found.
- skips out-of-bounds map coordinates before tile lookup so preview and map asset scans cannot alias negative coordinates into high map regions.
- ignores tiles whose minimap color is `0` so they do not overwrite the chunk background with black pixels.
- writes filenames whose embedded hash matches the generated content.
- emits `1/64`, `1/32`, and `1/16` minimap/satellite layers.
- stores backups in timestamped `bkps/export-*` snapshots instead of replacing an existing `.bkp` file.
- provides a `File > Client Assets > Restore Client Assets Backup` menu action that restores a selected snapshot and first moves overwritten current files into a `bkps/restore-*` safety snapshot.
- uses `GameSprite::getSpriteID(...)` for exported sprite image lookup; `GameSprite::getHardwareID(...)` is a GL texture/atlas id and must not be serialized or used as an asset sprite id.
- preserves full-size sprite images for export sampling so multi-tile sprites can be sampled with draw offsets instead of being cropped to one tile.
## 8. Validation Checklist
1. verify final data file hashes.
```sh
sha256sum assets/staticmapdata-<hash>.dat
sha256sum assets/staticdata-<hash>.dat
sha256sum assets/map-<hash>.dat
```
2. computed hash must match `<hash>` in each data filename.
3. verify `catalog-content.json` references the current `staticdata`, `staticmapdata`, and `map` files.
4. inspect `map-<hash>.dat` and verify each `MapData.mapassets[].filename` exists under the assets directory.
5. validate reference houses in client:
- `40503`
- `40211`
- `40510`
- `10301`
- `10302`
6. if house preview mismatch appears, compare:
- `origin`
- `dimensions`
- serialized tile count
- `skip` progression semantics
7. if map image mismatch appears, compare:
- emitted scale list
- `topleft`
- `widthsquare`
- `heightsquare`
- generated chunk filename
- BMP dimensions after LZMA decode
## 9. Regression Prevention Rules
1. use hash-valid CIP templates for original CIP-compatible maps.
2. do not reintroduce per-tile house/context mask serialization in the compatibility path.
3. do not shift template framing during merge/remap.
4. keep linear decode semantics (`+ skip + 1`).
5. keep custom-map exports independent from unmatched CIP templates.
6. keep minimap and satellite `mapassets` metadata aligned with the generated chunk dimensions.
7. keep the `1/16` layer when changing map asset generation.
8. keep Surface View sprite-based: draw ground, borders, and items into `SATELLITE` assets in tile draw order, using minimap colors only as fallback terrain.
9. do not use GL texture/atlas ids as exported sprite ids; use `GameSprite::getSpriteID(...)`.
10. validate changes on a real CIP-compatible client after exporter modifications.
## 10. Summary
Cyclopedia quality depends on:
1. correct static files (`staticdata` + `staticmapdata`).
2. correct map files (`mapdata` + referenced minimap/satellite assets).
3. CIP-preserving house framing (`origin/dimensions/skip`) so house and outside context share the same projection space.
4. complete, correctly scaled map asset layers so the client does not over-magnify a lower-resolution chunk.
When export preserves this contract, house geometry, outside blur/context, minimap, and satellite chunks align with expected CIP behavior.

View file

@ -126,6 +126,7 @@ void BaseMap::setTile(TileLocation* location, Tile* new_tile, bool remove) {
}
void BaseMap::setTile(int x, int y, int z, Tile* new_tile, bool remove) {
ASSERT(z >= 0 && z < rme::MapLayers);
ASSERT(!new_tile || new_tile->getX() == x);
ASSERT(!new_tile || new_tile->getY() == y);
ASSERT(!new_tile || new_tile->getZ() == z);

View file

@ -1318,12 +1318,12 @@ GLuint GameSprite::NormalImage::getHardwareID() {
return atlasTextureId;
}
uint32_t GameSprite::getSpriteID(int _layer, int _count, int _pattern_x, int _pattern_y, int /*_pattern_z*/, int _frame) {
uint32_t GameSprite::getSpriteID(int _layer, int _count, int _pattern_x, int _pattern_y, int _pattern_z, int _frame) {
uint32_t v;
if (_count >= 0) {
v = _count;
} else {
v = ((_frame * pattern_y + _pattern_y) * pattern_x + _pattern_x) * layers + _layer;
v = static_cast<uint32_t>(getIndex(0, 0, _layer, _pattern_x, _pattern_y, _pattern_z, _frame));
}
if (v >= numsprites) {
if (numsprites == 1) {

View file

@ -100,7 +100,7 @@ public:
int getIndex(int width, int height, int layer, int pattern_x, int pattern_y, int pattern_z, int frame) const;
GLuint getHardwareID(int _layer, int _count, int _pattern_x, int _pattern_y, int _pattern_z, int _frame);
SpriteUV getAtlasUVs(int _layer, int _count, int _pattern_x, int _pattern_y, int _pattern_z, int _frame);
uint32_t getSpriteID(int _layer, int _count, int _pattern_x, int _pattern_y, int /*_pattern_z*/, int _frame);
uint32_t getSpriteID(int _layer, int _count, int _pattern_x, int _pattern_y, int _pattern_z, int _frame);
virtual void DrawTo(wxDC* dc, SpriteSize sz, int start_x, int start_y, int width = -1, int height = -1);

View file

@ -1134,13 +1134,18 @@ void GUI::RefreshView() {
}
void GUI::CreateLoadBar(wxString message, bool canCancel /* = false */) {
CreateLoadBar(message, canCancel, true);
}
void GUI::CreateLoadBar(wxString message, bool canCancel, bool appModal) {
progressText = message;
progressFrom = 0;
progressTo = 100;
currentProgress = -1;
progressBar = newd wxGenericProgressDialog("Loading", progressText + " (0%)", 100, root, wxPD_APP_MODAL | wxPD_SMOOTH | (canCancel ? wxPD_CAN_ABORT : 0));
const long style = (appModal ? wxPD_APP_MODAL : 0) | wxPD_SMOOTH | (canCancel ? wxPD_CAN_ABORT : 0);
progressBar = newd wxGenericProgressDialog("Loading", progressText + " (0%)", 100, root, style);
progressBar->SetSize(280, -1);
progressBar->Show(true);
@ -1177,10 +1182,11 @@ bool GUI::SetLoadDone(int32_t done, const wxString &newMessage) {
}
bool skip = false;
bool continueProcessing = true;
if (progressBar) {
progressBar->Update(
continueProcessing = progressBar->Update(
newProgress,
wxString::Format("%s (%d%%)", progressText, newProgress),
wxString::Format("%s (%d%%)", progressText.c_str(), newProgress),
&skip
);
currentProgress = newProgress;
@ -1196,7 +1202,7 @@ bool GUI::SetLoadDone(int32_t done, const wxString &newMessage) {
}
}
return skip;
return continueProcessing && !skip;
}
void GUI::DestroyLoadBar() {

View file

@ -170,6 +170,7 @@ public:
* The default scale is 0 - 100
*/
void CreateLoadBar(wxString message, bool canCancel = false);
void CreateLoadBar(wxString message, bool canCancel, bool appModal);
/**
* Sets how much of the load has completed, the scale can be set with

File diff suppressed because it is too large Load diff

View file

@ -19,6 +19,10 @@
#define RME_OTBM_MAP_IO_H_
#include "iomap.h"
#include <functional>
#include <string>
#include <utility>
#include <vector>
enum OTBM_ItemAttribute {
OTBM_ATTR_DESCRIPTION = 1,
@ -112,16 +116,43 @@ struct MapVersion;
class NodeFileReadHandle;
class NodeFileWriteHandle;
class Map;
using CyclopediaExportProgressFn = std::function<bool(int32_t, const std::string &)>;
class IOMapOTBM : public IOMap {
public:
struct StaticHouseExportReport {
bool success = false;
bool filtered = false;
size_t selectedFilterCount = 0;
size_t matchedFilterCount = 0;
size_t mapHousesTotal = 0;
size_t staticDataGeneratedHouses = 0;
size_t staticDataFinalHouses = 0;
size_t staticMapAttemptedHouses = 0;
size_t staticMapGeneratedHouses = 0;
size_t staticMapFinalHouses = 0;
std::vector<std::string> failedStaticMapHouses;
std::vector<std::string> errors;
std::string outputBasePath;
std::string staticDataFileName;
std::string staticMapDataFileName;
};
IOMapOTBM(MapVersion ver);
~IOMapOTBM() { }
~IOMapOTBM() = default;
static bool getVersionInfo(const FileName &identifier, MapVersion &out_ver);
virtual bool loadMap(Map &map, const FileName &identifier);
virtual bool saveMap(Map &map, const FileName &identifier);
bool saveStaticData(Map &map, const FileName &dir, const std::vector<std::string> &houseNamesFilter = {});
bool saveCyclopediaMapData(Map &map, const FileName &dir, const CyclopediaExportProgressFn &progress = CyclopediaExportProgressFn {}, int satellitePixelsPerSquare = 2);
const StaticHouseExportReport &getLastStaticHouseExportReport() const {
return staticHouseExportReport_;
}
private:
StaticHouseExportReport staticHouseExportReport_;
protected:
static bool getVersionInfo(NodeFileReadHandle* f, MapVersion &out_ver);
@ -145,6 +176,18 @@ protected:
bool saveSpawnsNpc(Map &map, pugi::xml_document &doc);
bool saveZones(Map &map, const FileName &dir);
bool saveZones(Map &map, pugi::xml_document &doc);
bool serializeStaticDataHouses(Map &map, std::string &buffer);
std::string getStaticDataFilename(const Map &map) const;
bool serializeStaticMapDataHouses(
Map &map,
std::string &buffer,
size_t* attemptedHouseCount = nullptr,
size_t* exportedHouseCount = nullptr,
std::vector<std::string>* failedHouseNames = nullptr
);
std::string getStaticMapDataFilename(const Map &map) const;
bool serializeCyclopediaMapData(Map &map, std::string &buffer, std::vector<std::pair<std::string, std::vector<uint8_t>>> &assets, const CyclopediaExportProgressFn &progress = CyclopediaExportProgressFn {}, int satellitePixelsPerSquare = 2);
std::string getCyclopediaMapDataFilename(const Map &map) const;
};
#endif

View file

@ -27,12 +27,29 @@
#include "result_window.h"
#include "find_item_window.h"
#include "settings.h"
#include "iomap_otbm.h"
#include "sqlite_materials_inspector.h"
#include "lua/lua_script_manager.h"
#include "lua/lua_scripts_window.h"
#include "gui.h"
#include <wx/chartype.h>
#include <wx/choicdlg.h>
#include <wx/dirdlg.h>
#include <wx/msgdlg.h>
#include <wx/textdlg.h>
#include <wx/tokenzr.h>
#include <algorithm>
#include <chrono>
#include <ctime>
#include <filesystem>
#include <format>
#include <iomanip>
#include <sstream>
#include <string_view>
#include <utility>
#include <vector>
#include "items.h"
#include "editor.h"
@ -40,6 +57,507 @@
#include "live_client.h"
#include "live_server.h"
namespace {
constexpr int CyclopediaExportStatusMinIntervalMs = 1000;
bool selectAssetsOrCustomFolder(
wxWindow* parent,
const wxString &title,
const wxString &message,
const wxString &directoryDialogTitle,
wxString &outputPath
) {
wxArrayString choices;
choices.Add("Current loaded client assets (recommended)");
choices.Add("Choose a specific folder");
wxSingleChoiceDialog choiceDialog(parent, message, title, choices);
choiceDialog.SetSelection(0);
if (choiceDialog.ShowModal() != wxID_OK) {
return false;
}
if (choiceDialog.GetSelection() == 0) {
const wxString clientPath = ClientAssets::getPath();
if (clientPath.empty() || !wxDirExists(clientPath)) {
g_gui.PopupDialog("Error", "Current client path is not configured. Load/select the client assets path first.", wxOK);
return false;
}
outputPath = clientPath;
return true;
}
wxDirDialog directoryDialog(parent, directoryDialogTitle, "", wxDD_DEFAULT_STYLE | wxDD_DIR_MUST_EXIST);
if (directoryDialog.ShowModal() != wxID_OK) {
return false;
}
outputPath = directoryDialog.GetPath();
return true;
}
bool selectAssetsOrCustomExportFolder(wxWindow* parent, const wxString &title, wxString &outputPath) {
return selectAssetsOrCustomFolder(parent, title, "Select where to export the files.", "Select output folder", outputPath);
}
bool selectAssetsOrCustomRestoreFolder(wxWindow* parent, const wxString &title, wxString &outputPath) {
return selectAssetsOrCustomFolder(parent, title, "Select which assets folder should be restored.", "Select assets folder to restore", outputPath);
}
FileName makeDirectoryFileName(const wxString &directoryPath) {
FileName directory;
directory.AssignDir(directoryPath);
return directory;
}
wxString resolveCyclopediaOutputDisplayPath(const wxString &directoryPath) {
FileName rootCatalog;
rootCatalog.AssignDir(directoryPath);
rootCatalog.SetFullName("catalog-content.json");
if (rootCatalog.FileExists()) {
return directoryPath;
}
FileName assetsCatalog;
assetsCatalog.AssignDir(directoryPath);
assetsCatalog.AppendDir("assets");
assetsCatalog.SetFullName("catalog-content.json");
if (assetsCatalog.FileExists()) {
return assetsCatalog.GetPath(wxPATH_GET_VOLUME);
}
return directoryPath;
}
wxString appendDisplaySubdirectory(const wxString &directoryPath, const wxString &subdirectory) {
FileName path;
path.AssignDir(directoryPath);
path.AppendDir(subdirectory);
return path.GetPath(wxPATH_GET_VOLUME);
}
struct AssetsBackupSnapshot {
std::filesystem::path rootPath;
wxString label;
size_t fileCount = 0;
bool legacy = false;
};
bool isSafeRelativeBackupPath(const std::filesystem::path &path) {
if (path.empty() || path.is_absolute()) {
return false;
}
for (const std::filesystem::path &part : path) {
if (part == std::filesystem::path(".") || part == std::filesystem::path("..")) {
return false;
}
}
return true;
}
bool isBackupFile(const std::filesystem::path &path) {
return path.has_extension() && path.extension() == ".bkp";
}
size_t countBackupFiles(const std::filesystem::path &rootPath, const bool legacy) {
if (rootPath.empty() || !std::filesystem::exists(rootPath)) {
return 0;
}
size_t count = 0;
if (legacy) {
for (const auto &entry : std::filesystem::directory_iterator(rootPath)) {
if (entry.is_regular_file() && isBackupFile(entry.path())) {
++count;
}
}
return count;
}
for (const auto &entry : std::filesystem::recursive_directory_iterator(rootPath)) {
if (entry.is_regular_file() && isBackupFile(entry.path())) {
++count;
}
}
return count;
}
bool getLocalBackupTime(const std::time_t value, std::tm &localTime) {
#ifdef _WIN32
return localtime_s(&localTime, &value) == 0;
#else
return localtime_r(&value, &localTime) != nullptr;
#endif
}
std::string buildBackupSnapshotName(const std::string_view prefix) {
const std::time_t now = std::chrono::system_clock::to_time_t(std::chrono::system_clock::now());
std::tm localTime {};
if (!getLocalBackupTime(now, localTime)) {
return std::string(prefix) + "-unknown-time";
}
std::ostringstream name;
name << prefix << "-" << std::put_time(&localTime, "%Y-%m-%d_%H-%M-%S");
return name.str();
}
std::filesystem::path createBackupSnapshotPath(const std::filesystem::path &backupRootPath, const std::string_view prefix) {
std::error_code ec;
std::filesystem::create_directories(backupRootPath, ec);
if (ec) {
return std::filesystem::path();
}
const std::string baseName = buildBackupSnapshotName(prefix);
for (int attempt = 0; attempt < 1000; ++attempt) {
std::string snapshotName = baseName;
if (attempt > 0) {
snapshotName += std::format("-{:03}", attempt + 1);
}
std::filesystem::path snapshotPath = backupRootPath / snapshotName;
ec.clear();
if (std::filesystem::create_directory(snapshotPath, ec)) {
return snapshotPath;
}
if (ec) {
return std::filesystem::path();
}
}
return std::filesystem::path();
}
bool collectBackupSnapshots(const std::filesystem::path &basePath, std::vector<AssetsBackupSnapshot> &snapshots) {
snapshots.clear();
const std::filesystem::path backupRootPath = basePath / "bkps";
if (!std::filesystem::exists(backupRootPath) || !std::filesystem::is_directory(backupRootPath)) {
return false;
}
const size_t legacyFileCount = countBackupFiles(backupRootPath, true);
if (legacyFileCount > 0) {
AssetsBackupSnapshot legacySnapshot;
legacySnapshot.rootPath = backupRootPath;
legacySnapshot.fileCount = legacyFileCount;
legacySnapshot.legacy = true;
legacySnapshot.label = wxString::Format("legacy flat backup (%llu files)", static_cast<unsigned long long>(legacyFileCount));
snapshots.emplace_back(std::move(legacySnapshot));
}
for (const auto &entry : std::filesystem::directory_iterator(backupRootPath)) {
if (!entry.is_directory()) {
continue;
}
const size_t fileCount = countBackupFiles(entry.path(), false);
if (fileCount == 0) {
continue;
}
const wxString snapshotName = wxstr(entry.path().filename().string());
AssetsBackupSnapshot snapshot;
snapshot.rootPath = entry.path();
snapshot.fileCount = fileCount;
snapshot.label = wxString::Format("%s (%llu files)", snapshotName.c_str(), static_cast<unsigned long long>(fileCount));
snapshots.emplace_back(std::move(snapshot));
}
std::ranges::sort(snapshots, [](const AssetsBackupSnapshot &left, const AssetsBackupSnapshot &right) {
if (left.legacy != right.legacy) {
return !left.legacy;
}
std::error_code leftEc;
std::error_code rightEc;
const auto leftWriteTime = std::filesystem::last_write_time(left.rootPath, leftEc);
const auto rightWriteTime = std::filesystem::last_write_time(right.rootPath, rightEc);
if (!leftEc && !rightEc && leftWriteTime != rightWriteTime) {
return leftWriteTime > rightWriteTime;
}
return left.label.Cmp(right.label) > 0;
});
return !snapshots.empty();
}
bool collectBackupFilesForSnapshot(const AssetsBackupSnapshot &snapshot, std::vector<std::filesystem::path> &backupFiles) {
backupFiles.clear();
if (snapshot.rootPath.empty() || !std::filesystem::exists(snapshot.rootPath)) {
return false;
}
if (snapshot.legacy) {
for (const auto &entry : std::filesystem::directory_iterator(snapshot.rootPath)) {
if (entry.is_regular_file() && isBackupFile(entry.path())) {
backupFiles.emplace_back(entry.path());
}
}
} else {
for (const auto &entry : std::filesystem::recursive_directory_iterator(snapshot.rootPath)) {
if (entry.is_regular_file() && isBackupFile(entry.path())) {
backupFiles.emplace_back(entry.path());
}
}
}
std::ranges::sort(backupFiles, [](const std::filesystem::path &left, const std::filesystem::path &right) {
const bool leftCatalog = left.filename() == "catalog-content.json.bkp";
const bool rightCatalog = right.filename() == "catalog-content.json.bkp";
if (leftCatalog != rightCatalog) {
return !leftCatalog;
}
return left.generic_string() < right.generic_string();
});
return !backupFiles.empty();
}
bool getRestoreRelativePath(const AssetsBackupSnapshot &snapshot, const std::filesystem::path &backupFile, std::filesystem::path &relativePath) {
std::error_code ec;
relativePath = std::filesystem::relative(backupFile, snapshot.rootPath, ec);
if (ec || !isSafeRelativeBackupPath(relativePath) || !isBackupFile(relativePath)) {
return false;
}
const std::string filename = relativePath.filename().string();
constexpr std::string_view backupSuffix = ".bkp";
if (filename.size() <= backupSuffix.size() || !filename.ends_with(backupSuffix)) {
return false;
}
relativePath.replace_filename(filename.substr(0, filename.size() - backupSuffix.size()));
return isSafeRelativeBackupPath(relativePath);
}
std::filesystem::path makeRestoreBackupPath(const std::filesystem::path &snapshotPath, const std::filesystem::path &relativePath) {
std::filesystem::path backupPath = snapshotPath / relativePath;
backupPath += ".bkp";
return backupPath;
}
bool backupExistingTargetBeforeRestore(
const std::filesystem::path &basePath,
const std::filesystem::path &targetPath,
const std::filesystem::path &safetySnapshotPath,
wxString &errorMessage
) {
if (!std::filesystem::exists(targetPath)) {
return true;
}
std::error_code ec;
std::filesystem::path relativePath = std::filesystem::relative(targetPath, basePath, ec);
if (ec || !isSafeRelativeBackupPath(relativePath)) {
errorMessage = wxString::Format("Unsafe restore target path: %s", wxstr(targetPath.string()).c_str());
return false;
}
const std::filesystem::path backupPath = makeRestoreBackupPath(safetySnapshotPath, relativePath);
if (!backupPath.parent_path().empty()) {
std::filesystem::create_directories(backupPath.parent_path(), ec);
if (ec) {
errorMessage = wxString::Format("Failed to create restore safety backup folder: %s", wxstr(backupPath.parent_path().string()).c_str());
return false;
}
}
std::filesystem::rename(targetPath, backupPath, ec);
if (!ec) {
return true;
}
ec.clear();
std::filesystem::copy_file(targetPath, backupPath, std::filesystem::copy_options::none, ec);
if (ec) {
errorMessage = wxString::Format("Failed to backup current file before restore: %s", wxstr(targetPath.string()).c_str());
return false;
}
ec.clear();
std::filesystem::remove(targetPath, ec);
if (ec) {
errorMessage = wxString::Format("Failed to remove current file before restore: %s", wxstr(targetPath.string()).c_str());
return false;
}
return true;
}
bool restoreBackupSnapshot(
const std::filesystem::path &basePath,
const AssetsBackupSnapshot &snapshot,
size_t &restoredFileCount,
std::filesystem::path &safetySnapshotPath,
wxString &errorMessage
) {
restoredFileCount = 0;
std::vector<std::filesystem::path> backupFiles;
if (!collectBackupFilesForSnapshot(snapshot, backupFiles)) {
errorMessage = "Selected backup snapshot has no restorable files.";
return false;
}
safetySnapshotPath = createBackupSnapshotPath(basePath / "bkps", "restore");
if (safetySnapshotPath.empty()) {
errorMessage = "Failed to create restore safety backup snapshot.";
return false;
}
for (const std::filesystem::path &backupFile : backupFiles) {
std::filesystem::path relativePath;
if (!getRestoreRelativePath(snapshot, backupFile, relativePath)) {
errorMessage = wxString::Format("Unsafe backup file path: %s", wxstr(backupFile.string()).c_str());
return false;
}
const std::filesystem::path targetPath = basePath / relativePath;
if (!backupExistingTargetBeforeRestore(basePath, targetPath, safetySnapshotPath, errorMessage)) {
return false;
}
std::error_code ec;
if (!targetPath.parent_path().empty()) {
std::filesystem::create_directories(targetPath.parent_path(), ec);
if (ec) {
errorMessage = wxString::Format("Failed to create restore target folder: %s", wxstr(targetPath.parent_path().string()).c_str());
return false;
}
}
std::filesystem::copy_file(backupFile, targetPath, std::filesystem::copy_options::overwrite_existing, ec);
if (ec) {
errorMessage = wxString::Format("Failed to restore backup file: %s", wxstr(backupFile.string()).c_str());
return false;
}
++restoredFileCount;
}
return true;
}
bool selectStaticHouseExportFilter(wxWindow* parent, std::vector<std::string> &houseNamesFilter) {
houseNamesFilter.clear();
wxArrayString choices;
choices.Add("Export all houses");
choices.Add("Export only specific house names");
wxSingleChoiceDialog modeDialog(parent, "Select which houses should be exported.", "Export Static House Data", choices);
modeDialog.SetSelection(0);
if (modeDialog.ShowModal() != wxID_OK) {
return false;
}
if (modeDialog.GetSelection() == 0) {
return true;
}
wxTextEntryDialog namesDialog(
parent,
"Enter one or more house names separated by comma, semicolon, or line break.\n"
"Example:\n"
"Coastwood 3\n"
"Coastwood 4",
"Specific House Names"
);
if (namesDialog.ShowModal() != wxID_OK) {
return false;
}
const wxString rawNames = namesDialog.GetValue();
wxStringTokenizer tokenizer(rawNames, ",;\n\r", wxTOKEN_STRTOK);
while (tokenizer.HasMoreTokens()) {
wxString token = tokenizer.GetNextToken();
token.Trim(true);
token.Trim(false);
if (token.empty()) {
continue;
}
houseNamesFilter.emplace_back(nstr(token));
}
if (houseNamesFilter.empty()) {
g_gui.PopupDialog("Error", "No valid house names were provided.", wxOK);
return false;
}
return true;
}
wxString buildStaticHouseExportSummary(const IOMapOTBM::StaticHouseExportReport &report) {
const auto asU64 = [](size_t value) {
return static_cast<unsigned long long>(value);
};
wxString summary;
summary += report.success ? "Static house export completed.\n\n" : "Static house export failed.\n\n";
if (!report.outputBasePath.empty()) {
summary += "Output base: " + wxstr(report.outputBasePath) + "\n";
summary += "Backups: " + wxstr(report.outputBasePath + "/bkps") + "\n";
}
if (!report.staticDataFileName.empty()) {
summary += "staticdata file: " + wxstr(report.staticDataFileName) + "\n";
}
if (!report.staticMapDataFileName.empty()) {
summary += "staticmapdata file: " + wxstr(report.staticMapDataFileName) + "\n";
}
summary += "\n";
summary += "Export mode: ";
summary += report.filtered ? "specific houses\n" : "all houses\n";
if (report.filtered) {
summary += wxString::Format("Requested house names: %llu\n", asU64(report.selectedFilterCount));
summary += wxString::Format("Matched house names on map: %llu\n", asU64(report.matchedFilterCount));
}
summary += wxString::Format("Total houses on map: %llu\n\n", asU64(report.mapHousesTotal));
summary += "Counts:\n";
summary += wxString::Format(" - staticdata: generated=%llu, final=%llu\n", asU64(report.staticDataGeneratedHouses), asU64(report.staticDataFinalHouses));
summary += wxString::Format(
" - staticmapdata: attempted=%llu, generated=%llu, final=%llu\n\n",
asU64(report.staticMapAttemptedHouses),
asU64(report.staticMapGeneratedHouses),
asU64(report.staticMapFinalHouses)
);
if (!report.failedStaticMapHouses.empty()) {
summary += wxString::Format("Houses with staticmap build failure (%llu):\n", asU64(report.failedStaticMapHouses.size()));
const size_t maxFailedLines = 20;
size_t shown = 0;
for (const std::string &houseName : report.failedStaticMapHouses) {
if (shown >= maxFailedLines) {
break;
}
summary += " - " + wxstr(houseName) + "\n";
++shown;
}
if (report.failedStaticMapHouses.size() > shown) {
summary += wxString::Format(" - ... and %llu more\n", asU64(report.failedStaticMapHouses.size() - shown));
}
summary += "\n";
}
if (!report.errors.empty()) {
summary += wxString::Format("Errors/Warnings (%llu):\n", asU64(report.errors.size()));
const size_t maxErrorLines = 20;
size_t shown = 0;
for (const std::string &errorMessage : report.errors) {
if (shown >= maxErrorLines) {
break;
}
summary += " - " + wxstr(errorMessage) + "\n";
++shown;
}
if (report.errors.size() > shown) {
summary += wxString::Format(" - ... and %llu more\n", asU64(report.errors.size() - shown));
}
}
return summary;
}
}
BEGIN_EVENT_TABLE(MainMenuBar, wxEvtHandler)
END_EVENT_TABLE()
@ -67,6 +585,9 @@ MainMenuBar::MainMenuBar(MainFrame* frame) :
MAKE_ACTION(IMPORT_BITMAP_TO_MAP, wxITEM_NORMAL, OnImportBitmapToMap);
MAKE_ACTION(EXPORT_MINIMAP, wxITEM_NORMAL, OnExportMinimap);
MAKE_ACTION(EXPORT_STATIC_HOUSE_DATA, wxITEM_NORMAL, OnExportStaticHouseData);
MAKE_ACTION(EXPORT_CYCLOPEDIA_MAP, wxITEM_NORMAL, OnExportCyclopediaMapData);
MAKE_ACTION(REVERT_CYCLOPEDIA_ASSETS, wxITEM_NORMAL, OnRevertCyclopediaAssets);
MAKE_ACTION(EXPORT_TILESETS, wxITEM_NORMAL, OnExportTilesets);
MAKE_ACTION(RELOAD_DATA, wxITEM_NORMAL, OnReloadDataFiles);
@ -351,6 +872,9 @@ void MainMenuBar::Update() {
EnableItem(IMPORT_MONSTERS, is_local);
EnableItem(IMPORT_MINIMAP, false);
EnableItem(EXPORT_MINIMAP, is_local);
EnableItem(EXPORT_STATIC_HOUSE_DATA, is_local);
EnableItem(EXPORT_CYCLOPEDIA_MAP, is_local);
EnableItem(REVERT_CYCLOPEDIA_ASSETS, true);
EnableItem(EXPORT_TILESETS, loaded);
EnableItem(FIND_ITEM, is_host);
@ -876,6 +1400,189 @@ void MainMenuBar::OnExportMinimap(wxCommandEvent &WXUNUSED(event)) {
dialog.ShowModal();
}
void MainMenuBar::OnExportStaticHouseData(wxCommandEvent &) {
if (!g_gui.IsEditorOpen()) {
return;
}
std::vector<std::string> houseNamesFilter;
if (!selectStaticHouseExportFilter(g_gui.root, houseNamesFilter)) {
return;
}
wxString outputPath;
if (!selectAssetsOrCustomExportFolder(g_gui.root, "Export Static House Data", outputPath)) {
return;
}
IOMapOTBM mapsaver(g_gui.GetCurrentMap().getVersion());
const bool exportOk = mapsaver.saveStaticData(g_gui.GetCurrentMap(), makeDirectoryFileName(outputPath), houseNamesFilter);
const IOMapOTBM::StaticHouseExportReport &report = mapsaver.getLastStaticHouseExportReport();
const wxString summary = buildStaticHouseExportSummary(report);
if (!exportOk) {
const wxString failureMessage = summary.empty() ? wxString("Failed to export static house data.") : summary;
g_gui.PopupDialog("Export failed", failureMessage, wxOK);
return;
}
if (!report.errors.empty() || !report.failedStaticMapHouses.empty()) {
g_gui.PopupDialog("Export completed with warnings", summary, wxOK);
} else {
g_gui.PopupDialog("Export completed", summary, wxOK);
}
}
void MainMenuBar::OnExportCyclopediaMapData(wxCommandEvent &) {
static bool cyclopediaExportRunning = false;
if (cyclopediaExportRunning) {
g_gui.PopupDialog("Export in progress", "Cyclopedia export is already running.", wxOK);
return;
}
if (!g_gui.IsEditorOpen()) {
return;
}
// CipSoft cyclopedia assets are expected around 2 px/tile for satellite (scale = 1/16).
const int satellitePixelsPerSquare = 2;
wxString outputPath;
if (!selectAssetsOrCustomExportFolder(g_gui.root, "Export Cyclopedia Map Data", outputPath)) {
return;
}
IOMapOTBM mapsaver(g_gui.GetCurrentMap().getVersion());
cyclopediaExportRunning = true;
struct CyclopediaExportGuard final {
bool &running;
explicit CyclopediaExportGuard(bool &value) :
running(value) { }
CyclopediaExportGuard(const CyclopediaExportGuard &) = delete;
CyclopediaExportGuard &operator=(const CyclopediaExportGuard &) = delete;
~CyclopediaExportGuard() {
running = false;
}
};
CyclopediaExportGuard exportGuard(cyclopediaExportRunning);
int lastCyclopediaExportStatusPercent = -1;
auto lastCyclopediaExportStatusUpdate = std::chrono::steady_clock::now();
auto updateCyclopediaExportStatus = [&](const int32_t done, const std::string &message, const bool force = false) {
const int clampedDone = std::max<int32_t>(0, std::min<int32_t>(100, done));
const auto now = std::chrono::steady_clock::now();
if (!force && clampedDone == lastCyclopediaExportStatusPercent && now - lastCyclopediaExportStatusUpdate < std::chrono::milliseconds(CyclopediaExportStatusMinIntervalMs)) {
return true;
}
lastCyclopediaExportStatusPercent = clampedDone;
lastCyclopediaExportStatusUpdate = now;
const wxString progressMessage = message.empty() ? wxString("Exporting cyclopedia minimap/satellite...") : wxstr(message);
g_gui.SetStatusText(wxString::Format("Cyclopedia export: %d%% - %s", clampedDone, progressMessage.c_str()));
return true;
};
updateCyclopediaExportStatus(0, "Preparing cyclopedia export...", true);
if (!mapsaver.saveCyclopediaMapData(
g_gui.GetCurrentMap(), makeDirectoryFileName(outputPath), [&](const int32_t done, const std::string &message) {
return updateCyclopediaExportStatus(done, message);
},
satellitePixelsPerSquare
)) {
g_gui.SetStatusText("Cyclopedia export failed.");
g_gui.PopupDialog("Error", "Failed to export cyclopedia minimap/satellite.", wxOK);
return;
}
updateCyclopediaExportStatus(100, "Cyclopedia export completed.", true);
const wxString outputBasePath = resolveCyclopediaOutputDisplayPath(outputPath);
const wxString backupPath = appendDisplaySubdirectory(outputBasePath, "bkps");
g_gui.SetStatusText(wxString::Format("Cyclopedia export completed: %s", outputBasePath.c_str()));
g_gui.PopupDialog(
"Export completed",
wxString::Format(
"Cyclopedia minimap/satellite exported successfully.\n\nOutput: %s\nBackups: %s (backup-only)",
outputBasePath.c_str(),
backupPath.c_str()
),
wxOK
);
}
void MainMenuBar::OnRevertCyclopediaAssets(wxCommandEvent &) {
wxString selectedPath;
if (!selectAssetsOrCustomRestoreFolder(g_gui.root, "Restore Client Assets Backup", selectedPath)) {
return;
}
const wxString outputBasePath = resolveCyclopediaOutputDisplayPath(selectedPath);
const std::filesystem::path basePath(nstr(outputBasePath));
std::vector<AssetsBackupSnapshot> snapshots;
if (!collectBackupSnapshots(basePath, snapshots)) {
g_gui.PopupDialog(
"No backups found",
wxString::Format("No restorable backups were found in:\n%s", appendDisplaySubdirectory(outputBasePath, "bkps").c_str()),
wxOK
);
return;
}
wxArrayString choices;
for (const AssetsBackupSnapshot &snapshot : snapshots) {
choices.Add(snapshot.label);
}
wxSingleChoiceDialog snapshotDialog(
g_gui.root,
"Select which backup snapshot should be restored.",
"Restore Client Assets Backup",
choices
);
snapshotDialog.SetSelection(0);
if (snapshotDialog.ShowModal() != wxID_OK) {
return;
}
const int selectedSnapshotIndex = snapshotDialog.GetSelection();
if (selectedSnapshotIndex < 0 || static_cast<size_t>(selectedSnapshotIndex) >= snapshots.size()) {
return;
}
const AssetsBackupSnapshot &snapshot = snapshots[static_cast<size_t>(selectedSnapshotIndex)];
const wxString confirmMessage = wxString::Format(
"Restore this backup snapshot?\n\nAssets folder:\n%s\n\nBackup:\n%s\n\nCurrent files that would be overwritten will be moved to a new restore backup snapshot first.",
outputBasePath.c_str(),
snapshot.label.c_str()
);
if (wxMessageBox(confirmMessage, "Confirm Restore", wxYES_NO | wxNO_DEFAULT | wxICON_WARNING, g_gui.root) != wxYES) {
return;
}
size_t restoredFileCount = 0;
std::filesystem::path safetySnapshotPath;
wxString errorMessage;
if (!restoreBackupSnapshot(basePath, snapshot, restoredFileCount, safetySnapshotPath, errorMessage)) {
g_gui.PopupDialog(
"Restore failed",
errorMessage.empty() ? wxString("Failed to restore selected backup snapshot.") : errorMessage,
wxOK
);
return;
}
g_gui.PopupDialog(
"Restore completed",
wxString::Format(
"Restored %llu files from backup.\n\nAssets folder: %s\nSafety backup: %s",
static_cast<unsigned long long>(restoredFileCount),
outputBasePath.c_str(),
wxstr(safetySnapshotPath.string()).c_str()
),
wxOK
);
}
void MainMenuBar::OnExportTilesets(wxCommandEvent &WXUNUSED(event)) {
if (g_gui.GetCurrentEditor()) {
ExportTilesetsWindow dlg(frame, *g_gui.GetCurrentEditor());

View file

@ -35,6 +35,9 @@ namespace MenuBar {
IMPORT_NPCS,
IMPORT_MINIMAP,
EXPORT_MINIMAP,
EXPORT_STATIC_HOUSE_DATA,
EXPORT_CYCLOPEDIA_MAP,
REVERT_CYCLOPEDIA_ASSETS,
EXPORT_TILESETS,
RELOAD_DATA,
RECENT_FILES,
@ -218,6 +221,9 @@ public:
void OnImportMinimap(wxCommandEvent &event);
void OnImportBitmapToMap(wxCommandEvent &event);
void OnExportMinimap(wxCommandEvent &event);
void OnExportStaticHouseData(wxCommandEvent &event);
void OnExportCyclopediaMapData(wxCommandEvent &event);
void OnRevertCyclopediaAssets(wxCommandEvent &event);
void OnExportTilesets(wxCommandEvent &event);
void OnReloadDataFiles(wxCommandEvent &event);

View file

@ -26,6 +26,13 @@ if(MSVC AND BUILD_STATIC_LIBRARY)
"MultiThreaded$<$<CONFIG:Debug>:Debug>")
endif()
if(MSVC)
# sccache tries to package the target PDB emitted by generated protobuf
# sources and fails when parallel MSVC compiles race on that file. Keep the
# launcher enabled elsewhere and compile only this generated target directly.
set_target_properties(${PROJECT_NAME} PROPERTIES CXX_COMPILER_LAUNCHER "")
endif()
set(PROTOBUF_GENERATE_DEPENDENCIES)
if(TARGET protobuf::protoc)
list(APPEND PROTOBUF_GENERATE_DEPENDENCIES $<TARGET_FILE:protobuf::protoc>)

View file

@ -0,0 +1,53 @@
syntax = "proto2";
package clienteditor.protobuf.mapdata;
message MapData {
repeated AreaData areadata = 1;
repeated NpcData npcdata = 2;
repeated MapAssets mapassets = 3;
optional Position topleftedge = 4;
optional Position bottomrightedge = 5;
}
message Position {
optional uint32 posx = 1;
optional uint32 posy = 2;
optional uint32 posz = 3;
}
message AreaData {
optional uint32 areaid = 1;
optional string name = 2;
enum AreaType {
UNDEFINED = 0;
AREA = 1;
SUBAREA = 2;
}
optional AreaType type = 3;
repeated uint32 subarea = 4;
optional Position areaposition = 5;
optional bool donate = 6;
optional string akaname = 7;
}
message NpcData {
optional string name = 1;
optional Position position = 2;
optional uint32 subareaid = 3;
}
message MapAssets {
enum AssetsType {
SUBAREA = 0;
SATELLITE = 1;
MINIMAP = 2;
}
optional AssetsType type = 1;
optional Position topleft = 2;
optional string filename = 3;
optional uint32 widthsquare = 4;
optional uint32 heightsquare = 5;
optional uint32 areaid = 6;
optional double scale = 7;
}

View file

@ -0,0 +1,68 @@
syntax = "proto2";
package clienteditor.protobuf.staticdata;
message StaticData {
repeated Monster monster = 1;
repeated Achievement achievements = 2;
repeated House house = 3;
repeated Boss boss = 4;
repeated Quest quest = 5;
}
message Colors {
optional uint32 lookhead = 1;
optional uint32 lookbody = 2;
optional uint32 looklegs = 3;
optional uint32 lookfeet = 4;
}
message Appearance_Type {
optional uint32 outfittype = 1;
optional Colors colors = 2;
optional uint32 outfitaddon = 3;
optional uint32 itemtype = 4;
}
message Monster {
optional uint32 raceid = 1;
optional string name = 2;
optional Appearance_Type appearance_type = 3;
}
message Achievement {
optional uint32 achievement_id = 1;
optional string name = 2;
optional string description = 3;
optional uint32 grade = 4;
}
message HousePosition {
optional uint32 pos_x = 1;
optional uint32 pos_y = 2;
optional uint32 pos_z = 3;
}
message House {
optional uint32 house_id = 1;
optional string name = 2;
optional string unknownstring = 3;
optional uint32 price = 4;
optional uint32 beds = 5;
optional HousePosition housePosition = 6;
optional uint32 size_sqm = 7;
optional bool guildhall = 8;
optional string city = 9;
optional bool shop = 10;
}
message Boss {
optional uint32 id = 1;
optional string name = 2;
optional Appearance_Type appearance_type = 3;
}
message Quest {
optional uint32 id = 1;
optional string name = 2;
}

View file

@ -0,0 +1,42 @@
syntax = "proto2";
package clienteditor.protobuf.staticmapdata;
message StaticMapData {
repeated HouseEntry house = 1;
}
message HouseEntry {
optional uint32 house_id = 1;
optional HousePreviewData data = 2;
}
message Position {
optional uint32 pos_x = 1;
optional uint32 pos_y = 2;
optional uint32 pos_z = 3;
}
message HousePreviewData {
optional Position origin = 1;
optional Position dimensions = 2;
optional HousePreviewTiles preview = 3;
}
message HousePreviewTiles {
optional HousePreviewLayer layer = 2;
}
message HousePreviewLayer {
repeated HousePreviewTile tile = 3;
}
message HousePreviewTile {
repeated HousePreviewItem item = 1;
optional uint32 skip = 2;
optional bool is_house_tile = 3;
}
message HousePreviewItem {
optional uint32 value = 1;
}

View file

@ -337,10 +337,10 @@ wxImage SpriteAppearances::getWxImageBySpriteId(int id, bool toSavePng /* = fals
}
// Cut duplicated image and sets to the selected bgshade the empty background
if (sprite->size.width > rme::SpritePixels && sprite->size.height <= rme::SpritePixels) {
if (!toSavePng && sprite->size.width > rme::SpritePixels && sprite->size.height <= rme::SpritePixels) {
// TWO_BY_ONE (64x32): keep right 32 pixels
image.Resize(wxSize(rme::SpritePixels, rme::SpritePixels), wxPoint(-(sprite->size.width - rme::SpritePixels), 0), bgshade, bgshade, bgshade);
} else if (sprite->size.height > rme::SpritePixels && sprite->size.width <= rme::SpritePixels) {
} else if (!toSavePng && sprite->size.height > rme::SpritePixels && sprite->size.width <= rme::SpritePixels) {
// ONE_BY_TWO (32x64): keep bottom 32 pixels
image.Resize(wxSize(rme::SpritePixels, rme::SpritePixels), wxPoint(0, -(sprite->size.height - rme::SpritePixels)), bgshade, bgshade, bgshade);
}

View file

@ -620,6 +620,7 @@ void Tile::finalizeLoadedState() {
void Tile::update() {
statflags &= TILESTATE_MODIFIED;
minimapColor = INVALID_MINIMAP_COLOR;
if (spawnMonster && spawnMonster->isSelected()) {
statflags |= TILESTATE_SELECTED;