### Summary
* Bumps to .NET 9 with updated dependencies
* Comparing a value type against null is no longer allowed
* CI/CD now uses the version specified in global.json
* Serialization generator updated to .NET 9 with bug fixes, fixes to turkish language, and parallelization
### Summary
- Puts back `EventSink.SocketConnect`.
- Reverts networking change to push the networking to a separate thread.
- Reverts changes to the firewall by removing the firewall queue.
- Fixes listeners not shutting down with the server.
- Fixes race condition causing connections to get stuck even after they are disposed.
> [!NOTE]
> **Developer Note**
> Networking has been reverted back to using the main thread instead of a background thread. This alleviated complexity and the requirement for concurrent queues all over the place.
### Summary
- Adds [Code Generated Events](https://github.com/modernuo/CodeGeneratedEvents)
- Removes EventSink.PlayerDeath
- Adds `PlayerDeathEvent` and `CreatureDeathEvent` using code generated events
> [!Important]
> **Developer Note**
> Use `[OnEvent(nameof(PlayerMobile.PlayerDeathEvent))]` instead of `EventSink.PlayerDeath` delegate
> Check this commit for examples of how to use this.
> [!Note]
> **Developer Note**
> Now developers will only need to build/run the Application project instead of everything.
> When adding new projects, make sure to:
> 1. Add a reference to that project in the Application project.
> 2. Add the dll file to the Distribution/Data/assemblies.json file
### Summary
- Adds an application project
- Consolidates process restarts to use `Core.Kill(true)`
- Removes some old messaging, for example processor optimization
- Fixes missing build cleanup
> [!Warning]
> Users on Linux/OSX will need to follow the Readme
> and make sure `libdeflate` is properly installed
> [!Note]
> **Developer Note**
> The API for compression has changed. Use `Deflate.Standard` for the same functionality.
### Summary
* Replaces Zlib with LibDeflate for a 50% performance improvement!
* Adds MacOS 14 to properly test Arm64
> [!CAUTION]
> **BREAKING CHANGE**
> Removed `OrderdHashSet` and `PooledOrderedHashSet` due to bugs.
> [!NOTE]
> **Developer Note**
> The OrderedSet is not a full data structure. It is not particularly efficient. Pull Requests are welcome for a better implementation, especially if it ends up supporting `ISet<T>` and `IReadOnlySet<T>`
## Summary
The ordered hash set was buggy. It's kind of painful to implement, so for now, I added a simple `OrderedSet` to suffice for gumps. Please reach out if this causes disruption!
# New Gump API
We are pleased to release a new API that is faster, allocates nearly zero memory, and still feels very similar to the original API. The API is broken into 3 types of gumps, dynamic, static with placeholders, and static without placeholders.
### Dynamic Gumps
These gumps will inherit `DynamicGump` and are meant for gumps that have a dynamic layout. This includes specifying dynamic arguments to HtmlLocalized entries.
## Static Gumps
Static gumps are those where the function to the build the layout is called only once and cached forever. They can optionally have placeholders. These placeholders allow the developer to specify the string values later, dynamically in a `BuildStrings` method on the gump. If a gump does not have any placeholders, the string entries will also be cached forever.
## Benchmarks
To make sure we were going in the right direction and not wasting time, we took copious benchmarks. Here are the final benchmarks for a really simple gump.
Note:
* The majority of creating a gump is compressing the layout and the strings. Compressing each section takes ~6,000ns (12us total).
```cs
| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|------------------------------- |-------------:|-------------:|-------------:|-------------:|------:|--------:|-------:|----------:|------------:|
| OldGump | 13,308.29 ns | 1,059.695 ns | 1,883.608 ns | 14,330.72 ns | 1.000 | 0.00 | 0.1526 | 2400 B | 1.00 |
| DynamicLayoutGump | 13,357.86 ns | 129.144 ns | 226.185 ns | 13,323.60 ns | 1.029 | 0.17 | - | 48 B | 0.02 |
| StaticLayoutDynamicStringsGump | 6,653.10 ns | 81.815 ns | 143.292 ns | 6,617.45 ns | 0.514 | 0.09 | - | 40 B | 0.02 |
| StaticLayoutGump | 86.33 ns | 0.760 ns | 1.350 ns | 86.07 ns | 0.007 | 0.00 | 0.0020 | 32 B | 0.01 |
```
# Non-Breaking Changes
* All gump components in the core have been moved to `Gumps/Legacy`.
* All legacy gumps will still inherit `Gump`, which now inherits `BaseGump`
# Special Thanks
Thank you to @stefanomerotta for considerable contributions/benchmarking/testing to make this effort a reality! We collectively went through over 10 iterations, but it is finally ready.
## Breaking Changes
* The Firewall and IP Limiter have been rewritten. Please read the notes carefully!
* `TcpServer.Instances` moved back to `NetState.Instances` - sorry - it was stupid to move it to begin with.
> [!Note]
> Sockets that fail the IP Limiter or Firewall will be immediately and forcibly disconnected.
> This means they will be stuck at "Verifying account..." if it was a real client.
### Summary
- Removes firewall wildcard support.
- Removes `AccessRestrictions`.
- Moves Firewall/IPLimiter to the core.
- Moves `TcpServer` to its own thread.
- Removes the `SocketConnect` and `SocketDisconnect` event sinks.
- Moves `Instances` back to `NetState.Instances`.
- Fixes a long standing bug with bad handling of duplicate listener addresses.
#### Firewall
The firewall has been completely rewritten. There is now an "Admin Firewall" which saves to the config file. Secondarily, there is an internal firewall used exclusively by the TcpServer while processing sockets. The Admin firewall mirrors it's additions/deletions to the internal firewall by adding requests to a queue.
> [!IMPORTANT]
> **Wildcard firewall entries, such as `X`, `*`, `?` are not allowed.**
> **Ranges in between IP classes or sextets are not allowed.**
> **Please make sure to use one of the following:**
> * IP Address - `192.168.1.1`
> * CIDR - `192.168.1.0/24`
> * Range - `192.168.1.1-192.168.1.100`
#### IP Limiter
The IP Limiter has been completely rewritten. The available configurations are:
```json
"ipLimiter.enable": "True",
"ipLimiter.maxConnectionsPerIP": 10,
"ipLimiter.clearConnectionAttemptsDuration": "00:00:00:10",
"ipLimiter.clearThrottledDuration": "00:00:02:00",
```
The IP Limiter is set up to prevent spamming connections from the same IP. Every time an IP connects, it is added to a connection list. After 10 attempts, the IP is added to the throttle list. To keep the system fast, the connection list is entirely wiped every 10 seconds, and the throttle list is entirely wiped every 2 minutes.
### Summary
- Fixes issues with non-_ISerializable_ objects having bad serialization (Skill/Stat. Mods)
- Fixes issues with MarkDirty and namespaces: https://github.com/modernuo/SerializationGenerator/issues/29
- Fixes missing dirty tracking for some data structures.
- Fixes cascading MarkDirty for non-serializable types that have owners that are also non-serializable types.
### New Codegenned API
When a data structure (Array, List, Dictionary, HashSet, etc) is source generated for serialization, new methods are added which will handle dirty tracking. Use these instead of the built in methods for that data structure.
_All Data Structures_
```cs
public void Clear<PropertyName>();
```
_Lists and Sets_
```cs
public void AddTo<PropertyName>(V value);
public void RemoveFrom<PropertyName>(V value);
```
_Lists_
```cs
public void InsertInto<PropertyName>(int index, V value);
public void RemoveFrom<PropertyName>At(int index);
```
_Dictionaries_
```cs
public void AddTo<PropertyName>(T key, V value);
public void RemoveFrom<PropertyName>(T key);
public void ReplaceIn<PropertyName>(T key, V value);
```
Over time, they will be cleaned up and more variants added such as `bool RemoveFrom<PropertyName>(T key, out V value)`
## BREAKING CHANGE
- Deletes `map.GetObjectsInRange` and `map.GetObejctsInBounds`
### Notes
Developers are expected to enumerate mobiles and items separately now using `map.GetMobilesInRange` and `map.GetItemsInRange`. This helps keep the code streamlined so we don't have to maintain multiple copies of ref struct enumerators that do the same thing.
### Fixes
- [X] Fixes bug with planks closing
- [X] Fixes issue with iterating items/mobiles from a null map
## Breaking Changes
Incoming packet registration signature has changed to:
```cs
delegate* void OnReceiveCallback(NetState state, SpanReader reader, int packetLength);
IncomingPackets.Register(int packetID, int length, bool ingame, OnReceiveCallback onReceive);
```
For example, an incoming packet handler signature would now look like this:
```cs
public static void SomeIncomingPacket(NetState state, SpanReader reader, int packetLength)
{
// Parse the data
}
```
## Summary
Updates the network Pipe class to use a mirrored memory technique. This technique involves mapping the same physical memory to two contiguous virtual memory spaces so the byte buffer appears duplicated. This allows writing to a double-sized array to wrap around without the need for the `CircularBuffer` classes.
In practice this allows us to use `Span<byte>` as if the buffer was a regular array.
### Bug Fixes
- [X] Fixes bad fixed length string parsing
## Overhaul to Stamina (Overweight) System
### Added configurations
```json
{
"settings": {
"stamina.enableMountStamina": "True",
"stamina.cannotMoveWhenFatigued": "True",
"stamina.stonesPerOverweightLoss": "25",
"stamina.stonesOverweightAllowance": "4",
"stamina.baseOverweightLoss": "5",
"stamina.additionalLossWhenBelow": "0.1",
"stamina.mountLastMoveStepsReset": "01:00:00:00",
"stamina.enableMountStamina": "True",
"stamina.useMountStaminaOnlyWhenOverloaded": "False",
},
}
```
- `stamina.cannotMoveWhenFatigued` - By default, Pre-AOS expansions will outright block a player if they are fatigued. A player is fatigued when they run out of stamina by any mechanism.
- `stamina.stonesPerOverweightLoss` - The amount of stamina lost for every X stones above overweight. Example, if a person is overweight 28 stones overweight, then there there is 1 additional stamina. 28 / 25 = 1 (no decimals, no rounding)
- `stamina.stonesOverweightAllowance` - The number of stones allowed before overweight penalties take affect. This _does not_ substract from the overweight stones calculation.
- `stamina.baseOverweightLoss` - The base amount of stamina loss for being overweight. While running, the final amount is multiplied by 2. If mount stamina is turned off, then the final amount is divided by 3 while mounted.
### Mount stamina
Mounts have a new property `StepsMax` to determine the maximum steps they can take before being fatigued. To regain steps, the player _must stay on the mount and not move_ (per OSI). The gain rates are configurable. If a mount is dismounted, or the player logs off, the mount is considered inactive and mounts regain all of their steps after _24 hours_.
### Changes to player stamina
Players now have a proper inactivity time reset for their steps. If a player is idle for 16 seconds (including logging off, or the server being offline), then the steps counter is rest.
### Developer Notes
- `StepsTaken` - This field has been removed. It was not serialized and could not be used reliably. If a developer was using it, then the recommendation is to build a mechanism to track total steps another way.
- `IHasStamina` - This new interface was added and currently `IMount` and `PlayerMobile` are valid types.
Important: Entities that are sent to the StaminaSystem for tracking (mounts, players, or something else), must be an `ISerializable` to be serialized properly. If they are not, then the serialization system will record a null, and nothing will be deserialized upon world load. No errors will be given.
## MAJOR CHANGE (API BREAKING)
Added a player murder system to facilitate reporting murders. This should make it easier to extend to create a bounty system or other related game content. Player murders will be saved in a folder called _PlayerMurders_.
### Motivation
The motivation was two-fold, performance, and bug fixes.
First, murders are one of two systems that do a pre-world-save check on _every mobile in the game_ to decay kills and set their expiring murders. This is taxing since it freezes the world and makes world saves take longer. Every mobile has ShortTermMurders even though it is a player concept. And next, 90%+ of players are not murderers but had an ever increasing MurderElapse time that was being tracked against GameTime. These properties were also serialized unnecessarily for all mobs.
Second, when I tried to optimize/refactor the code, it was obvious that the system has bugs.
### Major API Changes
- [X] Created a player murder system and moved `ShortTermMurders`, `ShortTermElapse`, and `LongTermElapse` to the system.
- [X] Added convenience property `PlayerMobile.ShortTermMurders`.
- [X] Added convenience properties `PlayerMobile.ShortTermMurderExpiration` and `PlayerMobile.LongTermMurderExpiration`
- [X] Moved ReportMurdererGump.cs
- [X] Adds an `EventSink.PlayerDeleted` event.
### Notes
The system currently does not support NPCs. To support expiring murders on NPCs I highly recommend a different architecture for large servers (500k+ mobs including players). Specifically switching from looping through all MurderContext to a time-order link list.
### Summary
- [X] Gets rid of the Dtos
- [X] Simplifies the json serializer registration
- [X] Adds a custom `RegionByName` JsonConverter that can look up other regions that have already been registered.
### BREAKING CHANGE
1. **Child regions must appear after their parents in the JSON file**
2. In the regions json file, _"Parent"_ can no longer be just a string. It must be an object that includes the map.
- ```json
"Parent": { "Name": "Britain", "Map": "Felucca" }
```
## BREAKING CHANGE
**Publishing for linux will no longer include runtimes**
## Major Changes
* Adds .NET 7 support.
* Adds ARM64 support (experimental).
* Changes linux support to be open-ended against anything .NET 7 supports.
* Fixes native library resolutions. (Make sure to install `dev` versions of the libraries)
* Updates README
* Adds Ubuntu 22, CentOS 8/9 Stream to CI/CD.
## TODOs:
* Update modernuo.com docs
Closes#1173Closes#1159
## Changes
* Improves type hashing by introducing xxHash3 (64bit)
* Removes individual `tdb` files in favor of a single `SerializedTypes.db` file. This file is only used to identify a type that is being deserialized, which doesn't exist.
* Adds duplicate type alias detection
* Adds `AssemblyHandler.FindTypeByHash`
View changed files whitespaces: https://github.com/modernuo/ModernUO/pull/1172/files?diff=split&w=1
## SerializedTypes.db
The serialized types file is used to get back the original name of a type in case it no longer exists in code. This can easily be necessary if a class is renamed in code and no `TypeAlias` is provided.
### Format
byte[4] - version
byte[4] - count
--array--
byte[8] - xxHash
byte[1] - flag, 0 - null, 1 - not null
byte[n] - Full class name in UTF8
### Example
<img width="472" alt="SerializedTypes_Example" src="https://user-images.githubusercontent.com/3953314/195255429-31d24293-6bd1-419e-811b-07874dd0f78d.png">
## Benchmarks
Serialized 500 Type fields. The 8192bytes comes from the _ConcurrentQueue_ that would later be used for SerializedTypes.
Note that the queue is never cleared, so it's size grew considerably.
```cs
| Method | Mean | Error | StdDev | Allocated |
|--------------------- |---------:|---------:|---------:|----------:|
| BenchmarkXXHash | 18.44 us | 0.278 us | 0.260 us | 8192 B |
| BenchmarkTypeStrings | 25.09 us | 0.292 us | 0.259 us | - |
```
TODO:
* Add support in the Serialization Generator for `ReadType()` and `Write(Type)`
* Remove `SetTypeRef` from Serialization Generator
## Breaking Changes (New API)
ObjectPropertyList supports the following API:
```cs
list.Add(500000);
list.Add(500001, stringArgument);
list.Add("Some text");
list.Add($"Some text with {argument}");
list.Add(500002, $"{arg1}\t{arg2}");
```
## Notes
1. All API uses that require a formatter like this:
```cs
list.Add(500002, "{0}\t{1}", arg1, arg2);
```
Should be changed to use string interpolation, for example:
```cs
list.Add(500002, $"{arg1}\t{arg2}");
```
2. The following paradigm should no longer be used:
```cs
list.Add(1061170, prop.ToString()); // strength requirement ~1_val~
```
The new string interpolation API will avoid having to convert the argument to a string before writing it to the packet. Instead use the following:
```cs
list.Add(1061170, $"{prop}"); // strength requirement ~1_val~
```
### Benchmarks
```cs
| Method | Mean | Error | StdDev | Gen 0 | Allocated |
|------------------------------- |---------:|--------:|--------:|-------:|----------:|
| BenchmarkOldOPL | 241.0 ns | 0.56 ns | 0.47 ns | 0.0105 | 88 B |
| BenchmarkStringInterpolatedOPL | 199.9 ns | 2.44 ns | 2.39 ns | - | - |
```
### Changes
- [X] Removes crash in STArray.Return when array is null.
- [X] Fixes NPE in OPL when entity is null. Serial in packet will be 0 when entity is null.
- [X] Fixes NPE in AosAttributes when Parent is null.
- [X] Changes OPL to use string interpolation.
- [X] Introduces `IPropertyList` to allow extending PropertyList for other uses.
* Adds the following configurations:
* `movement.delay.npcMinDelay` - 0.1 - Pets or Non-monster NPCs
* `movement.delay.npcMaxDelay` - 0.4 - Pets or Non-monster NPCs
* `movement.delay.monsterMinDelay` - 0.4 - Non-pet Monsters or NPC vs Player Combat
* `movement.delay.monsterMaxDelay` - 0.8 - Non-pet Monsters or NPC vs Player Combat
* `movement.delay.monsterMinDex` - 150 - Dex maximum for delay by dex
* `movement.delay.MinDex` - 190 - Dex maximum for delay by dex
- [X] Fixes map issues at New Haven by turning off static diffs
- [X] Some code cleanup and reformatting of tile matrix, tile matrix patch, and tile data
- [X] Adds NetState.Flush for generating spawners so it doesn't feel like the server is frozen
- [X] Reverts NativeReader changes from a while back.
- [X] Adds more string reading for BufferReader.
Notes:
BufferReader is still `little endian` compared to `SpanReader` which is `big endian` (for packets). To that end, the RunUO deserialization `ReadString()` was made obsolete since it is ambiguous, and contains extra fields other than simply reading a string. Furthermore, we shouldn't be using UTF8 (for now) since it is slow.
- [X] Speeds up world loading
- [X] Removes PeekChar cause it wasn't used
- [X] Adds a BufferReader
- [X] Removes BinaryFileReader
- [X] Removes reading/writing `char` since it is not consistent and will cause a deserialize issue if used.
Bumps version release
- [X] Removes all save strategies
- [X] Removes duplicate file writer that won't be used
- [X] Removes persistence (it will become a duplicate system)
- [X] Add a save position variable to skip serializing a clean item/mobile
- [X] Update Guilds/Accounts to be IEntity types
- Because guilds are abstract, this may make serialization tricky.
- [X] Update the generalized IEntity writing
- [X] Update the load/save to write to buffers then to files in background
Bumps release version
- [X] Removing Kestrel & Libuv
- [X] Cleaning up NetState
- [X] Removing System.IO.Pipelines
- [X] Cleaning up packet reading
- [X] Adds a maximum of 5000 sockets (configurable) to prevent OOM
- [X] Replaces the AsyncState with a thread-safe wrapped boolean called NetworkState
- [X] Removes Parallel.ForEach (no perf gain)
- [X] Removes custom houses compression on another thread
- [X] Test high load scenarios
Bumps release version
