### Summary
- `GenericEntityPersistence` is now a type of `GenericPersistence`. This allows developers to serialize both entities and non-entities in the same system. 🎉
- Each `SerializationThreadWorker` now allocates 1MB of heap for serialization _permanently_. If more memory is needed, that thread will double it's memory, not to exceed increments of 64MB.
- Several bugs with serialization introduced with the pure MMF implementation have been fixed.
- `BinaryFileReader` has been added back. 🎉
- Adds `world.useMultithreadedSaves` to allow disabling threaded saves.
> [!IMPORTANT]
> **Developer Note**
> The split file serialization has been deprecated and is no longer used. We have effectively gone back to the same file writing we had before the pure MMF implementation.
> [!Important]
> **Developer Note**
> This code change will **completely move gumps out of the core**
### Summary
- Adds `GetGumps()` convenience which exposes methods to Find/Close/Send multiple gumps. This helper is a performance improvement by eliminating the Dictionary<Player, List> lookup for gumps.
### Summary
* Added World.NewVirtual for creating virtual serial numbers
* Reserved range 0x7EEEEEEE to 0x7FFFFFFF for virtual serials
* Hair and Facial hair (for mobiles) now use virtual serials instead of FakeSerial() functions
* Consolidated virtual hair to a single `VirtualHairInfo` class.
Corpse hair and facial hair now persists across save/load and hair and facial hair no longer teleport to newest corpse.
## Summary
- Removes allocation of a `List<ContextMenuEntry>` every time a context menu is created.
- Moves packet/context menu creation logic out of the core
- Fixes tame entry
## BREAKING CHANGE
> [!Important]
> **Developer Note**
> ```cs
> public virtual void GetContextMenuEntries(Mobile from, List<ContextMenuEntry> list)
> ```
> and similar functions changed to
> ```cs
> public virtual void GetContextMenuEntries(Mobile from, ref PooledRefList<ContextMenuEntry> list)
> ```
### Summary
Updates the serialization strategy to use `MemoryMappedFile` instead of thick buffers. This has the benefit of being on-par with the current implementation (based on hardware/OS), however won't incur the double-memory issue.
> [!Important]
> **Developer Note**
> The `BinaryFileWriter` and `BinaryFileReader` has been removed in favor of `MemoryMapFileWriter` and `UnmanagedDataReader`
### Summary
- Generalizes the On/Off toggle items concept
- Updates the OnOffGump so it is static
- Standardizes OnOff items so they can be used by staff
### Notes
Decided to not fix#1417 because it is not clear that the clilocs or errors are for that purpose. Can't test this on OSI anyways.
> [!Warning]
> Users on Linux/OSX will need to follow the Readme
> and make sure `libdeflate` is properly installed
> [!Note]
> **Developer Note**
> The API for compression has changed. Use `Deflate.Standard` for the same functionality.
### Summary
* Replaces Zlib with LibDeflate for a 50% performance improvement!
* Adds MacOS 14 to properly test Arm64
# New Gump API
We are pleased to release a new API that is faster, allocates nearly zero memory, and still feels very similar to the original API. The API is broken into 3 types of gumps, dynamic, static with placeholders, and static without placeholders.
### Dynamic Gumps
These gumps will inherit `DynamicGump` and are meant for gumps that have a dynamic layout. This includes specifying dynamic arguments to HtmlLocalized entries.
## Static Gumps
Static gumps are those where the function to the build the layout is called only once and cached forever. They can optionally have placeholders. These placeholders allow the developer to specify the string values later, dynamically in a `BuildStrings` method on the gump. If a gump does not have any placeholders, the string entries will also be cached forever.
## Benchmarks
To make sure we were going in the right direction and not wasting time, we took copious benchmarks. Here are the final benchmarks for a really simple gump.
Note:
* The majority of creating a gump is compressing the layout and the strings. Compressing each section takes ~6,000ns (12us total).
```cs
| Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|------------------------------- |-------------:|-------------:|-------------:|-------------:|------:|--------:|-------:|----------:|------------:|
| OldGump | 13,308.29 ns | 1,059.695 ns | 1,883.608 ns | 14,330.72 ns | 1.000 | 0.00 | 0.1526 | 2400 B | 1.00 |
| DynamicLayoutGump | 13,357.86 ns | 129.144 ns | 226.185 ns | 13,323.60 ns | 1.029 | 0.17 | - | 48 B | 0.02 |
| StaticLayoutDynamicStringsGump | 6,653.10 ns | 81.815 ns | 143.292 ns | 6,617.45 ns | 0.514 | 0.09 | - | 40 B | 0.02 |
| StaticLayoutGump | 86.33 ns | 0.760 ns | 1.350 ns | 86.07 ns | 0.007 | 0.00 | 0.0020 | 32 B | 0.01 |
```
# Non-Breaking Changes
* All gump components in the core have been moved to `Gumps/Legacy`.
* All legacy gumps will still inherit `Gump`, which now inherits `BaseGump`
# Special Thanks
Thank you to @stefanomerotta for considerable contributions/benchmarking/testing to make this effort a reality! We collectively went through over 10 iterations, but it is finally ready.
### Summary
* Removes old gump packet support
* Removes support for v4 clients
* Removes `Unpack` flag and assumes it is always true.
* Removes StringToBuffer since this is built into .NET now.
> [!Note]
> View the file changes with white space off: https://github.com/modernuo/ModernUO/pull/1739/files?diff=split&w=1
## Breaking Changes
* The Firewall and IP Limiter have been rewritten. Please read the notes carefully!
* `TcpServer.Instances` moved back to `NetState.Instances` - sorry - it was stupid to move it to begin with.
> [!Note]
> Sockets that fail the IP Limiter or Firewall will be immediately and forcibly disconnected.
> This means they will be stuck at "Verifying account..." if it was a real client.
### Summary
- Removes firewall wildcard support.
- Removes `AccessRestrictions`.
- Moves Firewall/IPLimiter to the core.
- Moves `TcpServer` to its own thread.
- Removes the `SocketConnect` and `SocketDisconnect` event sinks.
- Moves `Instances` back to `NetState.Instances`.
- Fixes a long standing bug with bad handling of duplicate listener addresses.
#### Firewall
The firewall has been completely rewritten. There is now an "Admin Firewall" which saves to the config file. Secondarily, there is an internal firewall used exclusively by the TcpServer while processing sockets. The Admin firewall mirrors it's additions/deletions to the internal firewall by adding requests to a queue.
> [!IMPORTANT]
> **Wildcard firewall entries, such as `X`, `*`, `?` are not allowed.**
> **Ranges in between IP classes or sextets are not allowed.**
> **Please make sure to use one of the following:**
> * IP Address - `192.168.1.1`
> * CIDR - `192.168.1.0/24`
> * Range - `192.168.1.1-192.168.1.100`
#### IP Limiter
The IP Limiter has been completely rewritten. The available configurations are:
```json
"ipLimiter.enable": "True",
"ipLimiter.maxConnectionsPerIP": 10,
"ipLimiter.clearConnectionAttemptsDuration": "00:00:00:10",
"ipLimiter.clearThrottledDuration": "00:00:02:00",
```
The IP Limiter is set up to prevent spamming connections from the same IP. Every time an IP connects, it is added to a connection list. After 10 attempts, the IP is added to the throttle list. To keep the system fast, the connection list is entirely wiped every 10 seconds, and the throttle list is entirely wiped every 2 minutes.
### Summary
- Removes old death packet that isn't used. Doubtful this causes issues with clients that are v4+.
- Removes duplicate incoming packets. Again, probably to fix some old client issues, doubtful it affects clients v4+.
- Fixes setting serials and entities in props/commands. Note: Disabled setting `Parent` since the new sector code has issues. We shouldn't rely on it anyway!
- Reverts a recent change to healthbars that should not have been made. Oops!
### Summary
- Fixes usernames not being `Intern`ed
- Reverts methods related to getting accounts from returning `Account` to `IAccount`.
- Makes `IAccount` also `ISerializable`
- Adds `IGenericReader.ReadAccount()` and `IGenericWriter.Write(IAccount)` -> The read method supports the original serialization of username, and using `IAccount.Serial`. The write method only serializes the `Serial`.
- Exposes `ReadStringRaw()` to allow some advanced scenarios.
## Breaking Changes
Incoming packet registration signature has changed to:
```cs
delegate* void OnReceiveCallback(NetState state, SpanReader reader, int packetLength);
IncomingPackets.Register(int packetID, int length, bool ingame, OnReceiveCallback onReceive);
```
For example, an incoming packet handler signature would now look like this:
```cs
public static void SomeIncomingPacket(NetState state, SpanReader reader, int packetLength)
{
// Parse the data
}
```
## Summary
Updates the network Pipe class to use a mirrored memory technique. This technique involves mapping the same physical memory to two contiguous virtual memory spaces so the byte buffer appears duplicated. This allows writing to a double-sized array to wrap around without the need for the `CircularBuffer` classes.
In practice this allows us to use `Span<byte>` as if the buffer was a regular array.
### Bug Fixes
- [X] Fixes bad fixed length string parsing
### Summary
- Removes Fastwalk system
- Removed the following settings:
- `movement.enableFastWalkPrevention`
- `movement.fastwalkExemptionLevel`
- Adds movement throttle system.
- Adds the following settings:
- `movement.throttleReset` - Default value is `1000` (1 second).
- `movement.throttleThreshold` - Default value is `400` (400ms).
### Movement Throttling
This new system will trigger if a player requests 400ms (configurable) worth of movements quicker than wall clock time. When this happens, the player is throttled (all incoming packets to the server are halted) until wall clock time catches up with the requests. Upon each throttle, the player receives enough credit to handle up to 400ms of "lag" as a grace/catch-up.
### Developer Notes
We use two throttle queues to prevent an infinite loop.
**Only one functional change**
* Fixes a bug in LogFactory where `Warning` is being logged as `Information`
Non-functional changes:
* Updates/Fixes copyright headers
* Removes namespace scopes for core files.
View with [whitespace off](https://github.com/modernuo/ModernUO/pull/1187/files?w=1).
- [X] Fixes NPE from account tags.
- [X] Fixes bad skill check due to missing cast to double.
- [X] Fixes water elemental duration.
- [X] Standardizes spell summon duration by expansion.
- [X] Fixes issue with wepoll losing GCHandle.
- [X] `NetState.Disconnect()` is no longer thread safe.
- Use `Core.LoopContext.Post()` to post disconnects
- [X] Optimizes PollGroup by not processing IntPtr -> GCHandle for discard polls.
Changes gump compilation to use string interpolation. .NET 6 uses code generation and compile time tricks to speed up string interpolation between 15 and 30% and reduce allocations dramatically.
- [X] Fixes map issues at New Haven by turning off static diffs
- [X] Some code cleanup and reformatting of tile matrix, tile matrix patch, and tile data
- [X] Adds NetState.Flush for generating spawners so it doesn't feel like the server is frozen
- [X] Reverts NativeReader changes from a while back.
- [X] Adds more string reading for BufferReader.
Notes:
BufferReader is still `little endian` compared to `SpanReader` which is `big endian` (for packets). To that end, the RunUO deserialization `ReadString()` was made obsolete since it is ambiguous, and contains extra fields other than simply reading a string. Furthermore, we shouldn't be using UTF8 (for now) since it is slow.
- [X] Removes network pause/resume
- [X] Adds back packet profiler
- [X] Adds state machine to keep track and trace netstates
- [X] Changes NetState.Running back to using an interlock exchange
- [X] Adds preliminary packet throttling support
### Packet Throttling
- `[GetThrottle <packetId>` to get the delay in milliseconds for that packet
- `[SetThrottle <packetId> <delay>` to set the delay in milliseconds for that packet
The settings are saved to `Configuration/throttles.json`
- [X] Converts boat packets
- [X] Makes a packet container builder for packet 0xF7
- [X] Generalizes world item packet so it works for mobiles too
Notes:
- This PR doesn't address proper smooth movement for boats.
- [X] Converts some player packets
Note:
- Not caching weather packet because it is copying directly from stackalloc using aggressive inlining. Don't need to do more optimizations.
