PlayerManager saves offline players in bulk, once very hour. This is done in bulk to help avoid desync between offline player updates.
Player caching is typically set to 30 min.
What can happen now is that PlayerManager may save a bunch of offline players, that will then be loaded into the player biota cache (for 30 min)... then after 30 min, they all get unloaded because nothign touches those biotas, then 30 min later again the PlayerManager performs its hourly maint of bulk save, thus, re-caching all those biotas.
Caching is not necessary for these biotas.
Caching is designed for biotas that exist in an online state.
* Step 1. Add new adapter classes and models
All classes and objects added are in new namespaces.
This add has 0 interraction on existing ACE code.
This sets the groundwork for Step 2. Moving World DB caching to the new models
* first part of the world entity usage
* almost there
* final part
* update WeenieExtensions
* fix
* merge fixes and improvements
* ACE.Database new code
* it compiles
* InitializePropertyDictionaries
* WOrldDatabaseWithEntityCache cleanup
* ShardDatabaseWithCaching
* ShardDatabase Cleanup
* todo
* cache counts reported in serverstatus
* House InitializePropertyDictionaries
* improve ServerStatus
* BiotaModel change HousePermissions to Dictionary<uint, bool>
* PropertiesAllegiance to dictionary
* BiotaUpdater improvements
* Shard Database Cache Reporting
* More code ported from Step 3
* Remove stub CharacterCache
* Update WorldDatabaseWithEntityCache.cs
* build fixes
* code audit
* Remove non-callback functions from SerializedShardDatabase
* BiotaUpdater progress.. only emote left
* Emotes done... now need to test
* fix
* Burden/Value fix for containers
* fat fingered EncumbranceVal
* add some safety
* Don't cache player biotas for initial PlayerManager load
* fix null spellbook
* Add another null check
This is redundant as the check is also upstream.
This is to prevent an exception in the future if this function is called from another source.
* prototype: adding account-wide house recall
* updating restriction db / has permissions
* additional ObjMaint refactoring
* re-adding CheckPlayers()
* the call to GetVisibleObjects() in Monster.FindNextTarget() is no longer needed in new design
* removing more unused code
* very preliminary testing
* prevent 1 character from purchasing multiple houses
* adding account/character houses
* further work on AccountHouses / CharacterHouses
* latest updates, account/character multihouse polish, apartment deed location
* adding /house-select process
* HouseManager refactoring, removing async, fixing PlayerHouse.CompareTo equals, fixed InventoryLoaded for weenies, added Player_House.GetHouseInstance()
* first posted draft
* wiring up the remaining HouseInstance -> GetHouseInstance()
* preventing maintenance items from being added for houses owned by players in multihouse state
* cleaning up log messages, handling some edge cases
* Add GetCharacterByGuid
* Adjust finger command to only return "playable" characters, and correct accounts for offline players
* Add IsDeleted and adjust IsPendingDeletion
* Rework Name and IsPlussed a bit
* Add basic support for player weenie changes when accesslevel changes
* Cleanup
* Update changelog
* cleanup
* clean up returns
* Return to Virtual + Override
* cleanup
* Restore @rename functionality
* Strip + from killer's name in Corpse.LongDesc for VTank compatiblity
* Add @pk command
* Add in support for PlayerKillerStatus.Free
* Add Chorzite to comps list
* If Level is greater than maxlevel, do not crash
* Fix @morph command
* Reuse available Dynamic GUIDs
On ACE startup, the shard database is queried for available id's in the dynamic guid range.
This helps further reduce (max + 1) id allocation.
It's a bit premature, but, based on current architecture, I don't know if we'll ever need to do a id defragment.
A requirement of this method is that ace_world may only contain guids in the static range.
* GetMaxGuidFoundInRange has no valid use in ace_world
All ids must be in the static range
* Add world update script
* Add Shard biota count to serverstatus
* Remove unneeded null check in Session.cs
* Progress
Remove Player.InWorld. This was only used for command manager, instead, we can check if CurrentLanblock != null
Improve Player.SaveToDatabase() processes
Add PlayerManager.ProcessDeletedPlayer stub
Reduce Player.EnqueueLogout code
* More Player Logout, Save and Session Drop progress
* fix admin boot command
* More cleanup
* Player_Inventory GetPickupItem() should not return error when item not found. Changed to debug
* PlayerManager initial checkin
* OfflinePlayer initial checkin
* More progress
* Comment out Console.WriteLine debug messages
* Add property dictionaries to OfflinePlayer
* IPlayer initial checkin
* Add allegiance vars to IPlayer
* Couple comments
* More progress
* Add more to IPlayer
* line endings fix
* More progress. Compiles. Friends list works again.
* Friend status updates now work (again)
* PlayerManager save OfflinePlayers every 1hr
* Bye bye legacy AllPlayers
* using cleanup
* Allegience switch to PlayerManager progress
* More Allegiance progress
* More progress
* More allegiance progress
* IPlayer adds
* Migrate inversefriends to PlayerManager
* Switch WorldManager GetPlayerByGuid over to PlayerManager
* WorldManager Find(ObjectGuid characterGuid) removed. Use PlayerManager instead
* WorldManager FindByPlayerName(string name) removed. Use PlayerManager instead
* PlayerManager thread safety switched to ReaderWriterLockSlim
* More progress
* ServerStatus added TotalAccountsCreated, TotalCharactersCreated
* No more Player access/management stuff in WorldManager. It's all in PlayerManager now
* Player.IsOnline is no more.
* Friends AppearOnline fix
* Handle dead session better to switch player from online to offline
* Remove adding the + to names in ChatChannels. Name property should be changed to not include +
* serverstatus total accounts and characters on same line
* Allegience initial fix, still more bugs though
* PlayerManager minor cleanup
uint should be faster key than ObjectGuid
Changed the dictionaries to private. We don't want anyone outside of this class referencing them.
* Thread safety added to LScape.get_landcell
* Landblock EnqueueBroadcast cleanup
Avoid double cast by using OfType
Change excludeList from IEnumerable to ICollection because it was being enumerated more than once.
* SerializedShard handle OperationCancledException
* SerializedShard break on the exceptions
* Landblock save progress
* Landblock merge fix
* Decayable -> Dynamic
* Improved ShutdownServer to include landblock unload and db queue wait
* Sleep during shutdown instead of using up all the CPU
* Shard GetDynamic was comparing against the wrong id... woops
* Fix shard biota caching
* Improve shutdown logging and actions
* cosmetic code order
* Fix crash when trying to add a bad wo to a landblock
* This fixes reloading dynamic objects
* Don't save item on pickup
* Don't save item on pickup from chest
* Don't save items on drop
* Set Home Position only for creatures (Used for navigation)
* Improve the landblock SaveDB code
* Don't save player corpse on death. Landblock will do that for us in bulk.
* Add the housing objects to statics that persist to the shard
* WorldObjectFactory dont overwrite location if object came from database
* Improved IsStaticThatShouldPersistToShard
* This should fix player dropped missiles
* Move PositionType into ACE.Entity.Enum.Properties
* Ephemeral support for Positions
* Decay code/saving works much better
* Filter out contained and wielded items from shard GetDynamicsByLandblock
* TimeToRot provision for -1 (No rot)
* Corpse decay is now controlled by landblock and uses the same decay code
* ShardDatabase: Fix add/remove biota bug
* WorldObject.Destroy now destroys contained and equipped items as well
* TryRemoveFromInventoryWithNetworking no longer removes the object from the db
* Player corpse items now remain when the corpse decays
* Cleaned up SetPropertiesForWorld
* RateMonitor created
* NetworkSession Update foreach change to help profiling
This just removes the lambda pattern for the foreach so profilers can monitor each line individually.
Functionality is unchanged.
* Reduce Landblock ctor async work to a single thread
This helps reduce ACE thread starvation
* NetworkSession cosmetic
* NetworkSession cosmetic
* NetworkSession currentBundles ConcurrentDictionary to Array
This significantly improves the performance of NetworkSession.
Arrays are faster than Dictionaries, and we manage concurrency to the elements via currentBundleLocks.
We're also able to get away with using an array because the number of elements is only 12.
Accessing the indexes of currentBundles should be atomic as it's an array of reference values, compiled as 64 bit.
* Add InParallel to database functions that do their underlying work in parallel
* DoSessionWork tick outbound messages in series, not in parallel.
* NetworkSession should not be interfacing with ActionChains
* Don't load landblock ctor resources in parallel
Save the threads in the pool for more important work
* LandblockManager removed check/recheck
This is legacy code from a pattern no longer used. It is no longer required.
* Cosmetic
* First batch of Character refactor
Characters are now cached with their dbContext, exactly like we do for Biotas. This greatly simplifies the way characters are pulled and saved, and is much more efficient.
Deleting and restoring characters have been migrated to the new SaveCharacter function. There is no need for separate functions to handle those specific tasks.
IsPlussed column added to Character. This removes the need to query the biota to see if we should display the + in front of the name at the character screen.
Character.LastLoginTimestamp, TotalLogins, CharacterOptions1, and CharacterOptions2 are now used. These property types were removed from the ACE.Entity.enum. (the 9000+ types)
Fixed a few places where ChangesDetected was being set insetad of CharacterChangesDetected. There is still more work to do to improve this pattern.
* Player_Client renamed to Player_Character
Plus some cleanup on the way CharacterOptions were get
* Remove duplicate appearance DID properties from WorldObject_Properties
* Removed the duplicate HairTexture DID properties
* Migrate HairTexture from biota property to Character property
* Forgot a CharacterChangesDetected
* Migrate Character specific functions to Player_Character
* Some more Biota wrappers added to BiotaExtensions
* CharacterPropertiesShortcutBar wrapped
* CharacterPropertiesSpellBar migrated to CharacterExtensions
* CharacterPropertiesTitleBook migrated to CharacterExtensions
* CharacterPropertiesFriendList migrated to CharacterExtensions
* drop AccountId from CharacterPropertiesFriendList
* SaveBiota major changes
SaveBiota now only updates values in the database that have changed instead of updating every single value.
In addition. SaveBiota detects added and removed linked entities.
* Replace AddBiota(s) with SaveBiota(s)
* This improves the way a few places referenced the biota
* SaveBiota switched to a diff method. Still some work to do
* More notes + fixes
* SaveBiota move GetBiota to before EnterWriteLock
* Added a note about the lock performance
* More summary improvements for BiotaDatabaseLock
* Thread safety added to the SaveBiotas and AddBiotas functions
SavePlayer also switched to use the SaveBiotas bulk action instead of saving each biota individually.
* Biota.Clone() added
* BiotaExtensions cosmetic only
* DbUpdateException fixed
If there were multiple new records in a collection, the first one was attached but with the last new records values.
This would end up resulting in property records with duplicate types id's.
* ReoveBiota check if biota actually exists in db before removal.
* /databasequeueinfo added
* New From-Weenie Containers should not be searching the shard for inventory
* DeveloperDatabaseCommands removed RequiresWorld flag
* Improved DatabasePerfTest results.
My test results:
Starting Database Performance Tests.
Biotas per test: 1000
This may take several minutes to complete...
1000 individual add . Duration: 29.9 seconds. Successes: 1000, Failures: 0. Average: 30 ms
1000 individual save . Duration: 23.9 seconds. Successes: 1000, Failures: 0. Average: 24 ms
1000 individual remove. Duration: 23.7 seconds. Successes: 1000, Failures: 0. Average: 24 ms
1000 bulk add . Duration: 8.5 seconds. Successes: 1, Failures: 0. Average: 9 ms
1000 bulk save . Duration: 6.2 seconds. Successes: 1, Failures: 0. Average: 6 ms
1000 bulk remove . Duration: 7.0 seconds. Successes: 1, Failures: 0. Average: 7 ms
Database Performance Tests Completed
Next step is to modify the save to detect added/removed entities and only update changed records (not all).
* Shard PopulatedCollectionFlags
This greatly improves GetBiota performance.
It does so by recording the biotas populated collections on save, and then on Get, only pulling those known populated collections back from the database.
* Note added about GetBiota improvements
* Shard Parallel loops added for GetPlayerBiotas
This improves performance signifcantly.
Previously, a full mule too 8s to load.
Now, it's uder 500ms.
Previously, PropertyManager used SerializedShardDatabase to wrap it's database calls.
This didn't make much sense because SerializedShardDatabase was basically just a wrapper that serialized calls to ShardDatabase.
So, instead, we now have a ShardConfigDatabase class as well that is held by DatabaseManager. This holds the property/config specific funcitons for the shard.
In addition, there were a couple bugs fixed in PropertyManager where a function migt have been GetLong() when it should have been GetDouble(), etc...
* Remove Parallel.ForEach from Fellowship
No need for parallel execution for these simple loops
* Cleanup Session/Player save call stack
* comment fix.
* Another comment added
* Config Properties cosmetic cleanup
This is in preparation for the properties/database refactor.
* Session.AccountCharacters renamed to Characters
* includedDeleted option added to Shard GetCharacters()
* change Shard GetCharacter includeDeleted parameter order
* Character pruning removed from GameMessageCharacterList
This class shouldn't be doing the pruning. It should be the caller.
This class should send the characters exaclty as they're passed. It's not good to have functions like this that are manipulating your data before sending it off when it's not obvious.
* Sorting characters by LastLoginTimestamp not needed
Client auto sorts first by isDeleted, then, by name.
* Logging off doesn't reload characters
No need to reload them from the database. The session already has all it's owned non-deleted characters.
We simply check the characters for deletion and remove them from the list if needed.
This reduces database overhead.
* using statement cleanups. Purely cosmetic
* CharacterHandler code reorder
I prefer top down.
Character create, enter world, log off, delete, restore
* AuthenticationHandler bug fix for characters that are marked deleted on initial login
* CharacterHandler Delete/Restore no longer reload entire character list
This improves DB efficiency
* Session code cleanup. Mostly just organization
* Remove IActor from Session and add InboundMessageQueue to WorldManager
* Improve WorldManager ActionQueue names
* Add back character sort on initial connect
This removes Character as the parent of a Biota, and instead, makes them side by side objects that are owned by Player.
Session no longer owns Character.
Session maintains a list of the Characters for the account.
Session holds a reference to the current Player being used.
Player holds a reference to Biota and Character objects.
What I said in discord:
We have the Player object (our god class).
To put a player in the world, it requires a biota. It doesn't require a character or session.
To manipulate the player, we need a session (but not a character).
To give the player a user interface, we need a character.
The character holds all the user progress of the particular biota it's associated with.
So, a session holds a link to the current player the session is managing.
Player holds a link to the biota and character that represent the player.
The biota represents the player from the worlds pov.
The character represents the player from the users pov.
* Adding landblock unloading system, and player corpses
* Adding dropped items to player corpse
* Adding landblock unloading system, and player corpses
* Adding dropped items to player corpse
* Adding permaload flag, and updating TimeToRot on landblock unload
* Adding corpse looting permissions
* Adding stack sizes
* Cleanup
* Allegiance WIP
* Ranks, XP passup calcs
* Granting Allegiance XP passup to patrons
* Commenting debug messages
* Fixing Allegiance tab
* Adding Allegiance exceptions
* Adding break allegiance
* add ability to /tele players by name (#823)
* add ability to /tele players by name
* command parsing resiliency
* Give Request fix (#824)
* Give Request fix
Selected Wielded Objects as well as Inventory Objects. This fixes crash of gioving wielded objects to the town crier.
* Deleted unnecessary semicolons
* requested return for null
* UI panel refresh, apply XP earned by vassals on login
* Unequip and burden correction (#826)
* Add a configurable property storing database to resolve issue #753 (#820)
* Added the configuration database base script.
This should initialize the config database to hold different values that can be changed during the server operation.
* Added database models and connection.
This should begin to add in the new configuration database and its models so that it can be accessed from the server.
* Added boolean statistic access methods.
Modified the database file so that it was properly casting to a boolean and added the methods into the server configuration database API.
* Added the ability to access/modify integers in the config database.
* Added the ability to access/modify floating point variables in the configuration database
* Added the ability to access/modify string values in the configuration database
* Added the ServerConfigurationDatabase to the DatabaseManager.
* Added in the initial caching layer for configuration
* Added the ability to modify cached items.
* Changed from regular dictionaries to concurrent dictionaries.
* Added in the ability for the ConfigurationManager to self-update from the database.
* Added fallback parameters instead of forcing a default
* Added customizable vitae statistics.
This is the first big stat that can be modified. It is a float, so more floats should now work when they have support added.
* Updated the configuration manager to separate methods and ensure semantics are correct.
* Added command to fetch float value from server
* Add property for XP/Lum multipler.
* Welcome message/MOTD is now configurable from the database.
* Added string modification commands
* Database is no longer required for server configuration.
* Added in all modification/fetch commands and a resync command
* Renamed ConfigurationManager -> PropertyManager as this removes ambiguity
* Changed logging string to be more consistent
* Pass w/ StyleCop
* Added comments to the property manager
* Added the ability to disable caching fallback values.
This can be used for default settings or if a fetch with the default fallback should not cache (ie fetching xp_multipler before xp_multiplier has been fetched from the DB/the default value has been given, and therefore xp_multiplier is fetched as 0 and cached as 0). This functionality should be used for server commands.
* The admin commands now properly obey cacheFallback restrictions
* Added initial default property handling.
* Moved to using constant values as defaults.
* Switched to using a Dictionary for resolution of default properties
* Added warning when the server configuration database is not loaded.
* Corrected MOTD default string
* Renamed some of the database connection objects and changed float -> double and int -> long
* Fixed admin commands not being named correctly
* Fixed spelling error
* Config tables now lie in the shard database
* Added descriptions to properties across the board that can be analyzed through admin commands
* Removed property from PlayerXP file
* Removed vitae properties from EnchantmentManager
* Added null checks to make output nicer
* Added command to modify property descriptions
* Fixed string not being stored properly in the DB and crashing the server
* Wrapped the old ServerPropertyDatabase into SerializedShardDatabase
* Fixed ShardDbContext having a modified OnConfiguring method
* Allegiance WIP
* Ranks, XP passup calcs
* Granting Allegiance XP passup to patrons
* Commenting debug messages
* Fixing Allegiance tab
* Adding Allegiance exceptions
* Adding break allegiance
* UI panel refresh, apply XP earned by vassals on login
* Added the configuration database base script.
This should initialize the config database to hold different values that can be changed during the server operation.
* Added database models and connection.
This should begin to add in the new configuration database and its models so that it can be accessed from the server.
* Added boolean statistic access methods.
Modified the database file so that it was properly casting to a boolean and added the methods into the server configuration database API.
* Added the ability to access/modify integers in the config database.
* Added the ability to access/modify floating point variables in the configuration database
* Added the ability to access/modify string values in the configuration database
* Added the ServerConfigurationDatabase to the DatabaseManager.
* Added in the initial caching layer for configuration
* Added the ability to modify cached items.
* Changed from regular dictionaries to concurrent dictionaries.
* Added in the ability for the ConfigurationManager to self-update from the database.
* Added fallback parameters instead of forcing a default
* Added customizable vitae statistics.
This is the first big stat that can be modified. It is a float, so more floats should now work when they have support added.
* Updated the configuration manager to separate methods and ensure semantics are correct.
* Added command to fetch float value from server
* Add property for XP/Lum multipler.
* Welcome message/MOTD is now configurable from the database.
* Added string modification commands
* Database is no longer required for server configuration.
* Added in all modification/fetch commands and a resync command
* Renamed ConfigurationManager -> PropertyManager as this removes ambiguity
* Changed logging string to be more consistent
* Pass w/ StyleCop
* Added comments to the property manager
* Added the ability to disable caching fallback values.
This can be used for default settings or if a fetch with the default fallback should not cache (ie fetching xp_multipler before xp_multiplier has been fetched from the DB/the default value has been given, and therefore xp_multiplier is fetched as 0 and cached as 0). This functionality should be used for server commands.
* The admin commands now properly obey cacheFallback restrictions
* Added initial default property handling.
* Moved to using constant values as defaults.
* Switched to using a Dictionary for resolution of default properties
* Added warning when the server configuration database is not loaded.
* Corrected MOTD default string
* Renamed some of the database connection objects and changed float -> double and int -> long
* Fixed admin commands not being named correctly
* Fixed spelling error
* Config tables now lie in the shard database
* Added descriptions to properties across the board that can be analyzed through admin commands
* Removed property from PlayerXP file
* Removed vitae properties from EnchantmentManager
* Added null checks to make output nicer
* Added command to modify property descriptions
* Fixed string not being stored properly in the DB and crashing the server
* Wrapped the old ServerPropertyDatabase into SerializedShardDatabase
* Fixed ShardDbContext having a modified OnConfiguring method
* World DB Rekeyed
* Rebase World DB
* Rebase Shard DB
* Update PK in EmoteAction
* Shard DB Rekeyed. This change creates lots of exceptions currently.
* Revert to Shard/World DB Rebase
* Restore Id to each table
* Restore Id to Shard tables
* Update World DB pr location
* Missed this ObjectId > Id change
* Rename Tables
* Updating Changlog and Readme. This PR fixes#782 and fixes#783
* Updating Changelog again to note that you must drop and re-create Shard and World DB due to changes
Records (properties) are removed from the db as they're removed from a biota. This is because the context can't track removed entries after the context has been disposed.
Adding new records (properties) does not automatically save to the db. Instead, periodic saves, or saves at critical points, will cause the entire biota to be saved to the db which will update any existing records (properties) as well as add any new ones.
