Batch updates and performance
Kris pointed out that I was handling batch updates differently; I had a batch for joining and a batch for those already in a region,
Where in osrs the batching packet is only for those in an area, those joining get everything sent as individual packets when they enter.
Which in theory is less efficient but there was a lot of extra complexity and storage used in the way I was doing it, so I opted for the simpler code.
This had the knock on effect of needing to modify FloorItems and GameObjects, systems which I wasn't overly happy with anyway so I decided to take the opportunity to rewrite those as well. FloorItems isn't that interesting just a tidy up but GameObjects came with a lot of improvements.
For context, on games first run it loads all the objects and tile data from the cache and writes it to a flat file, this means on subsequent runs it can be read faster, and while the game is running it can load map data for instances/dynamic maps without needing to hold the cache in memory (cache is discarded after startup).
Prior to recent changes, loading the full map from this flat file into memory would take on average 668.4ms and the server was using 1.1GB ram. This is much faster than using the cache directly, however still quite slow for startup. So I added a toggle which would choose to store only objects which have options, or have data in the server's objects config yaml file. Essentially only objects which are currently have content and are in use.
That cuts the stored object count from ~3.5m objects down to ~74k, and load times down to an average of 398ms and 454.68MB ram. This was the start point.
First thing to go was GameObject's as Entity's. They were heavy as each object had an event bus and a temp properties map and lots more unnecessaries, storing them instead as a Long as an inline value.
Now GameObjects were numbers they could be stored more efficiently too, I figured if you're using the objects location as the storage index you only need to store the objects id, type and rotation which all easily fit into an integer; not unlike how collision flags are stored but with the addition of the object group added to the index.
I added two implementations for storage, a simple fastutils Int2Int map which has a much lower memory footprint for when loading 75k objects. And a multi dimensional Array<IntArray> which is much faster but uses up a lot more memory, suitable for loading 3.5m objects.
The flat map file format also had some improvements, before it was storing and loading at a bit level, which is great for storage sizes but no so much for data throughput and complexity. I threw that out in favour of standard readInt/Long's.
Overall it didn't increase the size too much; the flat file is now 14.87MB instead of 14.1MB but it simplified the code and increased speed a lot.
And the final changes were making sure that the data loaded goes through transformations as infrequent and as late as possible not only in the object storage but collision storage too, along with some general refactoring and adding a checksum file so that the flat file is always correct.
Memory Usage
Total sever usage after startup has completed
Object Count | Before | After | Memory Saved
75k | 454.68MB | 438.01MB | 3.6%
3.5m | 1.1GB | 677.69MB | 38.4%
Loading speed
Map loading only, average of 5 cold starts
Object Count | Before | Array Init | Loading | After (Total) | Performance Gain
75k | 398ms | 30.6ms | 139.8ms | 170.4ms | 57%
3.5m | 668.4ms | 29.6ms | 202.2ms | 231.8ms | 65.3%
Which is really good, as you can see on the optimised version it barely peaks 500mb on startup before dropping to sub 400mb (700/500 with 3.5m objs). It's no longer a significant contributor to startup times and memory overhead although wasn't enough to breach below the 4s mark.
I'll have to do something about my yaml files if I want that.
