Play Now

Yaml parser

Yaml parser

The majority of the games content data I store in yaml files and they've slowly been growing in size; currently 30 files at 3.2mb. Loading them all on start up was starting to become noticable; taking between 50-500ms per file.

The second issue was Jackson doesn't support yaml anchors which are a very neat and useful feature which allow you to reuse data in multiple places.

Code:

duck_grey:
  &duck_grey
  id: 6113
  hitpoints: 30
  race: duck
  examine: "Waddle waddle waddle quack."
duck_swim:
  <<: *duck_grey
  id: 46
  swim: true
  examine: "Quackers."


& marks an item with a name, in this case: Map<String, Any>
<< is merge key which combines the two maps (while overriding ids, examines)
* marks an alias which gets replaced with the anchor item when parsed

SnakeYAML supports anchors but it's a bit slow so I was using a mix of both libraries



So naturally I wrote my own yaml parser.


It only supports a subset of the yaml specification but has all the important parts, more importantly it's fast. This is mainly because it doesn't have any reflection usage, the downside of which is converting values to objects you have to do with a manual bit of code in a configuration class. This is an acceptable tradeoff imo.

A basic config which converts direction strings to enums during parsing:

Code:

val config = object : YamlReaderConfiguration() {
    override fun set(map: MutableMap<String, Any>, key: String, value: Any, indent: Int, parentMap: String?) {
        super.set(map, key, if (key == "direction") Direction.valueOf(value as String) else value, indent, parentMap)
    }
}


Read speed
Average of 1k runs after 10 warmups loading all the config files before/after they used anchors

Parser | Without anchors | With anchors | Json**
SnakeYAML | 206ms | 201ms | 3404ms
Jackson | 109ms | * | 130ms
Mine | 14ms | 13ms | 84ms


* Jackson doesn't support anchors
** YAML is a superset of JSON so I included the 58MB US gov electric vehicle population dataset too

The write speed is probably as equally as fast but I haven't benchmarked it as it's not as important.

7-8x faster than Jackson and over 14x faster than SnakeYAML


Switching over decreased server load times from 4.2s down to 2.9s. and has knocked off an extra 11MB ram overhead which is a nice unexpected consequence, probably due to using more efficient maps.
There's a few other places I can improve to get load times down to ~2s, not that it's really necessary but optimising stuff is fun




A few other changes; refactored a bunch of stuff to make naming more consistent (and closer to jagex), added book interfaces for quests, and jerry has added stairs for the majority of f2p areas.

« Back to Dev Blog