There are several optimizations at play here:
- RAM usage: only load the assets you actually need and free them as soon as possible
- HDD seeking: keep related assets next to each other on disk
- bandwidth for patches: patches only include the stuff that actually changed
Before I go into detail, one could argue that these things aren't all that relevant today. And one would be correct. But Bitsquid was founded in 2009, meaning that this whole packaging stuff was design for hardware requirements from about 10 years ago.
Why didn't Fatshark change it? Because it's still "good enough", especially since current gen consoles are still running HDDs and the major downside of high storage usage is pitted against a mere €0.02 per GB on HDD, about 1/6th of 2009's prices. There is very little financial benefit in investing the huge amount of time it would require to change such an integral part of the engine. We might see it with Darktide and its focus on next gen consoles, but I doubt it.
On to details:
In order to keep RAM usage as low as possible, the developer wants the engine to only load the assets that the game actually needs at that moment, no extras, and unload an asset as soon as it becomes unused. To do that, the developer splits all assets into separate packages, which can be (un-)loaded individually. E.g. if I want to spawn a specific weapon, I don't want to load all weapons into memory, just the one I'm spawning.
However, loading just the weapon's mesh is not enough. Many assets have dependencies on other assets. E.g. for a weapon to spawn, I need a mesh, multiple textures, animations, hit sounds, etc. And there can be multiple things depending on the same asset, e.g. illusions for a specific weapon might have different meshes and textures, but they share the same animations and sound.
The obvious answer: multiple packages for each illusion's specific assets and one shared package for animations and sound. Sounds like a good idea, but that's slow, thanks to HDD seek times:
On HDD a software has no guarantee (at least with popular file systems) where a blob of data is stored on the physical disk. But in order to read data as fast as possible, you want it to sit right next to each other, otherwise the read head is spending precious time jumping (seeking
) between various locations on the disk to pick up all the bits and pieces. And the best way to ensure that your data is stored in a continuous block, is to put all related data into a single file. E.g. one package for each illusion that holds both the unique and shared data.
(Yes, even then NTFS doesn't guarantee it until defragmentation)
And what about "small patches"? 10 years ago download bandwidth was very precious and usually limited. So in order to keep patch files as small as possible, they only include the stuff that's actually changed. The downside is that the smaller patch bundle cannot replace the original one, but has to be saved as an addition. Which means more duplicated data. E.g. if the texture of a weapon illusion changes, you will have both the original full package and the additional one that only includes the changed texture on disk.