Sparrow WebGL Devlog 10: Major Refactoring

Consistency, Transforms, Matrices, Shaders, and glTF files

20.09.2023 - 09:37
I have added a lot of features to my WebGL engine over the last few months including models, animated models, shadows, color picking, and more. However, I mostly focused on implementing the functionalities and didn't consider the overall usage patterns and consistency of the engine, which is also the biggest weakness of my C++/OpenGL code base. For the past few weeks, I have overhauled many of the core systems of the engine and made it easier to use and more consistent overall.

Transforms and Matrices

When working in 3D you can control the position, rotation, and scale of the models. This is true for modeling programs like Blender and game engines like Unreal or Uni..., I mean, Godot. Position, rotation, and scale are represented as 3-element vectors, but they can also be combined into a 4x4 matrix. This matrix is called the model matrix and it controls the position, rotation, and scale of objects in a 3D scene.

There is also the view matrix, which defines the position and view direction of the camera, and the projection matrix which projects the 3D scene to a 2D monitor. All 3 of these matrices are multiplied together into the model-view-projection matrix(MVP), which is sent to the shaders to render the scene.

There are a few ways you can handle the matrices and send them to the shaders. By far the easiest is to transfer the matrices individually and multiply them together in the shader. However, this means that the matrices are multiplied on the GPU for every vertex of every mesh.

Generally, it's preferred to premultiply the matrices on the CPU. This also has the benefit that you don't have to recalculate the matrices every frame. You only have to update the model matrix if the position, rotation, or scale has changed (which for most static scenery objects, it never does). The MVP matrix, however, has to be recalculated every time the camera is moved and its view matrix changes.

I don't know how bad the performance hit from multiplying the matrices for every vertex is. I assume the shader compilers can do some code optimizations like C/C++ compilers, so maybe they are smart enough to know that they can multiply them once and cache the result for a bit. However, I still decided to multiply the matrices on the CPU, because that's the more common approach.

Because I was undecided about how I wanted to do this in the beginning, my existing code was a bit of a mess. Some classes had their own internal model and MVP matrices while others expected external ones. Some shaders wanted them individually and others combined. This had to become more consistent.

The first thing I did was add an Object3D class that all other 3D-renderable classes inherit from. Because all meshes need these matrices, the matrices are now stored and handled in the parent class. I also added an object manager and all objects add themselves to the manager when they are created. This means the object manager can take care of the matrix recalculations. Whenever the camera changes its view matrix, it triggers the MVP recalculations and when an individual object changes its position, rotation, or scale, it also triggers its matrix recalculations.

Sadly, the matrix problems don't stop here. Only handling the model and MVP matrices is still very easy, but many features and effects require additional matrices. Whenever you work with lighting, you also need a model-view matrix(MV) that converts everything into the camera space because that's where lighting calculations are typically done. The hardest thing, however, is features like shadow mapping or reflections, because they require matrices that are calculated differently.

I came up with a solution for this that I call matrix slots. Besides the model, MVP, and MV matrices, the objects also have an array of custom matrices. When a feature like shadow mapping is enabled, it registers new matrix slots that store the required matrices. It also specifies callback functions for how the custom matrices are calculated.

However, I'm not completely happy with this solution. It's quite complicated and not easy to understand and debug. It works quite well for engine-internal features, but I eventually want to implement custom effects and I don't think the matrix-slot system is easy to extend and customize.

Shader Modules and Includes

Another system that I have never found a good solution for is handling shaders. I always end up with many similar shaders with large sections of identical code. Unfortunately, while GLSL supports some preprocessor commands, it does not support #include commands like the C/C++ preprocessor does. After some research into different methods of handling shaders and shader code reusability, I decided to implement a custom version of a #include preprocessor command.

Before the GLSL code is compiled and linked into a shader program, the custom preprocessor parses the code and replaces the #include commands with code. This was quite easy to program. The shader source code is split into lines and the preprocessor checks whether a line begins with #include. If it does, it uses the string that follows to look up what to include.

Unlike C/C++, the preprocessor does not include files directly, because most of my WebGL shaders are just strings inside of other JavaScript files. Instead, I created shader modules, which are sections of shader code, split into different parts depending on where they need to be inserted. For example, a lighting module or a fog module. Modules can have a header and a main for the vertex and fragment shader. The header includes input and output definitions, uniforms, and function definitions, while the main is code that's called inside of the main function of a shader.

#include <sparrow_lighting/header>
void main()
{
#include <sparrow_lighting/main>
}

This should make it easier to create custom shaders too because they can also use the existing modules.

While working on the shaders, I also created a new default 3D shader for the engine that can be used to render meshes when no custom shaders are defined.

glTF improvements

Finally, I completely overhauled how glTF files are parsed and handled, which turned out to be the most difficult task. First of all, I created an animated model class, which is used to render an animated or vertex-skinned model. This is similar to my normal model class, which is used to render static models. Both of these can be loaded from glTF files and are only used for one model.

glTF files, however, can include multiple models with an arbitrary mix of animated and static models. For a very long time, I went back and forth on whether I wanted to support these complicated scene setups. It's very convenient to load everything from a single file, but it's also quite difficult to implement. My first glTF implementation was able to handle it for the most part, but it was not consistent with the rest of the engine, so it had to be redone.

It took more than a week before I had a decent implementation. It parses the glTF file to create a list of static and animated models (it uses the same classes mentioned above). Because glTF files contain parent-child relationships between the models, I also extended the aforementioned Object3D system to include children. The children take the transformations of their parents into account when their model matrices are calculated. I also had to add a new empty object (objects that aren't rendered) because glTF can have empties as part of the chain.

Some glTF tests:


I created some glTF test files in Blender with different setups and model combinations and it worked for all of them, but I am 100% certain that there are some scene setups that will break my glTF parser. Because the engine is only intended to be used by me at this point, I'll just deal with it when I encounter something that doesn't work. For the same reason, it's also far less likely to break because it can already handle the way I typically export models and scenes from Blender.

---

This was by far the most difficult part of working on my WebGL engine. Adding new features is fun, but refactoring major essential systems is complicated, time-consuming, and boring. However, it needed to be done and the earlier the better before it became even more of a mess than it already was. There is still more code cleanup and testing to do, but I should be done with most of the difficult parts for now.

It's also not a great blogpost topic because it's very technical and you cannot take a lot of pretty screenshots, but the next one will be better.

---