Sparrow WebGL Devlog 2: UI

Adding UI Elements, Processing Inputs, and Minimization

25.04.2023 - 22:14
In most cases, native user interfaces for WebGL engines are not needed, as HTML already provides this functionality. You cannot tell the difference between a button that is rendered in WebGL and an HTML <button> element on top of the WebGL canvas. If anything, the WebGL button probably looks worse.

However, I was never a fan of layering different systems on top of each other. Although it's possible to use Qt UI elements and OpenGL together, I have always preferred the minimalistic window management offered by the GLFW library over the more complex Qt system. Additionally, designing and implementing a UI system is challenging and fun.

I didn't have to start from scratch with the UI of my WebGL engine. It's heavily inspired by the one from my OpenGL engine. It works very well and has proven itself in many projects. Another advantage of using a similar system is that switching between them is much easier. However, there are also a few improvements I made in the WebGL version.

UI System

All UI elements share a common superclass. For Sparrow, I am using the newer JavaScript class syntax. While I got used to the prototype syntax, using classes is much nicer and it works better for inheritance, which came in handy for the UI elements. All elements like buttons, labels, or edit fields inherit from the superclass and implement only functions based on their unique characteristics.

The main UI class has a list of elements and handles all higher-level functions of the UI system. However, the elements are also organized in a tree structure. You can add elements directly to the main UI class, but you can also add children to every element. The children functionality is implemented in the superclass, which means it's independent of the individual components. For example, when the main UI class receives a mouse event, it checks all elements which are directly attached to it and if one of them catches the event, the element then checks whether one of its children or itself gets to process the event.

Drawing is handled similarly. The main UI class only calls the draw methods of its direct elements and then every element recursively calls the draw methods of its children. In 2D rendering, triangles are always rendered on top of everything else (if depth testing is disabled), which means children are always drawn on top of their parents. The elements also have a z-index, which can be used to sort them too. Another advantage of the parent calling the draw methods of its children is the scissor test. It's a WebGL function to limit the rendering to a given rectangle. This is very useful to prevent overflows. For example, when scrolling the children in a container, they are just moved according to the scrollbar position. Without the scissor test, they would go outside of the container.

Speaking of scrollbars, they are always the most annoying functionality to implement in a UI system. They have to support many different input methods like the mouse wheel, clicking the up and down buttons, dragging the scrollbar, and clicking into the empty scrollbar area. Then they have to dynamically adjust the children of the scrolled container, which always requires exceptions because the scrollbar itself, which is also a child of the container, must not move itself. It took me a very long time to get scrollbars working properly when I worked on the OpenGL UI library. Luckily, for the WebGL version, I could use my existing solution.

One of the biggest differences between the OpenGL system and my new WebGL implementation is the integration into the engine itself. The OpenGL system was a bit of an afterthought and I only added it later, which meant it wasn't well integrated and always required a lot of extra code when using it in a project. This time around, I worked on the UI system immediately and in parallel to the main WebGL engine, which means it's much better integrated. When creating a new UI in a project, it only requires a single line of code to add the UI to the engine, and from there everything else is handled automatically.

All currently added UI Elements:

Styles

Besides the functionality, the UI elements also have to look good. Additionally, it should be possible to change their appearance easily in the code. Just like in my OpenGL UI system, I took some inspiration from CSS. Every element has a style that is initialized with some global default values. Next, every element has a custom style applied based on its type, and finally, you can pass a style argument to the element when creating it. Later values override earlier ones, similar to the CSS hierarchy.

However, unlike the OpenGL engine where all styles are string-based and require a special parser, I used JavaScript key-value objects for the WebGL version, which can be parsed very easily. The style system is similar to CSS and some options are identical, but there are a few differences, especially the naming scheme (backgroundColor instead of background-color, or just font instead of font-family). I considered changing the names to their CSS equivalents for the WebGL version but decided to keep the old names. This is something that would be confusing for somebody else who wanted to use my engine, but switching to the CSS names with some slightly different behavior would be confusing too. At least, the different names indicate that it's different from CSS.

UI scaling

In the last devlog, I already mentioned high-resolution display scaling. This is especially important for user interfaces. Text has to remain legible and sharp on all types of screens. However, in addition to adjusting for the pixel density, users also have different preferences and possible accessibility restrictions. Because of this, I split the scaling into two parts for the UI system. First, the user interfaces are automatically scaled based on the pixel density, which you can easily get in JavaScript with the window.devicePixelRatio variable. Secondly, the UI also has a separate scaling value that can be set independently. Both values are then combined into a final UI scale, which is used to adjust all elements. To maintain sharp texts, all auto-generated textures are recalculated when the UI scale changes (the scale changes very rarely, so recalculating all UI textures isn't a big problem).

Issues

Because of its C++/OpenGL origin, the WebGL UI system already works quite well. However, there are also some issues. Some of them already existed in the original version, others were only introduced after porting it to JavaScript/WebGL. The OpenGL version only had very basic support for UI scaling, so this is something that caused a lot of problems in the WebGL implementation with the most problematic element being the scrollbar once again. I have fixed most obvious issues, but I'm sure there are some edge cases that don't work. For example, what happens with an edit field in a scrollable container inside a window? I'm not sure, it's impossible to test every combination of elements, which means I'm only going to discover these problems when I need to use that exact combination in a project.

Minimization

Finally, I also worked on merging and minimizing all engine files. I wasn't sure which tool to use for this, but after a bit of research, I found UglifyJS. It's quite minimalistic, which is the main requirement for me to avoid too many dependencies. In fact, it doesn't have any 3rd party dependencies and the node_modules folder is only ~1MB and 24 files, which is amazing, more node modules should be like this. I wrote a few small scripts that list all files from a directory and convert them to the input format that UglifyJS expects. There may be an option to automatically merge all files from a directory, but for some of my files the inclusion order matters so I had to write a custom script for this anyway. I half expected there to be some weird issues, but I tried my test project with the minimized files instead of the individual ones and everything still worked.

Minimized files:

---

All games require user interfaces, which makes them an important part of every game engine. The main reason why I am creating my own WebGL engine is game development, which is why I already worked on UI elements very early on in the development cycle. I already liked how the UI system worked in my OpenGL engine and I still like it after porting it to WebGL, some aspects are even better because of JavaScript's flexibility.

---