GameDev 50: Pixel Perfect Picking

Color Picking, Noise Based Placements

10.05.2020 - 16:30
Color Picking

In my editor I was using a ray cast and hitbox based method do decide which model was hit when the user clicked his mouse. For the most part this worked well, but there are a few cases where this method has its limitations. The main problem occurs when the hitbox of a smaller model is inside of the hitbox of a bigger model. In order to prevent clicking through walls the algorithm has to find the closest hitbox that is hit and therefore there is no way to select the smaller model. There are a few ways you could solve this problem, for example you could go down a level if a hitbox is hit and calculate the ray triangle intersection for all triangles of the model, but that seems like a lot of math and ain't nobody got time for that. I had a button to disable the hitbox of the bigger model temporarily, but that isn't a great solution either. Therefore I decided to implement another method: color picking.

The idea of color picking is actually very simple: the ID of every clickable model is encoded in a unique color, all models are rendered to a texture and when the mouse is clicked the color of the pixel at the mouse position is retrieved and decoded back to the model ID.

Trees with the model ID encoded as their color:


The most trivial way to implement color picking would be to send the model color via a uniform value to the shader. However, for performance reasons I am using instanced models, which can render a large number of instances in a single draw call, but there is no way to send a uniform per instance. I was thinking about using an extra instanced buffer to store the color for every instance, but I found a better way that requires no extra buffers and only a single uniform per instanced model. Every model gets an 8 bit base ID, which leaves 16 bit for the instances. The base model ID is send to the shader as a uniform and in the vertex shader there is a built in variable gl_InstanceID, that counts up for every instance of the model. With it the model color can be calculated in the vertex shader as such: modelID * 2^16 + gl_InstanceID. This gives me a maximum of 256 different models with 65k instances each. I would have liked to have 32 bits of RGBA instead of only 24 bits of RGB to encode the ID, but including alpha would have introduced a bunch of problems and I'm not sure whether it is possible at all. However, even 256 models with 65k instances are more than enough for my current needs.

Hitbox based picking (right) vs color picking(left):


One thing to mention is that glReadPixels is generally slow, because the CPU has to wait for the GPU to finish. However, I don't think this is a big issue, because I only use it for model selection and therefore only draw the colored models and call glReadPixels when the mouse is clicked. If I wanted to implement some kind of mouse hover effect with this method it may be an issue though.
Also I lied in the title, I render the models only at a quarter resolution to save a little bit of performance, so it's not quite pixel perfect, but it's still hard enough to hit a 2x2 pixel area with the mouse.

Noise Based Model Placement

Another feature I worked on this week was noise based model placement. My editor has a few ways to randomly place models on the map, but nature is rarely completely random. Instead, the vegetation is governed by different factors such as elevation, moisture, soil, wind, etc. And while I didn't want to model all these factors, I thought that noise is an easy way to achieve a similar result. I added a new window to the editor to create a noise image based on 2 noise frequencies layered on top of eachother and then place models according to that noise image. While working on it I was about to add all the different model placing options into the window, but I realized that it was getting ridiculous. There were already 3 different places where I had very similar model placing options and I was about to make it 4. So instead, I forced myself to clean it up and refactor a bunch of code. I removed the model placing options from all previous locations and created a new class to store the options and one singular window to change them. In all places where the model placing options are needed I added a button to open the window.

The new model placing options window:


With this cleaned up I was able to finish the noise based model placement both on the map and on the distant terrain next to the playable area. I also figured I should add noise based texture tile placement so there can be dirt under trees and grass on the clearings between them. Another feature I implemented for the models was the ability to alter the scale of the models based on the noise value, so there can be smaller trees near the edges of a forest and taller ones in the middle. I am quite happy with the result and I think it'll make maps a little bit more realistic and interesting.

Noise based placement:

---