Development adventures

Spatial mapping (Part 2)

After a few weeks of travel, I am back on track and back to working on the spatial mapping. Last time, I left off with the ability to view the debug information but things were visibly wrong. We were able to see the walls and part of the ceiling but there was no floor or any part of the couches. Its time to figure out why and do some serious debugging!

Before we jump to the debugger its good to come up with a few theories as to why we are experiencing these issues. This way we have a method to our process instead of blindly running through the code until we stumble on the problem.  This becomes more difficult as the project gets bigger, but it is still a good mental exercise.   If we look at the previous rendering results, things were starting to look good, but a lot of information was still missing.  While developing the initial code base I primarily focused on getting the data out to the screen as fast as possible which is always guaranteed to produce a number of bugs.  A project this large really needs short cycles to make sure I keep constant progress but it is necessary to do a lot of testing.  Programmers in general don't do enough testing and in this instance its very easy to see that there s a problem and more testing is indeed necessary.


Previous post mesh data


The first question that comes to my mind, does the data conversion work correctly? We're able to see the vertex points but they only overlap the walls and the ceiling. I wasn't sure what the data was doing or if the data was being skewed, flattened or corrupted.  Let's compare the vertex position values from the spatial mapping and the debug points. Time to pull out the graphics debugger and analyze the input buffers.


Visual studio graphics debugging


Since we are dealing with vertical data issues, Y position values are a good start. With a frame snapshot, I extracted the raw values from the couch arm which we can use as a base for comparison. Now we can go back to the debug meshes to find the same area and compare.  Is it possible that they are being flattened and combined into one plane? I don't know and it's too early to tell.


Buffer data example


The raw values stated that the position for the couch arm were in the area of (.00415, .0149, .0112). The raw data extracted from the buffer as SHORT4 normalized were roughly (138,504,392,32767). W is max 2 byte value, which means its invalid and simply not used. If we denormalize the data by dividing by 32767 we get float values approximately (.0042, .015, .011) which is really close. So we know that our conversion is good. So that theory is wrong. 

The spatial mesh is provided by the SpatialObserver which provides multiple meshes that represent a complete room. The meshes are streamed in an async process one by one as requested. Possible theory, are certain meshes from the spatial observer failing to complete the processing or never got processed?  To verify this, I setup an output log each time update is called on the Spatial Observer. This should help me see the function execution and get the counts of meshes as each process completes. Sometimes its hard to track multiple threads, so an output log is a good way to see what is happening in a real time application. Here are the results: 


debug output


Looks like the total count of spatial meshes is equal to total count of game objects. For each mesh created by the spatial observer a game object is generated with a matching id. Well, that looks like its working but we still don't know what is happening to the actual data. Time to go back to the visualization tools. Maybe the graphics analyzer can help us see a potential problem and possibly develop theories from any new clues.


Raw data

Looking at and comparing meshes between the spatial observer and the debug data, something is odd and the debug meshes look a little anemic. Considering how much data is processed, the debug mesh just seems empty. Maybe not all of the vertices are being processed. Going back to the mesh processing I started looking directly at the vertex conversion code. It took mere seconds to see the problem. The vertex counts didn't match at all. Finally, a "doh" moment here! We're not processing all the vertices! 

So what happened?  Well, last time I wrote the buffer conversion, I started with (SHORT*) buffer casting so that I can check individual values. To simplify the loop I changed the buffer casting to (SHORT4*) without removing the stride division. This in result gave me 1/8 of the actual vertexes to be processed!  So what does our debug data look like now?


Look a couch!


We have a couch! Each dot is a debug point that we will use for our terrain.  We have a floor and sides of our couch! Nice! I added keyboard toggles so that I can turn the solid mesh on and off. Here is some more spatial mapping. 


Dots Dots Dots Dots And More Dots Dots Overlayed


The dots alone can be hard to visualize without overlaying the spatial mesh, but we can get an idea of how the room is laid out. This will help guide us during terrain generation and help make sure it aligns properly to the room. 



So to recap, we have spatial mapping, we have debug data and everything lines up. Next comes the hard part, building the actual terrain.  I feel pretty confident that the debug data will help us stay on track and aligned.  Now we need to examine the data and start stripping things out like the ceiling, walls and any surfaces that are too high. In reality this is a game and we need to look at terrain, not the ceiling. Although we could use the ceiling for placing clouds and other sky stuff, but that may come at a later time.

Stay tuned, as we continue down this road and hopefully make something of a rough terrain in the next post. I'm thinking we might implement hardware tessellation of a plane as our next step. Please stay tuned. 

Comments are closed

Copyright © 2020 - Design by Francis