Studying with The Graphics Codex

This is a post where I talk about the projects of The Graphics Codex by Morgan Mcguire and present my results for each one. I was having a bit of a trouble focusing on one thing at a time because I wanted to do so many things! So, I decided to end my struggle by following a specific “curriculum” which has concrete goals to narrow down my selection. Every project requires to read up on specific subjects that are more beginner oriented but it is a nice refresher of the basics.

I talk about the strengths and weaknesses of each project, what I’ve learned and what I’ve achieved following the guidelines of each one. I hope to inspire anyone who happened to stumble upon this post and manages to finish it.

For the watered down version with less text and all the screenshots, please visit my Github page which, also, has the source code of all the projects. I will introduce more chapters as I complete each project.

1. Cubes

The “Cubes” project is the first one in the series and serves as an introduction to many things such as coordinates systems and positioning in 3D space and the G3D Innovation Engine. This is a very good starting point to learn Computer Graphics because the basic rendering is already implemented, there is a streamlined way to add GUI elements and provides the way of structuring the code when working with a real-time engine.

The pros of this project are:

  1. It is a great introduction which doesn’t require any previous knowledge; only C++ familiarity.
  2. Already implemented controls (FPS camera), debug GUI and camera effects adjustments.
  3. It touches on procedural generation without being too complicated (with the generation of a staircase).

The cons of the project are:

  1. The bloom effect is bugged and it doesn’t work well with spot lights. Since it is enabled by default, the reader needs to fiddle with the debug GUI and disable/enable everything until they find the culprit.
  2. The documentation on how to write in the .Any “language” doesn’t refer to every object and can be a huge pain for beginners. The only hope is to scour through the samples in hopes of finding something that fits our needs.

The final “assignment” of this project was to create a complex scene using only cubes. My “deliverable” read an image and created a cube grid where each cube corresponds to one pixel and the cube’s height depended on the luminance of the corresponding pixel. Below are two renders of a scene rendered in G3D.

Overall, I think it is a great learning experience but the cons become more important the less someone is experienced with C++ and reading documentation of software. It should, at least, mention some sample files in the G3D repository to introduce this way of thinking and searching to the beginners.

2. Meshes

The next project speaks about topology and expands on procedural generation by requiring the creation of a wine glass. The reader should first create a cylinder, then create a heightfield and, lastly, create a wine glass using only it’s contour from a half cross-section. My “deliverable” for the Cubes project was like a heightfield but with many more objects and way more image sampling because it is a recreation of the image with cubes (it can get out of hand with high resolution images and G3D didn’t like that).

The pros of the project are:

  1. It speaks about topology and when someone finally understands that a mug is the same as a torus topologically because they have only one hole, a very big “AHA!” arises.
  2. Procedural generation.

The cons of the project are:

  1. The setup of the existing GUI is not “correct” and the reader needs to debug it in order to place more items in the same style. This is something that I would normally classify as a “pro” because it is a learning experience but the reader already has many things to keep track of; putting more obstacles does not help.

The requirement for the wine glass is that the glass should be created only from a contour and an optional GUI element would be the number of rotational slices to generate. Since I didn’t want to give the numbers by hand or interpolate a Bezier curve and import the points, I decided to (again) use an image as input. The specification for that image is that it has to be grayscale with black indicating point of mass and white indicating surrounding space. After generating the wine glass, I had to present it in a stylish way.

For the generation of the glass, I simply read the image lines to find start and end points of mass and then translate them to a normalized system to preserve the aspect ratio. The initial results were encouraging but I didn’t like that there was too much generated geometry (one ring for every pixel line). So, I added a “Quality” slider which translates to a distance and an angle criterion that evaluate if a ring should be inserted or not. The lower the “Quality” the less rings are inserted and the higher the “Quality” the more faithful is the result to the input. I ended up liking the lower quality results more because they were cleaner.

From left to right: i) the grayscale image input, ii) the raw result where every pixel line generates a cylindrical ring, iii) a low quality result with the naive angle criterion and iv) the low quality result with the distance and angle criteria.

I decided to produce a more sophisticated presentation for this project with a more refined glass along with some plates and place them in a bar setting. The plates are generated with the same method as the glass. Below are the render in G3D, the input images and the resulting meshes.

I felt a great sense of accomplishment when I saw my creations sitting there and pretending to be realistically rendered. It is a very nice feeling to have something of a “closure” and see every bit fit together to produce a final result. This chapter was very good.

3. Geometry Design or How I spent a day to track a bug that turned out to be a feature.

Having finished the fourth project “Rays”, I returned to this one to support Instancing on top of the previous deliverables. The implementation was not hard but the way the project definition is worded makes it extremely difficult to find the exact G3D classes to use; and this wording is what made me spend that day for the feature. The way Instancing was implemented is:

  1. Keep one version of each model in the scene.
  2. For each Instance hold:
    • It’s Axis Aligned Bounding Box.
    • It’s TriTree which is a CPU tree with all the model’s triangles.
    • The model’s inverse transform – needed to convert the rays from world to object space.
    • The model’s transform – needed to convert the hit surface information back to world space.
  3. When a ray needs to be intersected:
    1. Intersect the bounding box of every instance.
    2. Intersect the triangles of every model whose bounding box is intersected.

With the simple addition of bounding boxes for the models instead of manually intersecting every triangle the performance boost can be insane depending on the complexity of the models. Having to load each model only once really helps with the memory footprint of the program and we can batch rays per model and triangle in order to have better cache coherency. Another needed improvement is a Binary Space Partitioning (BSP) tree to hold all the bounding boxes and, again, intersect only those needed.

The pros of this project are:

  1. The reader can learn about much needed optimization techniques. If they are fiddly, they will have to debug the same thing as I did which, in the end, is a good way to learn about how G3D does things.
  2. Depending on the project selected, a range of computational geometry problems can be tackled which are not too hard but not too easy either.

The cons of this project:

  1. The wording. Starting with the Rays project, there are certain things that are asked to be implemented and some rendering of expected results. The Rays project presents renderings with shadows from Area lights which are specifically excluded from the required implementation. This project required to make Instances for each Model but as will be shown below, we needed to make instances of Lights (or manually intersect with the Lights) otherwise the results would not be the same as the Rays project.

And now about the feature.

Looking at the next slideshow, the first image is what is generated with 1 indirect ray from the Rays project and the second image is the result with 1 indirect ray from this project. The first thing that can be noticed is that the light is getting bigger and I noticed it too now that I’ve made the two results fade between each other. The difference I noticed at first and decided to investigate is that the noise in the second image has an orange hue in some clusters.

The first thing I thought is that I have messed something in the ray calculations. Indeed, I was not converting the hit surface information back to world space but it didn’t solve the issue which lead me to inspect the value that these pixels are reporting with the G3D image inspector. These pixels were reporting a very strange hue which I couldn’t justify based on the lights in the scene.

I believed that it was, again, a rendering error due to some mis-transformed variable from object space. I removed each part of the rendering equation, biradiance and the scattering density function and it seemed that the biradiance was, somehow, the culprit even though the variables that were being fed were exactly the same. I started stepping into every G3D lighting function that was called in order to find that mis-transformed variable but to no avail. Alright then, if the equations are correct then the scene is not correct.

A new round of stepping into the function with the debugger was underway in the part of G3D that loads the scene and after some hours I saw it: the model that was being loaded contained an emissive surface; but why is it different from the other light? Why does it have an orange hue? Recalling some images of the Cornell Box and after checking online, the original model has a light with an orange hue, like below.

Image from Wikipedia.

So, the model that was loaded contained a surface that has white diffuse but orange emissive color. This is done in order to simulate the over saturation of the real lamp on it’s plastic part at the top of the box and emit the correct light in the scene. Primary rays that hit the surface will report the color white but some indirect rays that happen to have a low scattering factor will get the orange hue because the factor affects only the diffuse.

But what went wrong? This is where the wording of the project comes into play. This particular problem occurred because the project mentioned that we need to create an Instance for each Model of the scene and discard Lights, Cameras, Particles and models that can be animated. BUT some G3D::Light objects can produce triangles, such as area lights, in order to intersect them.

The Model that is loaded contains an emissive surface but the Scene that is loading that model contains one extra area light that is placed at almost the exact same position as the emissive surface, effectively covering it. Since I loaded the triangles from the model only, that surface had the correct diffuse color (fully white) but some indirect rays were carrying an orange hue because they happened to intersect it. Including the G3D::Light geometry in the computations, the noise went back to the original behaviour.

From the project deliverable point view, the result with the orange hue is the correct one because it does what is asked of it: keep only the models. From a correctness standpoint based on what the Scene contains, the Rays result is the correct one.

Personally, I found it a great mental exercise but it highlighted the problems of the project definitions that started with the Rays project. Let’s hope number 5 improves that.

4. Rays

The rays project asks of the reader to implement a CPU ray tracer. The features that will be implemented are:

  1. Multi-threading (easily implemented through G3D’s API).
  2. Shoot rays that pass through the center of each pixel and calculate their Radiance.
  3. Implement ray-triangle and ray-sphere intersection. The algorithms are discussed in the reading material and an optimized version of the ray-triangle one is provided.
  4. Perform α-testing on the triangles.
  5. The illumination must take into account:
    1. The emitted radiance,
    2. An ambient term,
    3. Direct illumination from point, spot and directional sources,
    4. Perform shadow test for lights that can cast shadows.
    5. Indirect radiance by sampling random directions around the normal of the hit.

This is the first project that requires a stronger will to complete because of the nature of the bugs and pitfalls that may be encountered. There are many artifacts due to the floating numbers precision and the implementation requires a lot of back and forth until the results are satisfying.

The pros of this project are:

  1. The reader gets to learn so many new things and how the light needs to interact with every surface to produce what we see as color. It’s very enlightening.
  2. The study material contains references of the very first papers in the research field and the curious reader will appreciate them.
  3. The exercise of deriving the intersection algorithms helps the reader develop intuition and skills for the next intersection algorithms they might need.
  4. It is a good challenge to complete and seeing the beautiful images at the end is a great reward for the effort.

The cons of this project are:

  1. The guidance it provides can be very lackluster for a beginner and a lot of time can be “wasted” on things that could have been explained clearer. This project can very hard to implement without some guidance.
  2. Some of the “results” pictures cannot be reproduced with this project. The Cornell Box scene cannot have dark shadows because the Area light is not marked to cast shadows and the shadowed parts receive direct illumination. Only with a lot of indirect rays will these areas appear darker.

The Advice section should be re-read every time the reader is stuck because they might have missed something valuable. I believe that this is going to hold true for all the remaining projects.

Below are the results I got for the “Cornell Box” and “Test Scene” G3D scenes. The first of each is rendered without indirect and the second with 2048 indirect rays from the surfaces. All the rays are one-bounce rays. The rainbow sky doesn’t affect the colors of the scene; it’s just a random hue based on the direction of a ray that doesn’t intersect anything.

Cornell Box: the indirect rays provide a more accurate representation because we can observe: a) the left side of the left rectangle has a red tint, b) the right side of the right cube has a green tint and c) there is a slight shadow at the bases of the rectangle and the cube. The shadows should be more prominent but we don’t perform correct shadow calculations for area lights in this project.

Car Scene: the indirect rays are not improving the image too much except for the cavities which are brightened and more details can be observed. The image with the indirect rays has noise because the car’s material is metallic and the finiteScatteringDensity of it’s surfaces is very high. Blender has a setting to reduce that source of noise.

The transition effect of this gallery is set to “Fade” to emphasize on the pixels that change.

The time it took to render the scenes in 1920×1080 with 2048 indirect rays was: 4m34s for the Cornell Box and 3h45m30s for the Car. Analyzing the performance for the hotspots of the computations, 50% of the time is spent in the ray-triangle intersection function. The code of this function is already optimized and can’t take it any further and considering that it takes literally half of the time, there are just too many ray-triangle intersections performed. The optimization for this case is to reduce how many triangles we check for intersection. The first step is to have bounding boxes around the each model to intersect these first and then build a BVH for the whole scene. We should perform the expensive calculations only when there is no other option.

5. Paths

Write a photorealistic renderer????

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s