**GPU** [A Graphics Codex Programming Project](../projects/index.html) ![Figure [teaser]: [Elephant](https://www.shadertoy.com/view/4dKGWm) by Iñigo Quilez, real-time ray marched from procedural surfaces and materials.](elephant.jpg border=1) Introduction ============================================================================================ Graphics processing units (GPUs) implement a variety of parallel evaluation strategies in hardware. These collectively present hundreds of thousands of concurrent virtual threads to a program. They also offer a fixed-function rasterization unit for high performance primary ray intersection with triangle meshes and a high throughput dedicated memory hierarchy. Fully leveraging the potential of a GPU hardware for graphics requires understanding many elements: - GPU programming languages such as GLSL - GPU APIs such as OpenGL - Hardware-aware algorithms - Rasterization-centric data structures - Adapting light transport algorithms to rasterization-based operations In this project you'll focus on the first one, applying what you already know about ray tracing to the GPU using GLSL to implement a real-time ray tracer. In later projects we'll explore the rasterizer and GPU architecture. Prerequisites -------------------------------------------------------------------------------------------- - Complete the [Rays](../rays/index.html) project - Read in the [Graphics Codex](http://graphicscodex.com): - Ray Marching You will find the "GLSL Functions" reference topic in the Graphics Codex very useful as you learn GLSL. Educational Goals -------------------------------------------------------------------------------------------- Learn: ![[Manta Ray](https://www.shadertoy.com/view/4ls3zM) by dakrunch](mantaray.jpg border=1 width=200px) - Practical aspects of GPU programming - Passing data between CPU and GPU - Programming with GLSL's constraints - Debugging without printf or breakpoints - Decypher "hacker" style GPU code by others - The sphere tracing algorithm - Implicit surface representations - Constructive solid geometry operations Explore: ![[Castle Argh](https://www.shadertoy.com/view/ldsGRX) by Antonalog](argh.jpg border=1 width=200px) - Creating shapes with mathematics instead of art tools - Noise functions for procedural texture - Creative shading approximations ...and work with BSDFs and vector and matrix math at a lower level than previous projects. Rules for this Project -------------------------------------------------------------------------------------------- There is no excluded code for this project. I encourage you to look at examples on [Shadertoy](http://shadertoy.com) and the G3D Ray March sample program. Just don't copy directly from them. Specification ============================================================================================ 1. Implement an implicit surface ray tracer with at least the following properties: 1. Renders an 848x480 window at a continuous rate better than 15 Hz 2. Shading algorithm: 1. Direct illumination from a single omnidirectional point source. 2. Shadows for direct light. 3. A lambertian BSDF (i.e., $f(\wi, \wo) = \rho / \pi$ for the positive hemisphere and zero otherwise), with different $\rho$ reflectivities ("colors") for different surfaces. 4. Constant ambient illumination. 5. Background skybox. 3. Uses the sphere tracing algorithm. 4. A scene containing at least four different geometric primitives and at least two operations on distance functions (e.g., union, subtraction, intersection, blending). 5. Uses the [G3D::GApp::activeCamera](http://g3d.cs.williams.edu/g3d/G3D10/build/manual/class_g3_d_1_1_g_app.html#a71e82db8c06ec12d86abbe8c7acb4251) for rendering. 2. Choose three Shadertoy shaders that appeal to you technically or artistically. Check in the source code for these shaders and low-resolution screenshots of them to your *journal* directory, with proper attribution to the authors and license statements (N.B. they will not run under G3D without some minor modification). 3. Structure your project as usual, with data-files/shader, data-files/image, source, journal, and other subdirectories (you must do this yourself for this project...the sample code does not have the correct structure). Report ========================================================================== 1. Describe how you *constructed* your scene, using images and diagrams. 2. *Correctness*: Demonstrate that your program is functioning correctly by rendering the following images. (You should do this as you are implementing, since if there are bugs, these images will help you to identify them): 1. The screen filled with a checkerboard of black and white squares that are each $16\times16$ pixels. 2. The screen filled with white "clouds" of three-octave noise over a blue background. 3. Eye ray directions (in world space) visualized as colors by $\vec{c} = (\w + 1) / 2$ 4. *Analytic* intersection with a sphere of radius 0.5 at center (0.5, 0.5, 0) visualized as: 1. World-space hit positions visualized as colors by $\vec{c} = P$ 2. World-space surface normals visualized as colors by $\vec{c} = (\n + 1) / 2$ 5. *Ray marched* intersections with the same sphere, visualized by the same two images 6. Ray marched sphere with direct illumination, casting a shadow on a ground plane and with a skybox in the background (the plane may be analytic or ray marched) 7. Separate images of each of your geometric primitives with direct illumination, casting a shadow on the ground plane. 3. *Demonstration Images*: Render at least five images of your scene from different, interesting viewpoints at full resolution, with no user interface visible in the image. Unless it is required by some aspect of your scene, keep the camera field of view around 50 degrees for a more photographic look. 4. *Questions*: Answer these questions in a paragraph each: 1. Choose one of the Shadertoy shaders that you liked. Show a screenshot, give attribution, and make a link to the live version on the web. Explain: 1. A single-sentence explaining why it appealed to you. 2. The core technique involved (in three or fewer sentences). If it uses ray marching, don't explain ray marching--just explain what the primitives and operations employed are. Some of the shaders have nothing to do with ray marching, and that's fine. 2. What online resources did you use to learn about GLSL (besides the Graphics Codex)? 3. Give a list of five important semantic or syntactic differences between C/C++ and GLSL that you encountered. 7. *Self Review*: Give yourself a nominal grade on this project (A+ through F) and critique your work in one sentence each on: code design/readability, expected code correctness, scene visual quality, and expected report correctness. (Restrict analysis to your GLSL code.) 8. *Skills*: Describe what you learned while working on this project. Distinguish software engineering, algorithmic, and mathematical skills. Include the reading and ideas that you had to implicitly work through even if not required. Example Results ------------------------------------------------------------------------------------- ![Checkerboard](checkerboard.jpg width=150px border=1) ![Clouds](clouds.jpg width=150px border=1) ![Eye ray directions](eye-rays.jpg width=150px border=1) ![Analytic positions](analytic-positions.jpg width=150px border=1) ![Analytic normals](analytic-normals.jpg width=150px border=1) ![A wheel](wheel.jpg width=150px border=1) ![Procedural textures](procedural-texture.jpg width=150px border=1) Advice ===================================================================================== Fully read the Graphics Codex Ray Marching chapter and the multiple trace-*.pix examples from the [included sample code](gpu-ray-march-tutorial.zip). They provide good base for starting this project. When you restructure the sample code, be sure to set the Visual Studio debug property for the working directory to $(ProjectDir)/data-files. In your main(), set settings.screenshotDirectory = "../journal/"; When taking your visualization screenshots, remember to disable bloom, antialiasing, and vignetting on the debug camera. Remember everything that you've learned about the practicalities of ray tracing. Although you're using a different ray-surface intersection method, issues like shadow rays, ray bumping, and transformations between coordinate systems still apply. ![[Fractal Land](https://www.shadertoy.com/view/XsBXWt) by Kali](fractalland.jpg border=1 width=200px) There is a large software engineering component to this project. Clever design of your functions for reuse will enable an elegant (and efficient) solution. Debugging is tricky in GLSL. While there are some advanced tools for inspecting shaders, a large amount of shader debugging is performed by rendering values (such as normals) to the screen...that's the only way to inspect millions of values efficiently. Coding in GLSL -------------------------------------------------------------------------------------- GLSL syntax resembles that of C (N.B.: _not_ C++) and should be easy for you to pick up. However, it is severely restricted in many ways. There are no pointers or classes, and the only first-class data types are booleans, numbers, vectors and matrices of numbers, and structs based on these. Functions can call each other, but not recursively (there is no stack!) The language [specification and manual](https://www.opengl.org/documentation/glsl/) are online, and the Graphics Codex contains a quick reference to all GLSL functions. Beware that GLSL 4.1 is the latest version that will run on OS X. Windows and Linux use GLGL 4.4 as of 2016. Shadertoy provides a wealth of sample code, although beware that most of it is horribly written by our standards. G3D's data/shader directory provides many more samples which are a bit better (although fairly convoluted because they make some extreme performance optimizations that you do not need for this project). ![[Relentless](https://www.shadertoy.com/view/lss3WS) by srtuss](relentless.jpg border=1 width=200px) G3D provides extensive GLSL helper functions and types (such as Point3, Ray, and Texture2D) for improving readability, however these do not have Doxygen documentation and are hard to discover except by looking at examples because documentation tools do not yet support GLSL. This programming environment is an accurate representation of the state of GPU programming in the industry. It is also quite challenging to encounter for the first time. Look at G3D's [g3dmath.glsl](http://g3d.cs.williams.edu/websvn/filedetails.php?repname=g3d&path=%2FG3D10%2Fdata-files%2Fshader%2Fg3dmath.glsl), [gradient.glsl](http://g3d.cs.williams.edu/websvn/filedetails.php?repname=g3d&path=%2FG3D10%2Fdata-files%2Fshader%2Fgradient.glsl), and [noise.glsl](http://g3d.cs.williams.edu/websvn/filedetails.php?repname=g3d&path=%2FG3D10%2Fdata-files%2Fshader%2Fnoise.glsl) for some particularly useful helper routines. Store your GLSL code in .pix files in your data-files directory. Comment your GLSL code heavily, including description of any unusual language features if you wish. These comments will help me to assess how well you understood what you were doing as well as helping you in the future to understand your own code in this slightly-foreign language. ![[Woods](https://www.shadertoy.com/view/XsfGD4) by Iñigo Quilez](woods.jpg border=1 width=200px) Use F5 to reload shaders without restarting your program. You can fix most shader errors by correcting the source and pressing the "Reload" button on the error dialog. In some cases, the error is unrecoverable and you'll have to restart your program (the driver can also get stuck in a bad state and sometimes needs a clean start after a long session of many reloads). Debug vs. Release mode won't affect the performance of your program much, since it is running on the GPU...except for loading and shader parsing time, which can be long under Debug mode. Examples of Objects ---------------------------------------------------------------------------------------- Some objects that would satisfy the specification include: - A *snow person* comprising at least: - Three balls of snow - Eyes - A carrot nose - Stick arms - A top hat - A *fluted column* comprising at least: - A capital with some form of volutes - A cylindrical shaft (it need not be bowed or tapered) - 24 flutes cut out of the shaft - A base - A *coffee mug* comprising at least: - A bowl - A handle - A design "printed" on the outside of the bowl Choose an appropriate target based on your aspirations for the course, artistic and programming ability, and the level of investment that you can afford for the project. Common Errors ------------------------------------------------------------------------------------- ![A correctly-rendered scene](good.jpg) ![The ground plane appears distorted by the nearby surfaces where they intersect in 2D. Increase the number of ray march iterations and switch to using an analytic ground plane to correct this.](bug-too-few-iterations.jpg) ![The bottom of the sphere has incorrect lighting because the programmer forgot to clamp$\wi \cdot \n\$ to zero in the direct illumination term.](bug-clamp-dot-product.jpg) ![The dark rings are self-shadowing from not bumping the shadow ray's origin, or from not bumping it far enough. Shadow rays should typically be bumped about 1% of the radius of objects in a sphere tracer like this.](bug-no-bump.jpg) ![In this case, the dark rings are caused by an excessively large minimum ray march step, putting the shadow ray origins _inside_ the surface. You can distinguish this from a shadow ray bug because the edges of the cube are slightly rounded and the shadow ray bump must be _huge_ to overcome the error in this case.](bug-excess-min-step.jpg) Distorting Shapes ------------------------------------------------------------------------------------------- The height field distance function approximation, for example, in [_Elevated_](https://www.shadertoy.com/view/MdX3Rr) simply assumes that the vertical distance above the height field is a good estimate to the closest distance to the height field in all directions. This is a good assumption for shallow slopes and a bad assumption in the presence of cliffs. Sophisticated sphere tracers (such as in _Elevated_) take the derivatives of the height field into account to compensate for this, and all tracers scale height field distances by a small constant to help make the assumption conservative in most cases. You can use the trick of scaling down poor distance estimates to make them conservative for many other applications of displacement mapping for sphere tracing. For example, simply adding or subtracting a noise vlaue from a sphere creates a bumpy sphere or [planet](https://www.shadertoy.com/view/ldyXRw)...as long as you scale down the distance estimate so that the marcher doesn't step through the surface due to the distorted estimate. Of course, scaling down distance estimates makes the tracer run slower, so if you can make a very accurate estimate of distance in some other way, that is always preferable. The benefit of sphere tracing is that we can spend performance (i.e., iterations) to trace surfaces that would be too hard to model in other ways. Gallery ============================================================================================== For more inspiration and some impressive extensions to the base specification, look at some examples of work created by my students for this project: - [Williams College CS371 2016](http://www.cs.williams.edu/~morgan/cs371-f16/gallery/5-raymarch/index.html) - [Williams College CS371 2014](http://www.cs.williams.edu/~morgan/cs371-f14/gallery/4-GPU/index.html)