Animation with Global Illumination

Computing the global illumination by random sampling light ray bounces is somewhat problematic for animations. Even when using a very high amount of light samples per pixel and heavy filtering, the computed illumination includes some random noise. Human eye easily notices even a low amount of noise. Just a couple of percents of noise from the full color intensity is still clearly visible. The constantly changing noise pattern usually appears quite unpleasant and distracting.

In this tutorial, we will examine how the noise problem can be solved using multi pass rendering. The big idea is to extend the illumination filtering over multiple frames. Also other random sampled shading components will be filtered in order to improve the quality.

To skip the details about how to configure multi pass rendering, go to the last section Multi Pass Rendering Summary. The summary tells you how to use the predefined components, which come with the installation.

Tutorial level: Advanced

Example projects: tutorprojects/rendering/gi_room.r3d, gi_post1.r3d, gi_post2.r3d

An example shot from the final animation

The First Pass: Ray Tracing Deep Files

1. Model a simple test scene or load the provided test scene 'tutorprojects/rendering/gi_room.r3d'. The test scene includes one area light source in the ceiling. Unless a high sampling rate and slow rendering is used, the shadows appear badly dithered. Nevertheless, you can set the area light's sampling rate to a low value, so that rendering time will be appropriate for animation use. If you model the test scene yourself, animate the scene for example by moving the camera along a path.

The test scene. The area light source is sampled with the minimal quality. Shadows are badly dithered.

2. We will apply post shading to the scene. Select the root level of the scene hierarchy. Go to the materials tab of the select window. select New/From Template from the popup menu and load in the file 'materials/illumination/capture_channels.r3d'. Then drag and drop the material to the view window to apply the material to the whole scene.

3. At this point, switch to the post effects tab of the select window. First select Delete All from the popup menu to remove the effects we will not need. Then select the popup menu's Paste from a file and pick a file realsoft3d/posteffects/post_shading_effects.r3d. This loads two effect configurations, post_shading and post_shading_draft. These are the example systems for filtering shadows, reflections etc. - see the tutorial Post Processing/Post Shading for details. Drag and drop post_shading_draft from the select window to the view window.

The hierarchy of the scene

4. Switch to the Render Settings tab of the select window. Drag and drop GI rendering to the view window. Make sure that View popup/Render/Backup Ray Trace option is enabled. Then select View/Render/Ray Trace. Adjust GI blurring and brightness parameters if necessary and apply View/Render/Post Process menu until the GI parameters seem to be suitable. You can also adjust other post shading components and shadow blurring until the shadows appear non-dithered. This is just a preliminary test stage - you can change all post processing parameters later. That is one of the main points of this tutorial.

So far, the workflow has been pretty similar to any project, which uses global illumination (or post effects in general). Next we prepare the first render pass. Instead of applying the post effects directly, we will collect all untouched raw data into very deep image files. This gives us a possibility to adjust the post effects later to produce variations from the same raw data. If the final animation shows some insufficient filtering quality, or if the customer suddenly wants a warmer lighting into the scene, this approach saves the most time consuming ray tracing phase.

5. To define the deep image file contents, go to the output objects tab of the select window. Select New/Realsoft 3D from the popup menu. Make sure that the new format object is selected and open the property window. Use the name field to rename the format object for example as 'postshade_raw'.

Post shading applied to filter shadows and reflections

We will have to add a large set of channels into this file format object. Select Distance from the Available Channels list and click Add. A new item labeled as Distance appears to the Image Channels box. Click the new image channel item and then change its Storage Type from Float to Integer. Very deep files consume huge amount of hard disk space. By optimizing the storage data type, we can reduce the file size. Word is usually enough to store distances in a scene which has a diameter of some meters, Integer is sufficient for several kilometers and Float is enough for astronomical scales. After adding the distance, add other required channels - the full list is given below. Order of the channels does not matter.

In the list above, the data types are mainly set to the minimal level in order to reduce the file size. If you have lots of free disk space available and feel uncertain about the right choice, just use one step better alternatives. Just remember that Float takes 8 times more space than Byte. The configuration below already consumes about 50 bytes per pixel. In the double PAL resolution, this means over 60 megabytes per image! A CD disk can only store about 8 of such images. For a 250 frame animation, you will need 20 GB of empty hard disk space. However, some of the channels are almost empty and contain only some local compact peaks, such as specular highlights. Therefore, the deep image files usually compress very well and a backup of the 20 BG raw data can most likely be stored to a DVD disk by using .zip compression.

Configuring the deep file format

The illumination components diffuse_GI, reflected_GI and diffuse shading are stored using Word data type. It may be possible to store them using Byte, if the lighting of the scene does not contain high intensity peaks. Insufficient data type increases sampling noise and therefore decreases rendering quality. If you use high intensity illumination effects, for example HDRI, bandwidth of the Byte data type is probably not enough.

[Note] Note
You can avoid the laborious file configuration step by saving predefined file format objects into a suitable folder. Realsoft 3D/imageformats/postshade_raw.r3d contains the required .r3i format object for this tutorial. Use Paste from a File popup menu to load it into the output objects library. You can check and optimize the data type for each channel if you like.

[Note] Note

The file format defined above does not include all post shading channels. Illumination and no_fog are not stored, because the example scene does not include fog. The example version postshade_raw, which can be found from the imageformats folder, is more extensive and includes all post shading channels. You can delete the mentioned two channels, as well as Glow size and Glow color channels to save some disk space. If you need these additional channels for other scenes, the recommended data types are:

  • Illumination - Word. The total result of shading. If you need to spare disk space and don't need to trace illumination peaks accurately, use Byte.

  • no_fog - Word. Illumination before a possible global fog. Use the same data type as for the Illumination channel.

  • Glow color and Glow size - Byte.

When rendering your own projects, you may have to expand the channel collection even more, depending on the applied post effects.

6. When the deep file format is ready, open the file rendering window (Ctrl+r). Add the deep file format, which we configured above, to the Active File Formats list and remove other formats. Give a suitable name and path for the image files (for example 'raytraced' because these files will hold purely raytraced information without any post effects). Select GI rendering as the applied Render Settings object. Clear all post effect selections. Finally, set the resolution to the double of the intended final resolution. If you plan to create a 480*360 pixel animation, double it to 960*720 pixels. The extra resolution will be used for antialiasing purposes. Save the project as pass1.r3d and then start rendering.

Rendering setup for the first pass

The Second Pass: Post Shading and GI Blurring Single Frames

With the second rendering pass, we will filter the noisy area lighting and the global illumination. GI, which is the noisiest and most problematic shading component, will not be added to the shading yet. It will be stored for another, multi-frame filtering pass.

7. Go to the geometric objects tab of the select window. We do not need the geometry in the second rendering pass, so you can either delete all objects or make them invisible in ray trace rendering. However, it is a good idea to leave the camera object intact, because some post effects (DOF, lens flares) depend on the camera. Although we do not use such effects in this tutorial, this is a safe working habit to learn.

8. We can optimize the render settings a bit. Go to the render settings tab, duplicate the GI rendering object and rename the copy as 'QuickRT' or something similar. Using the property window's Ray Tracing tab, set Horizontal and Vertical Undersampling to 4*4 pixels and turn Interpolation off. Switch to the Distr tab and turn box rendering off. In the Post Proc tab, turn the Safety area option off ( an additional continuity area around the image would not improve anything, because the rendering data comes from the deep files, which do not cover the extension region). This kind of render setting configuration initializes the empty black image in a minimal time. You can also load in the optimized settings from the 'realsoft3d/rendersettings/QuickRT' file.

[Note] Note
The render settings object above uses only one single rendering thread. Because of filtering continuity issues, use of several threads would not speed up rendering much. Later, when using frame blur effects, hard disk IO for several rendering threads reading simultaneously the same, very large data files might actually slow down the rendering.

9. Go to the Image Effects tab. We will now configure a post effect system, which filters the shading information from the deep ray traced files.

10. Because the filtering system will include the post shading components, one of the existing objects is a good starting point for the new system. So, select post_shading_draft and rename it as 'PostShadeFile'.

11. You can optimize the post effect system slightly to gain some rendering speed:

  • On the select window, select the VSL effect 'evaluate_shading'. On the property window, put the effect to Advanced mode and delete the following VSL objects from the shader tree:

    • Color fc - unused variable
    • Color+=Linear(Illumination) - custom shading not used
    • Color+=Linear(no_fog) - fog shading not used
    • fc=multiply(diffuse_color, rn) - no flashligt needed
    • Color+=Linear(fc) - no flashligt needed
    • fc=(1-p1)*p2+p1*p3(GI_contrast, one, rn) - GI will not be added in this pass
    • fc*=multiply(diffuse_color, diffuse_GI) - GI will not be added yet
    • Color+=Curve(fc) - GI will not be added yet
    • Color+=Linear(reflected_GI) - reflected GI will not be added yet

Remove inactive post shading components

12. One important effect is still missing: the one, which loads the channels to be processed from the deep files. Apply New\VSL Effect from the popup menu of the select window. Rename the new effect as 'LoadPass1'. Check the Advanced button of the property window. Activate the root node Image Processing and select New/Texture from the popup menu. Then use the browse button to select the first image of the deep image sequence. Switch to the Animation tab of the property window and change the Animation Type to Automatic. Open the popup menu of the VSL tree and change the Output parameter to first one of the required channels, Image:Distance.

13. Duplicate the first Texture object of the VSL tree using the popup menu. Change the output parameter to the next channel, Image:distance2. Repeat this for the remaining channels: normal1, normal2, diffuse_color, diffuse_shading, specular_shading, postreflection, posttransparency, self_illumination, diffuse_GI, reflected_GI and rn. Reading order does not matter.

14. Select again the effect PostShadeFile. Drag and drop the new VSL effect 'LoadPass1', which we just created, to the beginning of the Active effects list. The post effect setup for the second pass is now ready.

15. Go to the output objects tab of the select window. Select New/Realsoft 3D from the popup menu to add a new native format object (you can also load imageformats/GI_raw file to skip this step). Rename the new object as 'GI_Raw'. Then add the following channels to the object the same way as instructed in the step 5 above:

Define a VSL effect, which reads channels, and add it to the beginning of 'PostShadeFile' configuration

  • Color - Byte
  • Distance - Integer
  • distance2 - Integer
  • normal1 - Byte
  • normal2 - Byte
  • diffuse_color - Byte
  • diffuse_GI - Word
  • reflected_GI - Word
  • rn - Byte

The new format no longer includes majority of the post shading channels, because they are already composited into the Color channel. The channels above store the GI information for the final filtering pass.

The file format which stores the channels for the final pass

16. Open the file rendering window. Change the file name and path so that images from the first pass won't get accidentally overwritten. Change the active file format to the new GI_raw object. Set the Render Settings selection to the QuickRT object, and Effect/Box to PostShadeFile. The resolution remains the same (=the final resolution doubled). Save the project as 'pass2' and start rendering the animation.

Rendering the second pass

The Third Pass: Multi Frame Filtering

17. When the second pass rendering is finished, go to the post effects tab of the select window. We will now configure the last filtering system. An example system is included in the file 'realsoft3d/posteffects/MultiFrameGI.r3d' - load it in to skip the steps 18-21 of the tutorial.

18. Select New/FrameBlur from the popup menu. Rename the created effect as FrameBlurDiffuse, because we will blur diffuse global illumination with it. Use the browse gadget to select an image from the second render pass. Switch to the Animation tab of the property window and change Animation Type to Automatic. Go back to the Image tab and set Blur Start and Blur End to 5 frames. This means that the effect will combine GI information from 5 frames before the current frame to 5 frames after the current frame, 11 frames total. In each frame, we will get ten times more information about the global illumination; this will help to reduce the noise significantly.

Set Source and Destination Channels to diffuse_GI. This means that the effect will read diffuse channel from the disk files and store the blurred result to the diffuse channel of the post processed image. Then set Geometric Weighting to 1.0 - frame blur should combine GI information only from geometrically continuous areas.

19. Duplicate the created FrameBlur effect. Rename the new effect as FrameBlurReflected. This time set the source and destination channels to reflected_GI. This part of the GI is usually much more coherent as the diffuse GI. Therefore, set Start and End to only 1 frames.

Frame blur effects for the diffuse and reflected GI

20. Create a new VSL effect and rename it as LoadPass2. Add 7 Texture objects, which read Color, diffuse_color, normal1, normal2, rn, Distance and distance2 from the pass 2 files. Remember to set the animation type of each Texture object to Automatic. Note that we do not have to read diffuse_GI or reflected_GI channels here - the two frame blur effects take care of that.

21. Duplicate the post effect PostShadeFile and rename the copy as 'MultiFrameGI'. Remove all effects above GI_blur from the MultiFrameGI configuration. Add the above created effects FrameBlurDiffuse, FrameBlurReflected and LoadPass2 to the beginning of it in this order. Leave GI_blur and GI_blur_reflected to the configuration. These effects will be used now again in order to eliminate illumination banding artifacts, which easily occur, when a moving camera reveals previously obscured areas of the scene (GI sampling within a frame is heavily randomized, whereas time stepping is not random at all but happend at regular intervals).

The VSL effect, which loads channels from the second pass images, and the post effect configuration for the final pass

So, we first blur GI over image area, then over time and finally again within each frame. However, you can and should reduce GI_blur values quite much for the second filtering round. For example, GI_blur Level can be 10 and Iterations 2. Similarly, GI_blur_reflected can be reduced to Level=5, Iterations=1.

Contine editing the contents of MultiFrameGI as follows: remove all effects after GI_blur_reflected. Then add GI_brightness_control to the end of the MultiFrameGI configuration. It will be the last effect. It adds the many times filtered global illumination to the images.

Select MultiFrameGI effect and go to the the scaling tab in the property window. Set Output Scale X and Y to 0.5. The last pass will shink resolution to one half, thus improving the anti-aliasing quality.

22. Open the file rendering window. Now set the Active Outputs to a suitable 'normal' image or animation format, such as .jpg or .avi. Define also a suitable file name. The render settings should be again QuickRT. Set Post effect/Box to the new MultiFrameGI. Drop the resolution to the actual final resolution (e.g. 480*360) from the doubled value. MultiFrameGI, which has Output scaling factors 0.5, halves the resolution from its source data to antialiase the images. Save the project as pass3 and start rendering.

The final rendering pass

Multi Pass Rendering Summary

A brief summary of the multi pass rendering steps using the predefined components is given below. You can naturally use your own components instead of the objects, which come with the standard installation.

1. Model the scene as usual

2. Default map the material 'realsoft3d/materials/illumination/capture_channels' to the root level.

3. Render using the deep file format 'realsoft3d/imageformats/postshade_raw' in doubled resolution and with no antialiasing.

4. Load in the post effect system 'realsoft3d/posteffects/PostShadeFile'. Select the included VSL effect 'LoadPass1' and using the property window, change the file name of every Texture object to the image sequence you rendered in the previous pass. Set the frame count in the Animation tab of each Texture VSL object to match the frame count of yor animation. Remove or disable the VSL object, which adds filtered GI to the image. Change Output Scale X and Y of the LoadPass1 configuration to 1.0.

5. Make all geometry objects except the camera RT invisible and render the second pass to 'realsoft3d/imageformats/GI_raw' format with an optimized render settings object 'realsoft3d/rendersettings/QuickRT' and 'PostShadeFile' as the post effect per render box.

6. Load in the post effect system 'realsoft3d/posteffects/MultiFrameGI'. Update the image file names of the two FrameBlur effects to match the image files from the second pass. Change the Frames value of the FrameBlur effects in the Animation tab. Then update the file names of the VSL Texture objects in the VSL effect 'LoadPass2', and set the Frames value in the Animation tab to a correct value.

7. Render the final pass to the desired format with the QuickRT settings and the MultiFrameGI post effects.