TL;DR of the paper 'Conservative Z-Prepass for Frustum-Traced Irregular Z-Buffers'

I will in this post write down a TL;DR of the paper 'Conservative Z-Prepass for Frustum-Traced Irregular Z-Buffers'.

The paper makes heavy use of irregular z-buffers, so first this concept shall be explained. Observe the below image

'

The blue points are the points that are actually seen by the camera. On the other hand, the yellow points are the points seen from a z-buffer rendered from the perspective of the light source(that is, a shadow map). A major issue with shadow maps is that we need a high resolution shadow map to achieve accurate shadows. And to achieve pixel-perfect shadows, very high resolutions are required. In this image, we can see that P1 is falsely considered to be in shadow, because the point P3 was rendered to the shadow map. P2 is however correctly classified to be in shadow(everything BELOW the green line is in shadow.). One simple solution is to simply increase the resolution of the shadow map, however, the paper presents an alternative solution.

The Irregular Z-buffer offers a solution to the problem: it means that we store the blue points in the Z-buffer! This is called an Irregular z-buffer(IZB). This means that every visible point will be stored in a texel in the Z-buffer. We simply reproject every blue point into light-space. The reprojected point will fall into one of the texels of the shadow map, and we store the point into that texel. As can be observed below, both P1 and P2 will fall into the same texel:

This means that a list of points must be stored in every texel of the Z-buffer. In the paper, this is achieved by means of storing a linked list for every texel in the IZB:

Now, we will need to determine which of the points stored in the IZB are actually in shadow. Frustum-tracing is used to accomplish this; we rasterize every shadow-casting triangle from the perspective of our light source, just like when we are rendering regular shadow maps. Now, every rasterized triangle will cover some texels in the IZB, and these texels contains a linked list of points. We need to check whether that triangle casts a shadow on those points.

As shown above, we can extend a frustum from the triangle, using the direction of the light source. The frustum will be built up from four planes(one plane for the triangle itself, and three planes for the three sides of the triangle.) Everything within this frustum, will clearly be in shadow. In the image, P1 is in shadow, while P2 is not. Thus our algorithm is simple: we rasterize all triangles in light-space. The rasterized fragments will cover some texel in the IZB, and we will loop over all the points in the linked list, and check if they are within the frustum. If they are, then they are marked as shadowed. The cool thing about using frustum-tracing, is that we will obtain results that are equivalent to as if we had raytraced the shadows instead. As a result, we get pixel perfect shadows!

Now, there is one important implementation detail to this algorithm: the triangles must be rasterized using conservative rasterization.

Using the default rasterization algorithm, means that fragments are only rasterized, if the fragment center is covered by the triangle(so red fragments in the image above are not rasterized, only green ones are.). However, if a triangle at all covers a texel, then we certainly want to perform frustum culling for the points in the linked list, so this default behaviour is not what we want. Instead, we must enable conservative rasterization, which ensures that fragments are always rasterized when it is covered by some triangle.

So that is how a IZB works. Next, we will explain what a conservative shadow map(CSM) is. The CSM gives us a conservative estimate of which points are in shadow; that is, it allows us to find points that we for sure know are in shadow, but it doesn't find all of them. The issue with IZB, is that it is expensive to create a linked list, and then loop over it, when computing the shadowed points. The CSM allows us to find points that for sure are in shadow, before we run the expensive IZB technique. These found points do not need to be checked in the IZB technique. Because less points now need to be checked for in the IZB calculations, we obtain a performance boost.

The below image illustrates the conservative shadow map. The green line is a triangle seen from the side. If a triangle fully intersects a texel, that is, if the triangle fully covers that texel in light-space, then we output the conservative depth of that triangle. This is the depth value of that triangle which is the most distant from the light source. If we now use this conservative shadow map like a regular shadow map, then we will find that all points within the pink volume will be considered to be in shadow. Because since the depth value is only outputted when the triangle fully covers the texel, all points that are within that texel, and at a greater distance from the stored z-value, must be in shadow. So P2 will be considered to be in shadow, but not P1 though. P1 should be considered in shadow, but it is not according to the CSM, since the CSM only provides a rough and conservative estimate. In order to correct classify P1 as a shadowed point, the authors use the IZB algorithm afterward.

To summarize, this is how it works: We want to find which pixels on the screen are in shadow. Now, we first use CSM, to find a conservative estimimate of pixels that must be in shadow. Next, for the pixels that were not classified as shadowed by CSM, we use the IZB technique. Since IZBs are expensive, using the CSM first as an early-out-test allows us to gain performance.