Icare3D

NVIDIA just launched the TU102/TU104 (GeForce RTX 2080ti/2080), first GPUs based on the Turing architecture. This new architecture brings hardware ray-tracing acceleration, as well as many other new and really cool graphics features. A good high-level overview of the architecture and new graphics features can be found in the Turing Architecture whitepaper as well as this blog post. Most of these features are exposed through both Vulkan and OpenGL extensions, and I will quickly go through each of them in this post. A big thanks to the many people at NVIDIA who worked hard to provide us with these extensions !

Ray-Tracing Acceleration

• VK_NVX_raytracing (GLSL_NVX_raytracing / SPV_NVX_raytracing)

Turing brings hardware acceleration for ray-tracing through dedicated units called RT cores. The RT cores provide BVH traversal as well as ray-triangle intersection. This acceleration is exposed in Vulkan through a new ray-tracing pipeline, associated with a series of new shader stages. This programming model maps the DXR (DirectX Ray-Tracing) model, which is quickly described in this blog post, and this blog post details the Vulkan implementation.

A GTC 2018 presentation about Vulkan Ray-Tracing can also be found there: http://on-demand.gputechconf.com/gtc/2018/video/S8521/ (Slides here).
This blog post details

Mesh Shading

• VK_NV_mesh_shader / GL_NV_mesh_shader (GLSL_NV_mesh_shader / SPV_NV_mesh_shader)

As well as in his Siggraph 2018 presentation: http://on-demand.gputechconf.com/siggraph/2018/video/sig1811-3-christoph-kubisch-mesh-shaders.html

A full OpenGL sample code which implements a compute-based adaptive tessellation technique can also be found there: https://github.com/jdupuy/opengl-framework/tree/master/demo-isubd-terrain

Variable Rate Shading

• VK_NV_shading_rate_image / GL_NV_shading_rate_image (GLSL_NV_shading_rate_image / SPV_NV_shading_rate)

This is a very powerful hardware feature which allows the application to dynamically control the number of fragment shader invocations (independently of the visibility rate) and vary this shading rate across the framebuffer. The shading rate is controlled using a texture image ("Shading Rate Image", 8b/texel) where each texel specifies an independent shading rate for blocks of 16x16 pixels. The rate is actually specified indirectly using 8b indices into a palette which is specified per-viewport and stores the actual shading rate flags.

Not only the feature allows to vary the MSAA shading rate per-pixel (allowing 1x,4x,8x, and now even 16x SSAA, but with a maximum of 8x depth test and color storage), but it also allows to drop the shading rate below one invocation per-pixel, down to one invocation per block of 4x4 pixels (through one per 1x2, 2x1, 2x2, 2x4 and 4x2 pixels) and even zero invocation. 

The GLSL extensions also exposes intrinsics allowing fragment shaders to read the effective fragment size in pixels (gl_FragmentSizeNV) as well as the number of fragment shader invocation for a fully covered pixel (gl_InvocationsPerPixelNV). This opens the road to many new algorithms and more efficient implementations of optimized shading rate techniques, like Foveated Rendering, Lens Adaptation (for VR), Content or Motion Adaptive Shading.

More info on Variable Rate Shading in this blog post: https://devblogs.nvidia.com/turing-variable-rate-shading-vrworks/

Exclusive Scissor Test

• VK_NV_scissor_exclusive / GL_NV_scissor_exclusive

This adds a second per-viewport scissor test, which culls fragments *inside* (exclusive) the specified rectangle, unlike the standard scissor test which culls *outside* (inclusive). This can be used for instance to implement more efficient multi-resolution foveated-rendering techniques (in conjunction with Variable Rate Shading), where raster passes fill concentric strips of pixels by enabling both inclusive and exclusive scissor tests.

Texture Access Footprint

• VK_NV_shader_image_footprint / GL_NV_shader_texture_footprint (GLSL_NV_shader_texture_footprint / SVP_NV_shader_image_footprint)

These extensions expose a set of GLSL (and SPIR-V) query functions which report the texture-space footprints of texture lookups, ie. some data identifying the set of all texels that may be accessed in order to return a filtered result for the corresponding texture accesses (which can use anisotropic-filtering and potentially cover large footprints). Footprints are returned and represented as an LOD value, an anchor point and a 64-bit bitfield where each bit represents coverage for a group of neighboring texel (in 2D, group granularity can range from 2x2 to 256x256 texels).

This is actually an important component for implementing multi-pass decoupled and texture-space shading pipelines, where a restricted set of actually visible pixels must be determined in order to efficiently perform shading in a subsequent pass.

Derivatives in Compute Shader

• VK_NV_compute_shader_derivatives / GL_NV_compute_shader_derivatives (GLSL_NV_compute_shader_derivatives / SPV_NV_compute_shader_derivatives)

These extensions bring Compute even closer to Graphics by adding support for Quad-based derivatives in Compute Shaders, using the x and y coordinates of the local workgroup invocation ID. This allows Compute Shaders to use both built-in derivative functions like dFdx(), as well as texture lookup functions using automatic LOD computation, and the texture level of detail query function (textureQueryLod()).
Two layout qualifiers are provided allowing to specify Quad arrangements based on a linear index or 2D indices.

Shader Subgroup Operations

• VK_NV_shader_subgroup_partitioned / GL_NV_shader_subgroup_partitioned (SPV_NV_shader_subgroup_partitioned)

These shader extensions provide a series of  ballot-based partitioning and scan/reduce operations which operate on "subgroups" of shader invocations. This can be used for instance to implement clustering and de-duplication operations on sets of values distributed among different shader invocations.

Barycentric Coordinates and manual attributes interpolation

• VK_NV_fragment_shader_barycentric / GL_NV_fragment_shader_barycentric (GLSL_NV_fragment_shader_barycentric / SPV_NV_fragment_shader_barycentric)

Illustration courtesy of Jean-Colas Prunier, https://www.scratchapixel.com/

This feature exposes barycentric coordinates as Fragment Shader input in GLSL (and SPIR-V), and provides the ability for a Fragment Shader to directly fetch raw per-vertex values in order to perform manual barycentric interpolation. 

A three-component vector built-in input gl_BaryCoordNV provides perspective-corrected barycentric coordinates (gl_BaryCoordNoPerspNV for non- perspective-correct). Per-vertex inputs use the same brackets array syntax as for Tesselation and Geometry Shader inputs, and a pervertexNV qualifier is added to identify input blocs and variables which read raw per-vertex values from the vertices of the original primitive.

This feature potentially allows more efficient data passing to the Fragment Shader using compact or compressed data formats for instance. It could also allow interpolation from vertex values fetched directly from memory, user defined interpolations, or various reconstructions and computations using raw attributes accessed from the three vertices.

Ptex Hardware Acceleration

• VK_NV_corner_sampled_image

An corner-sampled image has texels centered on integer coordinates instead of being halfway, which allows edge sampling coordinates to filter to the exact texels on the edge of the texture. This facilitates implementing Ptex (Per-face Texture [Burley and Lacewell 2008], cf. https://developer.nvidia.com/sites/default/files/akamai/gamedev/docs/Borderless%20Ptex.pdf) texturing in real-time applications by providing proper filtering and interpolation. Ptex uses separate textures for each face of a subdivision surface or polygon mesh, and sample locations are placed at pixel corners, maintaining continuity between adjacent patches by duplicating values along shared edges.

Representative Fragment Test

• VK_NV_representative_fragment_test / GL_NV_representative_fragment_test

Multi-View Rendering

• VK_KHR_multiview / GL_OVR_multiview

List of new extensions

• EXT_post_depth_coverage
• EXT_raster_multisample
• EXT_sparse_texture2
• EXT_texture_filter_minmax
• NV_conservative_raster
• NV_fill_rectangle
• NV_fragment_coverage_to_color
• NV_fragment_shader_interlock
• NV_framebuffer_mixed_samples
• NV_geometry_shader_passthrough
• NV_sample_locations
• NV_sample_mask_override_coverage
• NV_shader_atomic_fp16_vector
• NV_path_rendering_shared_edge
• NV_viewport_array2

Quick description of the new extensions

Multisampling

• Target-independent multisampling control and mixed samples (NV_framebuffer_mixed_samples + EXT_raster_multisample):

This feature adds a lot of flexibility to the multi-sampled rasterization. It decouples the rasterization sampling frequency (which can be set explicitly) from the actual framebuffer storage. This enables rasterization to operate at higher sampling frequency than the one of the target render color buffers. It supports both depth and stencil testing at this frequency, if the corresponding depth and stencil buffers are sampled accordingly (it must be a multiple of the number of samples in the color buffers).
There are still some constraints; All color buffers must have the same number of samples, and the raster sample count must match the depth and stencil buffer sample count if depth or stencil test is enabled, and it must be higher or equal to the color buffer sample count.

A new “coverage reduction stage” is introduced in the per-fragment operations (after the fragment shader in early-z mode, after the depth-test in late-z), which converts a set of covered raster/depth/stencil samples to a set of covered color samples. There is an implementation-dependent association of raster samples to color samples. The reduced "color coverage" is computed such that the coverage bit for each color sample is 1 if any of the associated bits in the fragment's coverage is set, and 0 otherwise. This feature can be used in conjunction with the coverage to color feature (cf. below), in order to get the FS output coverage mask automatically transformed into a color by ROP. According to AnandTech, when rasterizing with explicit multisampling and no render-target, Maxwell allows evaluating primitive coverage at 16x MSAA.

Note that EXT_raster_multisample is equivalent to "Target-Independent Rasterization" in Direct3D 11.1, which allows using multiple raster samples with a single color sample, as long as depth and stencil tests are disabled, and it is actually a subset of NV_framebuffer_mixed_samples which is more general and exposes more flexibility.

• Coverage to color conversion (NV_fragment_coverage_to_color):

• Post-depth coverage (EXT_post_depth_coverage):

When operating in early-depth mode (layout(early_fragment_tests) in;, see here for more information), this extension allows the fragment shader to get the post depth-test coverage mask of the current fragment as input (gl_SampleMaskIn[], for which only sample passing the depth-test are set), unlike the standard GL 4.5 behavior which provides the pre- depth-test coverage (actual triangle coverage).

• Multisample coverage override (NV_sample_mask_override_coverage)

With standard OpenGL, the Fragment Shader output coverage mask (gl_SampleMask[]) is ANDed with the actual primitive input coverage mask before being used in subsequent pipeline stages. This extension disables this AND operation, which allows the fragment shader to entirely override the primitive coverage, and enables setting coverage bits that are not present in the input mask. This is actually very nice, because it allows using the output coverage as a way to dynamically route color output values into arbitrary sample locations inside a multisampled render target.

• Programmable sample locations (NV_sample_locations):

Allows applications to explicitly set the location of sub-pixel samples for multisample rasterization, providing fully programmable sampling patterns. Sampling patterns can be defined within a grid of adjacent pixels, which depends on the number of samples. According to provided queries, the sub-pixel positions are snapped to a 16x16 sub-pixel grid.

Rasterization

• Conservative rasterization (NV_conservative_raster):

This is a really great feature. It allows Rasterization to generate fragments for any pixel touched by a triangle, even if no sample location is covered on the pixel. A new control is also provided to modify the window coordinate snapping precision in order to allow the application to match conservative rasterization triangle snapping with the snapping that would have occurred at higher resolution. Polygons with zero area generate no fragments. Any location within a pixel may be used for interpolating attributes, potentially causing attribute extrapolation if outside the triangle. This can be useful for binning purpose for instance (using one pixel per-tile).

• Fragment Shader Interlock (NV_fragment_shader_interlock):

• Screen-space bounding-box rasterization (NV_fill_rectangle):

Geometry processing

• Geometry Shader Passthrough (NV_geometry_shader_passthrough):

• Viewport Multicast (NV_viewport_array2):

Viewport multicast allows automatically broadcasting the same primitive to multiple viewports (and/or multiple layers when using layered render-targets) simultaneously, in order to be rasterized multiple times. It is exposed through a new gl_ViewportMask[] GLSL output attribute which is available in both the vertex shader and the geometry shader. This can be especially powerful when combined to the new passthrough geometry shader. A sample using it for speeding-up cascaded shadow maps is available here.

Texturing

• Enhanced sparse textures (EXT_sparse_texture2) :

This extension improves on ARB_sparse_texture, which separates the allocation of virtual address space from the physical memory of textures, and provides the ability to sparsely allocate the physical backing-store of 2D/3D/2DArray textures on a per-tile basis. This new extension adds the ability to retrieve texture access residency information from GLSL, to specify minimum allocated LOD to texture fetches and to return a constant zero value for lookups into unallocated pages. It also adds support for multi-sampled textures.

• Texture Filter min/max (EXT_texture_filter_minmax):

This exposes a new sampler parameter which allows performing a min or max reduction operation on the values sampled inside a texture filtering footprint, instead of the regular linear interpolation. It is supported for all kind of textures, as well as anisotropic filtering.

Atomics 

• FP16 global atomics (NV_shader_atomic_fp16_vector) :

I defended my Ph.D thesis on GigaVoxels last July, and the document is now online.

You can download it there:
GigaVoxels: A Voxel-Based Rendering Pipeline For Efficient Exploration Of Large And Detailed Scenes

You can also check my other publications on my Ph.D webpage.

I started to compile links to websites where free 3D models can be found. If you know other good websites, feal free to post them in the comments :-)

Static models and scenes:

• Great collection of models for scientific publications on Morgan McGuire webpage: http://graphics.cs.williams.edu/data/meshes.xml
• 3D Render challenge: http://www.3drender.com/challenges/
• Crytek: http://www.crytek.com/cryengine/cryengine3/downloads
• Keenan Crane : http://www.cs.caltech.edu/~keenan/models.html
• Sibenik model: http://hdri.cgtechniques.com/~sibenik2/download/
• AIM@Shape : http://shapes.aimatshape.net/
• Characters Creative Common: http://artist-3d.com/
• Characters: http://www.3dvalley.com/3d-models/characters
• Blender files: http://www.blendswap.com/3D-models/category/featured/
• Archive 3D: http://archive3d.net/

• Ingo Wald: http://www.sci.utah.edu/~wald/animrep/
• MIT CSAIL 1: http://people.csail.mit.edu/drdaniel/mesh_animation/index.html
• MIT CSAIL 2: http://people.csail.mit.edu/drdaniel/dynamic_shape/index.html
• MIT Animals and Face: http://people.csail.mit.edu/sumner/research/deftransfer/data.html
• Face data: http://grail.cs.washington.edu/software-data/stfaces/index.html 
• Pants: http://www.ryanmwhite.com/research/cloth_cap.html

Full story here :-D

1. We want the code to be as generalized as possible.
2. We design everything future-proof and extendible.
3. When a feature request arrives, we’re doomed we need to change a lot of code.
4. Why?
5. Because everything was designed as generalized as possible.
6. goto 1;

GDC 2011 is over now and presentations start to appear on-line :-)

DICE: http://publications.dice.se/
NVIDIA: http://www.nvidia.com/object/gdc2011.html
AMD: http://developer.amd.com/documentation/presentations/Pages/default.aspx#GDC
INTEL: http://software.intel.com/en-us/articles/intelgdc2011/
Khronos on OpenGL: http://www.khronos.org/library/detail/2011-gdc-opengl

More links can be found on this blog: http://msinilo.pl/blog/?p=724

This is the season of the new game engines : Dice Frostbite 2, Crytek CryEngine 3 and Epic Unreal Engine 3 !

Here it is, CUDA 4.0 RC just got released to NVIDIA Registered developers.

Interesting stuff from the CUDA manual:

• Layered Textures Support (GL_TEXTURE_1D/2D_ARRAY)  : New tex.a1d/.a2d modifiers in PTX. But unfortunately the surface instruction do not support them yet, Grrrr
  Layered textures are created using cudaMalloc3DArray() with the cudaArrayLayered flag. New cudaTextureType2DLayered/ cudaTextureType2DLayered texture sampler types and tex1DLayered()/tex2DLayered() access intrinsics.

• New .address_size PTX specifier : Allows  to specify the address size (32b/64b) used throughout a PTX module.
• Inline PTX assembly: This feature was already present since CUDA 2.x but was not officially supported. It's now fully supported and documented :-D
• Driver API, new thread-safe stateless launch API function cuLaunchKernel(): cuLaunchKernel(kernelObj,   blocksPerGrid, 1, 1,   threadsPerBlock, 1, 1,   0, 0, args, 0);
• FERMI ISA documented and supported by cuobjdump.
• Enhanced C++: Support for operators new  and  delete, virtual functions.