Why Raytracing won't simplify AAA real-time rendering

"The big trick we are getting now is the final unification of lighting and shadowing across all surfaces in a game - games had to do these hacks and tricks for years now where we do different things for characters and different things for environments and different things for lights that move versus static lights, and now we are able to do all of that the same way for everything..."

Who said this?

Jensen Huang, presenting NVidia's RTX? 

Not quite... John Carmack. In 2001, at Tokyo's MacWorld, showing Doom 3 for the first time. It was though on an NVidia hardware, just a bit less powerful than today's 20xx/30xx series. A GeForce 3.

Can watch the recording on YouTube for a bit of nostalgia.

And of course, the unifying technology at that time was stencil shadows - yes, we were at a time before shadowmaps were viable.

Now. I am not a fan of making long-term predictions, in fact, I believe there is a given time horizon after which things are mostly dominated by chaos, and it's just silly to talk about what's going to happen then.

But if we wanted to make predictions, a good starting point is to look at the history, as history tends to repeat. What happened last time that we had significant innovation in rendering hardware? 

Did compute shaders lead to simpler rendering engines, or more complex? What happened when we introduced programmable fragment shaders? Simpler, or more complex? What about hardware vertex shaders - a.k.a. hardware transform and lighting...

And so on and so forth, we can go all the way back to the first popular accelerated video card for the consumer market, the 3dfx.

Memories... A 3dfx Voodoo. PCem has some emulation for these, if one wants to play...

Surely it must have made things simpler, now having to program software rasterizers specifically for each game, for each kind of object, for each CPU even! No more assembly. No more self-modifying code, s-buffers, software clipping, BSPs... 

No more crazy tricks to get textures on screen, we suddenly got it all done for us, for free! Z-buffer, anisotropic filtering, perspective correction... Crazy stuff we never could even dream of is now in hardware.

Imagine that - overnight you could have taken the bulk of your 3d engine and deleted it. Did it make engines simpler, or more complex? 

Our shaders today, powered by incredible hardware, are much more code, and much more complexity, than the software rasterizers of decades ago!

Are there reasons to believe this time it will be any different?

Spoiler alert: no. At least not in AAA real-time rendering. Complexity has nothing to do with technologies. 

Technologies can enable new products,  true, but even the existence of new products is always about people first and foremost.

The truth is that our real-time rendering engines could have been dirt-simple ten years ago, there's nothing inherently complex in what we got right now.

We can solve shadows with large shadowmaps, easily. We can do shading easily, just loop over all the lights in the scene, it's fairly trivial. We can do antialiasing easily, just rely on MSAA or if you want to be even simpler, full-screen supersampling...

Of course, you would need to trade-off performance for such relatively "brute-force" approaches... But it's doable, you can write a slow, photorealistic real-time renderer in relatively few lines of code, today. 

The only reason we do not... is in people and products. We like our graphics to be cutting edge as graphics still sell games, sell consoles, are talked about.

And in order to have that cutting edge advantage, we easily "sacrifice" engineers, really, they don't matter in the grand scheme of things, at all.

So pretty... Look at that sky. Worth its complexity, right?

In AAA is perfectly ok to have someone work for say, a month, producing new, complicated code paths to save say, one millisecond in our frame time. It's perfectly ok often to spend a month to save a tenth of a millisecond!

Until this equation will be true, we will always sacrifice engineering, and thus, accept bigger and bigger engines, more complex rendering techniques, in order to have larger, more beautiful worlds, rendered faster!

Another point of view, useful to make some prediction, comes from the classic works of Clayton Christensen on innovation. Christensen posits that technologies evolve in cycles of commoditization, bringing costs down and scaling, and de-commoditization, leveraging integrated, proprietary stacks to deliver innovation.

In AAA games, rendering has not been commoditized, and the trend does not seem going towards commoditization yet. 

Innovation is still the driving force behind real-time graphics, not scale of production, even if we have been saying for years, perhaps decades that we were at the tipping point, in practice we never seemed to reach it quite yet.

We are not even, at least in the big titles, close to the point where production efficiencies really matter that much for artists! - i.e. our rendering still teams still typically dwarf the efforts put into tooling and asset production efficiency. Think about that - how much production teams with bespoke AAA engines actually tailor to production efficiency, how many tradeoffs we make - or not - to improve efficiency versus either quality or performance...

We do not like to accept tradeoffs on our stacks, we run on tightly integrated technologies because we like the idea of customizing them to the game specifics - i.e. we have not embraced open standards that would allow for components in our production stacks to be shared and exchanged.

Alita - rendered with Weta's proprietary (and RenderMan-compatible) Manuka

I do not think this trend will change, at the top end, for the next decade or so at least, the only time horizon I would even care to make predictions. 

We see just recently, in the movie industry (which is another decent way of "predicting" the future of real-time) that production pipelines are becoming somewhat standardized around common interchange formats. For the top studios, rendering itself is not, with most big ones running on their own proprietary path-tracing solutions...

So, is it all pain? And it will always be?

No, not at all! 

We live in a fantastic world full of opportunities for everyone. There is definitely a lot of real-time rendering that has been completely commoditized and abstracted.

People can create incredible graphics without knowing anything at all of how things work underneath, and this is definitely something incredibly new and exciting.

Once upon a time, you had to be John friggin' Carmack (and we went full circle...) to make a 3d engine, create Doom, and be legendary because of it. Your hardcore ability of pushing pixels made entire game genres that were impossible to create without the very best of technical skills.

Today? I believe a FPS templates ships for free with Unity, you can download Unreal with its source code for free, you have Godot... All products that invest in art efficiency and ease of use first and foremost.

Everyone can create any game genre with little complexity, without caring about technology - the complicated stuff is only there for cutting-edge "blockbuster" titles where bespoke engines matter, and only to some better features (e.g. fidelity, performance etc), not to fundamentally enable the game to exist...

And that's already professional stuff - we can do much better!

Three.js is the most popular 3d engine on github - you don't need to know anything about 3d graphics to start creating. We have Roblox, Dreams, Minecraft and Fortnite Creative. We have Notch, for real-time motion graphics...

Computer graphics has never been simpler, and at the same time, at the top end, never been more complex.

Roblox creations are completely tech-agnostic.

Conclusions

AAA will stay AAA - and for the foreseeable future, its bespoke engines, fuelled by armies of rendering engineers, will keep being wonderfully complicated.

Slowly we will invest more in productivity for artists and asset production - as it really matters for games - but it's not a fast process.

In fact, it's easier for AAA to become relatively irrelevant (compared to the overall market size - that expands faster in other directions than in the established AAA one) - than for it to radically embrace change.

Other products and other markets is where real-time rendering is commoditized and radically different. It -is- already, all these products already exist, and we already have huge market segments that do not need to bother at all with technical details. That might make games with zero rendering engineers... And the quality and scope of these games grows year after year.

And yes, this market was definitely facilitated by the fact that we have 3d hardware acceleration pretty much in any device now - but at the same time new hardware is not going to change any of that.

Raytracing will only -add- complexity at the top end, we'll see incredible hybrid techniques, and if we have today dozens of ways of doing shadows and combining signals to solve the rendering equation in real-time, we'll only grow these more complex - and wonderful, in the future.

And it will eventually percolate to the "bottom-end" eventually, but that won't change complexity or open new products either because people who are making real-time graphics with more user-friendly tools already don't have to care about the technology that drives them - technology there will always evolve under the hood, never to be seen by the users...

from Hacker News https://ift.tt/2JolS8Z