CitizenCon 2951: Gen 12 & The Multicore of Vulkan

Transcript — every row opens the video at that moment

1. 00:00:14

   hi i'm ali brown director of graphics engineering here at cloud imperium games and today we're going to be talking about gen 12 our new multi-core renderer in addition to myself we'll also be hearing from christopher bolte our core engine architect and dell barnes our graphics programmer and today we're going to be talking about the need for our new renderer the architecture and how it's built vulcan which is the back end api we use and the progress of how we're doing so far and then what's next for the graphics team so to help understand the need for the gen 12 renderer i first want to take a look at the high level structure of our existing rendezvous to try and understand how the new render will differ so here you see a diagram very simplified of our overall architecture and on the left hand side you can see

2. 00:01:02

   the 3d engine and this is what manages all the visible objects in the scene for example their position their size and ultimately is responsible for the culling of them objects and deciding what should be sent through on screen each frame to the renderer and obviously the job at the render is just to feed out the image at the end of the frame and to do this to manage this process all these objects have to pass through the single universal pipe this conduit of information in the center and as a result of that every object has to come through of a certain amount of settings or paperwork and baggage to describe how they should be configured and stuff represented on the diagram by these switches so this type of design initially seems very flexible having this universal pipe you can very easily toggle certain settings and get a very different rendered result for each object but the

3. 00:01:50

   reality is actually that the objects and the render both end up extremely complicated because they have to decipher this list of instructions and distribute these objects through the various pipelines and stages inside the renderer and this has to happen for every object every frame and this deciphering ends up resulting in quite a significant performance problem and it also makes the renderer quite inflexible because it's quite complicated and hard to change the architecture so how does our gen 12 renderer differ from this well the methodology we're taking on is to try and make everything as explicit as possible and we're trying to minimize all these redundant connections and switches and configuration options and try and get things much more streamlined so to do this each object in the world will directly communicate with the render pass responsible for drawing that object at startup time so when the object is

4. 00:02:38

   first spawned it will communicate directly with the path responsible for it and pre-configure everything possible at that stage now this has a bunch of benefits so by speaking directly to the thing that's rendering it it doesn't have to worry about all the other rendering systems so we end up with a much more limited set of parameters both on the object and on the rendering pass and all the settings we do have are very relevant for the task which makes things much less complicated and it's easier to optimize more flexible and more modular so each of these passes then goes on to generate lower level commands that are actually going to map directly to our graphics api which is the piece of software that sits just above your gpu driver and this back end collects and executes all of these instructions and because there is no high level knowledge of the rendering in this back end it's just simply just issuing commands it ends up

5. 00:03:27

   very simple and streamlined and much simpler than the equivalent on our old renderer so the benefit of this simpler code is much easier to multi-thread which is to say to run on multiple cpu cores in parallel something that is crucial for modern cpus so multi-threading is a complex and often misunderstood paradigm so i thought it was worth us talking briefly about how we optimize multi-threaded code and for that i'm going to use an analogy of building a house which is another complex engineering task with dependencies so here you can see a list of tasks for a builder a joiner and an electrician or running one after another and there's dependencies between them showing in red and in this example the house is equivalent to rendering our frame each tradesman might be a different system and then the capacity of three people in our house is equivalent to the free cpu cores we might have on our system

6. 00:04:15

   and the occupancy as in the average number of people in the house is effectively the same as the cpu or gpu utilization numbers you might see in task manager so in this example we can see the project takes 11 units of time and is on average 60 full for this house now this occupancy of 60 isn't the direct reason that it takes 11 years of time and that is in fact down to the critical path of work where the builder is blocking the joiner and the joiner is then blocking the electrician so the obvious answer here is that you need another builder or multiple builders to work on this in parallel to unblock the joiner and and so on you can keep on paralyzing this work to try and reduce the critical path and that is our focus when we're optimizing multi-threaded code however not all tasks are trivial to run in parallel and sometimes it's impossible to actually run them in parallel so

7. 00:05:02

   you'll end up with these unallocated capacity space and these bubbles and in this case if we imagine the electrician was in fact our gpu then it fact that it takes longer is just going to result in a cpu bubble which is unavoidable and this type of thing happens very frequently if you have a configuration for example a very fast gpu and a very slow cpu then unavoidably one of them's two systems is going to have a bubble so we can fill these bubbles with non-critical work like streaming meshes or textures and this can help fill the unused cpu capacity and this can be useful to try and get some extra work done but ultimately it's not going to improve the performance of the frame and the point i really wanted to drive home was that the cpu and gpu utilization which are often looked at as key indicators of performance aren't actually the direct thing we should be looking at they are useful at statistics but ultimately the frame time is the

8. 00:05:50

   only thing that matters and the critical path that resulted in that frame time so as we roll out gen 12 we'll hopefully put in some new statistics in there that the players can see to help them understand the performance and understand these bubbles and that should hopefully give a little bit clearer insight into the performance of the renderer so how we can actually achieve this parallelism in practice that's going to come down to two major changes one is the architectural changes which christopher will talk us through and the second is the vulcan graphics api that darwell is going to talk us through we come to the next section of our gen 12 vendor presentation my name is christopher bolton co-engine architect here at cloud imperium games and i would like to spend the next few minutes introducing the high-level architecture of our in-development gen 12 vendor for the section i will focus on the

9. 00:06:42

   rendering of object instances in the world for example chairs words characters or spaceships this part of the rendering problem is called scene object rendering and has the largest impact on runtime performance there's also a lot of work happening on the architecture to manage operations which work on all pixels on the screen so called post effects but i won't cover those the current slide shows our existing vendor code setup we have a main thread which does all game simulation as well as figuring out what objects we should draw for every frame and we have a window thread which takes all those objects and translates the description into gpu commands to render them on the screen the system is set up to dub buffer the data in other words the vendor thread is

10. 00:07:31

    working on data from the previous frame where the main thread produces data for the next firm such a setup allows easy performance improvements in some situation but it other has two issues on modern hardware first the rendering code won't scale over multiple cpu cores which can result in a bottleneck during execution in other words every visible object at a certain cost the more objects we render the higher the cost and as a single cpu process is discussed no matter how well we optimize at some object count we will run into performance issues second since the main thread and render sweat must be synchronized with vsync we can end up with very bad load balancing as shown in the slide if the main thread takes longer than the render thread the render switch has to be idle and weight

11. 00:08:20

    and vice versa if the window sweat takes longer then the main switch has to wait during such wait time the cpu is under utilized especially if waiting for the single core render sweater one goal for the gen 12 vendor is to remove this kind of bottleneck and stand architecture as a system without those weights and allowing every operation to utilize all cpu cores when we utilize all cpu cores we would still have an object limit as every visible object must be processed but we can possess a higher object count and at the same time reduce the latency on the main thread until all objects are processed as we will be making better use of modern multicore cpus let's take a look at some details for example please keep in mind that the size of the sections are chosen to

12. 00:09:08

    visualize the cost relation and size doesn't necessarily translate to the same relation in cpu cost when looking at the cost of the operations done by the vendor threat a pattern quickly emerges you pay a similar cost for every rendered object this is called a drawcard for every draw call some time that is spent inside our own window code and some part of it is spent in the gpu driver code in the next slide we will cover our cool process as well as the next steps to move the trucker cost out of the vendor sweat or to multiple cpu cores we already have a level of parallelization on the main thread used to find out what objects are visible there we use our batchworker job system this is a parallelization system to execute the same code on a different

13. 00:09:58

    object instance of all cpu cores to give an example checking phone objects can be split over 10 spreads now that every thread will be processed 40 objects by doing that the latency on the main thread until all font objects are done is divided by the number of shreds reducing set latency to 40 objects only roughly at least as in the xv system several lower level factors affect the execution making this statement not fully true but covering those would be out of scope for this presentation the visible check itself happens multiple times per object in a frame as an object can be visible in the main camera but it can either be visible in shadows or via remote location rendering the so-called vendetta texture or rtt

14. 00:10:45

    rendering the slides show those pairs simplified as we can have more than one shadow pass for example those visibility checks or culling operations are performed on all cpu cores with a worker system when an object is determined to be visible its rendering description is copied into multiple temporary buffers those temporary buffers are processed on the next vendor sweat frame to submit every object's draw call to the gpu in other words object cutting is already at the point where we want to have the draw coil processing right now we are in the first implementation phase we have to find our lower level code building blocks ensured our apis work and are now in the process of moving our own rendering code out of the windows

15. 00:11:33

    thread into the existing batchwork execution this is a very time consuming vector as we need to change every rendering feature in a very old and large code base but we have set it up in a way to allow us to greatly move over parts step by step after this operation is done we still copy state to a temporary buffer to be processed by the render thread but the state which we copy is prepared in a way that we can directly send to the gpu with minimal processing on the vendor thread doing this step will already give us performance benefits when we are vendor sweatband as less code will be run on the vendor thread additionally this is a necessary stepping stone for the next phase after we managed to move our own rendering code to multiple cpu cores we

16. 00:12:22

    would start to utilize the working api one major selling point of the newest generation of graphics apis like virkon is a possibility to generate gpu comments on multiple threats that is something which wasn't possible before and mostly the course for the existing vendara design the catches to allow efficient parallel generation of gpu commands the data must be prepared in a certain way and that is what we are doing right now as part of porting the scene object rendering to change birth and moving our renderer code to the patchworker system when this is done we can implement a parallel working backend and remove the render thread after all that work is done our vendor should be able to process a very high number of visible objects at lower

17. 00:13:10

    impact on the frame time at the same time it will make better use of the available cpu resources and have less idle time when major systems wait for each other thank you for your time daru will take over to cover the working side of the gen 12 vendor hi my name is daryl i'm a graphics programmer here at cloud imperium games and i work closely with vulcan and our graphics renderer to make the game look as good as it does so what exactly is vulcan well vulcan is a modern graphics api that allows us as developers to take greater control over what you as a player sees

18. 00:13:58

    and also affects performance greatly as you know we already have a few areas of bottlenecking on the cpu but the design of vulcan allows us to alleviate these bottlenecks by submitting work in parallel to the gpu so i would like to explain a bit more about the software stack that's involved with our graphics in the engine so from the image you can see that we have a renderer front end to render a back end and as well the graphics driver the vulcan api generally sits at the renderer back end and allows us a fine-grained control over what we develop and how we develop for it it also gives us flexibility for cross-platform including windows and

19. 00:14:47

    linux and anything we may want to look at in the future the graphics driver stage is not managed by us but we look at that and we gather information and process any crashes or any issues that might happen and then we can deal with those further down the line so you can see now how graphics apis have changed over time we're of course in the more modern section where vulcan is on feature parity essentially with other apis such as directx 12 ultimate you might also be wondering what a graphics api actually is and we see that as a tool that is used for development that can interface between your graphics card as a player and us as developers so that we can give

20. 00:15:35

    you the latest and greatest vulcan also has many features and extensions available to it that we will be exploring in the future such as variable rate shading bindless resources and gpu accelerated ray tracing to address any issues we also need to collect data from our players hardware so that we can use that to target specific features and extensions we're not aware of any large-scale multiplayer games that captures vulcan data live in the exact same way we do capturing this data allows us to plan ahead for any optimizations and then leverage that for the larger majority of players so we can bring you the latest and

21. 00:16:22

    greatest on screen at the moment is a diagram that shows the distribution available for vulcan api versions amongst currently active players this was captured in the last three months and as can be seen there is 98 of players that are able to use vulcan fully in 1.2 we did see a negligible amount of 1.0 and some that were unavailable we are actively looking into these cases especially those that cannot currently run vulcan as it is seen as unavailable i'd like to take this opportunity to also say please update your drivers as we do see a few cases where this can take you straight to the latest version i'd like to now explain a bit more about

22. 00:17:10

    the render graph and how this works hand in hand with the vulcan api in order to improve our usage of vulcan so a render graph can be seen as a collection of stages that depend on each other this then determines the ordering the scheduling and the flow of the actual frame this allows us also to then achieve any synchronization that we need to during that frame it also helps us from a design point of view as we can look at the render graph and see where there may be issues or potential optimizations the render graph also allows us to keep track of any state of resources

23. 00:17:59

    and we can also validate against those resources as well now i would like to explain the render graph as a whole and give you an overview of how it works between the gpu and also how it works in terms of our frame so a frame is made up of a collection of passes and sometimes we require a texture to switch states perhaps between read and write the graphics driver used to do this for us but now with modern apis such as vulkan we need to carry out this work ourselves what the render graph will then do is insert a pipeline barrier into the render graph and pass this work along to the gpu and switch the state we can also cache the render graph

24. 00:18:49

    and use any data we had from a previous frame or perhaps similar resources as an optimization i'd now like to talk a bit about the synchronization that happens within the render graph and how resources change state so the idea is that we want to switch the state of a resource as early as possible we need to then also validate that this resource is in the correct state so from the diagram we can see that the depth pre-pass writes to image a but then later on we require to read from image a in the g buffer so to this end we insert a pipeline barrier for this transition between the two passes these barriers are scheduled as gpu work following a strict ordering dependent on

25. 00:19:39

    other work that may have happened previously all the work for a single stage within the frame can be carried out in a single pipeline barrier i'd now like to talk about some of the more nuanced areas of our vulcan implementation and how this affects us as developers and you as players we use the directx compiler for our shaders and this can compile our hlsl code into spear v dxe is a more modern compiler and has features that span both the d3d set of apis and vulcan hlsl is a shader programming language that we as developers can utilize and

26. 00:20:29

    read in order to make work happen on your gpu this hlsl is compiled down then into spear v itself is not as readable as it is seen as an intermediate language between hlsl and shader micro code spear v gives us less drive overhead at compile time we can use this then to create our shader modules in vulkan and optimize any dead code away dxe also gives us shader model 6. shaded models have progressed over time in hlsl with shader model 6 now giving language support for gpu parallelism as well as

27. 00:21:17

    variable rate shading amongst many other features we can see here now a diagram of how our shader compilation happens it starts at the shader author level this is where one of our many developers will write a shader using hlsl code with some markup integrated into that hlsl this hlsl is then passed to the preprocessor the preprocessor then removes any of the additional code that we don't need and is dealt with accordingly whilst then outputting just hlsl code the hlsl code is then passed to dxc and dxe will change this into spear v

28. 00:22:06

    it's at this stage that we can carry out any additional compiler optimizations and improve compile time performance even more spear-v is then passed to the driver and the driver will then change this into microcode which will run specifically on your gpu as i've previously mentioned there have been several vulcan core versions over time each one has added new features and extensions for us to reach into and develop with two of these i'd like to explain a bit more in detail now that we would like to look into and develop for in the future these are both bindless resourcing and also fragment shading rate

29. 00:22:56

    fragment shading rate can be seen as the same as variable rate shading as you may have already seen what this does is works on groups of pixels instead of a singular pixel at a time within the shader this allows less overhead in the frame whilst at the same time allowing variable amounts of these groups including variable sizes in order to have less fidelity where may not be as important to look at bindless resourcing is where we can take large groups of textures for example and reach in and grab one of these inside the group this gets rid of the overhead of

30. 00:23:43

    specifically binding two slots within the shader this also extends to other resources including buffers video ram involved in your graphics card vram used to be managed by your graphics but with vulcan and other modern apis this is now managed by the developers there are existing solutions towards this however we have decided to use our own this is because we know exactly what resources are used up front and how much memory these resources use using this we can potentially beat what the graphics driver used to be capable of

31. 00:24:30

    as we know all of our resource life cycles we can take advantage of that from the diagram we can see memory being paged in and out as it used to be when it was managed by the graphics driver on the other side we can see memory being allocated and then sub-allocated and we can then offset into the data from this sub-allocation the memory paging of years past could potentially result in a performance loss because this was seen as a more one size fits all we of course want to avoid any of that performance loss and the memory cost involved by offsetting into these larger buffers this can also help mitigate cache misses this then leaves us with all of the

32. 00:25:17

    allocation freeing usage and reusage of i'd now like to pass you back to alistair brown the director of graphics engineering following on from what dallas explained there with the explicit memory management and vulcan we're intending to expose some of this memory management to the player through the advanced graphics options we're going to let you tweak the memory assigned to each system so you can balance the preferred visuals and performance for your experience for example you may want to balance the output resolution of your game to a higher resolution but then sacrifice some shadow quality or maybe you want to use a lower internal resolution and rely on up sampling to achieve a higher texture quality these options will all be available to you and obviously only capped by the hardware that you possess so to summarize with our gen12 renderer we're hoping to achieve something that

33. 00:26:04

    is more efficient modular flexible and minimal abstraction to the hardware and uses modern graphics apis like vulcan so now you know a bit more about what gen 12 is i'll try and let you know where we're up to so we've done a huge amount of work already the architecture is all in place and we're using this hybrid rendering approach where we're combining elements of the old and new render at the same time to allow us to move piecemeal to the new system all of the post effects fog and lighting have been converted over and they're all enabled by default in 315 and the fundamentals of scene and geometry rendering are all in place but they're still being worked on so our main focus is finishing that off at the moment and once that's done our folks will shift to the remaining major systems which are gas clouds the render detector system and a few special cases

34. 00:26:53

    for transparency after this that's where we'll start seeing the public milestones and the first of that will be a 100 usage of gen 12 and none of the hybrid approach this will still be at directx 11 our current graphics api at this point and then our second milestone will be the vulcan api release that will be optional at first and then mandatory after we've removed all the bugs and then our final milestone will be when we have performed the optimizations for multi-threaded so that will only happen once the vulcan is in place and we can finally look at the performance on the final graphics api and optimize all the remaining code so after gen 12 i just briefly wanted to

35. 00:27:44

    touch on what comes next for the graphics team so a lot of gen 12 has been focused on cpu performance so after gen 12 we really want to start looking at the gpu performance the first few things we'd look at here were things that wouldn't change the visuals just improve the frame rate things like dlss fsr async compute and variable rate shading after we've kind of improved the steep performance we want to start looking at some of the more exciting visual features so then there is also mesh shaders and primitive shaders which is technology we can use to generate procedural geometry and this type of thing could be really exciting for things like the planets or asteroid fields where procedural geometry is critical and then there's the big one ray tracing we're very excited to get onto ray tracing especially to use it for lighting such as global illumination but also reflections and shadow quality there's a lot of exciting areas for us to look into and we can't wait to get

36. 00:28:31

    into it so that's it i just wanted to say a big thanks to everyone's been involved in this technology it's involved the graphics team engine team vfx and planetech all working together on a huge piece of code and we can't wait to get into your hands as soon as possible