AMD has an answer to DLSS, DirectML Super Resolution

tommybhoy · 2 Nov 2020 at 18:45

Nvidia's DLSS (Deep Learning Super Sampling) technology the gaming market's most disruptive technology, enabling higher performance levels by using machine learning to upscale images to higher resolutions without visual downgrades.

To many gamers, DLSS offers Nvidia RTX users a free graphics card performance upgrade in supported titles. This delivers Nvidia a huge performance advantage over their rivals.

AMD needs an answer to Nvidia's DLSS, and that answer is DirectML-powered Super Resolution. Like Nvidia, AMD plans to utilise Machine Learning improve the visual quality of games, but AMD's solution will have the backing of Microsoft.

In a recent disclosure regarding their Xbox Series X and Xbox Series S system, Microsoft confirmed that both of their next-generation consoles would support Machine learning for games with DirectML. Through their collaboration with AMD when creating their new consoles, AMD has a chance to replace DLSS with an alternative which will apply to gamers on both PC and Xbox, giving developers little reason to support DLSS over DirectML Super Resolution.

"Through close collaboration and partnership between Xbox and AMD, not only have we delivered on this promise, we have gone even further introducing additional next-generation innovation such as hardware accelerated Machine Learning capabilities for better NPC intelligence, more lifelike animation, and improved visual quality via techniques such as ML powered super resolution."

Click to expand...

We have discussed the potential of DirectML previously at OC3D, a component of Microsoft's DirectX feature set which has been overlooked by many enthusiasts. DirectML isn't a Radeon-only technology, and its applications extend far beyond Super Resolution functions. Over the coming years, future PC and Xbox games will bring Machine Learning into games in several new and innovative ways, impacting all gamers with supported hardware.

DirectML was revealed at the same time as DXR (DirectX Raytracing), and its image-enhancing capabilities have already been showcased at events like SIGGRAPH 2018. DirectML will create an open alternative to Nvidia's DLSS technology, allowing gamers outside of Nvidia's RTX ecosystem to benefit from Machine Learning.

https://www.overclock3d.net/news/software/amd_has_an_answer_to_dlss_directml_super_resolution/1

Wonder what the end result will be.

diamount · 2 Nov 2020 at 18:51

This has been around since early this year, why wasn't it mentioned at all?

JamesPliskin · 2 Nov 2020 at 19:06

DLSS has already been implemented in a number of games and proven to be pretty amazing. We'll just have to wait and see if DirectML can match that, and how many developers actually add support for it.
That being said, I doubt Nvidia is going to suddenly discontinue DLSS, so developers (at least the big ones) will likely have to support both technologies.

opethdisciple · 2 Nov 2020 at 19:09

PlayStation support?

Nablaoperator · 2 Nov 2020 at 19:11

I have seen some leaked slide suggesting that Nvidia gave away their dlss model to MS .. which I feel will become a directx super res extension rather than something owned by AMD

Though AMD/Nvidia can still train with different samples, but eventually this looks like ihvs pushing training responsibility to developers like they did with dx12.

Nvidia still seems to have a perf advantage stemming from tensor cores.

Someone should probably try large matrix operations on both kinds of hardware and confirm this.

KentMan · 2 Nov 2020 at 19:13

JamesPliskin said:
DLSS has already been implemented in a number of games and proven to be pretty amazing. We'll just have to wait and see if DirectML can match that, and how many developers actually add support for it.
That being said, I doubt Nvidia is going to suddenly discontinue DLSS, so developers (at least the big ones) will likely have to support both technologies.

Not necessarily true, it should be easy to port from console to AMD hardware and Nvidia can choose to utilise DX12Us DirectML in place of DLSS, why program a game twice for a feature that works on all hardware already?

Quite easy for a game dev to say "Were using DirectML on console and it will be available on PC" makes their porting much easier and quicker, to get the game to market.

Developers supporting DLSS will most likely have been working alongside Nvidia to do so in the future. And will most likely be games that are on PC first.

Anything that's on Console first or Console / PC at release with ML upscaling is now likely to be using DirectML, especially on the Xbox

JamesPliskin · 2 Nov 2020 at 19:20

KentMan said:
Not necessarily true, it should be easy to port from console to AMD hardware and Nvidia can choose to utilise DX12Us DirectML in place of DLSS, why program a game twice for a feature that works on all hardware already?

Quite easy for a game dev to say "Were using DirectML on console and it will be available on PC" makes their porting much easier and quicker, to get the game to market.

Developers supporting DLSS will most likely have been working alongside Nvidia to do so in the future. And will most likely be games that are on PC first.

Anything that's on Console first or Console / PC at release with ML upscaling is now likely to be using DirectML, especially on the Xbox

That would certainly be preferable. @opethdisciple raises a good point, though, what about PlayStation? Xbox already supports DirectX, so DirectML should be no problem. But there is no DirectX support on Sony's hardware afaik.

PrincessFrosty · 2 Nov 2020 at 19:23

I mean we all knew super resolution was a thing, it was in the slides on the AMD launch video it just wasn't mentioned or displayed. We know they need a solution for this for the same reason Nvidia needed it, which if you're going to do any ray tracing at all, it'll be at 720-1080p tops, and all the 1440p and 4k users won't buy into that, you need a way of getting that up to a decent resolution that's not expensive and looks good.

So it'll all come down to 2 things, quality of the upscaling and performance. Quality I think is just a matter of learning what works and time training a super computer, like with DLSS 1.0 it really sucked, DLSS 2.0 is very good, which gives you an idea of what is doable with that. But...where Nvidia might have an advantage is performance, they have dedicated Tensor cores which accelerate the kind of math you do when implementing a machine learned algorithm and so they might have an upper hand on performance.

Thing is that vendor agnostic Anti-Aliasing is kinda dead anyway, long gone are the days where developers could just put MSAA in the options and be done. All of the AA types need to be "implemented" these days, and I think DLSS is now no longer trained on specific games it's general, and so in theory much easier to support, so we'll have to see in future.

Nablaoperator · 2 Nov 2020 at 19:27

JamesPliskin said:
That would certainly be preferable. @opethdisciple raises a good point, though, what about PlayStation? Xbox already supports DirectX, so DirectML should be no problem. But there is no DirectX support on Sony's hardware afaik.

There should be open source alternatives or PS can license the tech from Nvidia (as per some leaked slide directx super res is actually a directml implementation of dlss).. no point reinventing wheels

MrPils · 2 Nov 2020 at 19:28

AMD are clearly behind with their value add software support. Their ray tracing and upscaling is likely to be very slightly ahead of Nvidias generation 1 DLSS and RTX as they have the benefit of hindsight learning, but nowhere near Nvidia's Generation 2 product. Catching up is going to be difficult and is likely to take a number of years.

The one advantage to a consumer they could have though is that as they're pushing the directx model generational improvements could backport onto previous hardware - the 6000 series in my opinion is likely to get performance improvements in upscaling and ray tracing as the software develops. Nvidia don't do that, you need to buy a new card each time. Whether the hardware will be able to keep pace with future implementations in future games is another question, but I would argue that upscaling becomes a more relevant technology if improvements over time in the implementation can be backported to cards released before those improvements were made.

As a 3070 owner I'd love to be wrong, but I really don't see any newer DLSS models Nvidia release alongside future cards being "compatible" with RTX 3000 series.

Grim5 · 2 Nov 2020 at 19:53

opethdisciple said:
PlayStation support?

Microsoft claims that only the Xbox has hardware accelerator units (AMD's version of Tensor Cores and RT cores) for ML up-sampling- Phil Spencer directly said the PlayStation 5 does not have ML accelerators - the PS5 can still do software based ML image up-sampling but it doesn't have the cores to accelerate it. The way I interpret this is that one day when ML upsampling is here, the Series X might be able to use it to pull off 8k 30fps games while the PS4 wont be able to

ML based DX12 features are a long way off, I wouldn't worry about it. By the time ML up-sampling becomes important, the PS5 Pro will be on the way.

Right now I'm going to enjoy 4k 120hz gaming for the next 3 or 4 years and after that when the PS5 Pro is out and the RTX6090, I'll upgrade to a 8k TV

Zeeflyboy · 2 Nov 2020 at 21:42

I remember getting downvoted and snarky comments on the r/AMD subreddit for suggesting AMD might work with Microsoft on DirectML based up sampling to have an answer for DLSS... also quite a few on this forum that would shoot you down for suggesting AMD needed to be working on something in that arena.

Some of those hardcore fanboys just really can't accept that machine learning up sampling technologies are going to be a big deal in the future even if they aren't necessarily right this second, and AMD needs to be on that train...

Anyhoo, I like the comments I've seen suggesting it will be a "broader" implementation than DLSS. Even if it only somewhat approaches the quality bar of DLSS2.1 but is actually available in a far wider range of games it would be a big win imo.

humbug · 2 Nov 2020 at 21:50

Good show AMD.

gpuerrilla · 2 Nov 2020 at 21:54

Grim5 said:
Right now I'm going to enjoy 4k 120hz gaming for the next 3 or 4 years and after that when the PS5 Pro is out and the RTX6090, I'll upgrade to a 8k TV

You buying a Radeon card then? What happened to the 3090?

Scougar · 2 Nov 2020 at 21:55

"..the gaming market's most disruptive technology.."

Somebody swallowed NVidia's marketing lines. (and I'm not talking about you tommy, the article).

It's barely in any games, same as RT. i.e. it isn't 'disruptive'.

Anywho.... I'll go read the article now lol.

Grim5 · 2 Nov 2020 at 21:58

Zeeflyboy said:
I remember getting downvoted and snarky comments on the r/AMD subreddit for suggesting AMD might work with Microsoft on DirectML based up sampling to have an answer for DLSS... also quite a few on this forum that would shoot you down for suggesting AMD needed to be working on something in that arena.

Some of those hardcore fanboys just really can't accept that machine learning up sampling technologies are going to be a big deal in the future even if they aren't necessarily right this second, and AMD needs to be on that train...

Anyhoo, I like the comments I've seen suggesting it will be a "broader" implementation than DLSS. Even if it only somewhat approaches the quality bar of DLSS2.1 but is actually available in a far wider range of games it would be a big win imo.

For some people, acknowledging that ML is a good thing would be admitting that DLSS is also a good thing, despite the months of misinformation campaigning by those people

and thats a hard pill to swallow

KiNgPiN83 · 2 Nov 2020 at 22:02

So is there a quick TLDR for what DLSS and ML actually does. I'm assuming quality is lowered somewhat? Or is it just FPS increasing magic?

Zeeflyboy · 2 Nov 2020 at 23:20

Scougar said:
"..the gaming market's most disruptive technology.."

Somebody swallowed NVidia's marketing lines. (and I'm not talking about you tommy, the article).

It's barely in any games, same as RT. i.e. it isn't 'disruptive'.

Talking about right now, no... but in the relatively near term it absolutely will be disruptive, which is why many are working on it. Once again the magic of Nvidia marketing is actually in making you believe they are the only ones working on doing a particular thing! For gaming consider power constrained devices like laptops/phones/mobileVR headsets... that is where this technology can really show huge potential. Pretty sure the rumours are true about the next gen Nintendo Switch for example, in that it will use a DLSS like technology to achieve high fidelity 4k graphics with a relatively slow (and power efficient for battery life) chip.

We are only at the beginning, the technology will improve both in quality and in how easy it is to implement. The gold standard would be a technique that will simply work at the driver level in any game that uses TAA for example, thus making it instantly applicable to a huge swathe of modern games. You can bet that is the goal these companies will be aiming for....

Grim5 said:
For some people, acknowledging that ML is a good thing would be admitting that DLSS is also a good thing, despite the months of misinformation campaigning by those people and thats a hard pill to swallow

Sadly it is certainly true that many can't see beyond the brand they dislike marketing the "gimmick" and look further into the future for the technology's potential than just in front of their noses. I completely agree with many that DLSS for example is functionally relatively useless at this point in time (due to availability in games rather than quality since the updates to 2.0 and 2.1), but that is completely missing the point.

KiNgPiN83 said:
So is there a quick TLDR for what DLSS and ML actually does. I'm assuming quality is lowered somewhat? Or is it just FPS increasing magic?

Super TLDR is that you use machine learning to provide a better upscaled image than you could otherwise achieve from low resolution source material. In very simple terms the algorithm learns what stuff should look like, and can use that knowledge to reconstruct high resolution images from lower resolution ones... imagine you were good at art (maybe you are lol) and you were shown a low res picture of a person's face... using your learned knowledge of what people look like, you could then draw a much higher fidelity picture from the original lower fidelity image by filling in the missing information yourself and it would probably look pretty accurate.

The general idea is that you can for example play at something that looks an awful lot like native 4k while only actually rendering at 1440p for example, which is obviously much easier to achieve high frame rates with - especially when you start to take the relatively poor ray tracing performance of current GPUs into account for example.

Less TLDR is that DLSS is Nvidia's own proprietary version of AI based upscaling. Originally for DLSS in particular this required training the algorithm using very high (16k I believe) source material from the game in question, and the AI would then learn to use more sparsely rendered data from lower resolution render targets and boost the resolution by filling in the blanks. More recently DLSS2.0 significantly altered how this works and is instead trained on a generic neural net which ditches the per game training requirement and works with TAA and motion vectors instead. It still requires per game implementation at this time, but is no where near as resource intensive.

DirectML super resolution I stand to be corrected but I don't believe we know much about yet in terms of the nuts and bolts of how it works, but one could take a stab at it being some sort of generic neural net TAA based solution too. DirectML itself though is much broader than just super resolution/DLSS and aims to incorporate a larger range of machine learning based systems for gaming which is pretty cool.

There are also others, FRL is also developing machine learning upsampling techniques, there are open sourced variants using GAN style training.

mrk · 3 Nov 2020 at 00:25

The image quality with DLSS 2.0 is also capable of producing a sharper than original image with more detail it should be noted.

TheRealDeal · 3 Nov 2020 at 00:35

When DLSS was announced i thought it was a much more relative tech than RT at this point in time. Then when we saw the results it was a proper let down due to the Blurr effect. DLSS 2.0 though is where it should have been in the first place.

Problem is DLSS is another locked in Nvidia tech so will fade out just like the others apart from the odd Nvidia title. Glad to see AMD and Microsoft are bringing in another open version which we will all benefit from. Here is hoping it's more DLSS 2.0 and not DLSS 1.0