Intel's Video Game Looks Like Reality! 🌴

Share
Embed
  • Published on Jun 4, 2021
  • ❤️ Check out Perceptilabs and sign up for a free demo here: www.perceptilabs.com/papers
    📝 The paper "Enhancing Photorealism Enhancement" is available here:
    isl-org.github.io/Photorealis...
    🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
    Aleksandr Mashrabov, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bryan Learn, Christian Ahlin, Eric Haddad, Eric Martel, Gordon Child, Ivo Galic, Jace O'Brien, Javier Bustamante, John Le, Jonas, Kenneth Davis, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Mark Oates, Michael Albrecht, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Ramsey Elbasheer, Steef, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi.
    If you wish to appear here or pick up other perks, click here: www.patreon.com/TwoMinutePapers
    Meet and discuss your ideas with other Fellow Scholars on the Two Minute Papers Discord: discordapp.com/invite/hbcTJu2
    Károly Zsolnai-Fehér's links:
    Instagram: twominutepa...
    Twitter: twominutepapers
    Web: cg.tuwien.ac.at/~zsolnai/
  • Science & TechnologyScience & Technology

Comments • 2 849

  • Marvin Polo
    Marvin Polo 11 months ago +3386

    i'm from germany and i can tell you that the blue/grey, dark, undersaturated, depressive look is spot on.

    • Bryan
      Bryan Month ago

      @BreadCrust you poor man

    • Bryan
      Bryan Month ago

      ew everything is blue there?

    • 雪|スノー
      雪|スノー Month ago

      @PJP K sauerland?
      du auch?
      armer

    • InSomnia DrEvil
      InSomnia DrEvil Month ago

      Yeah, that depressing tone really hits home :)

    • Keith Price
      Keith Price Month ago

      We use that colour palette a lot in the UK too!

  • Aaron Boomhower
    Aaron Boomhower 11 months ago +1382

    This technology would be absolutely perfect for a horror game.

  • Z0BI
    Z0BI 11 months ago +947

    I think that the desaturated look of the AI could also be in part because the video dataset was be filmed with dashcams and similar, which film stuff less saturated than we percieve it.

    • XaverDerSchnitzelFan
      XaverDerSchnitzelFan 2 months ago +1

      if we did some color correction on the training data that would help a lot

    • HellerArt
      HellerArt 3 months ago

      Of course but easy fix, just one post process stepp needed...

    • Z0BI
      Z0BI 5 months ago +1

      ​@Toothless Beaver I live in germany... Even though it's sometimes drab here, it's definitely not that bad.
      At least from around late spring until early fall.

    • Toothless Beaver
      Toothless Beaver 5 months ago

      You obviously dont know how the weather is in germany. its actually spot on!

    • Bill Kong
      Bill Kong 6 months ago

      yeah it looks like dashcam footage not what eyes see

  • THOMAS JAKOB
    THOMAS JAKOB 11 months ago +152

    This is nuts...truly next gen. Every time you think technology has reached its peak there's a new crazy innovation.
    Also it felt more German than Germany itself lol. Also made me realise real life places have their own real life colour schemes.

    • Peter Belanger
      Peter Belanger 11 months ago +4

      I agree. it's a whole new take on rendering. With a decade or so of continued development, there could be significant bonuses to using AI in the whole process. may turn out to be a good shortcut to get large scale physics without the massive calculation overhead. Right now, things like fluid dynamics are rather limited. But applying the AI process to not just the still 2D frame of the video, but effects in time as well.

  • AZUR3RUN
    AZUR3RUN 7 months ago +59

    I spent the whole video repeating to myself: “what a time to be alive”, this is the most inspiring thing I’ve seen this month, thank you so much for sharing it this well !!!

  • Baby Dance
    Baby Dance 11 months ago +76

    This technology will find its way to film industry as well. Imagine that you could make a whole realistic scene with only 3D graphics ... that's absolutely incredible.

    • bighand69
      bighand69 Month ago

      I am afraid that digital does not looks as good as film and until they can develop ai systems that can make imagines look film it will not be considered a contest.

    • random browser
      random browser 8 months ago +11

      @Icaras12x i think what the other person was trying to say is that cgi can look absolutely realistic, with no flaws and with half the budget

    • Icaras12x
      Icaras12x 8 months ago +4

      That's how they do it... since the invention of cgi

    • random browser
      random browser 8 months ago +5

      Imagine how much money that would save

  • Howard Kong
    Howard Kong 11 months ago +406

    Year 2030: This new gaming neural processing card is expensive.

    • The Barret
      The Barret 2 months ago

      @Sentinel Aerospace Official there is a movie made about that exact story called 'The surrogate" worth a watch!

    • Mode
      Mode 5 months ago

      Imagine if, in the future actual mini-brains made out of living neurons would be used for computing rather than chips.

    • KTK :O
      KTK :O 6 months ago

      @dob 0 infrastructer still needs alot of work, before it can actually be practical.

    • Z3RO notfund
      Z3RO notfund 6 months ago

      neat

    • Backfischritter
      Backfischritter 7 months ago

      They are already realtity dude. RTX card for example

  • Forbidden Flame Studios
    Forbidden Flame Studios 11 months ago +52

    this for realtime gameplay + VR will literally turn games into reality in the future

  • Andro Matthew
    Andro Matthew 10 months ago +14

    Man this guy nailed it. When I saw this in tech news outlets, MY JAW DROPPED. Some people were complaining that it was bland and didn't feel real at all. It was clear a lot of people didn't understand what they saw. All the footage really needed was some color grading and we're living the in science fiction. Loved this video explanation.

  • Anon archist
    Anon archist 11 months ago +1681

    imagine how much modeling time we could save for the artist?
    *Take Two heard:* imagine how many artist we could fire?

    • alexpanter
      alexpanter 12 days ago

      Not many. With AI powers we could generate _more_ art. Something that all gamers want. Which also means that producing new games would take far shorter time.

    • foodank_atr
      foodank_atr Month ago

      Hah and now look at the Trilogy remaster and it's subpar AI upscaling

    • HellerArt
      HellerArt 3 months ago

      @A G I wonder wheter there are english word to destinguise artists like vang gogh, Neo Rauch or Roberto Matta from artists doing starchips, monsters, game and movie SciFi/Fantasy work.
      I am not saying one of them is art and the other not. But These are quite different approaches, fields and targets. Artists looking for new styles and ways of showing the world like Neo Rauch and Game Artists being more constrained by an industry setting standards...
      Maybe 3D Artist, Concept Artist and FineArt-Artist could be used?
      I wonder what would happen when an IA is trained on faking Neo Rauch Style are inventing new art styles. Or even combining with a painting robot arm.

    • HellerArt
      HellerArt 3 months ago

      Well modelling will be erased completely, too. Perfect 3D Model creation from photos is already there, combine it with random variation, or train an AI on doing star wars style ships and it will deliver 200 Million ships a day... in 10 years maybe it will deliver 2000 AAA Games a day ... why not? Would be a absurd situation where there are more games then we could ever play. It would be more about FINDING the best games in an endless number of 1000000000000000000 AI created Games...
      I think AI can soon pull of some War Game for example. By soon I think of 10-20 years, but maybe it is only 5.

    • Disliker
      Disliker 8 months ago +1

      @Cyrus Balabanov-Gebreab That's only true when its a single seller, e.i. a monopoly.
      If there are two companies, one uses the extra money for a better or cheaper product, the other feed billionaires. Who would sell more and eventually outcompete the other?
      Most big companies are public (anyone can buy stocks), so unless the shareholders (can be you) wants stupidly overpaid executives, there wont be any.

  • ChillCoaster
    ChillCoaster 10 months ago +15

    I can't wait to see this in any game that I am playing! This seems like RTX on steroids. Your idea about freeing up game design by requiring minimal base graphics and letting the AI system do the graphics is amazing!

  • Michael Atorn
    Michael Atorn 11 months ago +56

    I would use this as an indi game developer to keep up with aaa studios and focus more on the gameplay.

  • MewBlood
    MewBlood 11 months ago +7

    "There's no way this will work" is really the common refrain for the Deep Learning/Differential Programming research field. Thanks for sharing!

  • Jordy Toke
    Jordy Toke 8 months ago +3

    This is incredible. You mention it works via post-processing and I'm curious when it renders the lush grass instead of the textured plane is this projected over the original game footage as essentially AI altered video footage? Would lack of persistence be a limitation, like if you were to go past the same location twice, might it still look similar but generate slightly different results?

  • Vincent
    Vincent 11 months ago +2205

    My guess is that it won't be long until the first AAA games will have a "neural rendering" alpha feature that you can switch on and off to try to play in a more realistic scenario in real time :)

    • pkFire
      pkFire 5 months ago

      the "holy sh!t" button under the graphics setting

    • Caralho
      Caralho 8 months ago

      @Jeroen You could try creating an exception in the code for those specific human patterns then just give the humans ultra realistic models to prevent them from standing out

    • koolerpure
      koolerpure 10 months ago

      one day they'll make a VR game that uses this kind of tech, just imagine a VR alien invasion with realistic textures

    • Peter Belanger
      Peter Belanger 11 months ago

      It'll be great if Unity or Unreal could have this as an add on.

    • Cedric lacrambe
      Cedric lacrambe 11 months ago

      @Jeroen maybe with a medieval scenes dataset if it ends up not requiring an huge one. It would be easier for old building ones as some part of the world have this look

  • hot gas stove!
    hot gas stove! 11 months ago +17

    The most important thing is that in like 10 years or something like that games would probably actually look like this because at that point that hardware will be advanced enough to actually do it at 60 frames.

    • Arrogance Official
      Arrogance Official 11 months ago +10

      @hot gas stove! bro the thing runs already on modern computers in real time, give the code say 2 years time to get more efficient and its gonna be a breeze to run, the question is if its gonna be useful to developers or not

    • hot gas stove!
      hot gas stove! 11 months ago

      @Arrogance Official that depends on if it’s going to be affordable to most people or cost over a thousand dollars! Just because it’s technically possible if you have a $3 000 computer doesn’t mean that it’s going to be the norm for games to look like that. And unless something impressive happens in the next 3 to 5 years it’s probably going to be a decade before the general public has access to hardware good enough and cheap enough to actually use this type of software enhancement at a reasonable frame rate.

    • Arrogance Official
      Arrogance Official 11 months ago +2

      way less than 10 years

  • モチポンズ
    モチポンズ 11 months ago +22

    The most mind-blowing thing about this is that it's rendered in real-time! The whole video I was thinking "Oh well but you can't use it in a game when it takes so long to create the image" but apparently no, it's real time.
    I hope the next big thing after raytracing will be AI enhanced visuals!

    • モチポンズ
      モチポンズ 8 months ago +1

      @vovalos These days, yes. Back in the 90s, that wasn't such an issue. The game won't run on your machine? Get an upgrade then. :P

    • vovalos
      vovalos 8 months ago

      one of the key limitations the video game developers must consider is hardware capabilities of it's client base. If it runs real time on a dual high end video card it's still not gonna change anything. Unfortunately hardware limitation hasn't been discussed at all

  • Auracle
    Auracle 10 months ago +6

    I love watching videos like this, then looking at games that were called "photo-realistic" in the 90s for just containing polygons.

    • DavidT
      DavidT 9 months ago

      And we were happy for those back then. When I was kid, we played with stuff before video games even existed. :)
      In the future, people will find it funny that we find photo realism exciting or even VR headsets. I expect direct sensory stimulation of the neural system, Total Recall effect and then we'll all turn out like the blobs in Wall-E.

  • Axeinus
    Axeinus 11 months ago +2

    Personally, I would do this as you said, ~25% representation of the setting and letting the AI fix it in post. This would open up a ton of possibilities for smaller teams or saving time doing things alone. Things wouldn't need to be as detailed ahead of time. Perhaps focus more on the character models to do close up where the AI wouldn't work as well. As an experiment, I would want to see what it does with forest/open world environments, as well as some mesh deforming for testing physics objects (car crashes, building collapse, etc)

  • Nick Piovesan
    Nick Piovesan 11 months ago +2075

    In other news: Rockstar delays any news of GTA VI once again, offering new "realistic rendering" pack upgrade.

    • bWWd
      bWWd 2 months ago

      they will notdo that, gta IV already had realistic enb mods before gta v was released , and gta v looked worse then these mods

    • Noran Zeur
      Noran Zeur 3 months ago

      LETS GO!!! The next GTA was just announced a week ago!!!!!

    • Just nothing
      Just nothing 5 months ago

      @iLearnCode that is very optimistic more like playstation 60 gta vi coming "soon"

    • jon d
      jon d 7 months ago

      if it has this ill happily wait lol

    • Timothy Ivaikin
      Timothy Ivaikin 7 months ago

      And Skyrim?

  • Axel Mertes
    Axel Mertes 11 months ago +2

    Wow. Amazing. It is about time that one of the big car games adds this feature and allows for real life car training at home to prepare for getting a drivers license. There is such a huge market for this. And visually this is exceptionally outstanding improvement over what has been there before. THIS is the next stepp after DLSS, obviously. Imagine rendering barely low quality graphics and an AI is creating beautiful stuff from it.

  • fakhouri kakeesh
    fakhouri kakeesh 11 months ago +2

    this seems very promising looking forward to the implementation in vr games

  • TheLyricsGuy
    TheLyricsGuy 10 months ago +1

    This looks amazing. It looks like real dash cam footage. I’m so excited for future gaming technology.

  • Enigmatic
    Enigmatic 10 months ago +16

    That's amazing, i hope it doesn't end up just being a good idea and actually changes the way we create video games in the future.

    • Alphoric Productions
      Alphoric Productions 7 months ago

      @Neyna oof

    • Neyna
      Neyna 7 months ago +1

      Just think 2s, it can be very very bad, if everyone uses this, all games will look the same.

  • eamonia
    eamonia 3 months ago

    Thank you for editing this so perfectly, as to truly encompass your ideas and content. Very immersive my dear Dr.

  • Thing482
    Thing482 11 months ago +2

    Imagine all the modelers we won't need! Oh wait... We can already implement real world objects into video games... Darn.

  • KanGar
    KanGar 10 months ago +1

    The physics of the game kind of break the immersion but still this incredible! Imagine just building plains and mountains with one single color and the ai makes everything more real! We have truly surpassed the need for moore’s law, DLSS is only the beginning of a new era.

  • David Schuhman
    David Schuhman 11 months ago +1

    I would love to take a castle and scenery asset set and use this for a fantasy setting. I think a few papers down the line we’ll see it as a game engine feature. Interpolating random textures in an open world (randomly generated) environment. Providing a truly “unique” experience. Possibly even having texture data shared server side so multiplayer is possible.

  • Tallywort
    Tallywort 11 months ago +3319

    Interesting how the algorithm seems to really want to add the blue EU flag to the licence plates.

    • Just nothing
      Just nothing 5 months ago

      Comrade AI forever with the European Union

    • GooZ
      GooZ 8 months ago

      Lol

    • Robert Hastings
      Robert Hastings 10 months ago

      @DrumtotheBass Woop Earthexit? Terraxit?

    • Rudolf Wickond
      Rudolf Wickond 10 months ago +2

      makes all rusty American cars suddenly have German TUV

  • Craxin01
    Craxin01 10 months ago +5

    Games, animations, hell even city planning could all be accomplished with this sort of image technology. Could Ready Player One style VR be on the horizon next?

    • Taki Tachibana
      Taki Tachibana 8 months ago

      A VRMMO like that is exciting
      We may expect it as early as the end of this decade
      (Zenith MMO is currently the best VRMMO out there (even if it's not nowhere near RP1), we are inching toward that future)

  • Master Champloo
    Master Champloo 11 months ago +6

    Looks real enough. I was sold in first few seconds when there was a damn car not going on green light.

  • Why Not
    Why Not 6 months ago +1

    5:40 So what happens with the trees when you get closer to the hill ? There are no tree models there, they will gradually fade away as you approach. Yes, I know they were not meant to be there, but it doesn't really matter. With this method things will look very different from up-close. I think the usage of it will be limited.

  • Luke Rawlinson
    Luke Rawlinson 11 months ago +3

    I would definitely use this as a masterpiece in acuteness in interior design.

  • Joannot
    Joannot 11 months ago +1089

    I was waiting for this one, ever since I've seen that demo a couple weeks back, I knew our fellow scholar would love it!

    • Dusty2455433
      Dusty2455433 11 months ago

      I was surprised when I clicked on the video and realized it wasn't a Two Minute Papers video

    • Muhammed Ali Inamdar
      Muhammed Ali Inamdar 11 months ago

      I had the exam same thought when I saw that video!

    • Joannot
      Joannot 11 months ago +6

      @Stirrcrazy perhaps I watched a couple of those and even one of their old robotics video but I haven't seen all of it. Two minutes paper is famous among researchers no doubt

    • Stirrcrazy
      Stirrcrazy 11 months ago +17

      Doesn’t the guy in one of the demo videos even make a reference to “hold on to your papers”?

  • timely skydive
    timely skydive 11 months ago

    Love these videos well edited and feel like an actual scientist made the video since he knows what he is talking about

  • Martin Gradwell
    Martin Gradwell 11 months ago +21

    02:42
    "Now, as you see, most of the generated photorealistic images are dimmer, and less saturated than
    the video game graphics. Why is that? This is because computer game engines often create a more stylized world where the saturation, haze, and bloom effects are often more pronounced."
    Or maybe it's because the AI was trained on German imagery where skies tend to be more overcast than in California. So it takes the unmistakably Californian views from GTA with their dawn/dusk pinkness and brightness, and transforms them into a cloudy German midday. Which is arguably more real, but only because Germany is arguably more real than California. Nothing to do with imagined failings in the game engine.
    This is a lot like the one where Abraham Lincoln's iconic wrinkled skin was smoothed out, by AI scientists who couldn't imagine that anyone might really have skin that wrinkled because today we have mostly eliminated such conditions through advances in medicine. They supposed they were fixing a fault in the photographic technique, even though that made no sense. The data scientists can't imagine a world that isn't totally homogenized and everywhere looks identical to the view outside their front door, so they use AI to totally homogenize the world and make it look exactly like the view from their front door, regardless of what it might actually look like.

    • CrazyGaming
      CrazyGaming 2 months ago +1

      "Nothing to do with imagined failings in the game engine.". yes it has, but calling them "failings" is maybe strong word. This is new technology that it would have been hard to expect already being developed and implemented back when GTA V was released. And it still isnt perfect. But you did see the video, right? It looks absolutely stunningly real, and that is not just because Germany on a cloudy day looks more real than California, it's because the tech is working, it just so happens to be that their datasets are from cloudy Germany, not sunny California.

  • Thunderson
    Thunderson 8 months ago +1

    If we already have this i can imagine future vr combat games with literally real life graphics

  • Crittek
    Crittek 11 months ago

    We've been effectively using computer learning for a short time yet with crazy results. Imagine what the next decade will be like.

  • FunkyPrince
    FunkyPrince 11 months ago +1394

    The undersaturated results look like that greenish filter in "The Matrix" applied on the inside-the-simulation scenes.

  • Lucawiouh
    Lucawiouh 4 months ago

    My goodness, this is absolutely mind blowing, amazing!

  • Origami Phoenix
    Origami Phoenix 7 months ago

    6:46 Holy cow! It's already almost a Photorealistic Post-Processing Injector. That would be incredible to see.

  • Vtmgll
    Vtmgll 11 months ago

    It has been a long time since I was amazed by realistic videogames, this video brings me back to early 2000's

  • WATEsub
    WATEsub 11 months ago +1

    Imagine playing games with similar AI turned on in real time!

  • Vincent Sanko
    Vincent Sanko 11 months ago +431

    This looks very promising. All we need to do is bump up that render quality and get better training data than just dash cam footage. The cityscape looks so green probably cause of the slightly green tint that is added to most windshields. When I think of photorealistic, I actually think of what the human eye would see. Can't wait to see how this improves!

    • eatingsushi
      eatingsushi 8 months ago

      @AlphaGarg couldnt you just use images from google street view theoretically?

    • Socks
      Socks 8 months ago

      yeah, and dashcams are often low quality, i wonder if that contributes to this looking 480p or if that is just a limitation of the ai.

    • AlphaGarg
      AlphaGarg 11 months ago

      @TyrannoFan Data sets are *very* expensive, though

    • 12 34
      12 34 11 months ago +1

      I’d still like to have video games that look like films, much the way this did. Real color recreation won’t be possible until we all have 12 bit displays.

  • 99thGamer
    99thGamer 11 months ago

    I think this would be a great thing to have included in some video games, maybe even trough AMD's Fidelity FX

  • FendrZwag
    FendrZwag 11 months ago +1

    As a Game Designer I can only imagine writing a script which processes every single texture from my games through this AI and create super realistic textures to use in the games

  • Vladimir Delecto
    Vladimir Delecto 11 months ago +1

    Good lord, this is AMAZING.
    Love your videos TMP. Keep it up!

  • xXinsaneboyXx
    xXinsaneboyXx 11 months ago +1

    Great! Seems we are one step closer to realize that everything goes full circle eventually.

  • Bruno
    Bruno 11 months ago +654

    The output looks kinda soft. Would love to see this put into DLSS 2.0 to regain some detail, as well as maybe try colour correcting the training dataset to look more like what humans see rather than what cameras see.

    • Tryphon, un ZOUAVE ??!
      Tryphon, un ZOUAVE ??! 11 months ago

      @Klin-Klin Plus the devs would have to kill thousands of people and animals for the training data set, not particularly worth it

    • Aron Septianto
      Aron Septianto 11 months ago

      @Cruz yeah, it does have that "dashcam" look
      would be interesting if the training material is from a film camera

    • Bruno
      Bruno 11 months ago

      @Cruz I was talking about the images on their website. Even when not viewed full screen they look quite soft imo. And yeah, I suspected the blue tint was due to the windshield, that shouldn't be too difficult to fix with a reshade.

    • Cruz
      Cruz 11 months ago

      Actually the fault's on the TheXvid compression algorithm! If you take a look at the image these guys have on their website, they look comparatively crisp (and some even look like real photos). Still, the unaltered material is not perfect, but after uploading it to their youtube channel, it looks way blurier.
      Also, some people have theorized in the original video that the "blueish" and "darker" tone of the videos are from the fact that the source imagery from german cities was recorded behind a car windshield, and they often have a slight tint on them.

    • Siana Gearz
      Siana Gearz 11 months ago +1

      @Aron Septianto The motion vector field is an intermediate output of the renderer in the engine, you can even gain access to it with basic engine-agnostic tools just by inspecting the render targets. So it doesn't need to access any underlying data like geometry or textures or normal maps and the like.

  • miss-sagemoon
    miss-sagemoon 5 months ago +1

    From a technology point of view, this is amazing. But I doubt this could be implemented without causing a bunch of gamers to complain about the realism. Still this would be great for simulators, especially city simulators!

  • Abstractn
    Abstractn 11 months ago

    Dude, no I cannot hold on to my papers! This is incredible! What a time to be alive indeed!

  • Sonny Dortmund
    Sonny Dortmund 10 months ago

    This has never been my field. I enjoyed this.
    Thankyou, good Sir 🙏🏻

  • Michael Crumpton
    Michael Crumpton 5 months ago

    I see so many of these papers you cover as having applications in budget filmmaking. For now they can use it to help create animated storyboards to mock up scenes, but as it improves it will make its way into the final product.

  • Daniel Wojciak
    Daniel Wojciak 11 months ago +173

    Since only driving footage was used, I'm interested to see how it would handle the player getting the camera up close to objects like trees and NPC's, and seeing how much detail comes through.

    • Randy Randerson
      Randy Randerson 11 months ago

      @YX Huang nah just break it into pieces. You can define it precisely like defining banks, hotels, apartments, ect

    • Jon Do
      Jon Do 11 months ago +2

      I'm interested how it looks if you kill someone since the real photo's shouldn't include such horrific scenes.

    • z beeblebrox
      z beeblebrox 11 months ago +4

      You can see how it handles rocks when the footage drives by boulders in the open landscape - the AI didn't have any rocky training sets, so they're all just the game's textures transposed into the color correction. That's the boring answer - what the AI doesn't understand it just uses the game's output for, so most objects will just be their regular models color corrected.

    • SOMNIAC EFFECTS
      SOMNIAC EFFECTS 11 months ago +15

      it actually won't work at all. and that's the reason why we won't see this in the games like gta, at least in the next 10-20 years. however, we might see this type of post proccesing in games with fixed camera angles starting from next generation of consoles, i.e. mostly sport games like NBA, FIFA, boxing, etc. (we have alot of training data on how a real football match looks like for example, so the future is bright for realistic sport simulators, but to move camera up or down in gta V, we have no dataset to figure out the proper way to draw it)

  • Billy
    Billy 11 months ago

    The hyperrealistic stuff looks like a dream. It actually looks really good!

  • Vidusha Thebuwana
    Vidusha Thebuwana 11 months ago

    This in VR would be mindblowing 🤯

  • Stepan Kracmar
    Stepan Kracmar 7 months ago

    I'd like to see this applied on some of the unreal engine games

  • Kantirant
    Kantirant 11 months ago

    now we need a new AI technique that will make things look more fluid

  • Kilgore Trout
    Kilgore Trout 11 months ago +1074

    "enhancing photorealism enhancement"
    "real reality is really real"

    • Socks
      Socks 8 months ago

      @w花b timestamp? darn i didnt notice that. kind of a deal breaker.. i guess they could iron those kinks out though.

    • Socks
      Socks 8 months ago

      @machbauer132 thats an amazing sentence.

    • Jon S
      Jon S 8 months ago

      I upvoted because your comment was at 999. Welcome to 1K.

    • PALKKI TT
      PALKKI TT 9 months ago

      🚩🏴

    • Igors123
      Igors123 10 months ago

      @Deborah Ajao Reality is really really really really real.

  • Hypnotoad
    Hypnotoad 10 months ago

    Funny to see this as a reality, I had thought about pretty much this exact same concept, only with Google Maps.
    I'm not a programmer/video game designer or anything near that, though, just a gamer lol.

  • X X
    X X 4 hours ago

    I love the development which is being made. This method works very well on roads and flat buildings, but is a lot less reliable when it comes to organic objects.

  • C V
    C V 10 months ago

    this could be really cool in the film industry. the amount of awesomeness gives me a headache

  • My M8
    My M8 11 months ago +2

    Every time Two Minute Papers uploads a new video:
    _Ah.. here my reality goes once again._

  • Offero 04
    Offero 04 11 months ago +282

    That bombshell at the end "this is real time" and hence can be used as a post processing effect.

    • Offero 04
      Offero 04 10 months ago

      @Mike Jenkins In theory it really shouldnt be a problem to get it working with any driving game etc. As a post processing effect its simply manipulating the pixels currently on the screen.

    • Mike Jenkins
      Mike Jenkins 10 months ago

      Can this be added as a plugin to ue4/5?

    • JICHAEL MORGAN
      JICHAEL MORGAN 10 months ago

      Could assetto corsa modders work with this technology??

    • DreadEx
      DreadEx 10 months ago +1

      @Just a dictator riding on a nuke oh wow, guess I'll have to start learning then 🙃 thanks for the heads up!

  • ChaZ-E
    ChaZ-E 11 months ago

    5:55 the value proposition mentioned here is mind blowing but also scary for World/Stage Designers

  • Hayden Hoodless
    Hayden Hoodless 11 months ago

    GTA V must go down as one of the most impressive games ever. These AI guys love messing around with it.

  • Alex Dalgård
    Alex Dalgård 10 months ago

    This is going to be really handy for concept art, and when protyping game maps

  • CMDR REFLIPD
    CMDR REFLIPD 11 months ago +1

    I am a developer for an upcoming game engine, I would love to implement this as a feature. The game engine is modular, I have already developed about 10 modules for it. I would love to create a module for this kind of feature, we already have realtime NPC interaction and communication and it's a module.
    The game engine uses Taskflow for Multi-threading and Dilligent Graphics for pre-processing. We also created an EOS module (epic online services), and a Quixel Mixer module.
    But this a post-processing capablity would be awesome.

  • anti/HUMAN Designs
    anti/HUMAN Designs 11 months ago +424

    This is really good because it means we could render the game in a much simpler way, only providing the kind of information that the post-processor needs, speeding it up significantly.

    • anti/HUMAN Designs
      anti/HUMAN Designs 11 months ago

      @Feroste "Two papers down the line". The idea is interesting.

    • Feroste
      Feroste 11 months ago +1

      Each time the effect runs, it will look different.
      You need most of the detail in the material itself otherwise your 'realism' will be inconsistent.
      Especially since each frame is rendered independently. Just reinviting flickering issues.

    • anti/HUMAN Designs
      anti/HUMAN Designs 11 months ago +1

      @Haiiry Cake Yes, this is obviously for games looking for pure realism, which is a fairly large part of games.
      This obviously has its place in racing games, for example, and any kind of simulator.

    • Haiiry Cake
      Haiiry Cake 11 months ago +2

      It's only good if you don't care at all about art.
      Art is not about perfection, it's about impressions, and if you have to rely on an arbitrary AI to create your expressions, the impressions they leave cannot be deliberate.

    • Stop Bugging Me Google I Hate You
      Stop Bugging Me Google I Hate You 11 months ago

      @anti/HUMAN Designs I don't know specifically about the ML technologies used in this paper, but anything based on a decision tree / random forest is going to have varying time per query due to different paths being taken through the tree. Something that's simply based upon fixed-time circuit evaluation would have the same performance, though that doesn't describe everything in the ML world.

  • Skeptic Mafia
    Skeptic Mafia 11 months ago

    2:14 I like how the main difference is how the hyper realistic one look more depressing.

  • Ash C
    Ash C 11 months ago

    I wonder how the ai would work learning and replicating underwater biomes and being used as post processing in subnautica or something

  • ParaPutty
    ParaPutty 11 months ago

    It's actually interesting how at 6:45 it actually gave me a realistic reaction of anger and shock before I remembered it was a game.

  • movax20h
    movax20h 11 months ago

    Wow. That is literally a game changer. Just imagine what will happen in 2-3 years, when this method is improved, data sets grow, and it also use depth buffer. Could be really real good.

  • Omar G
    Omar G 10 months ago +1

    It may not look that great, but remember, they used google maps images as reference, imagine using enhanced images or just better ones.
    This may be the future of not only gaming but graphics!

  • Chandramouli Roy
    Chandramouli Roy 11 months ago

    Imagine this being applied in Crysis 3, with Wuhan scape as the real world data set!

  • Logic_missing
    Logic_missing 10 months ago

    Imagine this being used in like 3d animated videos/movies

  • the takburger
    the takburger 10 months ago +1

    Next time you go on character creation in a game, you just can take a pic of yourself and the AI will get you in the game

  • Dan Heather
    Dan Heather 11 months ago +592

    LITERALLY a "Game" Changer!

    • Matías Navarrete
      Matías Navarrete 11 months ago

      @bibbly bobbly GTA XVII

    • lazylillies
      lazylillies 11 months ago +1

      You a journalist friend?

    • Allan Cristhian Kath Kipfer
      Allan Cristhian Kath Kipfer 11 months ago

      oh my!
      I see what u did here!!!

    • zdenek burian
      zdenek burian 11 months ago +1

      imagine if there is a similar module of AI postprocessing in real time like in that paper, applied for example to a 3D program like blender or 3dsmax, this is the end for the profession of architectural rendering intended as the search for perfect photorealism, like using corona renderer; Anybody capable of model some walls and insert some furniture blocks in sketchup could obtain a perfectly real image, using beautiful photos from architectural magazines as refences for interiors. And this is only the start, because after this step will come obviously the most compelling task, where an AI module will create directly the 3D models after submitting a simple plan and photos of interior examples, ths will wipe out an entire category of professionals, I left that field 20 years ago exactly because of this, and consider that I was thinking to first global illumination innovations like vray which I sensed as an impossible menace to my skills of artistic visualizer...

    • Nirmal Singh
      Nirmal Singh 11 months ago +2

      @Ny-kel Cameron you still need a good gpu for unreal 5, this is just a post-processing filter...

  • BluestoneCreeper
    BluestoneCreeper 11 months ago

    Just wait for this to get implemented into big games so when you turn off post prossessing it just looks like a tone of blocks

  • Outrider
    Outrider 11 months ago

    Im getting more and more convinced that we live in a simulation

  • DistortedV12
    DistortedV12 11 months ago

    I feel like you need some kind of intermediary interpretability nob like an editor in a film to really benefit from this approach. This is super interesting and is a good example of what AI can do for newcomers.

  • Joshua Latham
    Joshua Latham 11 months ago

    You have to be kidding me. This is usable right now?! I'm super impressed.

  • Mr Slime
    Mr Slime 11 months ago +505

    Reshade: I make games more realistic with raytracing!
    This AI: I'm about to end this man's whole career

    • Ark Solitude
      Ark Solitude 10 months ago

      @InfinitySwitch Which is great

    • Heisse Scheisse
      Heisse Scheisse 11 months ago +1

      @Cedric lacrambe They could maybe train this with shooting HDRI photos. These are photos that have lighting information. Skyboxes in games are many times made with HDRI's as well.

    • Cedric lacrambe
      Cedric lacrambe 11 months ago

      I suppose the neural network still need to get information about lighting to get the output right, you still need an approximate rendering even if I suppose it could be done low res

    • Mr Slime
      Mr Slime 11 months ago

      @Heisse Scheisse it legit says "ReShade is a generic post-processing injector"

    • Heisse Scheisse
      Heisse Scheisse 11 months ago +1

      @Mr Slime duh? like if you think that this stuff is common sense...

  • Natan Br
    Natan Br Month ago

    Next step, use game scenarios to test self driving car reaction to "crazy drivers". Also, it can be applied not only to the graphics bug also to the physics engine to generate extra realistic behavior.

  • Nixel
    Nixel 11 months ago

    And here I was thinking games already looked realistic. Now that I've seen what it _should_ look like, I can never unsee it.
    The fact it already runs in real-time makes me very excited!

  • zelest
    zelest 11 months ago

    I'd love to see this run on something it was never trained to just because I'm a fan of stressing algorithms. Like... what would Cyberpunk2077 look like through this?

  • Aroos Productions
    Aroos Productions 11 months ago +1

    This is insane! :O But any info regarding the speed?

  • GreenDave113
    GreenDave113 11 months ago +190

    Someone needs to make this actually available to try in GTA in real time.

    • Kuri0
      Kuri0 11 months ago

      It will be 1fps on a 3090

    • Kenneth
      Kenneth 11 months ago

      @Devan Mallory they said it can currently be added to the game

    • La Quica
      La Quica 11 months ago +3

      In the original video they said something along the lines of "can produce images at an interactive rate", so no real time. Also the limitations of this is pretty visible in the video.. low frame rate, low resolution and only car footage is provided which might be due to the fact that only dash cam videos were provided to the algorithm to work with, whose results may not translate well in Third Person Perspective.

    • Devan Mallory
      Devan Mallory 11 months ago

      @Kenneth I thought they said that it wasn’t real time yet

    • Kenneth
      Kenneth 11 months ago

      @Gile this is crazy, i've just watched the dev video i still can't believe it can run in realtime

  • Thial
    Thial 10 months ago

    Now just wait for the AAA companies to use that in their E3 presentations. 16x the detail.

  • B. S.
    B. S. 11 months ago

    If this is implemented soon, it will bring mind blowing graphics but also the craziest bugs we've ever seen 🤣

  • Mahabub Alam
    Mahabub Alam 10 months ago +1

    It's a great improvement of photorealism techniques.

  • Min percent
    Min percent 11 months ago

    Woah I like how new method got so much better :O
    Love it, cheers

  • Rickydo
    Rickydo 11 months ago

    This technology would be perfect for VR tbh

  • Philippe Dumonet
    Philippe Dumonet 11 months ago

    No wonder GTA VI is taking so long, if they incorporate some of these methods the game will be absolutely amazing!

  • Averagegamer 16
    Averagegamer 16 11 months ago

    I'd say merge this with the dream AI and also the predictive simulations ans you can have photo realistic simulations of completely made up events grounded by real world physics and universal laws as we know them

  • Fryncyar Yorvjink
    Fryncyar Yorvjink 9 months ago

    So if less grunt work has to be done, could you try this method on an early 3d driving game, like Driver 1?

  • CommandCube
    CommandCube 11 months ago +133

    Between this, ray tracing, and unreal engine 5's lumen and nanite, I am excited for the future graphical fidelity of video games.

    • Socks
      Socks 8 months ago

      @Sonic Broom its going to be a lot longer than 10 months. until they have some new manufacturing plants up and running, there is going to be no change in supply. all the tech out there all needing their chips made in the same places... and it will take years before the planning and construction etc are done, then several more years after that to get the process nodes working, and by then the node will be outdated and the will need to start again... i mean, look at how much trouble intel has been having. stuck on 14nm or whatever it was for like 5+ years now. (thats a different situation though i suppose).

    • Socks
      Socks 8 months ago

      ray tracing makes a minimal difference, its 1000x more realistic, but 'realistically' you're not really going to notice the difference. im far more interested in what dlss can do. especially in vr where we need all the performance we can get. or at least, if you are going to mention raytracing, include dlss in that sentence. ray tracing on its own is just a way to play at 10fps, or 40fps with all settings on low and 720p....

    • blaze smooth
      blaze smooth 9 months ago

      @Sonic Broom ?

    • Sonic Broom
      Sonic Broom 10 months ago +3

      Yeah, just have to wait 10 months to pay $2000 for a low-mid tier GPU. Great time to be a gamer... :/

    • VioletStorm
      VioletStorm 11 months ago +13

      It really feels like we are at the brink of a new generational leap doesn't it?

  • Patrick Kelly
    Patrick Kelly 11 months ago +1

    This is amazing, if this can be adapted as a tool with some artistic control then I can definitely see this becoming a literal game changer. I would love to see it paired with unreal engines lumen, as lumen struggles with foliage, trees, grass and leaves etc, but removes the need for lods, making high fidelity scenes more palatable on lower end machines, I can imagine it could be unstoppable duo. I can't wait to see what people can make of it, especially a few papers down the line.

    • TiagoTiago
      TiagoTiago 11 months ago

      I imagine it would just be a matter of having a "Hollywood" version of the game assets, leave some bot running around trying everything possible on some render farms; and use the resulting frames to train the AI that will be shipped in the final game, with the assets shipped with the game being much simpler stuff.

  • DefB
    DefB 11 months ago +4

    I'd love to see this algorithm applied to a completely different game, say Minecraft or something!

    • V mike
      V mike 8 months ago

      Wind waker

  • Art Curious
    Art Curious 2 months ago

    Great idea for travel agencies to give customers a preview of their destinations maybe