Finally, Beautiful Virtual Scenes…For Less! ☀️

  • Published on Oct 11, 2021
  • ❤️ Check out Weights & Biases and sign up for a free demo here:
    ❤️ Their mentioned post is available here:
    📝 The paper "DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks" is available here:
    Meet and discuss your ideas with other Fellow Scholars on the Two Minute Papers Discord:
    🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
    Aleksandr Mashrabov, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bryan Learn, Christian Ahlin, Eric Haddad, Eric Martel, Gordon Child, Ivo Galic, Jace O'Brien, Javier Bustamante, John Le, Jonas, Kenneth Davis, Klaus Busse, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Mark Oates, Michael Albrecht, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Rajarshi Nigam, Ramsey Elbasheer, Steef, Taras Bobrovytsky, Thomas Krcmar, Timothy Sum Hon Mun, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi.
    If you wish to appear here or pick up other perks, click here:
    Thumbnail background image credit:
    Thumbnail background design: Felícia Zsolnai-Fehér -
    Károly Zsolnai-Fehér's links:
    Instagram: twominutepa...
    Twitter: twominutepapers

Comments • 209

  • Bryce
    Bryce  +77

    That is an AMAZING paper! It's papers like these that really reaffirm my love for ML and it's implications for our future! I was genuinely gripping my hands with excitement for this one!

  • Andreadel
    Andreadel  +230

    Truly stunning progress, I did not squeeze my papers tight enough for 50 times less input and running in real time. Very impressive.

  • Chr0my
    Chr0my  +189

    Google maps is going to look amazing if they add it

  • Benji C
    Benji C  +114

    Using this to be able to create scenes in vr from just a photo would be mind blowing!

  • Cosmos Code

    The implications for games in general but especially vr are astonishing, For example you could double your frames by using this method to interpolate every other frame

  • Jack Gruber

    Imagine being able to walk around in a photographed space in vr. Amazing!

  • Tesserex
    Tesserex  +31

    Does this mean games in the future could save rendering time by, instead of rendering every frame, train the ai on just a few frames from some select nearby views, and then have it interpolate the rest of the views? Maybe it will become more and more useful as frames get more and more expensive to render, with raytracing and such.

  • Rupert Jung

    Can't wait to see that integrated in game engines.

  • Brodie Eaton


  • Stuffy stuff

    I cant wait for this to make its way to 3D packages, just imagine just imagine the renaissance that would happen once an individual or small group creates totally original IPs with blockbuster quality.

  • J M
    J M  +9

    The applications for this in image stabilization in video recording seem promising.

  • VcSaJen
    VcSaJen  +3

    I remember old quest games made from photos and videos. Looks like it's already usable for much more improved version of that, with free smooth walking!

  • Nelson Cabrera

    This is going to revolutionize 3D VR videos. Finally able to correct convergence disparities and provide (bounded) translational movement. I wonder if this could run in realtime on mobile hardware like the XR2 (which powers the Quest 2 headset). I hope down the line we'll be able to extract full 3d viewports in realtime from a 2d video. Those virtual tourism videos on TheXvid (ie: "Walking around Tokyo at night") are going to become epic in VR.

  • Alex Tuduran

    Wondering if it has enough temporal cohesion to be used in videos. Would be trully awesome to walk with your VR headset in a video. Can't wait to try it on photos in VR. It should be like a genuine lightfield VR experience.

  • JordanCo
    JordanCo  +42

    They should combine this tech with the paper that extends images past their borders so you can have an infinite AI generated 3D world to explore

  • Mark Dustin

    Wish you could feature more AI progress in other fields such as health, environment, and society. Even if they're just compilations

  • Sadiinso Snowfall

    What's the result if we feed the same amount of information to both "algorithms" ? Does this new method not scale as well with the amount of info ? If so, is that why the researchers only showcased it on a smaller input size compared to the previous technique ?

  • Jacques Broquard

    Everything you cover consistently blows my mind. Thank you my friend. Never stop doing what you do.

  • Pierce
    Pierce  +2

    could this be used as an alternative to Blender’s Cycles engine so we can work in real time?

  • KCJbomberFTW

    I’d love to have access to this when looking at 3D houses in Zillow/Redfin