This AI Learned Physics...But How Good Is It? ⚛

I have been thinking about this too. What if instead of trying to piece-by-piece write an entire physics engine, which will inevitably always fall into the uncanny valley, we trained an AI on inputs of real world collisions and interactions. For example, we feed a car racing game footage of crashes, skidding, and rolling, and it improvises in-game animations and even body deformations

Actually, this kinda stuff was possible in 2016 - and funnily enough, used the exact same models as this demo. It’s an engine called FleX, and I believe it only ever got used in one demo and the Nvidia VR Funhouse game. interesting tech, totally wasted opportunity.

@Deivison Carvalho To avoid actually computing physics interactions and make physics simulation degrees of magnitude lighter in terms of computing power, is the point of using AI-based physics simulators, isn't it? So the time for a 'world' with many simulated objects interacting with each other, may not be far away.

@Sitter lol in 50 years you'll be able to run these simulations on your wristwatch

@Sitter Yeah and 640K is enough for everybody.

the models are much slower than actually simulating some aproximative physics... these papers are kind of pointless

@Braxbro addition and subtraction are as fast as bit-wise operations, multiplication is slower and division is much slower AFAIK, however you shouldn't worry about those things, the compiler most probably will change multiplications/division to bit-shifts where it's wise to do so

@Daniel Yeah, my bad. There's one bit you can flip to go from lower to upper. Not sure addition and bit-wise ops are equally fast, but both are optimized to the point that any difference is probably negligible, so.

@Braxbro I think you are mistaking bit-shift for bit-wise operations, you can flip a bit by using the XOR bit-wise operation, and that can be used to go from upper to lowercase or vice versa.
It doesn't really matter because iirc, XOR and addition are equally fast, if not the compiler would use the correct one for you, in fact I did use both in my code and I didn't notice a difference.

@Daniel Iirc you can, because that's how the ASCII number allocations are set up, there's one bit difference between upper and lowercase if I recall.

uh

Hmmm, why not feed it real data? Maybe it could figure out quantum gravity.

Yes, especially if you like abusing rabbits by throwing them down the stairs :D

Plot twist: The Ai controls him.

@Benjamin Rood Or you could just feed it some rallying or air display show footage and get a working car or a plane sim. But if you are into that...

@skierpage considering the way that we humans treat the world, it only seems just.

Profound!

@Benjamin Rood Impaled, torn to pieces, shoved down stairs, crash tested, forced to sumo wrestle and ride trains across wonky bridges.

Yes, I look forward to being twisted and distorted and pinned to a wall for AI to learn physics.

What a time to be alive

@{Bread boi} not really .it's just that a single person would be able to do multiple things at once, which means working time would be reduced

@Dat On3 everyone will be unemployed then this is the consequence

Now, imagine that but in a brain interface where you have a program using AI assisting you. Being able to do multiple tasks while doing it solo without the need of other people is going to be fun while making creativity-based stuff.

hmm

@Matthias Max A degree says nothing about your skills. Open source projects created from the work of volunteers are repeatedly outperforming corporate solutions as we speak. So don't be afraid to learn :)

why do I feel like you are describing an adult game XD

I think this paper is enough to do everything you described, since a complex VR Game world can be made with less than 10,000 physics vertices. The real models with over 100,000 vertices would just be used for rendering. In game engines, physics calculations are done using far less complex models than the rendering. Now, all that is needed is talented developers willing to code this paper into a game engine or game.

It's already being done my friend, AI is the future.

@MusicDev
Exciting ! I always wanted to play with a chemical simulator like I can play now with fluid simulations or electrical circuit simulations or optical simulations, etc.
For me this is very interesting because I always hated working with organic perchlorates in the lab. Designing a safe chemical process in a simulator before attempting it in reality would be a dream come true.

Yes. So we currently have an AI that can predict protein folding INCREDIBLY well.Take it from a biochemist, this is a huge deal in our field. The program in question is called AlphaFold. We’re already using AI to simulate molecular interactions without having to create the molecules ourselves, and can even roughly approximate some of these interactions in fairly mild conditions. We’re not quite on the level of what you’re talking about, but we are quickly approaching it.

Yes (maybe), but that's too complex for now

@Adam Sobol you are probably right.. only time will tell though :)

@serloinz Simulating models isn't equivalent of actually understanding them with logical tools such as mathematics. So if anything those simulations will open a whole new frontier for physicists :)

i think you're on a lost cause there.. considering how fast AI is progressing in this field ..almost doing it all now in almost real-time.. maybe become a chemist or biologist ;p

Considering they're doubling down on getting into movie level VFX, I've little doubt they'll be exploring into that.

@Étienne Vanier Agreed.

@VanillaAbstract AI was involved in the folding process to identify the proteins we target with vaccines, so in a way, yes... It's up to each of us to actually do our part and get vaccinated.

@VanillaAbstract -- I don't see how that relates to my comment at all, or even who you're referring to with the word "they".

I think that would require extensive, detailed training data from situations that are very extreme and beyond the ability of humans to currently observe. (e.g. what happens right near a black hole's event horizon) Given that scientists themselves are (in many cases) being held back by a lack of data, I don't think that the AI would accomplish much that humans haven't already.
Furthermore, because neural networks don't care about elegance or understanding of the thing they're approximating, only the accuracy of the results, we wouldn't be able to derive any new physics equations from the output, even if the output was highly accurate.

Maybe if we gave it random data it would create a model of physics that has nothing in common with ours but is consistent enough to be explored and have fun with. Like that 4D Toys game.

@skierpage I agree with you. However, there are all these examples where you add some noice to a picture or you just change a few pixels and your ML model cannot cope with that. For you it looks indistinguishable, for the model it becomes a totally different thing. Maybe these small perturbations are the things that make it complex. What comes to my mind is, that the use of manly synthetic data might make your model more prone to willful manipulation.
I'm not a scholar in machine learning, I just try to puzzle together all the bits I've heard and remember.

@Harald Töpfer i watch a of Lex ..but i disagree that we will never get to a 'real world footage' simulation.. surely it's inevitable, regardless of how slow we progress ?

Question is, is your simulation sufficient enough to catch the edge cases.
If I remember correctly, some researcher on the Lex Fridmann podcast mentioned that he doesn't believe that simulation will ever replace real world footage.

@P E Thankyou it makes a bit more sense now.

as I understand it, each particle in the sim gets itself and its neighboring particles' data (velocity, mass, acceleration, gravity, other stuff) and sends to the neural network, getting a result back that tells it how to act.
basically while normal sims calculate long form, this algo reads the multiplication table. a really, really big and elaborate multiplication table.

@Yo Él Thankyou man, that i have watched. What i meant was how does the tech in this paper work
Like a channel that explains each paper in detail.

This video explains quickly how do neural networks work

We are heading toward this, but it will take a shift in people perception of computers. These AIs can do incredible things, but they always have a chance to blowup too. And when it does, it often catastrophic, like the dino bones turning into a santa-claus by mistake.
Those AI need human supervisors. We need people to accept that their blender module might go awry and spit gibberish. And it's quite different of what we are used from computers.

@Get Sideways That is basically exactly what I'm saying. The algorithm hits a point in the simulation that it just can't handle mathematically. The example I gave was from observation of where the failure most often occurs, and that is when the mesh ends up clipping through either itself or the underlying object during the simulation.

@Dilligff Are you sure that's not just a result of division by zero or a similar showcase of Mathematics being but a crutch?

Not too far off. In my limited use of simulation (primarily dforce in daz studio) the explosions generally happen when the verts clip through the mesh. The algorithm doesn't understand the 'negative space' calculation and it basically throws everything off. I'd equate a pure simulation as flying by instrumentation alone where you set a course and it follows through as best it can, where the neural net is like having a pilot on standby to make course corrections based on 'knowing' where certain landmarks are supposed to be.

@Lysander Dusseljee I'm afraid it's not as easy as you put it, and all these Two Minute Papers videos are enough proof. Apparently, neural network calculations take a lot of overhead themselves. And that's with the specifically designed "Tensor" units on the RTX branded GPUs. The videos on this channel also show that the resulting 'emulations' are still far from being real-time (still, 1 to 2 frames per second is vastly better than one frame in one hour).
As I understand this, to make the best use of NNs we need high qubit count personal or easily accessible quantum computers, so that's still relatively far off in the future. At least the "simulating the entire universe" task.

@Get Sideways Thanks for putting a name to it. This allowed me to google it and see that RTX is used in 150 apps. If I understand it correctly, RTX is slightly different from this physics simulation, but both use the same principle; the principle of running a complex simulation and training an AI to replicate the results with minimal processing.
If they added a temporal component, where the AI predicts process states at time x without going through the intermediate steps, it should be able to run off-screen simulation using microscopic amounts of processing. I think this technique could probably simulate an entire universe in real time. The larger the object, the less details matter (until you zoom in). You'd just need to estimate a plausible end-state, not necessarily the precise outcome.

RTX is basically ray-tracing approximation using neural networks and "far fewer light rays".

@NukeMarine interesting

They're already using AI to predict protein folding. That's a BIG field with enormous medical applications.

Of course it shouldn't since it learns "visually". A mathematical model - however good it is - has tolerance zones where it goes haywire (like dividing by zero), while a neural network simply goes, "ok, with those limbs being there and with that much bending the whole system should look like this..."
Basically, you could do the same thing. Human brain is also a neural network, and having had watched a whole lot of physical interactions you can also try to predict how the thing is going to behave.
When you watch a car or a flight sim doing its thing, you do have the episodes of "well... that doesn't seem realistic" or "whoa, it flies exactly as I expected it to!" from time to time, right?

I can almost guarantee UE has plugins for that sort of thing

reminds me of spinning a ragdoll too fast

Never seen cars launch themselves into the sky in a racing sim before?

I believe it is just slow framerate to display more precision

Division by zero? Or maybe sqrt(-1)

yes, take a look at Dall-E to see what one which was trained on the whole google image can achieve.

@SomeRandomAutisticGuy Personally, I'm waiting for neural networks to start emulating physics in simulations (like racing sims or flight sims).

Yeah Ray tracing is really promising it's much cheaper than simulating every single photon
if i see an ai doing ray tracing or simulating photons it would be sight to behold

@SomeRandomAutisticGuy As far as I'm aware, it's either for improving low ray count ray-tracing or indeed for upscaling, but it's still ray-tracing specific (trained on the traced data).

The ai in RTX are for upscaling low res ray tracing

@Get Sideways but rtx is just an ai
upscaling a low resolution ray tracing

IKR? Go back and watch it frame-by-frame. It's even more unsettling.

@skierpagePS - I’m no scientist… I’ve been looking at this for a while, and it’s just a feeling, that this is an interface thing. The object in space. The stuff in between…like what’s going on with the Fourth Phase of Water. I am just a guy who reads stuff… take it for what ever you will. Pull the POV back a little. Look at more than the focus. Keep at it. Never stop. We are all behind you.

@skierpage Science has done a tremendous job with DNA. I know that they’ll get to it sooner or later… but nature sure has you biting your tails on this. There is a answer, but it’s going to be a revelation… I think the Electric Universe model has a lot of insight.

They are already using them in a similar fashion, but for the graphics with RTX...

Footage from real life would work better, but you can't always see what were the input values (or output coordinates for all the vertices in question) in that case. Not to mention it could simply be prohibitive to present ALL the needed situations in real life to the network.

@mvmlego1212 oh right, that totally makes sense ! Thanks

From my understanding, the ground truth is a simulation in which the laws of physics have been explicitly programmed, and then applied to a virtual object. The positions of vertices are determined by spring equations in the same way that a physics student would determine them with a paper and pencil.
The neural network is trained on data from those simulations--specifically, the positions of the vertices over time, but _not_ the equations that govern those positions. From that data, it attempts to derive physical laws that would reproduce that output. The laws won't match the actual laws of nature exactly, so it's not a rigorous simulation, but I think the advantage is that it has the potential to be faster and more versatile.

Probably simulated with a most correct traditional simulation method without much of acceleration means.

Not gonna happen...