this post was submitted on 28 Aug 2024

50 points (100.0% liked)

Gaming

30424 readers

614 users here now

From video gaming to card games and stuff in between, if it's gaming you can probably discuss it here!

Please Note: Gaming memes are permitted to be posted on Meme Mondays, but will otherwise be removed in an effort to allow other discussions to take place.

See also Gaming's sister community Tabletop Gaming.

This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

founded 2 years ago

MODERATORS

[email protected]

New AI model can hallucinate a game of 1993’s Doom in real time (arstechnica.com)

submitted 2 weeks ago by [email protected] to c/[email protected]

30 comments fedilink hide all child comments

top 30 comments

sorted by: hot top controversial new old

[–] [email protected] 63 points 2 weeks ago* (last edited 2 weeks ago) (4 children)

"The potential here is absurd," wrote app developer Nick Dobos in reaction to the news. "Why write complex rules for software by hand when the AI can just think every pixel for you?"

"Can it run Doom?"

"Sure, do you have a spare datacenter or two full of GPUs, and perhaps a nuclear powerplant for a PSU?"

What the fuck are these people smoking. Apparently it can manage 20 fps on one "TPU" but to get there it was trained on shitload of footage of Doom. So just play Doom?!

The researchers speculate that with the technique, new video games might be created "via textual descriptions or examples images" rather than programming, and people may be able to convert a set of still images into a new playable level or character for an existing game based solely on examples rather than relying on coding skill.

It keeps coming back to this, the assumption that these models, if you just feed them enough stuff will somehow become able to "create" something completely new, as if they don't fall apart the second you ask for something that wasn't somewhere in the training data. Not to mention that this type of "gaming engine" will never be as efficient as an actual one.

[–] [email protected] 18 points 2 weeks ago

Apparently it can manage 20 fps on one "TPU" but to get there it was trained on shitload of footage of Doom. So just play Doom?!

Shhhh! Are you nuts? People are going to start realizing this is another tech bubble, like Blockchain...

[–] [email protected] 16 points 2 weeks ago

To be fair, half of the AAA gaming industry is all about trying to clone the latest successful game with a new coat of paint. Maybe using AI to make these clones will mean that the talented people behind the scenes are free to explore other ideas instead.

Of course in reality, it just means that the largest publishers will lay off a whole lot of people and keep churning out these uninspired games in the name of corporate profits, but it's nice to dream sometimes.

[–] [email protected] 10 points 2 weeks ago (1 children)

big weird flex but okay vibes except actually not okay

[–] [email protected] 2 points 2 weeks ago

left me stunlocked with this one icl

[–] [email protected] 8 points 2 weeks ago (1 children)

I mean, you've never seen a purple elephant with a tennis racket. None of that exists in the data set since elephants are neither purple nor tennis players. Exposure to all the individual elements allows for generation of concepts outside the existing data, even though they don't exit in reality or in the data set.

[–] [email protected] 15 points 2 weeks ago* (last edited 2 weeks ago) (1 children)

Ok.

Try to get an image generator to create an image of a tennis racket, with all racket-like objects or relevant sport data removed from the training data.

Explain the concept to it with words alone, accurately enough to get something that looks exactly like the real thing. Maybe you can give it pictures, but one won't really be enough, you'll basically have to give it that chunk of training data you removed.

That's the problem you'll run into the second you want to realize a new game genre.

[–] [email protected] 6 points 2 weeks ago (1 children)

There are more forms of guidance than just raw words. Just off the top of my head, there's inpainting, outpainting, controlnets, prompt editing, and embeddings. The researchers who pulled this off definitely didn't do it with text prompts.

[–] [email protected] 16 points 2 weeks ago* (last edited 2 weeks ago) (1 children)

Obviously.

But at what point does that guidance just become the dataset you removed from the training data?

To get it to run Doom, they used Doom.

To realize a new genre, you'll "just" have to make that game the old fashion way, first.

[–] [email protected] 6 points 2 weeks ago (1 children)

But at what point does that guidance just become the dataset you removed from the training data?

The whole point is that it didn't know the concepts beforehand, and no it doesn't become the dataset. Observations made of the training data are added to the model's weights after training, the dataset is never relevant again as the model's weights are locked in.

To get it to run Doom, they used Doom.

To realize a new genre, you’ll “just” have to make that game the old fashion way, first.

Or you could train a more general model. These things happen in steps, research is a process.

[–] [email protected] 8 points 2 weeks ago* (last edited 2 weeks ago) (1 children)

You are completely missing what I'm saying.

I know the input doesn't alter the model, that's not what I mean.

And "general" models are only "general" in the sense that they are massively bloated and still crap at dealing with shit that they weren't trained on.

And no, "comprehending" new concepts by palette swapping something and smashing two existing things together isn't the kind of creativity I'm saying these systems are incapable of.

[–] [email protected] 4 points 2 weeks ago (1 children)

What kind of creativity are you talking about then? I've also never heard of a bloated model. Which models are bloated?

[–] [email protected] 3 points 2 weeks ago* (last edited 2 weeks ago) (1 children)

Bloated, as in large and heavy. More expensive, more power hungry, less efficient.

I already brought it up. They can't deal with something completely new.

When you discuss what you want with a human artist or programmer or whatever, there is a back and forth process where both parties explain and ask until comprehension is achieved, and this improves the result. The creativity on display is the kind that can unfold and realize a complex idea based on simple explanations even when it is completely novel.

It doesn't matter if the programmer has played games with regenerating health before, one can comprehend and implement the concept based on just a couple sentences.

Now how would you do the same with a "general" model that didn't have any games that work like that in the training data?

My point is that "general" models aren't a thing. Not really. We can make models that are really, really big, but they remain very bad at filling in gaps in reality that weren't in the training data. They don't start magically putting two and two together and comprehending all the rest.

[–] [email protected] 1 points 2 weeks ago (1 children)

Do you have any examples of how they fail? There are plenty of ways to explain new concepts to models.

https://arxiv.org/abs/2404.19427 https://arxiv.org/abs/2406.11643 https://arxiv.org/abs/2403.12962 https://arxiv.org/abs/2404.06425 https://arxiv.org/abs/2403.18922 https://arxiv.org/abs/2406.01300

[–] [email protected] 2 points 2 weeks ago* (last edited 2 weeks ago) (1 children)

In a couple sentences? In a way that doesn't approach, equal or exceed the effort of training the model with that data to begin with?

You insist these models can do new things out of nothing, and you keep saying "all you have to do, is give them something".

[–] [email protected] 2 points 2 weeks ago (1 children)

You keep moving the goal posts and putting words in my mouth. I never said you can do new things out of nothing. Nothing I mentioned is approaching, equaling, or exceeding the effort of training a model.

You haven't answered a single one of my questions, and you are not arguing in good faith. We're done here. I can't say it's been a pleasure.

[–] [email protected] 1 points 2 weeks ago* (last edited 2 weeks ago)

My argument was and is that neural models don't produce anything truly new. That they can't handle things outside what is outlined by the data they were trained on.

Are you not claiming otherwise?

You say it's possible to guide models into doing new things, and I can see how that's the case, especially if the model is a very big one, meaning it is more likely that it has relevant structures to apply to the task.

But I'm also pretty damn sure they have insurmountable limits. You can't "guide" and LLM into doing image generation, except by having it interact with an image generation model.

[–] [email protected] 28 points 2 weeks ago (1 children)

Game code you can't debug because you don't understand it. Brilliant.

[–] [email protected] 5 points 2 weeks ago

Honestly the let’s plays will be epic tho. Pure chaos.

[–] [email protected] 18 points 2 weeks ago (1 children)

The author seems to have written endless amounts of "AI" articles. Do they really not know what "hallucinate" means in such a context?

[–] [email protected] 20 points 2 weeks ago

Well you see, they wrote the articles with AI.

[–] [email protected] 18 points 2 weeks ago

It's basically like. Someone drawing a picture. Then watching the buttons you're pressing on a controller. And then drawing a new picture. And based on the game that they think you're playing in their head trying to guess what the next picture ought to look like. With no error correction and no conceptualization other than what the next picture should look like.

The... many limitations of this is the inability of image generators to rationalize 3 dimensional space. It can only approximate it based on what it thinks should appear on the screen. It lacks any ability to keep track of variable information. It really is more like a Doom-style hallucination than anything else. Some of the videos on that article are truly bizarre looking. I'd imagine after a few minutes every single one of them would devolve into an endless loop of being trapped in non-sensical geometry or killing the same enemy over and over again as the AI has no way of remembering the enemy existed to begin with, let alone that you killed it.

I'll be honest I don't think there is much use in this at all. It suffers from the same limits as any other model AI. Believability at a glance is not believability under scrutiny and if it's only believable at a glance then there's not much practical use in it. The advance in computational power and model sophistication required to stand up under scrutiny is massive.

[–] [email protected] 11 points 2 weeks ago

DOOM runs on everything, even on a hallucinations of artificial intelligence. It's getting beyond ridiculous.

[–] [email protected] 10 points 2 weeks ago* (last edited 2 weeks ago)

- "Can artificial hallucination run Doom?"
- "What kind of stupid question is that??? Ofc it can, let me show you ..."

[–] [email protected] 8 points 2 weeks ago

Proof that if a system or technology exists, nerds will try to run Doom on it.

[–] [email protected] 5 points 2 weeks ago* (last edited 2 weeks ago) (1 children)

This is perfect for capitalism with Matrix bio-fuel-cells-human/battery tech!

It would have been too easy to just chill peacefully and unbothered in my cozy pod - they would feed me a hallucination of a dead-end job the whole time, complete with all the stupid office buttons I have to press.

[–] [email protected] 3 points 2 weeks ago (1 children)

Wake up Neo

[–] [email protected] 3 points 2 weeks ago

No fucking way, I have to go to work tomorrow, my soulless cubicle needs me.

[–] [email protected] 4 points 2 weeks ago (1 children)

Interesting thought for a game to not be 3D modeled and programmed, but to be "modeled".

[–] [email protected] 2 points 2 weeks ago

Envisioned in the general direction.

Like holodeck tech.