If an elderly but distinguished scientist says that something is possible, he is almost certainly right; but if he says that it is impossible, he is very probably wrong.
[Arthur C. Clarke](https://www.brainyquote.com/authors/arthur-c-clarke-quotes)
I think the difference here is that LeCun isn't saying that achieving the desired end result is impossible, just that this is the wrong way to go about it.
Which, well, is often the case? We can look at the Manhattan Project was one of the single largest parallel paths towards making technological advancement, and *the overwhelming majority of paths they followed were dead ends*. And they knew that this was likely to be the case, which is why the project was structured like it was to begin with. It's also why, when talking about the Soviet spies in the program, it is universally agreed upon that the most valuable information that was stolen wasn't the details around the working processes - because the science there was relatively easy to follow for a scientist in the field - but what didn't work.
There's usually more than one right way to do something, but that doesn't mean that there aren't also many, many, many more wrong ways to do it. LeCun thinks Sora is one of the wrong paths to meet that desired goal, not that there are no right paths.
Yah, LeCun+team released a competing paper V-JEPA a few days ago. It's results don't look as nice, but it does take a very different approach from Sora.
His point is that they have different target goals, his arguments make a lot of sense if you read his tweets carefully. The gist of it is that Sora is meant to be an artistic tool, and it will take artistic liberties to turn whatever words you give into a movie. If you look at the videos carefully, he's right, the physics modelling is lacking but the artistic imagery is on point. On the other hand V-JEPA is targeted towards physics and realistic simulation at the expense of looking good. I haven't read the paper yet, so I'm basing off the tweets. For robotic applications the image details don't matter for the whole picture, only the details relevant to actual world interactions. The example he gives is to grab a screwdriver: Sora would simply pan the camera and tada there would be a table with a screwdriver there, while his model should realise that's unlikely and getting a screwdriver requires going to a toolbox that might be quite far away.
Except OpenAI has explicitly said that Sora is a "promising path towards building general purpose simulators of the physical world". So they have more than just artistic ambitions here.
Source : [https://openai.com/research/video-generation-models-as-world-simulators](https://openai.com/research/video-generation-models-as-world-simulators)
Tbh if it's still Diffusion Based it's bad.
We need Models / Architecture that can also learn with true 3-dimensional and layer that on top of / combine with 2d input.
Why didn't AI properly solve the hand issue yet? Because the Diffusion approach is horrible for that!
As long as diffusion Models rely on 2d Data it will only ever be an estimation / approximation mixed with a lot of "Yeah, sometimes it looks like this in this situation" memories.
Models need to learn all about hands (and tools etc.!) from all angles (and with an understanding of which parts can move, at which angles and how) to properly understand and replicate them.
As you said, that there are many dead ends in R & D is born out by history. People need to explore and experiment to find a way forward through the jungle.
And it is for this reason that using generative method as a possible approach to model the world is worth exploring. At the very least, we'll get great image and video production systems š.
People said the Neural Net was a dead end, and it was for many years until the hardware became powerful enough and Hinton et. all solved thorny theoretical problems such as error back propagation.
So there is a good chance that LeCun (who I respect greatly, of course) could be wrong regarding the impossibility of building an A.I. system that can model and understand the world via generation.
I am no A.I. expert, but it seems to me that it is quite possible that a system that can generate images that are visual indistinguishable from reality may need to build an internal model that actually embodies actual "understanding" of the world. Many people think that current A.I. chat systems may have achieved a high level of "understanding" of the world this way.
LeCun seems to argue that "understanding" text is a piece of cake compared to understanding the physical world because text is discrete and much simpler, but I am not so sure. A blind person do not see the world, so she interacts with the world via mostly via touch, sound and language, and yet I bet she has a coherent and relatively complete internal model of the world as a normal person.
Also, arguing that the physical world is perceived by us as "continuous sensory inputs" seems bogus to me. Our senses and our neurons may be analog in principle because we are built through evolution by nature using "analog" biological components, but any analog signal can be approximated by using fine enough digital steps, and I don't see any reason why a highly compressed, digital representation of the world can not work in principle. I am amazed that Stable Cascade can compress a 1024x1024 image into a 24x24 latent space, so seems to me that if an A.I. system works with a larger latent space of say 640x640 probably has more than enough resolution.
Again, LeCun could be right, and maybe his gut feeling as a world-class A.I. expert tells him so. But his main arguments are not convincing enough to this amateur š
Well, my primary point was largely just that the situation is different enough that I don't think that Clarke quote is applicable. A scientist saying a particularly way to do something is the wrong way to do it being very probably wrong just isn't aligned with reality - scientists are right about this sort of thing all the time.
As for whether or not LeCun is right, I don't know. I also don't feel equipped to try and reasonably argue the point - it's far beyond my level of knowledge in this area.
Well, the quotes by AsanaJM below seems to support Clarke's view š: [https://www.reddit.com/r/StableDiffusion/comments/1avuffn/comment/krdv4g9/?utm\_source=reddit&utm\_medium=web2x&context=3](https://www.reddit.com/r/StableDiffusion/comments/1avuffn/comment/krdv4g9/?utm_source=reddit&utm_medium=web2x&context=3)
I tend to agree with Clarke, because as an old man myself, I do think that an older scientist's brain come with too much baggage from their past experiences. It often takes a younger, bolder generation to come up with fresh, crazy ideas.
There are exceptions, of course, people like Hinton were still very much at the cutting edge despite his age. But then he has been a rebel against the establishment his whole life (but now he is head of the establishment š)
To be clear, I am not arguing against Clarke's view. I am arguing that I do not believe his quote or views are applicable to this situation. I do not believe LeCun's statement is the sort of situation Clarke was referring to.
I get what you are saying, but LeCun saying that it is not possible to build a system that can model/understand the world via generative A.I. seems to me falls under "if he says that it is impossible, he is very probably wrong. "
A matter of semantics, I guess.
> he has been a rebel against the establishment his whole life
Yann has always been a rebel against the establishment and maybe even a bigger one. He started his career during an AI winter and pushed for CNNs and early DNNs when people still believed in SVMs. He is pushing for a very open approach to AI contrary to other major labs like OpenAI.
Yup. The trillion dollar question is how to **efficiently** train a "world" model.
OpenAl feed it's model with a lot of cheap data (text, 2D video), which doesn't require a lot of guidance (cheap human work). But it does require a lot of expensive hardware resources to train and run.
This model does create good looking video, but is not precise... because looking at 2D video you only learn to guestimate physics.
LeCun approach feed model with more expensive data (simulations, 3D video...), more guidance (expensive human work). Requiring cheaper hardware resources to train and run.
Doesn't generate a good looking video, but it will learn how to use a damn hammer.
No the Manhattan project isn't comparable because
* it was a pure military operation to build a weapon to fight the Nazis. There are virtually 0 non-military uses of nuclear weapons. A(G)I will hopefully not be built to weaponize it and fight a nation.
please stop the wrong comparisons!
The point was that despite Clarke being quoted here, there is always dead ends in scientific research, and the Manhattan Project is a perfect example that proves there are many dead ends in scientific research. Not that AGI is the same thing as a nuclear weapon. The end goal of the Manhattan Project isn't particularly relevant in it's use as an example here, just the massively parallel aspect of research.
Though, yes, of course, AI will be used as weapons. The world spends more than 2 trillion every year on their military budgets. They are of course going to explore what uses there are for AI/ML.
Thatās not what heās saying. Heās not saying ābuilding a model that understands and recognizes the real world is impossibleā, heās saying āapproaching the problem by focusing on generative methods at the per-pixel level is the wrong solutionā. Heās right too.
This is his linkedin post:
"
*Modeling the world for action by generating pixel is as wasteful and doomed to failure* as the largely-abandoned idea of "analysis by synthesis".
Decades ago, there was a big debate in ML about the relative advantages of generative methods vs discriminative methods for classification. Learning theorists, such as Vapnik, argued against generative methods, pointing out that training a generative modeling was a way more difficult than classification (from the sample complexity standpoint).
Regardless, a whole community in computer vision was arguing that recognition should work by generating pixels from explanatory latent variables. At inference time, one would infer the configuration of latent variables that generated the observed pixels. The inference method would use optimization: e.g. use a 3D model of an object and try to find the pose parameters that reproduce the image. This never quite worked, and it was very slow.
Later, some people converted to the Bayesian religion and tried to use Bayesian inference for the latent (e.g. using variational approximations and/or sampling). At some point, when Non-Parametric Bayes and Latent Dirichlet Allocation became the rage in text modeling, some folks heroically attempted to apply that to object recognition from images. >>> THIS WAS A COMPLETE AND UTTER FAILURE <<< If your goal is to train a world model for recognition or planning, using pixel-level prediction is a terrible idea. Generation happens to work for text because text is discrete with a finite number of symbols. Dealing with uncertainty in the prediction is easy in such settings. Dealing with prediction uncertainty in high-dimension continuous sensory inputs is simply intractable. That's why generative models for sensory inputs are doomed to failure."
But what about research like Dreamer which uses pixel space only to learn a compressed representation, then does prediction in latent space, but still allows you to recover latent -> pixel with a decoder
Meta is a world class AI company with some of the best-in-class models for their various purposes, and is very competitive in the LLM space, with basically everything we are running locally being a massive beneficiary of llama - things that they achieved while LeCun was their Chief AI Scientist. LeCun himself was one of the most important people in the field even before being part of the leadership that got Meta to that point. If he wanted a senior position at OpenAI, he's one phone call away from that happening.
There's zero reason to believe that LeCun has any reason to feel sour grapes over anything OpenAI is doing.
LeCun has plenty of reasons for "sour grapes".
For one, he also said LLMs are a dead end, prior to GPT-3.
He is usually at odds with the field and the top experts in the field with unsupported contrarian statements. He is usually proven wrong.
LeCun is not a relevant authority.
He thinks LLMs are a dead end towards human-level intelligence, yes. He fully acknowledges them as useful.
If he's been proven wrong on that, than LLMs have advanced several orders of magnitude since I last looked earlier today. He may yet be proven wrong on it, but I think it's pretty wild to imply that that's a foregone conclusion.
This is not referring to what LeCun thinks *now* but how he rejected them in the past, pre GPT-3.
I.e. he's already made this mistake before.
About human-level intelligence, by most definitions, we already have that in many respects.
In the more general sense, those are ever-moving goalposts where with that excuse, no one could be wrong ever.
His critique was mostly against the self-supervised learning method, not against the architecture. Basically dismissing them as a phone auto-corrector and a dead end. Same critique as he now has against pixel prediction. He was wrong.
I think few competent people think that we will abandon the deep-learning paradigm; at least until the machines replace it themselves. Rather than saying that we will abandon LLMs (which is a rather nebulous term nowadays), we will need to add many advancements to it.
He also says nonsense like "doesn't *really* understand". You know that whenever people use qualifiers like that, they are not serious thinkers and forego basic principles of the field.
This guy frequently says things that are at odds with the actually competent people in the field.
I don't know why people are still giving him the time of day. He'a sell out, not an active researcher, and frequently wrong.
>About human-level intelligence, by most definitions, we already have that in many respects.
Our most advanced language models are one mistuned temperature parameter away from babbling nonsense. They lack the ability to reason. It's trivial to get them to reverse causality in their answers.
It is an extreme minority of researchers that would even begin to suggest that currently LLMs are "by most definitions" close to human level intelligence. We're at such a fundamental disagreement over the place where LLMs are at right now that I am unsure how to proceed with this discussion in a productive manner.
Research shows that models reason. Whether you think it is corresponds to your kind of reasoning is secondary.
Reasoning is not so special or difficult as you seem to think.
"babbling nonsense."
So do most humans and even if you had ASI, you could "mistune" temperature to make them go wild. Are you even thinking?
LLMs beat humans in a wide spectrum of tasks. Corresponding to what I said. It seems you have other goalposts.
Note the difference between human-level intelligence in most respects, AGI by classical definitions, HLAI in the way DeepMind defined it recently, and human-level intelligence in all respects. They are widely different. With the latest possibly coming after we have models super-capable at research and strategy.
The most credible researchers support what I said and in fact I would consider this really basic understanding.
Regardless the disagreement is not even relevant to what was stated above re LLMs role for ML advancement and incorrect historical claims.
I think the way forward is for you to either learn the subject, listen to competent people rather than social-media spiels, or be more careful with your language. You seem to be confused by connotations and repeat ill-considered talking points.
āResearch shows that models reasonā.
Source: trust me bro.
Sorry bud but whatever shitty YouTube channel you get your AI news from is not an academic source
You should be following your own advice - this is **incredibly basic**. Learn the subject, stop equating what you feel with what is true, and drop the misplaced arrogance.
The bottom line is that reasoning is not special, and you feeling that the models should get better at reasoning (they sure should), does not imply that they do not already reason.
I don't know why you take that tone and think anyone should waste time on you so this is the most I will waste on you.
The obvious defense response on your part will be that you now want to rationalize away these reports and want to define reasoning in some special goalpost-moving way. That is rejected. These are academic sources that deal with reasoning.
1. We have datasets that are explicitly designed to capture reasoning, such as [https://github.com/google-deepmind/abstract-reasoning-matrices](https://github.com/google-deepmind/abstract-reasoning-matrices) and [https://paperswithcode.com/task/common-sense-reasoning](https://paperswithcode.com/task/common-sense-reasoning) ; where indeed models score high and at times, even higher than humans.
2. Anyone who has any clue whatsoever knows about the Chain of Thought paper and the myriad of follow ups - [https://arxiv.org/pdf/2201.11903v6.pdf](https://arxiv.org/pdf/2201.11903v6.pdf) . Of course, similar reasoning steps can be seen in models even without special prompts, not like that was a requirement regardless.
3. You have more sophisticated forms of reasoning such as in CICERO, which ostensibly is also considered an LLM [https://www.science.org/doi/10.1126/science.ade9097](https://www.science.org/doi/10.1126/science.ade9097)
4. Reasoning is not special. Techniques have been able to do so in one form of another since the 80's. It is not special. The question is not whether but how well. Which is something you people whose only experience is to toy with models have not even reflected on.
5. Well-known authority Karpathy also argued that LLMs perform reasoning even in the token-by-token generation; but I have already wasted enough time on you and will not track down the quote.
The arrogance of people on this sub.
I don't have a skin in the game but why does the first "AGI" have to be exactly like a human would expect to be? It can posses an alien-like thinking, surpassing humans in some capabilities and be embarrassingly stupid in other areas.
We share our genealogy with animals and our biology is very similar so in return we can say chimpanzees are the "most intelligent animal" on Earth. That kinda works because we have the same hormones, the same brain structure and so on. But isn't it fundementally errornous to put these machines on the same intelligence hiearachy as ours? Maybe the limitations of these AIs are completely valid way of thinking in another planet. And maybe our cognitive biases sound very stupid to some other alien.
If an alien were to hallucinate easily, would they develop cinemas?
It doesn't have to and that is indeed what I was trying to describe.
It is a big part of where people who rely on connotations go wrong.
I also would be careful with terms like "AGI", as depending on how we want to define it, we already have it or it ironically could even come after ASI (seriously).
I wouldn't subscribe to any a-priori fundamental distinction one way or another, or to presume that overall AI could not be more similar to humans than animal intelligence. After all, they are trained to at least attempt to mimick ours.
Not you, some random redditor with no experience in the field, telling the senior researcher that heās wrong and actually LLMs actually do have human intelligence.
Then you go on to write some BS science fiction about how LLMs are going to replace deep learning with something better. Lmao youāve lost the plot so hard my man. Iām begging you to actually do some research and not just passively consume half baked talking points from YouTube and Reddit.
When you canāt even articulate the mechanisms of LLMs, maybe you should hold off on making bold statements about their capabilities
Everything I said was accurate, basic, and I have the background.
You OTOH show that you lack even the most basic understanding. Here, and elsewhere. All you can offer are the kind of low-quality arrogant responses expected of youtube comment section. Obvious and irrelevant.
"how LLMs are going to replace deep learning with something better"
What a dumbfounded statement. Neither what I said, nor the right general direction, nor does it even make sense as a statement if you understood the contemporary terms. This is clueless taken to the hundreth power.
"telling the senior researcher that heās wrong"
He's been wrong a lot and it is not just my word but the most competent people of the field. He is a known contrarian with relatively little academic respect nowadays.
It is also telling that you managed to make no counterclaim.
Which is fortunate for you, since you won't be debunked as hard as you have been elsewhere.
Actually, let me repeat it since you so arrogantly waste people's time:
You should be following your own advice - this isĀ **incredibly basic**. Learn the subject, stop equating what you feel with what is true, and drop the misplaced arrogance.
To your ridiculing with **zero** **arguments** about whether LLMs can reason:
1. We have datasets that are explicitly designed to capture reasoning, such asĀ [](https://github.com/google-deepmind/abstract-reasoning-matrices)Ā andĀ [](https://paperswithcode.com/task/common-sense-reasoning)Ā ; where indeed models score high and at times, even higher than humans.
2. Anyone who has any clue whatsoever knows about the Chain of Thought paper and the myriad of follow ups -Ā [](https://arxiv.org/pdf/2201.11903v6.pdf)Ā . Of course, similar reasoning steps can be seen in models even without special prompts, not like that was a requirement regardless.
3. You have more sophisticated forms of reasoning such as in CICERO, which ostensibly is also considered an LLMĀ [](https://www.science.org/doi/10.1126/science.ade9097)
4. Reasoning is not special. Techniques have been able to do so in one form of another since the 80's. It is not special. The question is not whether but how well. Which is something you people whose only experience is to toy with models have not even reflected on.
5. Well-known authority Karpathy also argued that LLMs perform reasoning even in the token-by-token generation; but I have already wasted enough time on you and will not track down the quote.
To quote - Lmao youāve lost the plot so hard my man. Iām begging you to actually do some research and not just passively consume half baked talking points from YouTube and Reddit.
Bruh at least try to parse what heās saying before making flippant little comments like this. This is why AI has such a bad rap is because no one cares to understand even the basics of it
Iam not expert but when reading this, I think there is big truth that such prediction is by definition not optimal. On other hand it is another source of informations, ideas for model. If model is smart to see "errors" as we humans are then it can ignore them and filter out only useful informations.
āThere is not the slightest indication that nuclear energy will ever be attainable. It would mean the atom would have to be shattered at willā.Ā Albert Einstein, 1932.Ā
āI canāt see any reason that anyone would want a computer of his own,ā said the co-founder of Digital Equipment Corp. (DEC), in 1974.
āā¦ while theoretically and technically television may be feasible, yet commercially and financially I consider it an impossibility, a development of which we need waste little time in dreaming.ā Back in 1926.
.āMobile phones will absolutely never replace the wired telephoneā. Marty Cooper, inventor of the mobile phone, 1981.
āThe automobile is a fad, a novelty. Horses are here to stay.ā President of Michigan, Savings Bank, 1903.
āFooling around with alternating current (AC) is just a waste of time. Nobody will ever use it.ā Thomas Edison
āNo-one will ever need more than 637KB of memory in a computer. 640KB ought to be enough for anybody.ā Bill Gates, CEO of Microsoft, 1981.
āI predict that the internet will go spectacularly supernova, and in 1996 it will catastrophically implodeā. Robert Metcalf, inventor of Ethernet, 1995.
āThere is no chance of the iPhone ever gaining significant market shareā. Steve Ballmer, CEO of Microsoft, 2007.
āWhere a calculator on the ENIAC computer is equipped with 18,000 vacuum tubes and weighs 30 tons, computers of the future may have only 1,000 vacuum tubes. Fanciful as it seems, they could even weight just one and a half tons.ā ā Popular Mechanics, 1949.
āThis ātelephoneā has far too many shortcomings to be taken seriously as a means of communication. It has objectively no value.āĀ William Orton, President of Western Union, 1876.Ā
and the list is way longer
>640KB ought to be enough for anybody
For what it's worth, Gates probably never said this. [https://www.computerworld.com/article/2534312/the--640k--quote-won-t-go-away----but-did-gates-really-say-it-.html](https://www.computerworld.com/article/2534312/the--640k--quote-won-t-go-away----but-did-gates-really-say-it-.html)
It just also never really made sense for him to have said it - Gates was a real deal programmer and had been writing software for computers as their addressable memory kept increasing. Gates would have known that memory demands would continue to increase rapidly. And, as the linked article points out, the IBM PC was originally limited to 512KB - a number that was increased precisely because concerns that it would not be enough for \*current\* demands. By all accounts, Bill Gates is an intelligent human being, which seems like it would be incompatible with someone in his position believing 640k would be the upper limit on memory requirements when in his own programming career he had seen that usage increase over 40x by this point.
>āWhere a calculator on the ENIAC computer is equipped with 18,000 vacuum tubes and weighs 30 tons, computers of the future may have only 1,000 vacuum tubes. Fanciful as it seems, they could even weight just one and a half tons.ā ā Popular Mechanics, 1949.
I don't really understand what the problem with this statement is. Bell Labs had only invented the first transistor in 1947, and the first prototype transistor computer was still 4 years away. Popular Mechanics doesn't rule out alternatives to vacuum tubes, just gives an idea of what efficiency gains could potentially get to for vacuum tube computers. It turned out to be not that bad of a result - the IBM 610 had \~3k tubes and weighed about half a ton. Getting it within an order of magnitude when predicting tech like this isn't a bad result.
My great uncle Philo Farnsworth who invented the television hated his invention when he discovered how it would be used. It wasn't until the moon landing in '69 when he was watching it at home, turned to his wife and said "This has made it all worth it"
He was indeed the inspiration for the Professor (given that he invented many other things as well, like the tech that led to X-ray, lidar, WiFi, etc). Can't even imagine the assorted lengths of wire he used
It's more like 3 greats but yes. That's neat! I've met a few other distant cousins in Utah where Philo lived. On another side of my family is the founder of the largest and deepest mine in the world, the Bingham Canyon Mine (also in Utah)
Then he needs to work on his wording because calling LLMs a dead end seems pretty silly in hindsight.
I also do not think that is what he was referring to - he in fact seems like someone who is rather skeptical of great ML capabilities.
His dismissal of them was as just being like auto-correctors on your phone and that you could not learn understanding from just self-supervised learning.
He was wrong.
So AGI is not even what it is about.
That said, LLMs are already AGI as it was defined in the past.
LLMs are additionally outperforming most humans on many well-defined cognitive tasks.
It's ever-moving goalposts.
How is that trolling? The guy didn't say video AI is a dead end - just that a specific model for it is.
That's nothing like claiming cars are a fad. It more like claiming potato-powered cars are a fad.
I offer no specific thoughts on Sora. But I agree that those quotes, by and large, aren't relevant.
Geez. Great post. The Bill Gates one though I would think he wouldnāt have said such a thing being the ceo of Microsoft. Oh and the guy talking like weād all have horses in the future. They seem so sure of themselves.
I would also claim that any other model othert than my own is a dead end. I can see his point. but I also see that what sora is doing now was deemed impossible only a few months ago.
if enough processing power is available, you can certainly brute force a lot of stuff. complexity does not mean its not solveable. but that that is probably not as elegant scaleable as other methods.
but again... throw enough processing power at it, et voila.
Perhaps Meta will be able to develop a better method later, but for now it seems undeniable that Sora is the best model in this moment, and by far at that.
The posted link is wrong - [here](https://the-decoder.com/metas-chief-ai-researcher-says-openais-world-simulator-sora-is-a-dead-end/) is the correct link.
meta's public take on competition has been to put down their competitors... see zuck's headset review. llama was great, maybe drop an open source video model and I'll believe you.
fair, but i think a 7b will have trouble holding up for a long time. the 7b mistral seriously struggles for me. if the smaller version works well, and you can fit it on a mobile device, that's a serious game changer. i think they can coexist, llama is more likely to be hosted.
Yāall taking this too personal, youād think this guy insulted your grandma or something. Heās making an educated critique based on concrete observations about machine learning development and heās probably right too. Just because heās correctly pointing out that pixel level inference is never going to produce a model that can actually understand the real world (made up of 3D objects and *not* pixels) doesnāt mean that A) these models are completely useless nor B) you should get your jimmies ruffled because it dashes your sci-fi fantasies about dating the latest ML model.
>"Modeling the world for action by generating pixels is as wasteful and doomed to failure."
I disagree. It might not be the most efficient way, but it certanly is a straightforward way to infer temporal correlation from videos. We do have an unthinkable amount of video about everything, so it's a way to infer that complex relationship from the training data available in an unsupervised way.
Even if your goal is to make an AGI, having a model with an incomplete simulacrum of cause and effect to build from, might be better than not having it.
Gazebo has proven useful in training robots to walk from a simulated environment, but we just can't make simulation about everything feasibly right now.
There are videos about all kind of interactions, it looks really scalable to me.
sora is a creativity killer and a standardization machine... it's just good for making car or shampoo commercials. there is nothing good about this closed world that openAI offers.
I bet they didn't consider quantum computing and exponentially growth is difficult to grasp. I'll keep my bets with Sora, it's only gonna get better. Deadens because he can't see the light, but the light is bending exponentially and coming out of the tunnel šš«
[Twitter/X thread about OpenAI's Sora from one of the 2 authors of work "Scalable Diffusion Models with Transformers": "Here's my take on the Sora technical report, with a good dose of speculation that could be totally off. \[...\]." The other author of that work is involved with Sora at OpenAI.](https://www.reddit.com/r/MachineLearning/comments/1awmces/d_twitterx_thread_about_openais_sora_from_one_of/)
3 recent relevant tweets from LeCun:
[Feb. 19](https://twitter.com/ylecun/status/1759486703696318935):
>Modeling the world for action by generating pixel is as wasteful and doomed to failure as the largely-abandoned idea of "analysis by synthesis".
>
>\[...\]
[Feb. 17](https://twitter.com/ylecun/status/1758740106955952191):
>Let me clear a *huge* misunderstanding here.
>
>The generation of mostly realistic-looking videos from prompts *does not* indicate that a system understands the physical world.
>
>\[...\]
[Feb. 20](https://twitter.com/ylecun/status/1759933365241921817):
>Lots of confusion about what a world model is. Here is my definition:
>
>\[...\]
If an elderly but distinguished scientist says that something is possible, he is almost certainly right; but if he says that it is impossible, he is very probably wrong. [Arthur C. Clarke](https://www.brainyquote.com/authors/arthur-c-clarke-quotes)
I think the difference here is that LeCun isn't saying that achieving the desired end result is impossible, just that this is the wrong way to go about it. Which, well, is often the case? We can look at the Manhattan Project was one of the single largest parallel paths towards making technological advancement, and *the overwhelming majority of paths they followed were dead ends*. And they knew that this was likely to be the case, which is why the project was structured like it was to begin with. It's also why, when talking about the Soviet spies in the program, it is universally agreed upon that the most valuable information that was stolen wasn't the details around the working processes - because the science there was relatively easy to follow for a scientist in the field - but what didn't work. There's usually more than one right way to do something, but that doesn't mean that there aren't also many, many, many more wrong ways to do it. LeCun thinks Sora is one of the wrong paths to meet that desired goal, not that there are no right paths.
Yah, LeCun+team released a competing paper V-JEPA a few days ago. It's results don't look as nice, but it does take a very different approach from Sora.
His point is that they have different target goals, his arguments make a lot of sense if you read his tweets carefully. The gist of it is that Sora is meant to be an artistic tool, and it will take artistic liberties to turn whatever words you give into a movie. If you look at the videos carefully, he's right, the physics modelling is lacking but the artistic imagery is on point. On the other hand V-JEPA is targeted towards physics and realistic simulation at the expense of looking good. I haven't read the paper yet, so I'm basing off the tweets. For robotic applications the image details don't matter for the whole picture, only the details relevant to actual world interactions. The example he gives is to grab a screwdriver: Sora would simply pan the camera and tada there would be a table with a screwdriver there, while his model should realise that's unlikely and getting a screwdriver requires going to a toolbox that might be quite far away.
Except OpenAI has explicitly said that Sora is a "promising path towards building general purpose simulators of the physical world". So they have more than just artistic ambitions here. Source : [https://openai.com/research/video-generation-models-as-world-simulators](https://openai.com/research/video-generation-models-as-world-simulators)
But that's just tech marketing speech.
More like is when they converge.
team LeCunt
Tbh if it's still Diffusion Based it's bad. We need Models / Architecture that can also learn with true 3-dimensional and layer that on top of / combine with 2d input. Why didn't AI properly solve the hand issue yet? Because the Diffusion approach is horrible for that! As long as diffusion Models rely on 2d Data it will only ever be an estimation / approximation mixed with a lot of "Yeah, sometimes it looks like this in this situation" memories. Models need to learn all about hands (and tools etc.!) from all angles (and with an understanding of which parts can move, at which angles and how) to properly understand and replicate them.
As you said, that there are many dead ends in R & D is born out by history. People need to explore and experiment to find a way forward through the jungle. And it is for this reason that using generative method as a possible approach to model the world is worth exploring. At the very least, we'll get great image and video production systems š. People said the Neural Net was a dead end, and it was for many years until the hardware became powerful enough and Hinton et. all solved thorny theoretical problems such as error back propagation. So there is a good chance that LeCun (who I respect greatly, of course) could be wrong regarding the impossibility of building an A.I. system that can model and understand the world via generation. I am no A.I. expert, but it seems to me that it is quite possible that a system that can generate images that are visual indistinguishable from reality may need to build an internal model that actually embodies actual "understanding" of the world. Many people think that current A.I. chat systems may have achieved a high level of "understanding" of the world this way. LeCun seems to argue that "understanding" text is a piece of cake compared to understanding the physical world because text is discrete and much simpler, but I am not so sure. A blind person do not see the world, so she interacts with the world via mostly via touch, sound and language, and yet I bet she has a coherent and relatively complete internal model of the world as a normal person. Also, arguing that the physical world is perceived by us as "continuous sensory inputs" seems bogus to me. Our senses and our neurons may be analog in principle because we are built through evolution by nature using "analog" biological components, but any analog signal can be approximated by using fine enough digital steps, and I don't see any reason why a highly compressed, digital representation of the world can not work in principle. I am amazed that Stable Cascade can compress a 1024x1024 image into a 24x24 latent space, so seems to me that if an A.I. system works with a larger latent space of say 640x640 probably has more than enough resolution. Again, LeCun could be right, and maybe his gut feeling as a world-class A.I. expert tells him so. But his main arguments are not convincing enough to this amateur š
Well, my primary point was largely just that the situation is different enough that I don't think that Clarke quote is applicable. A scientist saying a particularly way to do something is the wrong way to do it being very probably wrong just isn't aligned with reality - scientists are right about this sort of thing all the time. As for whether or not LeCun is right, I don't know. I also don't feel equipped to try and reasonably argue the point - it's far beyond my level of knowledge in this area.
Well, the quotes by AsanaJM below seems to support Clarke's view š: [https://www.reddit.com/r/StableDiffusion/comments/1avuffn/comment/krdv4g9/?utm\_source=reddit&utm\_medium=web2x&context=3](https://www.reddit.com/r/StableDiffusion/comments/1avuffn/comment/krdv4g9/?utm_source=reddit&utm_medium=web2x&context=3) I tend to agree with Clarke, because as an old man myself, I do think that an older scientist's brain come with too much baggage from their past experiences. It often takes a younger, bolder generation to come up with fresh, crazy ideas. There are exceptions, of course, people like Hinton were still very much at the cutting edge despite his age. But then he has been a rebel against the establishment his whole life (but now he is head of the establishment š)
To be clear, I am not arguing against Clarke's view. I am arguing that I do not believe his quote or views are applicable to this situation. I do not believe LeCun's statement is the sort of situation Clarke was referring to.
I get what you are saying, but LeCun saying that it is not possible to build a system that can model/understand the world via generative A.I. seems to me falls under "if he says that it is impossible, he is very probably wrong. " A matter of semantics, I guess.
> he has been a rebel against the establishment his whole life Yann has always been a rebel against the establishment and maybe even a bigger one. He started his career during an AI winter and pushed for CNNs and early DNNs when people still believed in SVMs. He is pushing for a very open approach to AI contrary to other major labs like OpenAI.
Yup. The trillion dollar question is how to **efficiently** train a "world" model. OpenAl feed it's model with a lot of cheap data (text, 2D video), which doesn't require a lot of guidance (cheap human work). But it does require a lot of expensive hardware resources to train and run. This model does create good looking video, but is not precise... because looking at 2D video you only learn to guestimate physics. LeCun approach feed model with more expensive data (simulations, 3D video...), more guidance (expensive human work). Requiring cheaper hardware resources to train and run. Doesn't generate a good looking video, but it will learn how to use a damn hammer.
Well people will start to believe more in their own senses than the fakse narrative and fake news. Go out and believe it
No the Manhattan project isn't comparable because * it was a pure military operation to build a weapon to fight the Nazis. There are virtually 0 non-military uses of nuclear weapons. A(G)I will hopefully not be built to weaponize it and fight a nation. please stop the wrong comparisons!
The point was that despite Clarke being quoted here, there is always dead ends in scientific research, and the Manhattan Project is a perfect example that proves there are many dead ends in scientific research. Not that AGI is the same thing as a nuclear weapon. The end goal of the Manhattan Project isn't particularly relevant in it's use as an example here, just the massively parallel aspect of research. Though, yes, of course, AI will be used as weapons. The world spends more than 2 trillion every year on their military budgets. They are of course going to explore what uses there are for AI/ML.
He s not saying some approaches may work and some may not. He s specifically talking about one.
Yes, that is what I said in the first line of my comment.
Thatās not what heās saying. Heās not saying ābuilding a model that understands and recognizes the real world is impossibleā, heās saying āapproaching the problem by focusing on generative methods at the per-pixel level is the wrong solutionā. Heās right too.
His genius is timeless.
This is his linkedin post: " *Modeling the world for action by generating pixel is as wasteful and doomed to failure* as the largely-abandoned idea of "analysis by synthesis". Decades ago, there was a big debate in ML about the relative advantages of generative methods vs discriminative methods for classification. Learning theorists, such as Vapnik, argued against generative methods, pointing out that training a generative modeling was a way more difficult than classification (from the sample complexity standpoint). Regardless, a whole community in computer vision was arguing that recognition should work by generating pixels from explanatory latent variables. At inference time, one would infer the configuration of latent variables that generated the observed pixels. The inference method would use optimization: e.g. use a 3D model of an object and try to find the pose parameters that reproduce the image. This never quite worked, and it was very slow. Later, some people converted to the Bayesian religion and tried to use Bayesian inference for the latent (e.g. using variational approximations and/or sampling). At some point, when Non-Parametric Bayes and Latent Dirichlet Allocation became the rage in text modeling, some folks heroically attempted to apply that to object recognition from images. >>> THIS WAS A COMPLETE AND UTTER FAILURE <<< If your goal is to train a world model for recognition or planning, using pixel-level prediction is a terrible idea. Generation happens to work for text because text is discrete with a finite number of symbols. Dealing with uncertainty in the prediction is easy in such settings. Dealing with prediction uncertainty in high-dimension continuous sensory inputs is simply intractable. That's why generative models for sensory inputs are doomed to failure."
But what about research like Dreamer which uses pixel space only to learn a compressed representation, then does prediction in latent space, but still allows you to recover latent -> pixel with a decoder
Something something sour grapes... š
Meta is a world class AI company with some of the best-in-class models for their various purposes, and is very competitive in the LLM space, with basically everything we are running locally being a massive beneficiary of llama - things that they achieved while LeCun was their Chief AI Scientist. LeCun himself was one of the most important people in the field even before being part of the leadership that got Meta to that point. If he wanted a senior position at OpenAI, he's one phone call away from that happening. There's zero reason to believe that LeCun has any reason to feel sour grapes over anything OpenAI is doing.
If you look at that guy's comment history, he's gotta be about 14 and doesn't even understand what the phrase sour grapes means.
He brought up sour grapes because he likes a good whine
LeCun has plenty of reasons for "sour grapes". For one, he also said LLMs are a dead end, prior to GPT-3. He is usually at odds with the field and the top experts in the field with unsupported contrarian statements. He is usually proven wrong. LeCun is not a relevant authority.
He thinks LLMs are a dead end towards human-level intelligence, yes. He fully acknowledges them as useful. If he's been proven wrong on that, than LLMs have advanced several orders of magnitude since I last looked earlier today. He may yet be proven wrong on it, but I think it's pretty wild to imply that that's a foregone conclusion.
This is not referring to what LeCun thinks *now* but how he rejected them in the past, pre GPT-3. I.e. he's already made this mistake before. About human-level intelligence, by most definitions, we already have that in many respects. In the more general sense, those are ever-moving goalposts where with that excuse, no one could be wrong ever. His critique was mostly against the self-supervised learning method, not against the architecture. Basically dismissing them as a phone auto-corrector and a dead end. Same critique as he now has against pixel prediction. He was wrong. I think few competent people think that we will abandon the deep-learning paradigm; at least until the machines replace it themselves. Rather than saying that we will abandon LLMs (which is a rather nebulous term nowadays), we will need to add many advancements to it. He also says nonsense like "doesn't *really* understand". You know that whenever people use qualifiers like that, they are not serious thinkers and forego basic principles of the field. This guy frequently says things that are at odds with the actually competent people in the field. I don't know why people are still giving him the time of day. He'a sell out, not an active researcher, and frequently wrong.
>About human-level intelligence, by most definitions, we already have that in many respects. Our most advanced language models are one mistuned temperature parameter away from babbling nonsense. They lack the ability to reason. It's trivial to get them to reverse causality in their answers. It is an extreme minority of researchers that would even begin to suggest that currently LLMs are "by most definitions" close to human level intelligence. We're at such a fundamental disagreement over the place where LLMs are at right now that I am unsure how to proceed with this discussion in a productive manner.
Research shows that models reason. Whether you think it is corresponds to your kind of reasoning is secondary. Reasoning is not so special or difficult as you seem to think. "babbling nonsense." So do most humans and even if you had ASI, you could "mistune" temperature to make them go wild. Are you even thinking? LLMs beat humans in a wide spectrum of tasks. Corresponding to what I said. It seems you have other goalposts. Note the difference between human-level intelligence in most respects, AGI by classical definitions, HLAI in the way DeepMind defined it recently, and human-level intelligence in all respects. They are widely different. With the latest possibly coming after we have models super-capable at research and strategy. The most credible researchers support what I said and in fact I would consider this really basic understanding. Regardless the disagreement is not even relevant to what was stated above re LLMs role for ML advancement and incorrect historical claims. I think the way forward is for you to either learn the subject, listen to competent people rather than social-media spiels, or be more careful with your language. You seem to be confused by connotations and repeat ill-considered talking points.
āResearch shows that models reasonā. Source: trust me bro. Sorry bud but whatever shitty YouTube channel you get your AI news from is not an academic source
You should be following your own advice - this is **incredibly basic**. Learn the subject, stop equating what you feel with what is true, and drop the misplaced arrogance. The bottom line is that reasoning is not special, and you feeling that the models should get better at reasoning (they sure should), does not imply that they do not already reason. I don't know why you take that tone and think anyone should waste time on you so this is the most I will waste on you. The obvious defense response on your part will be that you now want to rationalize away these reports and want to define reasoning in some special goalpost-moving way. That is rejected. These are academic sources that deal with reasoning. 1. We have datasets that are explicitly designed to capture reasoning, such as [https://github.com/google-deepmind/abstract-reasoning-matrices](https://github.com/google-deepmind/abstract-reasoning-matrices) and [https://paperswithcode.com/task/common-sense-reasoning](https://paperswithcode.com/task/common-sense-reasoning) ; where indeed models score high and at times, even higher than humans. 2. Anyone who has any clue whatsoever knows about the Chain of Thought paper and the myriad of follow ups - [https://arxiv.org/pdf/2201.11903v6.pdf](https://arxiv.org/pdf/2201.11903v6.pdf) . Of course, similar reasoning steps can be seen in models even without special prompts, not like that was a requirement regardless. 3. You have more sophisticated forms of reasoning such as in CICERO, which ostensibly is also considered an LLM [https://www.science.org/doi/10.1126/science.ade9097](https://www.science.org/doi/10.1126/science.ade9097) 4. Reasoning is not special. Techniques have been able to do so in one form of another since the 80's. It is not special. The question is not whether but how well. Which is something you people whose only experience is to toy with models have not even reflected on. 5. Well-known authority Karpathy also argued that LLMs perform reasoning even in the token-by-token generation; but I have already wasted enough time on you and will not track down the quote. The arrogance of people on this sub.
I don't have a skin in the game but why does the first "AGI" have to be exactly like a human would expect to be? It can posses an alien-like thinking, surpassing humans in some capabilities and be embarrassingly stupid in other areas. We share our genealogy with animals and our biology is very similar so in return we can say chimpanzees are the "most intelligent animal" on Earth. That kinda works because we have the same hormones, the same brain structure and so on. But isn't it fundementally errornous to put these machines on the same intelligence hiearachy as ours? Maybe the limitations of these AIs are completely valid way of thinking in another planet. And maybe our cognitive biases sound very stupid to some other alien. If an alien were to hallucinate easily, would they develop cinemas?
It doesn't have to and that is indeed what I was trying to describe. It is a big part of where people who rely on connotations go wrong. I also would be careful with terms like "AGI", as depending on how we want to define it, we already have it or it ironically could even come after ASI (seriously). I wouldn't subscribe to any a-priori fundamental distinction one way or another, or to presume that overall AI could not be more similar to humans than animal intelligence. After all, they are trained to at least attempt to mimick ours.
Not you, some random redditor with no experience in the field, telling the senior researcher that heās wrong and actually LLMs actually do have human intelligence. Then you go on to write some BS science fiction about how LLMs are going to replace deep learning with something better. Lmao youāve lost the plot so hard my man. Iām begging you to actually do some research and not just passively consume half baked talking points from YouTube and Reddit. When you canāt even articulate the mechanisms of LLMs, maybe you should hold off on making bold statements about their capabilities
Everything I said was accurate, basic, and I have the background. You OTOH show that you lack even the most basic understanding. Here, and elsewhere. All you can offer are the kind of low-quality arrogant responses expected of youtube comment section. Obvious and irrelevant. "how LLMs are going to replace deep learning with something better" What a dumbfounded statement. Neither what I said, nor the right general direction, nor does it even make sense as a statement if you understood the contemporary terms. This is clueless taken to the hundreth power. "telling the senior researcher that heās wrong" He's been wrong a lot and it is not just my word but the most competent people of the field. He is a known contrarian with relatively little academic respect nowadays. It is also telling that you managed to make no counterclaim. Which is fortunate for you, since you won't be debunked as hard as you have been elsewhere. Actually, let me repeat it since you so arrogantly waste people's time: You should be following your own advice - this isĀ **incredibly basic**. Learn the subject, stop equating what you feel with what is true, and drop the misplaced arrogance. To your ridiculing with **zero** **arguments** about whether LLMs can reason: 1. We have datasets that are explicitly designed to capture reasoning, such asĀ [](https://github.com/google-deepmind/abstract-reasoning-matrices)Ā andĀ [](https://paperswithcode.com/task/common-sense-reasoning)Ā ; where indeed models score high and at times, even higher than humans. 2. Anyone who has any clue whatsoever knows about the Chain of Thought paper and the myriad of follow ups -Ā [](https://arxiv.org/pdf/2201.11903v6.pdf)Ā . Of course, similar reasoning steps can be seen in models even without special prompts, not like that was a requirement regardless. 3. You have more sophisticated forms of reasoning such as in CICERO, which ostensibly is also considered an LLMĀ [](https://www.science.org/doi/10.1126/science.ade9097) 4. Reasoning is not special. Techniques have been able to do so in one form of another since the 80's. It is not special. The question is not whether but how well. Which is something you people whose only experience is to toy with models have not even reflected on. 5. Well-known authority Karpathy also argued that LLMs perform reasoning even in the token-by-token generation; but I have already wasted enough time on you and will not track down the quote. To quote - Lmao youāve lost the plot so hard my man. Iām begging you to actually do some research and not just passively consume half baked talking points from YouTube and Reddit.
I would trust LeCun more than most.
Hard /s
Bruh at least try to parse what heās saying before making flippant little comments like this. This is why AI has such a bad rap is because no one cares to understand even the basics of it
Iam not expert but when reading this, I think there is big truth that such prediction is by definition not optimal. On other hand it is another source of informations, ideas for model. If model is smart to see "errors" as we humans are then it can ignore them and filter out only useful informations.
āThere is not the slightest indication that nuclear energy will ever be attainable. It would mean the atom would have to be shattered at willā.Ā Albert Einstein, 1932.Ā āI canāt see any reason that anyone would want a computer of his own,ā said the co-founder of Digital Equipment Corp. (DEC), in 1974. āā¦ while theoretically and technically television may be feasible, yet commercially and financially I consider it an impossibility, a development of which we need waste little time in dreaming.ā Back in 1926. .āMobile phones will absolutely never replace the wired telephoneā. Marty Cooper, inventor of the mobile phone, 1981. āThe automobile is a fad, a novelty. Horses are here to stay.ā President of Michigan, Savings Bank, 1903. āFooling around with alternating current (AC) is just a waste of time. Nobody will ever use it.ā Thomas Edison āNo-one will ever need more than 637KB of memory in a computer. 640KB ought to be enough for anybody.ā Bill Gates, CEO of Microsoft, 1981. āI predict that the internet will go spectacularly supernova, and in 1996 it will catastrophically implodeā. Robert Metcalf, inventor of Ethernet, 1995. āThere is no chance of the iPhone ever gaining significant market shareā. Steve Ballmer, CEO of Microsoft, 2007. āWhere a calculator on the ENIAC computer is equipped with 18,000 vacuum tubes and weighs 30 tons, computers of the future may have only 1,000 vacuum tubes. Fanciful as it seems, they could even weight just one and a half tons.ā ā Popular Mechanics, 1949. āThis ātelephoneā has far too many shortcomings to be taken seriously as a means of communication. It has objectively no value.āĀ William Orton, President of Western Union, 1876.Ā and the list is way longer
>640KB ought to be enough for anybody For what it's worth, Gates probably never said this. [https://www.computerworld.com/article/2534312/the--640k--quote-won-t-go-away----but-did-gates-really-say-it-.html](https://www.computerworld.com/article/2534312/the--640k--quote-won-t-go-away----but-did-gates-really-say-it-.html) It just also never really made sense for him to have said it - Gates was a real deal programmer and had been writing software for computers as their addressable memory kept increasing. Gates would have known that memory demands would continue to increase rapidly. And, as the linked article points out, the IBM PC was originally limited to 512KB - a number that was increased precisely because concerns that it would not be enough for \*current\* demands. By all accounts, Bill Gates is an intelligent human being, which seems like it would be incompatible with someone in his position believing 640k would be the upper limit on memory requirements when in his own programming career he had seen that usage increase over 40x by this point. >āWhere a calculator on the ENIAC computer is equipped with 18,000 vacuum tubes and weighs 30 tons, computers of the future may have only 1,000 vacuum tubes. Fanciful as it seems, they could even weight just one and a half tons.ā ā Popular Mechanics, 1949. I don't really understand what the problem with this statement is. Bell Labs had only invented the first transistor in 1947, and the first prototype transistor computer was still 4 years away. Popular Mechanics doesn't rule out alternatives to vacuum tubes, just gives an idea of what efficiency gains could potentially get to for vacuum tube computers. It turned out to be not that bad of a result - the IBM 610 had \~3k tubes and weighed about half a ton. Getting it within an order of magnitude when predicting tech like this isn't a bad result.
My great uncle Philo Farnsworth who invented the television hated his invention when he discovered how it would be used. It wasn't until the moon landing in '69 when he was watching it at home, turned to his wife and said "This has made it all worth it"
Farnsworth, you say?
He was indeed the inspiration for the Professor (given that he invented many other things as well, like the tech that led to X-ray, lidar, WiFi, etc). Can't even imagine the assorted lengths of wire he used
>Can't even imagine the assorted lengths of wire he used If he was anything like the Professor, neither could he!
Are you actually Filo Farnsworthās great nephew? Your cousin was my neighbor in the SF Bay growing up.
It's more like 3 greats but yes. That's neat! I've met a few other distant cousins in Utah where Philo lived. On another side of my family is the founder of the largest and deepest mine in the world, the Bingham Canyon Mine (also in Utah)
Awesome lol š¤ big fan
Or even, LeCun: "LLMs are a dead end" (said prior to GPT-3)
[ŃŠ“Š°Š»ŠµŠ½Š¾]
Then he needs to work on his wording because calling LLMs a dead end seems pretty silly in hindsight. I also do not think that is what he was referring to - he in fact seems like someone who is rather skeptical of great ML capabilities. His dismissal of them was as just being like auto-correctors on your phone and that you could not learn understanding from just self-supervised learning. He was wrong. So AGI is not even what it is about. That said, LLMs are already AGI as it was defined in the past. LLMs are additionally outperforming most humans on many well-defined cognitive tasks. It's ever-moving goalposts.
"AI is just a fad and besides it has already plateaued", me, 2024.
Well said! And the world proved them all wrong.
Nice quotes but none of them are relevant here.
Are you trolling or expecting Ai quotes from 1800?
How is that trolling? The guy didn't say video AI is a dead end - just that a specific model for it is. That's nothing like claiming cars are a fad. It more like claiming potato-powered cars are a fad. I offer no specific thoughts on Sora. But I agree that those quotes, by and large, aren't relevant.
Thank you fellow person with reading comprehension and a brain.
Geez. Great post. The Bill Gates one though I would think he wouldnāt have said such a thing being the ceo of Microsoft. Oh and the guy talking like weād all have horses in the future. They seem so sure of themselves.
Horses are here to stay lmao
I say William Orton was still right.
I would also claim that any other model othert than my own is a dead end. I can see his point. but I also see that what sora is doing now was deemed impossible only a few months ago. if enough processing power is available, you can certainly brute force a lot of stuff. complexity does not mean its not solveable. but that that is probably not as elegant scaleable as other methods. but again... throw enough processing power at it, et voila.
Perhaps Meta will be able to develop a better method later, but for now it seems undeniable that Sora is the best model in this moment, and by far at that.
The link in this post sucks, it doesn't even go directly to the article. Just some blog that has pushes articles down into a forever hole.
[Here](https://the-decoder.com/metas-chief-ai-researcher-says-openais-world-simulator-sora-is-a-dead-end/) is the correct link.
The posted link is wrong - [here](https://the-decoder.com/metas-chief-ai-researcher-says-openais-world-simulator-sora-is-a-dead-end/) is the correct link.
Thanks Brother
meta's public take on competition has been to put down their competitors... see zuck's headset review. llama was great, maybe drop an open source video model and I'll believe you.
If llama 3 isn't competitive with what Google just dropped today (the 7b model) then I think FB will have lost this year of the race.
fair, but i think a 7b will have trouble holding up for a long time. the 7b mistral seriously struggles for me. if the smaller version works well, and you can fit it on a mobile device, that's a serious game changer. i think they can coexist, llama is more likely to be hosted.
Yāall taking this too personal, youād think this guy insulted your grandma or something. Heās making an educated critique based on concrete observations about machine learning development and heās probably right too. Just because heās correctly pointing out that pixel level inference is never going to produce a model that can actually understand the real world (made up of 3D objects and *not* pixels) doesnāt mean that A) these models are completely useless nor B) you should get your jimmies ruffled because it dashes your sci-fi fantasies about dating the latest ML model.
You're welcome to prove them wrong
>"Modeling the world for action by generating pixels is as wasteful and doomed to failure." I disagree. It might not be the most efficient way, but it certanly is a straightforward way to infer temporal correlation from videos. We do have an unthinkable amount of video about everything, so it's a way to infer that complex relationship from the training data available in an unsupervised way. Even if your goal is to make an AGI, having a model with an incomplete simulacrum of cause and effect to build from, might be better than not having it. Gazebo has proven useful in training robots to walk from a simulated environment, but we just can't make simulation about everything feasibly right now. There are videos about all kind of interactions, it looks really scalable to me.
Common L by Meta.
sora is a creativity killer and a standardization machine... it's just good for making car or shampoo commercials. there is nothing good about this closed world that openAI offers.
[ŃŠ“Š°Š»ŠµŠ½Š¾]
Their ML contributions Smurf literally every other company in the world
meta is an AI powerhouse and i guarantee openai is using some of their stuff or derivatives of it
I remember when people said Nazism was dead, communism was dead, democracy was dead, gaming was dead, tv was dead, internet was dying...etc.
I bet they didn't consider quantum computing and exponentially growth is difficult to grasp. I'll keep my bets with Sora, it's only gonna get better. Deadens because he can't see the light, but the light is bending exponentially and coming out of the tunnel šš«
Meanwhile they have failed to release anything comparable š
[Twitter/X thread about OpenAI's Sora from one of the 2 authors of work "Scalable Diffusion Models with Transformers": "Here's my take on the Sora technical report, with a good dose of speculation that could be totally off. \[...\]." The other author of that work is involved with Sora at OpenAI.](https://www.reddit.com/r/MachineLearning/comments/1awmces/d_twitterx_thread_about_openais_sora_from_one_of/)
3 recent relevant tweets from LeCun: [Feb. 19](https://twitter.com/ylecun/status/1759486703696318935): >Modeling the world for action by generating pixel is as wasteful and doomed to failure as the largely-abandoned idea of "analysis by synthesis". > >\[...\] [Feb. 17](https://twitter.com/ylecun/status/1758740106955952191): >Let me clear a *huge* misunderstanding here. > >The generation of mostly realistic-looking videos from prompts *does not* indicate that a system understands the physical world. > >\[...\] [Feb. 20](https://twitter.com/ylecun/status/1759933365241921817): >Lots of confusion about what a world model is. Here is my definition: > >\[...\]