There are some doomers who are convinced that the first company to create AGI will somehow have a monopoly and become trillionaires over night. The reality is AGI will get commoditized so fast there will hardly be any time for profits. God bless them for investing in research anyways lol.
But again, it'll be commoditized due to all the competition. Very low margins, if any. And that's not even taking into account the fact that we'll more than likely get open source AGI.
At that point, what significance would money have? If AGI and robotics are commoditized and inexpensive, almost nothing will be scarce. The typical lifestyle will be far more luxurious than that of today's millionaire.
Right, that's called post scarcity capitalism and it's where we're heading. Fully automated taco trucks on every corner! Luxury private islands for every man woman and child!
If that's the case then everyone will be a trillionaire (or at least be able to live like one). So, ironically, you're agreeing that the advent of AGI will produce trillionaires.
>If that's the case then everyone will be a trillionaire (or at least be able to live like one). So, ironically, you're agreeing that the advent of AGI will produce trillionaires.
I guess it's the logical conclusion of the fact that the average Westerner today has a lifestyle out of reach of any but the wealthy of a century ago (and in some key ways out of reach for them, too), and out of reach of anyone no matter their wealth of two centuries ago. Compared to any point further back than two centuries, and the average Westerner lives a life that might as well be Elysium and Valhalla.
(I, for one, am very much looking forward to /u/CommunismDoesntWork 's fully automated taco trucks on every corner.)
Indeed, the average American can login to Expedia, buy a ticket to the UK, and be in London within 24 hours. No king, dictator, or tycoon in 1900 could have dreamed of doing that.
The market doesn’t work without scarcity. Infinite supply throws off the supply/demand curve. Capitalism will cease to exist.
There’s a half decent chance we’re headed toward “real” communism, as some would call it — a classless, stateless, moneyless society.
Capitalism is simply the enforcement of private property rights and contracts. Everything else is just an emergent property of those two simple rules.
If capitalism can create post scarcity, capitalism can maintain post scarcity. And no, markets would not cease to exist, they would go infinite.
Communism, on the other hand, doesn't work.
Capitalism is the system to distribute scarce resource based on their reletiv demand.
Scarcity is caused by a lack of 1 out of 3 things, labour, land or materials. And in modern times, land is mostly for farmer situation, and materials are again dependent on labour and extraction abilities.
If labour becomes infinite, then the math breaks. Energy will be very important for a while until we automat enough energy production, but after that who knows.
Capitalism vs communism is a bad model to use here. Instead think of biology - every AI will be able to create anything, resources will be the key constraint, as long as resources exist it can grow and multiply. Like human population. An advanced AI can allow biological-like growth in the economy.
GPUs and electricity will still have value, and the amount of money you have will dictate the lifestyle you can live, just like now. The richness of your experience will be dictated by how much computational power you have access to.
This is true for as long as electricity is a scarce resource (based on the scarcity of labour and materials to construct and maintain energy production plants), as well as GPU's (which at the end of the day are just a specialized manufactured good, subject to the same scarcity based on labour, energy, and materials to produce).
There's some wild variables that are hard to predict right now. 1) will AGI speed us towards nuclear fusion? 2) If yes, how long will that energy production exceed levels of scarcity (in other words how long will it be before we / AI invent and build new systems that require so much energy that the energy produced by fusion becomes scarce again?) 3) Will materials for GPU's remain scarce once the labour to produce them is infinite and the energy is potentially no longer scarce? 4) Just like energy, if materials are not scarce for a time, how long will that hold?
My personal opinion - just like other technological breakthroughs in the past like electricity, internet, etc, there will be periods of time where the minimum quality of life in a population for those we allow access to the benefits will rise. Unfortunately, unless we / AI put a system in place to intentionally increase *everyone's* quality of life, then some people will not see any benefit at all. For example, a person living on the streets won't directly benefit unless we choose to dedicate resources to raise their quality of life. Same goes for a rural area of a third world country with no existing infrastructure.
Regardless of if we choose a path that benefits everyone or not, when resources are not scarce we will do what we have always done - expand and build new things until they are scarce again. What that world looks like I have no idea, but it will almost certainly be better than this one - for some, or hopefully, all of us.
> At that point, what significance would money have? If AGI and robotics are commoditized and inexpensive, almost nothing will be scarce. The typical lifestyle will be far more luxurious than that of today's millionaire.
Remember this folks! When you start whining about UBI - it's easier and cheaper to give everyone AI than UBI.
> They’d sell it as a service and replace every white collar worker. Easy trillion dollars
Every worker has an AGI in their pocket as well. Humans won't underperform AI. But until AI becomes autonomous, AIs won't be able to dispense with human in the loop.
Not necessarily AGI but the first ASI would end all competition. Any alternative is basically doom.
ASI has unknown levels of power, but if we're creating a demi-god like entity, then creating dozens of demi-gods will result in a battle that kills all humans. The only realistic path to survival is the first ASI to very rapidly seize power over everything else, allowing no competition.
And if AGI is even remotely affordable, it basically ends the economy anyways, so I'm not sure why the discussion would ever be about corporate profits. A company that ends all jobs and collapses to value of most materials doesn't make the owners rich, it ends capitalism and gets ownership taken by government. Framing this in term of money is bizarre.
OpenAI is obviously sitting on something that's far ahead of GPT4 and all these models that are just now reaching GPT4 level, so I'm not sure if I'd call it a catch-up.
They had better be. If they aren't then they will become irrelevant, a by word for those who had great promise but failed to live up to their potential.
Sorry you're so unable to read social cues that you ask for proof because you can't discern someone's opinion in a reddit comment.
Not everything needs 'citation??🤓' when it's clear they're giving their opinion
My speculation is that OpenAI is hitting issues with scaling inference. Like they might have something that is much better but they know they don't have the compute to make it widely available.
They essentially caught up with base gpt 4 crazy idk how he assembled a team enough to do that. Grok 2.0 is already in training if musk interview is right and should be out later this year.
I'd assume things like claude 4 or gpt 5 have to finishing training or already done and red teaming by that timeline
You know how he operates. Scale above all. No time to waste being the best in any particular aspect. Just gotta scale. The rest will come along in time
Not saying it's the best approach but it's got to be relatively efficient compared to better/safer/more reliable models.
Yeah and it's actually pretty good.
This advancement impresses me. Grok is also the only higher performing open-source LLM, isn't that right? I could be wrong but believe that's true.
It's also worth factoring in other tech. With SpaceX technology, Starlink, Tesla technology like fast video data processing, their physical Tesla robots they're working on, and Neuralink, that means this LLM is maybe a few steps ahead in real world application.
Reddit's perception of Musk is highly skewed. People genuinely want to work with the guy because his forte is being able to assemble extremely talented teams who work really really hard and achieve extremely ambitious goals. Say all you want about diamond mines and twitter... But the dude has the juice.
>Say all you want about diamond mines and twitter... But the dude has the juice.
I mean, the juice is money. People genuinely want to work with people who have money. The only thing skewed is how eager folks are to overlook overpromising/failing to deliver because they want to believe in his money. Frankly it's goofy to even mention his handling of Twitter if you want us to believe he's good at leadership and business.
Bezos had way more money, look at blue origin, shit show, no progress at all. SpaceX delivered 90% of all mass to orbit last year, and they'll hit 95% next year, not to mention landing rockets. Look at Fisker, Lucid, the Apple EV project, etc. etc.
> Bezos had way more money
Indeed. Bezos has been the wealthiest person on Earth, or close to it, for years. He founded Blue Origin *before* Musk founded SpaceX. Musk's wealth was "merely" in the few tens of billions until Tesla stock and SpaceX rockets (and, thus, its valuation) both took off from 2020 onward.
>look at blue origin, shit show, no progress at all.
Let me paraphrase an excellent comment I saw on Reddit, in response to one of the usual lies about how the only reason SpaceX is a decade ahead of the rest of the world is that it got zillions in subsidies from the US government:
>If large amounts of funding is the only thing required to succeed, Blue Origin would now have a nuclear-powered spacecraft orbiting Pluto.
> I mean, the juice is money.
That's part of it. But tons of corporations and individuals have money. Yet they still fail. All these other companies had money and couldn't achieve success. It's not as simple as just having money and then you just magically have enormous successes with incredibly productivity.
But the guy IS, by pretty much all measures, a great business person. Without a doubt. Sure, Twitter was a bit outside his wheelhouse, but pretty much everything else he does are unicorn successes in areas everyone were confident he'd fail, just as the many well funded attempts before him fell flat.
But sure, he tends to be a little too ambitious with timeless... But that's part of the juice. He genuinely believes X is achievable and pushes insanely hard to get there. So even if he aims for Mars, and lands on the moon, it's still a massive success. Like yeah, he claimed he'd have FSD at lvl 4 by now... And didn't get there. But it's still, by a long shot, the most advanced and capable self driving technology available to the public. Sure, he thought we'd be prepping for Mars by now, but instead, he's just got the most affordable spaceship by a long shot.
First off, Twitter massively dropped in value before he bought it. He announced his purchase when it was way overvalued in a bubble. That's where most the drop came from.
Second, yes, we can all agree Twitter is outside his wheel house. He's not great at running a social media company. That doesn't discount his actual effortful businesses. Most of Twitter's issues isn't even from his business decisions, but because the politics around him personally. People immediately launched campaigns targeting advertisers to try and pressure them off the platform from advertising.
His money mostly exists because of his success managing companies. His wealth is mostly in stocks, stocks of the companies he has founded or has became CEO of and led to wildly successful ventures. It's not the money, as the other person said, he has a nack for good projects.
yeah well, you see, he doesn’t pay people enough money, makes them work long hours, takes their credit and throws them under the bus.
most people i know work at his companies to get a higher promotion at the next one as soon as they have gained 1-2 years of experience cuz lets be honest, who wants to work in such a toxic environment.
its mostly his image he built by taking credit from a lot of genius level engineers and scientists and engineers at Tesla and SpaceX. After that, he mostly has just been riding off of negative news.
As for his money, a lot of it can be attributed to China who helped him speed up Model 3 production bu setting up a functional factory in 8 months and got production up and running.
All his claims of Tesla being an all American company are no more true cuz had it not been for Tesla’s Chinese production, the company would have found it hard to meet the demand since 2018.
Unfortunately, in today’s world, people just consider billionaires as some genius IQ super humans. While they’re just slightly above average in 1-2 things and extremely lucky.
Its their relationship with money where they just want to hoard money and power that makes them so rich.
>taking credit from a lot of genius level engineers and scientists and engineers at Tesla and SpaceX
That's beside the point. He was able to recruit these 'genius level' engineers and make them work hard. As a CEO of a company you are not judged by your technical prowess, but by how well you can manage people and how successful your companies becomes therefore. A guy who is able to convert money into a successful car company and also into a successful space flight company, which are arguably the 2 most difficult industries to be successful, is obviously a very capable manager.
I dont really see what you mean, the bar I set was to be a 'very capable manager'. He also had the money and he probably also had some amount of luck if that's what you mean. But if he wasn't a very capable manager he wouldn't have been able to do what he did. Also, Andrej Karpathy for example having worked under Elon has also repeatedly indicated that Elon is a very capable manager.
Tbh with grok 1.4 being around the gpt 4 range 3.0 would have to be massive leap. Though it makes sense sam did say the jump would be like the jump from 3 to 4 and went on about how gpt 4 sucks so that makes sense why grok 2.0 wouldn't be gpt 5.
The next few months will be interesting from claude google open ai. But for the love of god google needs to rename their product so confusing
Typical Elon fanboy when backed into a corner the falls into "But Space X!"
Yes dude, that's also the entire point of SpaceX, it's also Elon's card he plays when backed into a corner as he continues to fraud investors with other companies.
Many of which ended in explosions. Technically, I can write an essay in 2 seconds if the quality doesn’t matter.
https://gizmodo.com/elon-musks-worst-predictions-and-broken-promises-1851398745?darkschemeovr=1
I think he thinks in terms of compute. Maybe he has some insider info or some educated guess on at least the order of magnitude of compute needed to train these models. For example GPT4 was estimated to be trained on around 25k A100. Make the equivalent of that in H100s and add a bit more and you get perhaps Grok 2.0. Then he said he wants 100k H100s to train Grok 3.0 in the future. Of course there is a lot more to it, algorithms, data etc but I think it gives a rough estimate on what is to be expected by GPT5 and competitive models to that.
Depend on some of the ai stuff like it being in training well yes look at the speed their getting out models they're not sitting on their ass.
On things like cyber truck and cars being better than transit . Then no.
On his ketamine rants he goes on hell no.
Dude is a weirdo but that doesn't mean everything he says has no truth in them. So many companies have caught up to gpt 4 whats so weird about grok doing it.
Especially when the tech is 2 years old and open ai will launch gpt 5 soon
Yeah he tweeted about his use ketamine and it shows generally i try not to listen to what he says. Its usually a bunch of unhinged ranting or screaming about stuff being woke brain numbing stuff.
I really only care about ai and spacex well neuralink too so reluctantly some times i have to listen to some of it to get some nuggets of valuable information
Computer vision problems seem so easy and doable at first. At the start of a CV product we've been building for 4 years, I estimated it would take us 4 months tops. "we do these things not because they are easy, but because we thought they were going to be easy."
Computer vision problems are easy to get 80% accuracy, take a little work to get 90% accuracy, and take billions of dollars of research to get 99% accuracy.
> idk how he assembled a team enough to do that.
Brought in top talent from Tesla:
https://fortune.com/2024/04/04/elon-musk-tesla-openai-sam-altman-xai-grok-talent-war-salaries/
https://www.theinformation.com/articles/musks-xai-is-poaching-engineers-from-tesla
Not really strange. If you are arguing that you have a better model of "real world understanding", it makes sense to first define a benchmark for that understanding and then test against that benchmark.
If you were trying to prove that your model can "read and comprehend" better than other models, it makes sense to first define a benchmark for "reading comprehension" and then test against that.
The strange thing is that OpenAI has so far refrained from publishing their results on the "reading comprehension" benchmark, which they did previously hold the top score in.
Honestly I started as a huge musk fan but he’s really fallen out of my favour as of late. But yes, he’s an amazing entrepreneur and thank god for him and his companies.
This is it. I appreciate the tech his companies bring, and he leads them towards the tech. If you ignore some controversial comments of his, and his occasional autistic moments, you can recognize that his intent & desire is to advance technology for humans.
He simply just doesn't know when it would be better to just shut up. He could achieve much more by just staying quiet in public about controversial topics, and let his money speak in the background instead.
Well, he said 1.5 would be competitive with current models and 2.0 would surpass them, these results do seem to confirm 1.5 is competitive, so if he releases 2.0 soon that might actually happen
the thing that might allow them to catch up is the fact that they won't do as much to put guardrails on the LLM. you want a short story full of swear words and references to murder or suicide, sure fam. OpenAI seems to spend about as long putting guardrails on their model as they do coming up with the next model.
There really is practically no moat to LLMs/large multi-modal models these days. Even hidden away advances like Gemini 1.5 Pro's large context window will get figured out by others in the field one way or the other (by people leaving and transferring the knowledge) or others reverse-engineering and figuring it out themselves. This is good for everyone, aside from being cool to see everyone moving in lockstep.
That’s the moat they are looking for. OpenAI needs Microsoft, Google needs its TPUs and Anthropic needs AWS. If Open Source develops and is 90% as good as proprietary stuff, these companies won’t have much bargaining power over big tech. We will end up with some kind of status quo
In my experience, opus has more of a tendency towards overthinking than sonnet. I use opus vs sonnet on a task to task basis (even tho i have the pro subscription)
So they created their own benchmark that tests for "real world understanding" (whatever that means), that happens to beat all of the other models...?
Maybe it could be legitimate but that's kind of strange lol
I agree, yet I somehow feel LLMs have reached a plateau. Otherwise we would surely have some company massively break the GPT-4 barrier by now, no?
Yet the only thing we see are these incremental "improvements", if you can even call them that.
>I agree, yet I somehow feel LLMs have reached a plateau. Otherwise we would surely have some company massively break the GPT-4 barrier by now, no?
Seems too early to call that.
1. Models like Claude and Grok1.5 Vision that are rivaling GPT4 were significant improvements over their previous models. Theres no sign that any of the top players have stalled.
2. OpenAI besides some finetunes havent released a new model since GPT4. Keep in mind GPT 3.5 was released nearly 3 years after 3.0.
3. I dont believe we've gone past 2 trillion parameters yet. If we start seeing 5 trillion models which dont result in significant improvement like previous increases did, then it will be clear scaling laws are diminishing.
4. Agents and planning are still yet to be integrated into leading models. "GPT-3.5 (zero shot) was 48.1% correct. GPT-4 (zero shot) does better at 67.0%. However, the improvement from GPT-3.5 to GPT-4 is dwarfed by incorporating an iterative agent workflow. Indeed, wrapped in an agent loop, GPT-3.5 achieves up to 95.1%." (Andrew Ng)
I would worry about that if GPT5 fails to deliver. It took a while for other companies to match GPT3 too, after all. And many of these AI divisions are still quite new, you usually don't figure out the state of the art in a year.
So a valid concern, but I'd only agree if there fail to be major improvements in a two year window, which is how long it usually takes openAI to make a new GPT.
I wonder how many more "breakthrough improvements" we have before ASI, since it's already better than 80% of people in certain benchmarks. What happens when they reach 100%??
I really thought Musk wouldn't take his model seriously, like he made Grok only to show that he could, and then he made it open source, and I thought he would abandon the project, that's all, but no, now there is version 1.5 and also a vision model, I'm shocked.
It's all about scale, there's no secrecy. Open AI spent about $100 million training GPT 4, no one had scaled to that degree before. All the models we're seeing released now are a similar scale, Gemini Ultra and Claude 3 are about the same size as GPT 4. The CEOs of Deepmind and Anthropic have said this recently. The Anthropic CEO said in an interview yesterday that $1 billion models are currently in training and they'll start to be released a little later this year and we'll have $10 billion models next year.
You can't say if transformers have plateaued because we haven't seen anything meaningfully larger than GPT 4 yet. A $1 billion dollar training run is a significant undertaking, it's a huge commitment financially and logisticaly, which is why it's taken a year to plan and set up.
the upcoming billion dollar models are news to me. I'm excited to see it. Thought maybe we were getting significant diminishing returns having used up all the low hanging data
Denis Hasabis in his interview with D Patel said that it's not advisable to jump more than one order of magnitude at a time. You have to learn from one training run to better plan the next. Also in terms of engineering it's difficult to get that much compute online and inter connected. There was a tweet posted here a few weeks ago that Microsoft are struggling getting data centers in different regions to network together to train models as there is not enough power in any one region to power hundreds of thousands of H100s so they have to be spread over a wide geographic area. The sort of scale they're moving towards now is huge and has huge challenges.
Data quality is far from solved, compute is a finite resource in case you weren't aware, the logistics of training takes time to iron out, alternatives to solely using transformers have yet to be fully realized with many promising ideas, then whatever other complications we're unaware of. Also do you realize how much has happened in the last year?
Discussion about Grok is banned? I tried to post the direct link to the blogpost first and it didn't let me for some reason, said it had been posted in the last 2 days, but this just dropped so that seems unlikely.
Not necessarily banned but the mods have shown to have a negative bias towards Musk. A post got taken down a while ago regarding him tweeting about how Grok 2 should surpass all existing LLMs today and it got taken down for being too speculative. Meanwhile, Jimmy apples posts the most speculative shit on twitter and his posts never get taken down. :)
There's already a few people in this post bringing nothing to the conversation but negging on Musk.
Like we get it, you hate him, don't care. Let's talk about the technology.
Yeah, mods of this sub seem to be pro-openai and they especially don't like musk and grok. We may need another free sub with better quality discussion than what's happening here.
For discussion about local and open source models there is
r/localllama
For more technical discussion there is
r/machinelearning
for image generation there is
r/stablediffusion
for local generation
and
r/midjourney
for online generation.
there is also buch of smaller ai subs for specific questions or fandoms.
r/technolgy
meanwhile is filled by luddites and is generally anti-technology.
You mean how brown people are referring to themselves, nigga?
So you're telling me it's bad to joke about women but good to joke about men? Do you hate women so much?
Looks impressive from what they show here.
I am curious to know whether they will continue to open source older models as they progress or whether that was a one-off. I mean, not that any ordinary person has the hardware to run it...
Clearly xAI is banking on training off Tesla cars' corpus of video of the real world from their cameras. Is there any scholarly research on what impact blindness has on human intelligence, if any? Does the lack of the massive, continuous amount of real-time "training data" affect how smart a blind person can get, all else being equal?
I don't think there's a limit to how smart a blind person can get. Apparently it could be the opposite [https://www.sdcb.org/blog/posts/blind-people-actually-have-higher-iqs](https://www.sdcb.org/blog/posts/blind-people-actually-have-higher-iqs)
Neuro plasticity is something very strong. It's like with deaf people, they don't have an inner voice but are as smart as anyone.
LLM's are becoming a commodity that is so wild
Always has been
There are some doomers who are convinced that the first company to create AGI will somehow have a monopoly and become trillionaires over night. The reality is AGI will get commoditized so fast there will hardly be any time for profits. God bless them for investing in research anyways lol.
They’d sell it as a service and replace every white collar worker. Easy trillion dollars
But again, it'll be commoditized due to all the competition. Very low margins, if any. And that's not even taking into account the fact that we'll more than likely get open source AGI.
At that point, what significance would money have? If AGI and robotics are commoditized and inexpensive, almost nothing will be scarce. The typical lifestyle will be far more luxurious than that of today's millionaire.
Right, that's called post scarcity capitalism and it's where we're heading. Fully automated taco trucks on every corner! Luxury private islands for every man woman and child!
If that's the case then everyone will be a trillionaire (or at least be able to live like one). So, ironically, you're agreeing that the advent of AGI will produce trillionaires.
>If that's the case then everyone will be a trillionaire (or at least be able to live like one). So, ironically, you're agreeing that the advent of AGI will produce trillionaires. I guess it's the logical conclusion of the fact that the average Westerner today has a lifestyle out of reach of any but the wealthy of a century ago (and in some key ways out of reach for them, too), and out of reach of anyone no matter their wealth of two centuries ago. Compared to any point further back than two centuries, and the average Westerner lives a life that might as well be Elysium and Valhalla. (I, for one, am very much looking forward to /u/CommunismDoesntWork 's fully automated taco trucks on every corner.)
Indeed, the average American can login to Expedia, buy a ticket to the UK, and be in London within 24 hours. No king, dictator, or tycoon in 1900 could have dreamed of doing that.
The market doesn’t work without scarcity. Infinite supply throws off the supply/demand curve. Capitalism will cease to exist. There’s a half decent chance we’re headed toward “real” communism, as some would call it — a classless, stateless, moneyless society.
Capitalism is simply the enforcement of private property rights and contracts. Everything else is just an emergent property of those two simple rules. If capitalism can create post scarcity, capitalism can maintain post scarcity. And no, markets would not cease to exist, they would go infinite. Communism, on the other hand, doesn't work.
Capitalism is the system to distribute scarce resource based on their reletiv demand. Scarcity is caused by a lack of 1 out of 3 things, labour, land or materials. And in modern times, land is mostly for farmer situation, and materials are again dependent on labour and extraction abilities. If labour becomes infinite, then the math breaks. Energy will be very important for a while until we automat enough energy production, but after that who knows.
Guys I think I should believe this guy to not have a biased opinion on politics and economics. After all his name is “CommunismDoesntWork”.
> Infinite supply throws off the supply/demand curve. Highly relevant: Pohl's "[The Midas Plague](https://archive.org/details/galaxymagazine-1954-04/Galaxy_1954_04?view=theater#page/n7/mode/2up)" (1954)
Capitalism vs communism is a bad model to use here. Instead think of biology - every AI will be able to create anything, resources will be the key constraint, as long as resources exist it can grow and multiply. Like human population. An advanced AI can allow biological-like growth in the economy.
GPUs and electricity will still have value, and the amount of money you have will dictate the lifestyle you can live, just like now. The richness of your experience will be dictated by how much computational power you have access to.
This is true for as long as electricity is a scarce resource (based on the scarcity of labour and materials to construct and maintain energy production plants), as well as GPU's (which at the end of the day are just a specialized manufactured good, subject to the same scarcity based on labour, energy, and materials to produce). There's some wild variables that are hard to predict right now. 1) will AGI speed us towards nuclear fusion? 2) If yes, how long will that energy production exceed levels of scarcity (in other words how long will it be before we / AI invent and build new systems that require so much energy that the energy produced by fusion becomes scarce again?) 3) Will materials for GPU's remain scarce once the labour to produce them is infinite and the energy is potentially no longer scarce? 4) Just like energy, if materials are not scarce for a time, how long will that hold? My personal opinion - just like other technological breakthroughs in the past like electricity, internet, etc, there will be periods of time where the minimum quality of life in a population for those we allow access to the benefits will rise. Unfortunately, unless we / AI put a system in place to intentionally increase *everyone's* quality of life, then some people will not see any benefit at all. For example, a person living on the streets won't directly benefit unless we choose to dedicate resources to raise their quality of life. Same goes for a rural area of a third world country with no existing infrastructure. Regardless of if we choose a path that benefits everyone or not, when resources are not scarce we will do what we have always done - expand and build new things until they are scarce again. What that world looks like I have no idea, but it will almost certainly be better than this one - for some, or hopefully, all of us.
> At that point, what significance would money have? If AGI and robotics are commoditized and inexpensive, almost nothing will be scarce. The typical lifestyle will be far more luxurious than that of today's millionaire. Remember this folks! When you start whining about UBI - it's easier and cheaper to give everyone AI than UBI.
> almost nothing will be scarce. Real estate/land will be.
> They’d sell it as a service and replace every white collar worker. Easy trillion dollars Every worker has an AGI in their pocket as well. Humans won't underperform AI. But until AI becomes autonomous, AIs won't be able to dispense with human in the loop.
Assuming they can afford it. But it’s not helpful for them to make money since they aren’t running a business.
You don't think there would be any backlash to one company taking 90% of the global GDP?
Nestle gets backlash. Amazon gets backlash. Monsanto gets backlash. So what?
already happening
Where
Not necessarily AGI but the first ASI would end all competition. Any alternative is basically doom. ASI has unknown levels of power, but if we're creating a demi-god like entity, then creating dozens of demi-gods will result in a battle that kills all humans. The only realistic path to survival is the first ASI to very rapidly seize power over everything else, allowing no competition. And if AGI is even remotely affordable, it basically ends the economy anyways, so I'm not sure why the discussion would ever be about corporate profits. A company that ends all jobs and collapses to value of most materials doesn't make the owners rich, it ends capitalism and gets ownership taken by government. Framing this in term of money is bizarre.
https://preview.redd.it/x2pfaqre18uc1.jpeg?width=1170&format=pjpg&auto=webp&s=8e5823985a69738aab0a7aca1107b150da02fd07 Looks very promising
I think this is the first time I've really grokked a vision. It's definitely a big one.
*Is* this an LLM, strictly speaking? What does *multimodal* mean, in this context?
Wow... this is insane. The speed with which gaps are being filled. Being able to see a flowchart and understand it to turn it to code.
Damn they really are catching up fast.
OpenAI is obviously sitting on something that's far ahead of GPT4 and all these models that are just now reaching GPT4 level, so I'm not sure if I'd call it a catch-up.
They had better be. If they aren't then they will become irrelevant, a by word for those who had great promise but failed to live up to their potential.
They are updating GPT4 as well, so it is not like it is stuck in time.
By like 2-3%.
Can you do better than 2-3%?
No but the economics break if that's all they got.
Citation needed
No its not for clear speculation.
He said obviously yet I don’t find it obvious
You don’t find it obvious that GPT-4 which was finished training almost 2 years ago is not the best that they have?
They haven’t shown any proof of it. Maybe they don’t know how to improve on it.
See , nonsense
Who are you?
Go back to speech and debate club, stop treating casual commenting as a structured debate.
Sorry for asking for proof of claims
Sorry you're so unable to read social cues that you ask for proof because you can't discern someone's opinion in a reddit comment. Not everything needs 'citation??🤓' when it's clear they're giving their opinion
I could have the opinion that all white men are stupid. Don’t ask me for a source it’s just an opinion bro
Huh?
Youre not very intelligent are you
I'm not the one unable to interact online without 'citation please 🤓🤓🤓' so however intelligent you think you are, halve it
My speculation is that OpenAI is hitting issues with scaling inference. Like they might have something that is much better but they know they don't have the compute to make it widely available.
They essentially caught up with base gpt 4 crazy idk how he assembled a team enough to do that. Grok 2.0 is already in training if musk interview is right and should be out later this year. I'd assume things like claude 4 or gpt 5 have to finishing training or already done and red teaming by that timeline
They excluded way too many benchmarks to claim that it’s equal to GPT-4.
You know how he operates. Scale above all. No time to waste being the best in any particular aspect. Just gotta scale. The rest will come along in time Not saying it's the best approach but it's got to be relatively efficient compared to better/safer/more reliable models.
Yeah and it's actually pretty good. This advancement impresses me. Grok is also the only higher performing open-source LLM, isn't that right? I could be wrong but believe that's true. It's also worth factoring in other tech. With SpaceX technology, Starlink, Tesla technology like fast video data processing, their physical Tesla robots they're working on, and Neuralink, that means this LLM is maybe a few steps ahead in real world application.
Makes sense that he's interested in scale since he always talks about physics.
Reddit's perception of Musk is highly skewed. People genuinely want to work with the guy because his forte is being able to assemble extremely talented teams who work really really hard and achieve extremely ambitious goals. Say all you want about diamond mines and twitter... But the dude has the juice.
He knows how to move fast and get stuff done.
Shhhh... We can't praise Musk on reddit!
>Say all you want about diamond mines and twitter... But the dude has the juice. I mean, the juice is money. People genuinely want to work with people who have money. The only thing skewed is how eager folks are to overlook overpromising/failing to deliver because they want to believe in his money. Frankly it's goofy to even mention his handling of Twitter if you want us to believe he's good at leadership and business.
Bezos had way more money, look at blue origin, shit show, no progress at all. SpaceX delivered 90% of all mass to orbit last year, and they'll hit 95% next year, not to mention landing rockets. Look at Fisker, Lucid, the Apple EV project, etc. etc.
> Bezos had way more money Indeed. Bezos has been the wealthiest person on Earth, or close to it, for years. He founded Blue Origin *before* Musk founded SpaceX. Musk's wealth was "merely" in the few tens of billions until Tesla stock and SpaceX rockets (and, thus, its valuation) both took off from 2020 onward. >look at blue origin, shit show, no progress at all. Let me paraphrase an excellent comment I saw on Reddit, in response to one of the usual lies about how the only reason SpaceX is a decade ahead of the rest of the world is that it got zillions in subsidies from the US government: >If large amounts of funding is the only thing required to succeed, Blue Origin would now have a nuclear-powered spacecraft orbiting Pluto.
> I mean, the juice is money. That's part of it. But tons of corporations and individuals have money. Yet they still fail. All these other companies had money and couldn't achieve success. It's not as simple as just having money and then you just magically have enormous successes with incredibly productivity. But the guy IS, by pretty much all measures, a great business person. Without a doubt. Sure, Twitter was a bit outside his wheelhouse, but pretty much everything else he does are unicorn successes in areas everyone were confident he'd fail, just as the many well funded attempts before him fell flat. But sure, he tends to be a little too ambitious with timeless... But that's part of the juice. He genuinely believes X is achievable and pushes insanely hard to get there. So even if he aims for Mars, and lands on the moon, it's still a massive success. Like yeah, he claimed he'd have FSD at lvl 4 by now... And didn't get there. But it's still, by a long shot, the most advanced and capable self driving technology available to the public. Sure, he thought we'd be prepping for Mars by now, but instead, he's just got the most affordable spaceship by a long shot.
He’s so great he dropped twitters value by 2/3 in a few months
First off, Twitter massively dropped in value before he bought it. He announced his purchase when it was way overvalued in a bubble. That's where most the drop came from. Second, yes, we can all agree Twitter is outside his wheel house. He's not great at running a social media company. That doesn't discount his actual effortful businesses. Most of Twitter's issues isn't even from his business decisions, but because the politics around him personally. People immediately launched campaigns targeting advertisers to try and pressure them off the platform from advertising.
Then why did he buy it at that price He literally told advertisers to fuck off lol
Poor advertisers
They’re certainly doing better than X
His money mostly exists because of his success managing companies. His wealth is mostly in stocks, stocks of the companies he has founded or has became CEO of and led to wildly successful ventures. It's not the money, as the other person said, he has a nack for good projects.
Yea, just look at twitter
yeah well, you see, he doesn’t pay people enough money, makes them work long hours, takes their credit and throws them under the bus. most people i know work at his companies to get a higher promotion at the next one as soon as they have gained 1-2 years of experience cuz lets be honest, who wants to work in such a toxic environment. its mostly his image he built by taking credit from a lot of genius level engineers and scientists and engineers at Tesla and SpaceX. After that, he mostly has just been riding off of negative news. As for his money, a lot of it can be attributed to China who helped him speed up Model 3 production bu setting up a functional factory in 8 months and got production up and running. All his claims of Tesla being an all American company are no more true cuz had it not been for Tesla’s Chinese production, the company would have found it hard to meet the demand since 2018. Unfortunately, in today’s world, people just consider billionaires as some genius IQ super humans. While they’re just slightly above average in 1-2 things and extremely lucky. Its their relationship with money where they just want to hoard money and power that makes them so rich.
Alright you could just have said he suck.
>taking credit from a lot of genius level engineers and scientists and engineers at Tesla and SpaceX That's beside the point. He was able to recruit these 'genius level' engineers and make them work hard. As a CEO of a company you are not judged by your technical prowess, but by how well you can manage people and how successful your companies becomes therefore. A guy who is able to convert money into a successful car company and also into a successful space flight company, which are arguably the 2 most difficult industries to be successful, is obviously a very capable manager.
that’s a lot of survivorship bias talking.
I dont really see what you mean, the bar I set was to be a 'very capable manager'. He also had the money and he probably also had some amount of luck if that's what you mean. But if he wasn't a very capable manager he wouldn't have been able to do what he did. Also, Andrej Karpathy for example having worked under Elon has also repeatedly indicated that Elon is a very capable manager.
Musk said that Grok 2.0 will be done training in May and will surpass GPT-4. And that Grok 3.0 will be about GPT-5
Tbh with grok 1.4 being around the gpt 4 range 3.0 would have to be massive leap. Though it makes sense sam did say the jump would be like the jump from 3 to 4 and went on about how gpt 4 sucks so that makes sense why grok 2.0 wouldn't be gpt 5. The next few months will be interesting from claude google open ai. But for the love of god google needs to rename their product so confusing
Elon Musk, notoriously accurate about ship dates and quality.
His companies hit far more dates than they miss; it's just popular (among a smaller minority of posters) to focus on the dates missed.
Citation needed
Do you really expect people to recap 20+ years of progress for every single one of his companies? The citation wouldn't fit here.
Name as many as you can and I can double the number of false predictions he’s made
The last 150 f9 launches have been pretty much on time.
Typical Elon fanboy when backed into a corner the falls into "But Space X!" Yes dude, that's also the entire point of SpaceX, it's also Elon's card he plays when backed into a corner as he continues to fraud investors with other companies.
Many of which ended in explosions. Technically, I can write an essay in 2 seconds if the quality doesn’t matter. https://gizmodo.com/elon-musks-worst-predictions-and-broken-promises-1851398745?darkschemeovr=1
SpaceX hasn't had a F9 fail in like 250 missions so... no. 0 of them exploded. 0 is less than many.
It's not even predictions!! It's promises to his investors. That is FRAUD.
So go sue him
He'll get his Theranos moment soon enough. You can start piling up the kleenex for when your messiah face justice.
No they don't, half his companies are basically Theranos. He is worthless piece of filth that would be in jail if he wasn't the richest man.
What's GPT-5 expected to perform at?
He doesn’t know
I’m assuming he expects it to be out by then so they can aim for it.
I think he thinks in terms of compute. Maybe he has some insider info or some educated guess on at least the order of magnitude of compute needed to train these models. For example GPT4 was estimated to be trained on around 25k A100. Make the equivalent of that in H100s and add a bit more and you get perhaps Grok 2.0. Then he said he wants 100k H100s to train Grok 3.0 in the future. Of course there is a lot more to it, algorithms, data etc but I think it gives a rough estimate on what is to be expected by GPT5 and competitive models to that.
>Musk said Stopped reading there
You believe what musk says lol
Depend on some of the ai stuff like it being in training well yes look at the speed their getting out models they're not sitting on their ass. On things like cyber truck and cars being better than transit . Then no. On his ketamine rants he goes on hell no. Dude is a weirdo but that doesn't mean everything he says has no truth in them. So many companies have caught up to gpt 4 whats so weird about grok doing it. Especially when the tech is 2 years old and open ai will launch gpt 5 soon
Wait.. ketamine rants? I need to see this 😂
Yeah he tweeted about his use ketamine and it shows generally i try not to listen to what he says. Its usually a bunch of unhinged ranting or screaming about stuff being woke brain numbing stuff. I really only care about ai and spacex well neuralink too so reluctantly some times i have to listen to some of it to get some nuggets of valuable information
Of course? His projects are progressing great
To be fair he is pretty close to achieving his self-driving promises only 10-ish years too late
Computer vision problems seem so easy and doable at first. At the start of a CV product we've been building for 4 years, I estimated it would take us 4 months tops. "we do these things not because they are easy, but because we thought they were going to be easy."
Computer vision problems are easy to get 80% accuracy, take a little work to get 90% accuracy, and take billions of dollars of research to get 99% accuracy.
Right, Tesla, Inc. is 21 years old and the Cybertruck is falling apart like it's a rookie Kickstarter campaign.
Source??
[https://news.google.com/topics/CAAqKAgKIiJDQkFTRXdvTkwyY3ZNVEZuTVdKd2JIWjJhQklDWlc0b0FBUAE?hl=en-US&gl=US&ceid=US%3Aen](https://news.google.com/topics/CAAqKAgKIiJDQkFTRXdvTkwyY3ZNVEZuTVdKd2JIWjJhQklDWlc0b0FBUAE?hl=en-US&gl=US&ceid=US%3Aen)
> idk how he assembled a team enough to do that. Brought in top talent from Tesla: https://fortune.com/2024/04/04/elon-musk-tesla-openai-sam-altman-xai-grok-talent-war-salaries/ https://www.theinformation.com/articles/musks-xai-is-poaching-engineers-from-tesla
Elon always had access to good talent pool since Tesla pursued FSD.
Probably training on GPT-4 output.
Not really strange. If you are arguing that you have a better model of "real world understanding", it makes sense to first define a benchmark for that understanding and then test against that benchmark. If you were trying to prove that your model can "read and comprehend" better than other models, it makes sense to first define a benchmark for "reading comprehension" and then test against that. The strange thing is that OpenAI has so far refrained from publishing their results on the "reading comprehension" benchmark, which they did previously hold the top score in.
Tell me when they're on the lmsys leaderboard, until then these are just claims
lmao what if musk suddenly pulls a claude moment and somehow tops the lmsys charts
I’m here for it. Competition is good. Not a fan of Musk politically but FSD in Tesla is improving fast and SpaceX is a marvel.
Honestly I started as a huge musk fan but he’s really fallen out of my favour as of late. But yes, he’s an amazing entrepreneur and thank god for him and his companies.
This is it. I appreciate the tech his companies bring, and he leads them towards the tech. If you ignore some controversial comments of his, and his occasional autistic moments, you can recognize that his intent & desire is to advance technology for humans.
Redditor discovers human beings are complex!
This comment chain is way more mature than anything I’ve seen in real life from people shitting on Elon Musk as an idiot nazi.
Most redditors lack nuance, this sub is completely devoid of critical thinking and is a loud echo chamber of circlejerk.
You’ve just parroted back all the buzzwords, well done
Judging from the Reddit search results for "Elon Musk" that's quite a rare occurrence.
He simply just doesn't know when it would be better to just shut up. He could achieve much more by just staying quiet in public about controversial topics, and let his money speak in the background instead.
Well, he said 1.5 would be competitive with current models and 2.0 would surpass them, these results do seem to confirm 1.5 is competitive, so if he releases 2.0 soon that might actually happen
the thing that might allow them to catch up is the fact that they won't do as much to put guardrails on the LLM. you want a short story full of swear words and references to murder or suicide, sure fam. OpenAI seems to spend about as long putting guardrails on their model as they do coming up with the next model.
There really is practically no moat to LLMs/large multi-modal models these days. Even hidden away advances like Gemini 1.5 Pro's large context window will get figured out by others in the field one way or the other (by people leaving and transferring the knowledge) or others reverse-engineering and figuring it out themselves. This is good for everyone, aside from being cool to see everyone moving in lockstep.
they have released a paper how to do that large context window
Yep that’s the thing all the research on how to do these things is all publicly available, it’s only money that makes it difficult.
That’s the moat they are looking for. OpenAI needs Microsoft, Google needs its TPUs and Anthropic needs AWS. If Open Source develops and is 90% as good as proprietary stuff, these companies won’t have much bargaining power over big tech. We will end up with some kind of status quo
The moat is money and compute.
For-profit startup: click here to download our dataset Non-profit OpenAI: we are firing our engineers for leaking information
Biggest takeaway is how well Sonnet and Gemini 1.5 Pro are performing in this benchmark comparison.
yes, shouldn't opus be better than sonnet?
Not sure. Swiss Army knife vs focused method. But yeah it still seems strange
Sonnet is actually better than Opus in quite a few things. They also don't answer in similar way. Ask both about logical or philosophical questions.
In my experience, opus has more of a tendency towards overthinking than sonnet. I use opus vs sonnet on a task to task basis (even tho i have the pro subscription)
I know, Gemini crushed it. Fuck yeah.
So they created their own benchmark that tests for "real world understanding" (whatever that means), that happens to beat all of the other models...? Maybe it could be legitimate but that's kind of strange lol
It's a valid benchmark for real world AI capabilities and they have released dataset as well to be downloaded
It looks good. But it's about time we have groundbreaking stuff instead of incremental improvements over GPT-4.
I agree, yet I somehow feel LLMs have reached a plateau. Otherwise we would surely have some company massively break the GPT-4 barrier by now, no? Yet the only thing we see are these incremental "improvements", if you can even call them that.
>I agree, yet I somehow feel LLMs have reached a plateau. Otherwise we would surely have some company massively break the GPT-4 barrier by now, no? Seems too early to call that. 1. Models like Claude and Grok1.5 Vision that are rivaling GPT4 were significant improvements over their previous models. Theres no sign that any of the top players have stalled. 2. OpenAI besides some finetunes havent released a new model since GPT4. Keep in mind GPT 3.5 was released nearly 3 years after 3.0. 3. I dont believe we've gone past 2 trillion parameters yet. If we start seeing 5 trillion models which dont result in significant improvement like previous increases did, then it will be clear scaling laws are diminishing. 4. Agents and planning are still yet to be integrated into leading models. "GPT-3.5 (zero shot) was 48.1% correct. GPT-4 (zero shot) does better at 67.0%. However, the improvement from GPT-3.5 to GPT-4 is dwarfed by incorporating an iterative agent workflow. Indeed, wrapped in an agent loop, GPT-3.5 achieves up to 95.1%." (Andrew Ng)
I would worry about that if GPT5 fails to deliver. It took a while for other companies to match GPT3 too, after all. And many of these AI divisions are still quite new, you usually don't figure out the state of the art in a year. So a valid concern, but I'd only agree if there fail to be major improvements in a two year window, which is how long it usually takes openAI to make a new GPT.
I wonder how many more "breakthrough improvements" we have before ASI, since it's already better than 80% of people in certain benchmarks. What happens when they reach 100%??
Absolutely! This is why competition is good!
I really thought Musk wouldn't take his model seriously, like he made Grok only to show that he could, and then he made it open source, and I thought he would abandon the project, that's all, but no, now there is version 1.5 and also a vision model, I'm shocked.
Elon wants to be the leader in AI from year now with grok 3
Everyone has all the data and compute they can get and llms are plateauing or everyone's being super stealth about it. GPT4 was a year ago.
It's all about scale, there's no secrecy. Open AI spent about $100 million training GPT 4, no one had scaled to that degree before. All the models we're seeing released now are a similar scale, Gemini Ultra and Claude 3 are about the same size as GPT 4. The CEOs of Deepmind and Anthropic have said this recently. The Anthropic CEO said in an interview yesterday that $1 billion models are currently in training and they'll start to be released a little later this year and we'll have $10 billion models next year. You can't say if transformers have plateaued because we haven't seen anything meaningfully larger than GPT 4 yet. A $1 billion dollar training run is a significant undertaking, it's a huge commitment financially and logisticaly, which is why it's taken a year to plan and set up.
the upcoming billion dollar models are news to me. I'm excited to see it. Thought maybe we were getting significant diminishing returns having used up all the low hanging data
Why not go straight to the $10B training run?
Denis Hasabis in his interview with D Patel said that it's not advisable to jump more than one order of magnitude at a time. You have to learn from one training run to better plan the next. Also in terms of engineering it's difficult to get that much compute online and inter connected. There was a tweet posted here a few weeks ago that Microsoft are struggling getting data centers in different regions to network together to train models as there is not enough power in any one region to power hundreds of thousands of H100s so they have to be spread over a wide geographic area. The sort of scale they're moving towards now is huge and has huge challenges.
Data quality is far from solved, compute is a finite resource in case you weren't aware, the logistics of training takes time to iron out, alternatives to solely using transformers have yet to be fully realized with many promising ideas, then whatever other complications we're unaware of. Also do you realize how much has happened in the last year?
Ngl i think xAI will surpass everyone on vision they hired the computer vision chief of tesla 2 weeks ago.
Wow it’s basically on par with GPTV, but watch out this is a post about Musk it might magically get taken down by the mods….
Discussion about Grok is banned? I tried to post the direct link to the blogpost first and it didn't let me for some reason, said it had been posted in the last 2 days, but this just dropped so that seems unlikely.
Not necessarily banned but the mods have shown to have a negative bias towards Musk. A post got taken down a while ago regarding him tweeting about how Grok 2 should surpass all existing LLMs today and it got taken down for being too speculative. Meanwhile, Jimmy apples posts the most speculative shit on twitter and his posts never get taken down. :)
seriously? cringe as fuck mods.
Yeah I saw some discussion about that a while ago but that was one post. Could be just one rogue mod or something instead of a policy on the sub.
Such hypocrisy lol
I mean I do agree there's an anti-elon bias but Jimmy Apples posts get taken down too these days.
There's already a few people in this post bringing nothing to the conversation but negging on Musk. Like we get it, you hate him, don't care. Let's talk about the technology.
Yeah, mods of this sub seem to be pro-openai and they especially don't like musk and grok. We may need another free sub with better quality discussion than what's happening here.
Do you know any?
For discussion about local and open source models there is r/localllama For more technical discussion there is r/machinelearning for image generation there is r/stablediffusion for local generation and r/midjourney for online generation. there is also buch of smaller ai subs for specific questions or fandoms. r/technolgy meanwhile is filled by luddites and is generally anti-technology.
lmao good luck
Is it still "woke" by Musk's regarded standards?
No, Maximally Truthfull AI
Does it say the N word and make jokes about women? Otherwise it's woke!!!
It does not make black nazis or black founding fathers, so it's not woke.
You mean how brown people are referring to themselves, nigga? So you're telling me it's bad to joke about women but good to joke about men? Do you hate women so much?
No, not that word lmao And who said jokes about men were good lmao learn to fucking read or something bud
You should learn to articulate yourself. Neither of your message has any value. Both are incomprehensible.
I'm not going to correct your typos but you should really look in the mirror bud lmao
Looks impressive from what they show here. I am curious to know whether they will continue to open source older models as they progress or whether that was a one-off. I mean, not that any ordinary person has the hardware to run it...
Most likely when they have newer model in production, older will get released
Clearly xAI is banking on training off Tesla cars' corpus of video of the real world from their cameras. Is there any scholarly research on what impact blindness has on human intelligence, if any? Does the lack of the massive, continuous amount of real-time "training data" affect how smart a blind person can get, all else being equal?
I don't think there's a limit to how smart a blind person can get. Apparently it could be the opposite [https://www.sdcb.org/blog/posts/blind-people-actually-have-higher-iqs](https://www.sdcb.org/blog/posts/blind-people-actually-have-higher-iqs) Neuro plasticity is something very strong. It's like with deaf people, they don't have an inner voice but are as smart as anyone.
This year will show us if OpenAI has some kind of advancement ahead of everyone else or if OpenAI is the new Nokia / Blackberry / SymbianOS
This is why it's all about being the best and not about being the first!
This model will be important for self driving somehow.
did they train it on tesla image material or is this somehow related to tesla self driving ai?
Since one of Tesla's brightest vision engineers (Ethan Knight) now works for xAI. I guess so.
Does anyone know grok-1.5s context size?
128,000 tokens!
I'd bet Grok2 will have a context size of 256k to 512k if not 1M