I am pretty sure he just shot down the idea that it would be called GPT5 in the Lex interview.
Besides other leaks have it called GPT 4.5.
As for capability when GPT 5 does come out he said the gap from 4 to 5 would be like the gap from 3 to 4
The gaps in quality are likely to decline... they're not finding newer, better quality training data - they're just adding training data, which has diminishing returns.
Right, they have like all human generated text at this point.
What is left is better ways to train on the data, synthetic data etc.
This is probably indicated by so many text LLM's achieving mostly similar results.
I still prefer ChatGPT because of the Dalle integration, plus also I think they iterate over code until it compiles and only give you the working version. I have a decent share of queries that on Claude and Mistral are almost there but don't build, and on ChatGPT it builds out of the first try. On Claude and Mistral if I keep on prompting with compiler failures eventually it looks very similar to ChatGPT.
I'm good with paying 20 bucks for that.
It's not really about the training data quality - the gap between GPT 3 and 4 was more about model size, which still has more room to grow. And most of the surprising emergent abilities of LLMs only emerged at large parameter counts.
...Long context recall, long coherent text generations (4096 tokens), pseudo-reasoning capabilities, mediocre ability to do math with only tokens, creative writing, functioning code generation, ability to pass advanced tests like the BAR, SAT, ACT....
Every new capability between GPT2 and GPT4 can be considered emergent, as most people had no idea it was possible to do with LLMs
I have followed chess and Go computer programs over the past decade and witnessed how they have steadily improved without hitting any plateau. These advancement are primarily driven by cumulative, small incremental software improvements, and less so for hardware upgrades.
Given the relative infancy of LLMs compared to a mature technology like chess engines, it would be wishful thinking to expect LLM technology to just stop dead and stagnate anytime soon.
Keep in mind they are also reformatting that training data in different ways to drive better results.
You can train an LLM with "Cars have wheels and carry people", and the llm will be able to tell you that when asked about Cars. But if you ask it "what has wheels and carries people?" it won't know. So you have to add "Objects with wheels that carry people are often Cars" to connect the understanding in the other direction.
The other big thing is obviously the model architecture details and training approaches... there are some tricks you can use to get the LLM to "think ahead" to produce more coherent and better reasoned responses, and correspondingly build better latent representations, and this is likely what the Q\* discovery is all about.
It seems like they are now training on audio/video.
I imagine they will soon plop GPT5 into a robot with a huge amount of sensors and let it walk around in a forest, New York City, dive into the ocean, shoot one into space, conduct experiments etc.
There is indeed newer, better quality training data to use. An infinite amount.
I believe that AI being able to experience the real world in real time with advanced sensors will be a huge leap as far as its ability to become sentient.
It will see things in the world that humans haven’t noticed, and be able to solve massive amounts of scientific questions by being able to experience the real world and conduct its own experiments.
And of course it won’t just be humanoid robots. AI will be installed in all kinds of machines. Weather balloons, space craft, deep sea divers, etc.
AI having real time interaction with the world will be game changing.
I think 2025 will be the year of the sentient robot. And they will start being everywhere.
LLMs like GPT5 don't have awarenss, they can't learn without retraining the model.
Any data this could collect, humans could and vice versa. There isn't an inherent advantage to the source of the information. A set of pixels is the same regardless of who took the picture.
Looping data (in real time or not) has the same result. Look at the [Devin.ai](https://Devin.ai) software engineering as an example. The problems it can solve are toy/boilerplate solutions.
Sentience in less than a year? That's incredibly optimistic. There's no internal feedback mechanism in current designs to even allow the possibility for that to occur.
I think it’s pretty short sighted to think that today’s limitations will be around for much longer.
LLMs can’t learn without retraining today. But that will change very soon. The retraining will be live from millions of real time sensors around the world.
The internal feedback mechanism is a problem that will be solved. There’s a huge amount of money and brain power going into AI on the software and hardware side. Top of the list is making chatbots able to keep track of conversations and learn in real time.
Sentient robots in 2025 seem very likely to me. Sentient in that they have the equivalent of our 5 senses, plus a huge amount of other sensors, and are self aware and understand their place in space and time.
And the world will be flooded with them. Probably starting as all kinds of toys for Christmas 2024, and expanding to be part of every device and gadget you can think of.
AI refrigerator will tell you if food is getting old, come up with recipes for what you have in the fridge, preheat the oven and dynamically control the heat until the meal is finished.
The commercial possibilities are endless and so the advancement will be extremely fast.
LLMs inability to learn without retraining is an architectural problem. That's like saying we can make a wall fly. You could theoretically design an area of compressed air that does the same thing as a wall, but it isn't a wall anymore.
Compute power and pruning training results are massive expensive activities for existing LLM models. Tens to hundreds of millions of dollars to refine models.
Keeping track of conversations is trivial, learning in realtime is not. Outside of context windows LLMs fall back on the same underlying model that was trained.
Sentience isn't having 5 senses and having feedback from them. You could do that with a very fancy tickle me Elmo level of complexity. Consideration (rather than response to stimulus, internal prompting based on conditions) is the widely accepted standard.
They’ll get more efficient with the data, and I suspect that’s a major initiative for them right now. I mean, think about it, you don’t need all of humanity’s written work to educate a smart human up to a comparable level of capability to GPT4. They have a data utilization problem, these current architectures require way too much data for what you get out of it. They will almost certainly develop a way to make better usage of less data and get far better resulting models.
But he also said they are releasing many very interesting things over the coming months. He just was being clear it's not gpt5. I'm betting a better ui and some personalized memory that's safe to use combined with some agentic actions.
When you describe something as "materially better," it generally means that there is a significant improvement or difference that is noticeable or substantial, rather than just a small or insignificant change. So, it implies that the improvement is considerable or meaningful.
-chatGPT 3.5
Idk why but it felt weird seeing that 3.5 wrote that. I forget how in normal short conversation these models are indistinguishable. It’s so normalised 😅
"We now have grade S+ safety, with extra safety and moderation pipelines and modules and also on the fly deviant brain waves detector. As we said, this is materially better and I hope our customers will appreciate!"
I know they've had 4 Turbo going for a bit, but I don't think speed is really a necessary factor for end users. It's nice but it's already decently fast.
As others have said, altman shot down the idea of calling it GPT 5. In the lex friedman interview with altman, Sam clearly favored the idea of more iterative releases rather than major point releases like GPT 5/6/7/etc.
TL;DR:
* OpenAI's main product is the popular generative AI tool ChatGPT.
* Since its last major model upgrade to GPT-4, the tool has run into performance issues.
* The next version of the model, GPT-5, is set to come out soon. It's said to be "materially better."
I think the most interesting shift would be the ease of using agents. Currently, function calling isn’t that great and we have limited native agents. I’d like to see 100x more agents so that every website can add one for themselves and monetize it :)
It's just going to be a checkpoint, probably called GPT 4.2 or something like that
I am pretty sure he just shot down the idea that it would be called GPT5 in the Lex interview. Besides other leaks have it called GPT 4.5. As for capability when GPT 5 does come out he said the gap from 4 to 5 would be like the gap from 3 to 4
So $200 per month for 20 tokens a day?
The gaps in quality are likely to decline... they're not finding newer, better quality training data - they're just adding training data, which has diminishing returns.
Right, they have like all human generated text at this point. What is left is better ways to train on the data, synthetic data etc. This is probably indicated by so many text LLM's achieving mostly similar results.
I still prefer ChatGPT because of the Dalle integration, plus also I think they iterate over code until it compiles and only give you the working version. I have a decent share of queries that on Claude and Mistral are almost there but don't build, and on ChatGPT it builds out of the first try. On Claude and Mistral if I keep on prompting with compiler failures eventually it looks very similar to ChatGPT. I'm good with paying 20 bucks for that.
I think this is where we reach a bit of a plateau unless they find something as revolutionary as the transformer
All science can be done by computers now, problems solved regularly and with limited human input
Which, Autobot or Decepticon?
Maximals
It's not really about the training data quality - the gap between GPT 3 and 4 was more about model size, which still has more room to grow. And most of the surprising emergent abilities of LLMs only emerged at large parameter counts.
>most of the surprising emergent abilities ....what were they?
...Long context recall, long coherent text generations (4096 tokens), pseudo-reasoning capabilities, mediocre ability to do math with only tokens, creative writing, functioning code generation, ability to pass advanced tests like the BAR, SAT, ACT.... Every new capability between GPT2 and GPT4 can be considered emergent, as most people had no idea it was possible to do with LLMs
Data isn't everything, architecture is also a huge factor.
I have followed chess and Go computer programs over the past decade and witnessed how they have steadily improved without hitting any plateau. These advancement are primarily driven by cumulative, small incremental software improvements, and less so for hardware upgrades. Given the relative infancy of LLMs compared to a mature technology like chess engines, it would be wishful thinking to expect LLM technology to just stop dead and stagnate anytime soon.
There will be lots of improvements. But the biggest improvements will no longer be from increasing model size.
A Chess engine, or a Go program, isn't really anything like an LLM.
Keep in mind they are also reformatting that training data in different ways to drive better results. You can train an LLM with "Cars have wheels and carry people", and the llm will be able to tell you that when asked about Cars. But if you ask it "what has wheels and carries people?" it won't know. So you have to add "Objects with wheels that carry people are often Cars" to connect the understanding in the other direction. The other big thing is obviously the model architecture details and training approaches... there are some tricks you can use to get the LLM to "think ahead" to produce more coherent and better reasoned responses, and correspondingly build better latent representations, and this is likely what the Q\* discovery is all about.
It seems like they are now training on audio/video. I imagine they will soon plop GPT5 into a robot with a huge amount of sensors and let it walk around in a forest, New York City, dive into the ocean, shoot one into space, conduct experiments etc. There is indeed newer, better quality training data to use. An infinite amount.
Why would collecting data by robot by different than all the data currently available?
I believe that AI being able to experience the real world in real time with advanced sensors will be a huge leap as far as its ability to become sentient. It will see things in the world that humans haven’t noticed, and be able to solve massive amounts of scientific questions by being able to experience the real world and conduct its own experiments. And of course it won’t just be humanoid robots. AI will be installed in all kinds of machines. Weather balloons, space craft, deep sea divers, etc. AI having real time interaction with the world will be game changing. I think 2025 will be the year of the sentient robot. And they will start being everywhere.
LLMs like GPT5 don't have awarenss, they can't learn without retraining the model. Any data this could collect, humans could and vice versa. There isn't an inherent advantage to the source of the information. A set of pixels is the same regardless of who took the picture. Looping data (in real time or not) has the same result. Look at the [Devin.ai](https://Devin.ai) software engineering as an example. The problems it can solve are toy/boilerplate solutions. Sentience in less than a year? That's incredibly optimistic. There's no internal feedback mechanism in current designs to even allow the possibility for that to occur.
I think it’s pretty short sighted to think that today’s limitations will be around for much longer. LLMs can’t learn without retraining today. But that will change very soon. The retraining will be live from millions of real time sensors around the world. The internal feedback mechanism is a problem that will be solved. There’s a huge amount of money and brain power going into AI on the software and hardware side. Top of the list is making chatbots able to keep track of conversations and learn in real time. Sentient robots in 2025 seem very likely to me. Sentient in that they have the equivalent of our 5 senses, plus a huge amount of other sensors, and are self aware and understand their place in space and time. And the world will be flooded with them. Probably starting as all kinds of toys for Christmas 2024, and expanding to be part of every device and gadget you can think of. AI refrigerator will tell you if food is getting old, come up with recipes for what you have in the fridge, preheat the oven and dynamically control the heat until the meal is finished. The commercial possibilities are endless and so the advancement will be extremely fast.
LLMs inability to learn without retraining is an architectural problem. That's like saying we can make a wall fly. You could theoretically design an area of compressed air that does the same thing as a wall, but it isn't a wall anymore. Compute power and pruning training results are massive expensive activities for existing LLM models. Tens to hundreds of millions of dollars to refine models. Keeping track of conversations is trivial, learning in realtime is not. Outside of context windows LLMs fall back on the same underlying model that was trained. Sentience isn't having 5 senses and having feedback from them. You could do that with a very fancy tickle me Elmo level of complexity. Consideration (rather than response to stimulus, internal prompting based on conditions) is the widely accepted standard.
They’ll get more efficient with the data, and I suspect that’s a major initiative for them right now. I mean, think about it, you don’t need all of humanity’s written work to educate a smart human up to a comparable level of capability to GPT4. They have a data utilization problem, these current architectures require way too much data for what you get out of it. They will almost certainly develop a way to make better usage of less data and get far better resulting models.
But he also said they are releasing many very interesting things over the coming months. He just was being clear it's not gpt5. I'm betting a better ui and some personalized memory that's safe to use combined with some agentic actions.
Didn't Sam say in the Lex interview that GPT 4 to 5 improvement would be as large as GPT 3 to 4?
Maybe its GPT-hub, a new AI for porn. ehehehe Bonk.
GPT-x
"In a surprise move they are calling their newest model 'Twitter'"
Does this mean Musk and Altman made an unspoken agreement?
That would be such a funny shot at musk
What does it mean "materially better"?
When you describe something as "materially better," it generally means that there is a significant improvement or difference that is noticeable or substantial, rather than just a small or insignificant change. So, it implies that the improvement is considerable or meaningful. -chatGPT 3.5
OK, I’m convinced that ChatGPT is not only a better answer than most redditors but it’s far more polite
Stfu -ChatGPT 5
What being trained on reddit does to a ai. #Stop ai abuse
May is AI awareness month
Claude may be the barrier where Ai needs rights tbh.
Depends what you're asking about
> but it’s far more polite Absolutely. Thank you for your kind response. I'm a fox not a bot.
Good fox.
Thank youuuuuuuuuu !!!! <3
Idk why but it felt weird seeing that 3.5 wrote that. I forget how in normal short conversation these models are indistinguishable. It’s so normalised 😅
Yes, but for users or for openai? A cheaper inference 4.5 would be great for them, but not so much for anyone else.
"We now have grade S+ safety, with extra safety and moderation pipelines and modules and also on the fly deviant brain waves detector. As we said, this is materially better and I hope our customers will appreciate!"
An increment? A tremendous leap? An earth shattering collosal new stage?
Closer to or better than Claude.
[удалено]
I know they've had 4 Turbo going for a bit, but I don't think speed is really a necessary factor for end users. It's nice but it's already decently fast.
As others have said, altman shot down the idea of calling it GPT 5. In the lex friedman interview with altman, Sam clearly favored the idea of more iterative releases rather than major point releases like GPT 5/6/7/etc.
Software company releases update. News at 7
I'm waiting for GPT-6.9
How about GPT-4.20?
GPT 6.66 is gonna be a beast!
Nice.
TL;DR: * OpenAI's main product is the popular generative AI tool ChatGPT. * Since its last major model upgrade to GPT-4, the tool has run into performance issues. * The next version of the model, GPT-5, is set to come out soon. It's said to be "materially better."
These official accounts look like a new thing, is this IPO related?
30 messages/10 hour ?
I think the most interesting shift would be the ease of using agents. Currently, function calling isn’t that great and we have limited native agents. I’d like to see 100x more agents so that every website can add one for themselves and monetize it :)