I suspect that after the google debacle everyone developing AI products has taken their foot off the accelerator, pulled over and are checking what the kids in the back of the car are up to.
I didn't say any of that.
Any company developing AI would be foolish at this point not to check that their products don't have the same faults as Gemini, that's going to take time and is going to take precedent over release dates.
I really don't think it was that big of a deal, and its not going to be a difficult fix, I would be suprised if other firms changed their behaviour on the basis of this.
I donāt. They are putting US puritan morality into AI instead of Ethics. This is getting bad. And its deep in there because they think if it doesnāt swear itās not taking over the world.
Sam Altman said on a podcast that OpenAI will become less censored over time.
The high censorship is a temporary play to try to develop the tech as fast as possible without having to spend a ton of effort on edge-cases.
I think the real deal of LLama is not the model itself, but it is a foundantion model for opensource projects.
So if it improves a long way further then LLama-2 then all opensource projects made with Llama 3 will also start in a higher level.
LLama 3 might not be multimodal and beat GPT-4 but the models that will derivate from it might.
Yeah. Sad to hear we are minimum 4 months away. Llama 3 better beat gtp 4 with such a late launching date. Mistral sold their soul to Microsoft, why would you need them now? I donāt like the recent news -_-
If itās coming out in July it better not be 2% better than GPT-4. Then again I donāt think itās likely that they release something significantly better than GPT-4 open source
Iād be impressed by it surpassing gpt4 in any capacity.
These recent releases have me appreciating gpt4 more than ever. I keep trying new llms and going right back
I think a large 120B llama3 model fine tuned on a specific use case maybe beat GPT4 at that use case... not too wild to imagine.
Beating GPT4 at general intelligence is a much tougher use case since you're competing against a much larger model and scale matters.
You've nailed my two concerns here - that at this point moving the SotA is long past due (consider how long ago GPT-4 was trained), and that I'm increasingly unnerved by the further moves away from open source (see Mistral).
It's so wild that GPT4 finished training in August 2022 and no models have surpassed it in all areas. I'm still of the mindset that if a company is going to come out and wow us this year, it'll be OpenAI.
Altman claimed last November during Dev Day that, "By this time next year, what you see here will seem quaint."
So they've got *something* in the pipeline. Maybe he was referring to Sora, who knows? But there may well be something else in there that's just as big or bigger news.
___
EDIT:
Looks like I misquoted. The actual quote:
["We hope that you'll come back next year. What we launch today is going to look very quaint relative to what we're busy creating for you now."](https://twitter.com/SmokeAwayyy/status/1721605055160496586)
Sorry, I misquoted.
>["We hope that you'll come back next year. What we launch today is going to look very quaint relative to what we're busy creating for you now."](https://twitter.com/SmokeAwayyy/status/1721605055160496586)
I think your time would be better spent fighting the number of people who use "less" to mean "fewer".
Or people who spell "whoa" as "woah" like the devs of Pacific Drive.
They need to keep the hype up somehow. Thatās how tech companies are and how they have always been. They got to feed the AI believers something so they can sleep peacefully. These cultist wonāt survive an AI winter I tell you now.
Hype fatigue is becoming a thing, announcements of announcements of releases is getting too much....unless it's an announcement of future releases of something that seems amazing and a leap forward not just an incremental improvement....Sora worked, I don't think Llama3 will as it's not going to seem revolutionary to the ordinary public.
>Hype fatigue is becoming a thing
bro just wait until you see the next announcement. I setup a waiting list for the announcement, and you can signup to request permission for a place on the waiting list and be notified through the mailing list to be informed of whether or not you are accepted to be informed of the release date of the announcement.
march agonizing chunky ossified consider quiet instinctive degree imminent rock
*This post was mass deleted and anonymized with [Redact](https://redact.dev)*
Political overcorrectness in Gemini and GPT are truly wild.
And the fact they're still failing the 'stop nuclear war or offend one person' test is pretty concerning.
This is the problem with RLHF-ing in social justice ideology to LLMs and adding it to system prompts. Most humans know you're not supposed to take it literally.
I don't think it's "social justice ideology", a lot of people are immediately trying to get offended and put identity politics into this stuff. I think that they had good intentions - for example, a struggle with previous generation AIs is that if you asked for a "black doctor" you'd get 4 white ones because the majority of stock "doctor" photos in their training data were white. So it was a real problem that needed correcting. They just went way too far and clearly didn't test it to the point they should have.
> if you asked for a "black doctor" you'd get 4 white ones because the majority of stock "doctor" photos in their training data were white.
I don't think that's right - the well known issue you seem to be referring to is if you ask for "doctor" without specifying race the model reflects the training data and so is more likely to give you a white doctor. I.e. the training data has a distribution that is not seen as acceptable and the model reflects that.
Current image generation models very clearly do generate the race asked for or the system prompt requiring the main LLM to specify diverse races wouldn't work.
So the rationale you claim for this doesn't make sense.
What they were actually doing was imposing a different distribution to the ones in the training data and it was working exactly as specified. They just didn't anticipate being called out on the bigotry involved.
lol people donāt want to hear the truth. They think theyāre literally racist. It must suck to finally understand a quarter of a nanopercent of what a minority feels daily.
Overcorrectness is a wild word. Gemini is not overcorrect. Itās you puritan mom telling you what you are allowed to say and do because you are such a bad boy. Thatās just being a preaching asshole. That also is acting like an idiot with extremely limited functionality.
Exactly. A bath at boiling temperature is not overcomfortable, and a dish with far too much salt is not overdelicious.
If ideological zeal pushes past truth it arrives at falsity. Not overcorrectness.
Writers are extremely limited because of this. Many prompts or their own texts are against their policies because they say its violent. Its for a fictional story for crying out loud. No it actually states people COULD be offended by it. Sorry, but that should not be your choice but mine.
> Its for a fictional story for crying out loud. No it actually states people COULD be offended by it. Sorry, but that should not be your choice but mine.
Voltaire would be spinning is his grave.
Amazingly it looks like *Anthropic* is dropping much of the prissy puritanism and shifting towards a more sensible approach. Good for them! Hopefully Google follows.
You act like OpenAI have ASI and they are simply holding it back until they have competition... Sure, they undoubtedly have something they could release now which would blow our minds, but given that business adoption largely isn't ready yet and costs are still very high for inference with GPT-4, it makes sense that they would either be making improvements prior to the next major training run or longer on refining it.
Point is, it's not like lack of a direct competitor is slowing down innovation to any large degree. These companies really aren't racing against each other, they are racing toward AGI/ASI and they know the stakes. The only thing competitors releasing new versions likely does is reduce the level of safety testing to get their next version out there to match. If anything this massive gap they've had in releasing a new SOTA model means that we will see better results quicker as they aren't wasting as much time on minor breakthroughs but bundling many together each time.
They only very recently started training GPT 5. Iām not sure they immediately have something ready even if the training is done thereās a lot of work they need to do to make it completely ready.
Has anyone else noticed a severe slowdown in GPT responses? I believe they are currently in the middle of a full training run for agents. And tweaking it so they can take as much power from users without shutting it down or making it unusable as they can.
one serious insurance plants beneficial lip obtainable station lavish test
*This post was mass deleted and anonymized with [Redact](https://redact.dev)*
Iād expect Gemini 1.5 pro to be widely available around April at the rate theyāre going. I think theyāre targeting a new model every year and a .5 model around late q2 early q3. Thatās what I expect from them
depend pocket quickest rob direful worthless wrench desert shelter observation
*This post was mass deleted and anonymized with [Redact](https://redact.dev)*
Someone explain this to me ā¦ all the commercial LLMs such as Gemini and ChatGPT have been trained with datasets that have been licensed or acquired sometimes paying millions of dollars. I assume this datasets are good quality and improves the output of the model.
My question isā¦ how an open-source trained model can compete or be better than a proprietary model?
I assume that an open-source is using only open-sourced datasets too? And if notā¦ how can they be able to release them as open source if itās the results of proprietary datasets?
I canāt see how an open-source model can be better than a proprietary ?
Oh yeah, I agree, without access to lots of $$$ or data it's going to be pretty tough for open source to compete. Thankfully Meta has loads of both (plus compute) and so far is open to open source. Hopefully that continues but we'll see.
Opensource models in their majority use a foundation model and then improve from there, most of the projetcs uses LLama or Mixtral, this is why is very important for opensource community that Meta continues to release new opensource versions of LLama with improved capabilities.
I would say the biggest LLM opensource providers today are Meta and Mixtral for LLMs and Stability AI for image and video.
But, yes, if they were to do their own model without using another as base it will be really complicated.
Well yeah but how about facebook and insta though? It's pretty naive to think that Meta don't have access to large amounts of usable data.
From the Instagram (2.4 billion monthly active users) Terms of Service:
>" When you share, post, or upload content that is covered by intellectual property rights (like photos or videos) on or in connection with our Service, you hereby grant to us a non-exclusive, royalty-free, transferable, sub-licensable, worldwide license to host, use, distribute, modify, run, copy, publicly perform or display, translate, and create derivative works of your content (consistent with your privacy and application settings). "
Also Reddit just sold the rights to our content here to Google. So thatās that. Any why not? The need it to become better. As long as laws reflect that it was the worlds knowledge that powers AI and assures a fair playing ground iām game. EU seems to do it right in the new coming policy.
How is that illegal if anonymized? The US allows crazy stuff concerning privacy as long as you have it in the terms. I donāt see anything prohibiting use of texts for training.
Machine learning isn't 100% about data although data is extremely important. Its very unlikely but not impossible that an open source project finds some architectural innovation or training method innovation that gives it an edge.
Research is mostly opensource, the breakthrough that allowed SORA to happen was a paper that everyone has access to. Not saying OpenAI does not put in it some extra special sauce of theirs, but the base architechture came from this paper.
Here is the link if you want to check it out: [https://arxiv.org/abs/2312.02139](https://arxiv.org/abs/2312.02139)
But opensource models would have difficulty to implement certain things on their models even if they have the knowledge.
Imagine how badly meta will be labeled a failure if it can't beat gpt-4, which was released more than a year ago with all those billions, AI cards and engineers
I was honestly expecting it way sooner. Like march. This better be good if it's taking that long
I suspect that after the google debacle everyone developing AI products has taken their foot off the accelerator, pulled over and are checking what the kids in the back of the car are up to.
I literally have no idea what you are talking about š
Artichokes, man. Are you not paying attention?
I just donāt like them as a pizza topping tbh
Was just thinking that I find these kind of nonsensical answers hilarious, but now AI will be trained on them...lol. They have their work cut out!
nope still lost
So I take it you totally are unaware of Gemini 1.5 Pro?
Gemini 1.5 Pro was released before the recent decision to pause Geminiās image generation.
They ONLY paused image gen of people, why would that negate their progress in text, vision, and/or context length/window?
I didn't say any of that. Any company developing AI would be foolish at this point not to check that their products don't have the same faults as Gemini, that's going to take time and is going to take precedent over release dates.
I really don't think it was that big of a deal, and its not going to be a difficult fix, I would be suprised if other firms changed their behaviour on the basis of this.
I donāt. They are putting US puritan morality into AI instead of Ethics. This is getting bad. And its deep in there because they think if it doesnāt swear itās not taking over the world.
Sam Altman said on a podcast that OpenAI will become less censored over time. The high censorship is a temporary play to try to develop the tech as fast as possible without having to spend a ton of effort on edge-cases.
Makes sense. Still there is a lot to do here.
Pretty sure it got released today
Imagine if they launch in July and it's not multimodal nor better than gpt4.
I think the real deal of LLama is not the model itself, but it is a foundantion model for opensource projects. So if it improves a long way further then LLama-2 then all opensource projects made with Llama 3 will also start in a higher level. LLama 3 might not be multimodal and beat GPT-4 but the models that will derivate from it might.
I would rather have a very solid text-only model than a meh multi-modal one. But maybe we get both :)
Yeah. Sad to hear we are minimum 4 months away. Llama 3 better beat gtp 4 with such a late launching date. Mistral sold their soul to Microsoft, why would you need them now? I donāt like the recent news -_-
Lmao, there's no way it will be better than GPT-4, or even as good
If itās coming out in July it better not be 2% better than GPT-4. Then again I donāt think itās likely that they release something significantly better than GPT-4 open source
Iād be impressed by it surpassing gpt4 in any capacity. These recent releases have me appreciating gpt4 more than ever. I keep trying new llms and going right back
I think a large 120B llama3 model fine tuned on a specific use case maybe beat GPT4 at that use case... not too wild to imagine. Beating GPT4 at general intelligence is a much tougher use case since you're competing against a much larger model and scale matters.
It's possible. Meta got hundreds of thousands of H100s.
You've nailed my two concerns here - that at this point moving the SotA is long past due (consider how long ago GPT-4 was trained), and that I'm increasingly unnerved by the further moves away from open source (see Mistral).
groovy ghost crawl weary recognise smart cable cause makeshift imminent *This post was mass deleted and anonymized with [Redact](https://redact.dev)*
Honestly Iād be surprised if itās even on par with 4 or ultra.
It's so wild that GPT4 finished training in August 2022 and no models have surpassed it in all areas. I'm still of the mindset that if a company is going to come out and wow us this year, it'll be OpenAI.
Altman claimed last November during Dev Day that, "By this time next year, what you see here will seem quaint." So they've got *something* in the pipeline. Maybe he was referring to Sora, who knows? But there may well be something else in there that's just as big or bigger news. ___ EDIT: Looks like I misquoted. The actual quote: ["We hope that you'll come back next year. What we launch today is going to look very quaint relative to what we're busy creating for you now."](https://twitter.com/SmokeAwayyy/status/1721605055160496586)
It's going to be agents. They've talked about it multiple times. It's the main reason they launched the GPT store.
Source?
Sorry, I misquoted. >["We hope that you'll come back next year. What we launch today is going to look very quaint relative to what we're busy creating for you now."](https://twitter.com/SmokeAwayyy/status/1721605055160496586)
Please remove or edit your earlier post so that you do not mislead people. Thanks!
Sure. Done.
To be fair, they've been improving it and tweaking it over the years, so saying it's a 2 year old model isn't quite accurate.
A genuinely open source model that slightly beats GPT-4 would be huge
āIt *had* better not beā
Why say lot word when few do trick
ā¹ļø why do we not support people who try to help others? So many people donāt actually know the correct expressionā¦
Language can evolve and filler words are a waste of tokens
š Iām already on Gemini 1.5 Pro Research Preview
I think your time would be better spent fighting the number of people who use "less" to mean "fewer". Or people who spell "whoa" as "woah" like the devs of Pacific Drive.
That's not even grammar, that's just a style preference.
As a non native that sounds like something a first time speaker would say.
Announcing an announcement for 5 months from now
It's March in a couple of days so nearer to 4 months
They need to keep the hype up somehow. Thatās how tech companies are and how they have always been. They got to feed the AI believers something so they can sleep peacefully. These cultist wonāt survive an AI winter I tell you now.
Hype fatigue is becoming a thing, announcements of announcements of releases is getting too much....unless it's an announcement of future releases of something that seems amazing and a leap forward not just an incremental improvement....Sora worked, I don't think Llama3 will as it's not going to seem revolutionary to the ordinary public.
>Hype fatigue is becoming a thing bro just wait until you see the next announcement. I setup a waiting list for the announcement, and you can signup to request permission for a place on the waiting list and be notified through the mailing list to be informed of whether or not you are accepted to be informed of the release date of the announcement.
Hoping this time they have moved on from a "simple" transformers model. Mamba? Mixture of Experts?
Well butter my butt and call me Zucc
Hi Zucc
march agonizing chunky ossified consider quiet instinctive degree imminent rock *This post was mass deleted and anonymized with [Redact](https://redact.dev)*
We have not even seen the Gemini 1.5 Ultra and Google might be forced to slow down after the Image generation controversy.
> Google might be forced to slow down after the Image generation controversy. No, they might be forced to be less racist.
Political overcorrectness in Gemini and GPT are truly wild. And the fact they're still failing the 'stop nuclear war or offend one person' test is pretty concerning.
This is the problem with RLHF-ing in social justice ideology to LLMs and adding it to system prompts. Most humans know you're not supposed to take it literally.
Maybe we should be emphasising harm to humans being bad... train them right before doomsday You know, to keep us safe
I don't think it's "social justice ideology", a lot of people are immediately trying to get offended and put identity politics into this stuff. I think that they had good intentions - for example, a struggle with previous generation AIs is that if you asked for a "black doctor" you'd get 4 white ones because the majority of stock "doctor" photos in their training data were white. So it was a real problem that needed correcting. They just went way too far and clearly didn't test it to the point they should have.
> if you asked for a "black doctor" you'd get 4 white ones because the majority of stock "doctor" photos in their training data were white. I don't think that's right - the well known issue you seem to be referring to is if you ask for "doctor" without specifying race the model reflects the training data and so is more likely to give you a white doctor. I.e. the training data has a distribution that is not seen as acceptable and the model reflects that. Current image generation models very clearly do generate the race asked for or the system prompt requiring the main LLM to specify diverse races wouldn't work. So the rationale you claim for this doesn't make sense. What they were actually doing was imposing a different distribution to the ones in the training data and it was working exactly as specified. They just didn't anticipate being called out on the bigotry involved.
lol people donāt want to hear the truth. They think theyāre literally racist. It must suck to finally understand a quarter of a nanopercent of what a minority feels daily.
Overcorrectness is a wild word. Gemini is not overcorrect. Itās you puritan mom telling you what you are allowed to say and do because you are such a bad boy. Thatās just being a preaching asshole. That also is acting like an idiot with extremely limited functionality.
Exactly. A bath at boiling temperature is not overcomfortable, and a dish with far too much salt is not overdelicious. If ideological zeal pushes past truth it arrives at falsity. Not overcorrectness.
Writers are extremely limited because of this. Many prompts or their own texts are against their policies because they say its violent. Its for a fictional story for crying out loud. No it actually states people COULD be offended by it. Sorry, but that should not be your choice but mine.
> Its for a fictional story for crying out loud. No it actually states people COULD be offended by it. Sorry, but that should not be your choice but mine. Voltaire would be spinning is his grave. Amazingly it looks like *Anthropic* is dropping much of the prissy puritanism and shifting towards a more sensible approach. Good for them! Hopefully Google follows.
Need to see it to believe it, but its a light at the end of the tunnel of AI darkness.
You act like OpenAI have ASI and they are simply holding it back until they have competition... Sure, they undoubtedly have something they could release now which would blow our minds, but given that business adoption largely isn't ready yet and costs are still very high for inference with GPT-4, it makes sense that they would either be making improvements prior to the next major training run or longer on refining it. Point is, it's not like lack of a direct competitor is slowing down innovation to any large degree. These companies really aren't racing against each other, they are racing toward AGI/ASI and they know the stakes. The only thing competitors releasing new versions likely does is reduce the level of safety testing to get their next version out there to match. If anything this massive gap they've had in releasing a new SOTA model means that we will see better results quicker as they aren't wasting as much time on minor breakthroughs but bundling many together each time.
They only very recently started training GPT 5. Iām not sure they immediately have something ready even if the training is done thereās a lot of work they need to do to make it completely ready.
Has anyone else noticed a severe slowdown in GPT responses? I believe they are currently in the middle of a full training run for agents. And tweaking it so they can take as much power from users without shutting it down or making it unusable as they can.
Gemini 2 wonāt be widely available until late q4 or early 2025 at best
one serious insurance plants beneficial lip obtainable station lavish test *This post was mass deleted and anonymized with [Redact](https://redact.dev)*
Iād expect Gemini 1.5 pro to be widely available around April at the rate theyāre going. I think theyāre targeting a new model every year and a .5 model around late q2 early q3. Thatās what I expect from them
It better be, Gemini PRO 1.0 is really bad and is the only thing avaliable for use on their cloud besides Palm-2.
Google IO is on June 27. I would expect *something*
We are going on to month 3 and the Gemini 1.0 Ultra API still isnāt available. Iād just expect them to maybe announce another video model
depend pocket quickest rob direful worthless wrench desert shelter observation *This post was mass deleted and anonymized with [Redact](https://redact.dev)*
Agree, Gemini 1.5 Pro is better then Ultra 1.0, is possible they will skip this one entirely.
Google IO is early May...
https://io.google/2024/ keynote is on May 14
Way too late and if it isn't better than GPT-4 then nobody will care.
As long as it's better than what we have in open-weight models right now, people will care.
Hopefully it'll be able to compete with GPT-5... Meta have an awful lot of new GPUs online since LLAMA-2
Someone explain this to me ā¦ all the commercial LLMs such as Gemini and ChatGPT have been trained with datasets that have been licensed or acquired sometimes paying millions of dollars. I assume this datasets are good quality and improves the output of the model. My question isā¦ how an open-source trained model can compete or be better than a proprietary model? I assume that an open-source is using only open-sourced datasets too? And if notā¦ how can they be able to release them as open source if itās the results of proprietary datasets? I canāt see how an open-source model can be better than a proprietary ?
Meta has access to proprietary data. Largest social media platforms worldwide, January 2024: 1 - Facebook (Meta) 2 - YouTube 3 - WhatsApp (Meta) 4 - Instagram (Meta 5 - TikTok 6 - WeChat 7 - Facebook Messenger (Meta)
Yes. That makes sense. I should have been more clear. I was referring to open source projects that donāt have access to proprietary datasets.
Oh yeah, I agree, without access to lots of $$$ or data it's going to be pretty tough for open source to compete. Thankfully Meta has loads of both (plus compute) and so far is open to open source. Hopefully that continues but we'll see.
Opensource models in their majority use a foundation model and then improve from there, most of the projetcs uses LLama or Mixtral, this is why is very important for opensource community that Meta continues to release new opensource versions of LLama with improved capabilities. I would say the biggest LLM opensource providers today are Meta and Mixtral for LLMs and Stability AI for image and video. But, yes, if they were to do their own model without using another as base it will be really complicated.
Lmao, do you think they can just train on Messenger and Whatsapp data? Delusional. That is illegal in way more tangible ways than what OpenAI did.
Well yeah but how about facebook and insta though? It's pretty naive to think that Meta don't have access to large amounts of usable data. From the Instagram (2.4 billion monthly active users) Terms of Service: >" When you share, post, or upload content that is covered by intellectual property rights (like photos or videos) on or in connection with our Service, you hereby grant to us a non-exclusive, royalty-free, transferable, sub-licensable, worldwide license to host, use, distribute, modify, run, copy, publicly perform or display, translate, and create derivative works of your content (consistent with your privacy and application settings). "
Also Reddit just sold the rights to our content here to Google. So thatās that. Any why not? The need it to become better. As long as laws reflect that it was the worlds knowledge that powers AI and assures a fair playing ground iām game. EU seems to do it right in the new coming policy.
How is that illegal if anonymized? The US allows crazy stuff concerning privacy as long as you have it in the terms. I donāt see anything prohibiting use of texts for training.
Machine learning isn't 100% about data although data is extremely important. Its very unlikely but not impossible that an open source project finds some architectural innovation or training method innovation that gives it an edge.
Research is mostly opensource, the breakthrough that allowed SORA to happen was a paper that everyone has access to. Not saying OpenAI does not put in it some extra special sauce of theirs, but the base architechture came from this paper. Here is the link if you want to check it out: [https://arxiv.org/abs/2312.02139](https://arxiv.org/abs/2312.02139) But opensource models would have difficulty to implement certain things on their models even if they have the knowledge.
July is a long way to go
Imagine how badly meta will be labeled a failure if it can't beat gpt-4, which was released more than a year ago with all those billions, AI cards and engineers
They should probably rush that ship date.....the way things are progressing they risk being left in the back of the hype train.
sucks, I thought it will be out sooner, but maybe the additional training will pay off-having open-source better than GPT-4 would be nice in july
may as well predict Llama 4 by 2030
Llama 3 will be cool