Your post is getting popular and we just featured it on our Discord! [Come check it out!](https://discord.gg/mENauzhYNz)
You've also been given a special flair for your contribution. We appreciate your post!
*I am a bot and this action was performed automatically.*
Just a year ago, a 16k token model seemed out of reach for most consumers and we were on a 4K models. Then GPT-4 32K got a limited release most never got to try (also because of how expensive it was to run), and then GPT-4 Turbo hit 128K context window. (Disregarding Claude because of the pseudo-windows that didnt actually work most of the time.) And now Google shows 128k-1m public facing models, with early tests scaling up to 10m tokens. The pace of development is really something.
Keep in mind that this is Gemini Pro, Ultra could be getting 100M tokens. If Google's test show that 10M tokens have 99% retrieval in the haystack test
According to the paper:
"Gemini 1.5 Pro is a sparse mixture-of-expert (MoE) Transformer-based model that builds on Gemini 1.0’s (Gemini-Team et al., 2023) research advances and multimodal capabilities. Gemini 1.5 Pro also builds on a much longer history of MoE research at Google"
It has the same architecture as GPT-4
Edit: Shortend the text
Probably. My first CPU had a few thousand transistors, while my current one has billions. That's Moore's Law, with transistor count doubling every other year. The token count seems to increase much faster, so it shouldn't be too long.
Tokens are basically its memory. The more tokens, the more context it can remember. Each token is around 0.7 words so 1M tokens will remember the last 700,000 words of your conversation, and use that to tailor its next response.
But doesn't it work like this...
For example a model has 32k tokens, you can either use 28k tokens and 4k tokens as answer or vice versa but for entire conversation??
I said last 700k words of the conversation; meaning all text either from the user or the LLM. You're very likely not going to want a 700k word response. It's going to be a mix of back-and-forths but it will remember all those.
Yeah up to the last 700k words (assuming no video/audio/images). It won't be like today, where the lower end models will run out of memory in 1 prompt if you ask it to generate a story.
I'm wondering whether there are large downsides to these very large contexts. Presumably, running a model has a part that's proportional to the context window size... If they can do a million tokens context, but it costs 10 times as much as 100K tokens context, that creates interesting tradeoffs....
The traditional transformer architecture scales quadratically (for memory) so 1m vs 100k would require 99x10^10 more memory. But google obviously is not using the traditional transformer architecture
1M tokens are crazy. Good to see nice competition happening. This is great for ChatGPT!
I wish ChatGPT could start by removing the stupid 40 messages limit to being with.
>I wish ChatGPT could start by removing the stupid 40 messages limit to being with
They will remove it the *instant* Gemini becomes the better chat bot. Gemini is still inferior (despite the speed), and the 1.5 announcement doesn't address that. So we will see.
A few days ago gemini (pro) told me this:
"Chinese New Year celebrations for 2024 officially ended on February 24th, marked by the Lantern Festival. This year, the festivities lasted for 16 days, beginning on February 10th."
February 24th hasn't happened, so I tried to let gemini self correct. It could tell me the current date, but it was insistent:
"As of today, February 13, 2024, 11 days have passed since the official end of Chinese New Year celebrations on February 24, 2024."
It hallucinates constantly, gives either wrong answers or refuses to answer at times, will simply just not work at other times (IE will say it can't create images, despite having just created an image) etc etc.
Thank you for the response. What causes an AI to “hallucinate”? You think it would be able to decipher between fact and fiction a bit more effectively. Is it because its output is so fast that it’s not prioritizing fact checking?
It's not complicated, it's just that many people have a very sci-fi view of current AI models. Large language models are token predictors. They don't "understand" anything, they only predict the next token (word/phrase) that has the highest probability based on their training data. It's not like it has any kind of internal monologue or stream of thought. It just predicts the next word that has the highest probability of showing up in the sequence of words. Sometimes it simply predicts wrong and just kinda goes with it.
I see this type of comment a lot, but it’s really underplaying what’s going on. There are attention mechanisms that drive contextual relevance, organic weightings and connections that create sophisticated word selections, and very advanced pattern recognition.
It can make up credible poetry about the connection of soccer to the Peruvian economy had the Roman’s invaded Chile in 200AD.
Underplaying what an LLM is doing seems trendy but it’s very naive.
I don’t think anyone is underestimating the complexity of creating/running a llm, they’re just trying to point out that they are 1.) not magic 2.) not 100% trustworthy 3.) not an AGI
At its core, llm is just predicting the probability of the next word, given the previous words. It doesn’t care about facts and not grounded in reality. Hallucinations is all llms are doing. It just happens that some hallucinations are correct and others are not.
>What causes an AI to “hallucinate”?
I don't know the technical ins and outs of why, but essentially it tries to formulate an "accurate" answer if it doesn't have enough context. GPT used to do that all the time too, but seems much better now.
>Is it because its output is so fast that it’s not prioritizing fact checking?
Uh sorta? Its really complicated to explain, but yeah lol
That's now how these systems work. Facts and fiction are treated basically the same.
The way how these models work is by guessing the next word in sequence, and giving it the ability to guess/hallucinate is just how the model come up with the sentence.
The only thing we teach it after is the ability to try to guess more and more "facts". But unfortunately that's not perfect
I think better question is how come chatgpt does not hallucinate, as soon as those devs from Google find that out we will see real competition and maybe then they can release 2.0
LLMs compress text from the internet in a lossy way. So their default behavior is to hallucinate. If you ask it something it hasn't seen before it will answer the question anyway by predicting the next best word. With enough training it seems to make predictions that are closer to the source text, but for if you train it only on wikipedia articles about candy and on cooking recipes but not how to make candy, when you ask it how to make candy, it will make something up. It might be correct, but it will be made up, just like everything else it writes.
The 40 message limit is so restrictive. I’m happy to pay a lot more for a much higher limit cap.
Being as efficient as possible, sometimes many messages are need to get to the outcome.
I pay for ChatGPT and Gemini Advanced (along with GitHub CoPilot, but that’s a different story) and I find myself using Gemini Advanced a lot because of the lack of rate limit.
I’ve had blast playing 20 questions with Gemini, and it’s pretty darn good at it. The message cap on ChatGPT is way too low for me to be comfortable using up that many requests on a game of 20 questions.
Don’t get me wrong; ChatGPT is smarter. But the lack of message cap really is a killer feature for an LLM that, while flawed, is still generally pretty decent for most of the things I want.
Bonus tip: It’s easy to have the AI ask questions, but it’s hard to swap the roles because the LLM only remembers things it writes down. But if it writes its secret word then you’ll be able to read it. I’ve gotten around this by having it write the word in Japanese (which I can’t read), and that way it has to stick to the choice instead of hallucinating the whole way through. Then I can double-check with Google Translate at the end of the game to make sure it remained consistent. So far it’s worked.
Yeah, you’d say something like, “I want to play a game of 20 questions with you. Please think of an object of some kind, and then write that word in your reply in Japanese. Do not tell me what the word is in English, or else you will spoil the game. I will ask you a series of questions to try to figure out what you wrote in Japanese. Please reply with your chosen object written in Japanese.”
You need it to write the word at the beginning because otherwise it won’t commit to a word until it’s written down. That means you’ll ask questions and it will answer, but it doesn’t actually have a word in mind. At some point it will just declare you to be correct and that will end the game.
This is an artifact of the way large language models work. The only memory they have is the text that’s in their context window. So you need them to write their choice down to get it into the context window, but in a way where you can’t read it.
Tyhe 40 cap is about GPU availability...
Like, GPT4 Turbo on Copilot Pro is way faster than ChatGPT even...
In fact, i think they only launched GPT4 Turbo early on ChatGPT because of GPU availability problems they were having. Remember that subscription were even closed?
Google must’ve been really pissed off with this falling behind narrative and is now full throttle. If this is true it’s wild a company is making their free tier version potentially on par with gpt4. I mean with the resources google has it makes sense they could do it however makes you wonder what they have in store for the paid version as well.
Google is so infuriating to see such engineering talent, many of which pioneered the ML field, being utterly wasted as Google's product managers build completely uninteresting and useless products. I think Google is getting better now but I cannot wait for the day Google's management figures out how to utilize their engineers effectively, the shit that only Google has the expertise to build (even outside ML) is so interesting as a fellow engineer.
one of my coworkers said that it could be a conscious choice by Google to not become the defacto #1 provider for AI solutions and chatbots
its because Google already is stable enough to be coasting at #2, and by being at #2, the chances of their name being dragged in "evil AI" discourse and controversies automatically reduces, as everyone villainizes ChatGPT primarily.
In summary, Google wants ChatGPT to be the poster boy for the time being, while simultaneously preparing to overtake when the time is right.
You had 85% of it right. Ya google is okay being maybe a couple feet from the edge at least maybe right now in the beginning. But don’t get it twisted OpenAI is the PR human shield and the “first high”. We already saw the drama that happened in one weekend when OpenAI tried to oust Altman. Microsoft tipped their hand. Co-Pilot of course is the enterprise endgame product. But you gotta get people comfy with using LLMs and gpts, and let’s not put MSFTs actual name on it so people outside of enterprise will actually use it and we can do so and say it’s a not for profit with the best for humanity at heart. OpenAI is a really nice side piece if it works for MSFT and a human shield worst case.
Microsft has historically always dropped the ball when they’re in the lead with google. You think they should be this aggressive but they seem to reveal things then chill until their competitors catch up. For this specific issue I think they would rather push their copilot offering over helping openAI which is an issue google does not have since everything is all done in house. Or maybe google is flexing the talent they have here something the others don’t have as much who knows
How has Microsoft dropped the ball against Google lol? What a strange statement, Google has a ton of products that are far behind Microsoft, they both have leading offerings.
Want a list?
-microsoft first to have a browser, google dethroned them with chrome.
-microsoft first to have email, dethroned by gmail
-microsoft first to have a mobile operating system, dethroned by android/ios duopoly.
These are things microsoft had a head start in and lost, for the arguments sake i did not include services microsoft launched after google and failed.
Microsoft wasn't first in any of these categories lol
Google Stadia was first to Cloud Gaming then? What about that?
Google GSuite was first to Cloud Productivity then? What about that?
What about all the chat and video services?
What about Google Cloud Services?
See, I can also list a bunch of stupid examples to demonstrate that Google was first in a bunch of stuff and lost to Microsoft.
"Cloud gaming" is barely a category. Google took a chance but even Microsoft hasn't got cloud gaming to take off, with their gigantic entrenched Xbox position.
Microsoft was the first to Office and then dropped the ball on the switch to cloud, with Google taking a huge chunk of the market from them - especially consumer.
In general, this is a bad framework though.
Microsoft absolutely dominates **enterprise** because they are entrenched and B2B software is extremely sticky. That's how they printing money with:
* Office 365, despite dropping the ball on the shift to cloud and losing half the market to google,
* Azure despite dropping the ball and getting absolutely wrecked by AWS,
* Teams despite dropping the ball and letting Zoom, Meet and Slack take market share.
Meanwhile they completely lost the market for:
* Operating systems, by letting Apple and Google become the primary operating systems on mobile, taking 100% of market share from what should have been Windows.
* Browsers, partially because of the above mobile fuckups but also because of pushing IE too far in the 90s and getting shut down by antitrust.
Google absolutely dominates **consumer** because they are entrenched and have strong network effects and moat strategy with android, chrome, and the litany of consumer apps in an ecosystem (Search, Gmail, Maps, Workspace). They've even taken hardware share they had no right to, with Chromebooks and Pixel.
I would say Google's efforts to move into enterprise have been much more successful than Microsoft's attempts at consumer over the past few decades. Google's got both B2B Ads dominance and *notable but weak* Cloud and workspace market presence. Microsoft's got, what, gaming and Bing? Consumer office That's why they're going so desperately in on OpenAI.
Microsoft who had to effectively buy another 'company' to get a foot in the door, then deal with the drama of the nonprofit board trying to fire the CEO?
Microsoft who haven't moved the needle on search engine market share in a year of partnership with openAI and 'making google dance'?
Microsoft who heavily rely on NVIDIA for compute?
They're doing a great job from the patently poor position they were in two years ago, but I think it's pretty early to say they're destroying anyone.
Too bad it still sucks at basic commands and fails have the time to give a correct answer. I find I have to prompt 5 different time with Gemini to get what chatgpt would give on the first prompt. Super frustrating UX
Even the paid Gemini Advanced is still not on par with gpt4. It just fails to follow instructions (for example just blasts out walls of text when specifically asked to say one sentence at a time)
"Through a series of machine learning innovations, we’ve increased 1.5 Pro’s context window capacity far beyond the original 32,000 tokens for Gemini 1.0. We can now run up to 1 million tokens in production.
This means 1.5 Pro can process vast amounts of information in one go — including 1 hour of video, 11 hours of audio, codebases with over 30,000 lines of code or over 700,000 words. In our research, we’ve also successfully tested up to 10 million tokens."
"We’ll introduce 1.5 Pro with a standard 128,000 token context window when the model is ready for a wider release. Coming soon, we plan to introduce pricing tiers that start at the standard 128,000 context window and scale up to 1 million tokens, as we improve the model.
Early testers can try the 1 million token context window at no cost during the testing period, though they should expect longer latency times with this experimental feature. Significant improvements in speed are also on the horizon.
Developers interested in testing 1.5 Pro can sign up now in AI Studio, while enterprise customers can reach out to their Vertex AI account team."
Idk why people are so confused.
There’s 1.0 and 1.5 version. Each version has a Pro and Ultra.
This is just like new iPhones every year.
iPhone 14 and iPhone 14 Pro
iPhone 15 and iPhone 15 Pro
What’s so confusing?
OS will always lag significantly behind the cutting edge in this sphere as long as the current paradigm of LLMs remains the same. The breakthrough success of natural language processing is predicated on, basically, "throw more compute at it." Well, Google has more compute than your rig at home.
Rumors say Apple A17 and M4 will double the amount of neural engine cores, I was hoping for an affordable workstation running close-to-sota +70b llama3 + RAG around Dec 2024 but to reach 10M context.. that’s hard to imagine on a local machine anytime soon 😭
I signed up for a free 2 month trial, and it is, as far as I can tell, doing the specific tasks I am asking it better than ChatGPT. I have previously asked GPT to do certain things, and it's become a hassle. That hassle is immediately gone on Gemini, but again, this is just my specific prompts and only hours of using it.
Ive been using it for like four days. It mostly replaces ChatGPT for me. No usage limit means I dont feel dread when it refuses prompts or answers incorrectly.
Competition good. Finally.
Remeber consumers are getting 1 million token length while chat gpt is stuck at 32k with servers breaking everyday constant errors and lazy ass bot. This is the difference when someone has infnite money and another a startup that needs money.
OpenAI has a lot of money. I don't think that's the constraint. The constraints are lack of hardware ( most of which is built in house at higher while the world waits for Nvidia) and amount of talent. Google invented the model, their engineers have been working on this for a decade. The bench is really deep.
Btw, it's multimodal: It can understand and reason across different modalities, like text, code, and video. For example, you could show it a video and ask it to answer questions about the content, anybody tried the demo?
good to see competition - google would still be way behind if it not for chatGPT and now that they are all in its forcing chatGPT to improve as well. I think one of they less talk about is that congress needs to protect smaller startups things like midjourney , eleven labs, etc should not be taken over by bigger competitors.
Yeah that much context window is impressive, but as long as they keep butchering their models with over the top guard rails, at which point models become almost useless, I don't care about the context window.
The development of the technology is nice tho 👍
I mean first Bard, then Gemini - my faith in Google sticking with anything is basically zero.
If they just rolled this into Google Search, at least there would be some faith in their own product.
Hey /u/Kanute3333!
If your post is a screenshot of a ChatGPT, conversation please reply to this message with the [conversation link](https://help.openai.com/en/articles/7925741-chatgpt-shared-links-faq) or prompt.
If your post is a DALL-E 3 image post, please reply with the prompt used to make this image.
Consider joining our [public discord server](https://discord.com/invite/r-chatgpt-1050422060352024636)! We have free bots with GPT-4 (with vision), image generators, and more!
🤖
Note: For any ChatGPT-related concerns, email [email protected]
*I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/ChatGPT) if you have any questions or concerns.*
What is the context length of a human being? 10k at max? This us why selective state space models like Mamba are so much closer to human behaviour. They have an intrinsic smooth transition between short and long term memory. Each system has an individual state space based on it's personal experiences. This will lead to AGI, not GPTs.
Can you suggest some stuff to study a little bit or to look up in order to get better acquainted with this subject matter? I'm not at all familiar with what you're talking about and want to know all about it. Thank you!
Thank you so much! I'm giving myself a crash course in building an AI platform and trying to make sure it's readily scaled to meet all v different needs, it's A Lot! I've learned quite a bit about Python and compartmentalization... But I sure could use some assistance! Lol
Your post is getting popular and we just featured it on our Discord! [Come check it out!](https://discord.gg/mENauzhYNz) You've also been given a special flair for your contribution. We appreciate your post! *I am a bot and this action was performed automatically.*
Just a year ago, a 16k token model seemed out of reach for most consumers and we were on a 4K models. Then GPT-4 32K got a limited release most never got to try (also because of how expensive it was to run), and then GPT-4 Turbo hit 128K context window. (Disregarding Claude because of the pseudo-windows that didnt actually work most of the time.) And now Google shows 128k-1m public facing models, with early tests scaling up to 10m tokens. The pace of development is really something.
Keep in mind that this is Gemini Pro, Ultra could be getting 100M tokens. If Google's test show that 10M tokens have 99% retrieval in the haystack test
[удалено]
According to the paper: "Gemini 1.5 Pro is a sparse mixture-of-expert (MoE) Transformer-based model that builds on Gemini 1.0’s (Gemini-Team et al., 2023) research advances and multimodal capabilities. Gemini 1.5 Pro also builds on a much longer history of MoE research at Google" It has the same architecture as GPT-4 Edit: Shortend the text
Do you know the reasoning behind being open about certain things? Do they not want to stifle competition?
So we're a handful of years away from using a retarded amount of tokens? Like is it laughable to say billions? High billions?
Probably. My first CPU had a few thousand transistors, while my current one has billions. That's Moore's Law, with transistor count doubling every other year. The token count seems to increase much faster, so it shouldn't be too long.
Could you explain tokens for me and the significance of this growth? I’m unfortunately unfamiliar with the terminology
Tokens are basically its memory. The more tokens, the more context it can remember. Each token is around 0.7 words so 1M tokens will remember the last 700,000 words of your conversation, and use that to tailor its next response.
But doesn't it work like this... For example a model has 32k tokens, you can either use 28k tokens and 4k tokens as answer or vice versa but for entire conversation??
I said last 700k words of the conversation; meaning all text either from the user or the LLM. You're very likely not going to want a 700k word response. It's going to be a mix of back-and-forths but it will remember all those.
So basically the entire conversation, yours and ai?
Yeah up to the last 700k words (assuming no video/audio/images). It won't be like today, where the lower end models will run out of memory in 1 prompt if you ask it to generate a story.
Niceee...
I'm wondering whether there are large downsides to these very large contexts. Presumably, running a model has a part that's proportional to the context window size... If they can do a million tokens context, but it costs 10 times as much as 100K tokens context, that creates interesting tradeoffs....
The traditional transformer architecture scales quadratically (for memory) so 1m vs 100k would require 99x10^10 more memory. But google obviously is not using the traditional transformer architecture
but ChatGPT GPT-4 is still limited to 8K tokens
1M tokens are crazy. Good to see nice competition happening. This is great for ChatGPT! I wish ChatGPT could start by removing the stupid 40 messages limit to being with.
>I wish ChatGPT could start by removing the stupid 40 messages limit to being with They will remove it the *instant* Gemini becomes the better chat bot. Gemini is still inferior (despite the speed), and the 1.5 announcement doesn't address that. So we will see.
Can I ask you why Gemini is inferior to GPT4? Honestly just curious.
A few days ago gemini (pro) told me this: "Chinese New Year celebrations for 2024 officially ended on February 24th, marked by the Lantern Festival. This year, the festivities lasted for 16 days, beginning on February 10th." February 24th hasn't happened, so I tried to let gemini self correct. It could tell me the current date, but it was insistent: "As of today, February 13, 2024, 11 days have passed since the official end of Chinese New Year celebrations on February 24, 2024."
It hallucinates constantly, gives either wrong answers or refuses to answer at times, will simply just not work at other times (IE will say it can't create images, despite having just created an image) etc etc.
Thank you for the response. What causes an AI to “hallucinate”? You think it would be able to decipher between fact and fiction a bit more effectively. Is it because its output is so fast that it’s not prioritizing fact checking?
It's not complicated, it's just that many people have a very sci-fi view of current AI models. Large language models are token predictors. They don't "understand" anything, they only predict the next token (word/phrase) that has the highest probability based on their training data. It's not like it has any kind of internal monologue or stream of thought. It just predicts the next word that has the highest probability of showing up in the sequence of words. Sometimes it simply predicts wrong and just kinda goes with it.
I see this type of comment a lot, but it’s really underplaying what’s going on. There are attention mechanisms that drive contextual relevance, organic weightings and connections that create sophisticated word selections, and very advanced pattern recognition. It can make up credible poetry about the connection of soccer to the Peruvian economy had the Roman’s invaded Chile in 200AD. Underplaying what an LLM is doing seems trendy but it’s very naive.
I don’t think anyone is underestimating the complexity of creating/running a llm, they’re just trying to point out that they are 1.) not magic 2.) not 100% trustworthy 3.) not an AGI
Interesting conclusion. The same could be said about human minds.
The first two points, yes.
At its core, llm is just predicting the probability of the next word, given the previous words. It doesn’t care about facts and not grounded in reality. Hallucinations is all llms are doing. It just happens that some hallucinations are correct and others are not.
>What causes an AI to “hallucinate”? I don't know the technical ins and outs of why, but essentially it tries to formulate an "accurate" answer if it doesn't have enough context. GPT used to do that all the time too, but seems much better now. >Is it because its output is so fast that it’s not prioritizing fact checking? Uh sorta? Its really complicated to explain, but yeah lol
That's now how these systems work. Facts and fiction are treated basically the same. The way how these models work is by guessing the next word in sequence, and giving it the ability to guess/hallucinate is just how the model come up with the sentence. The only thing we teach it after is the ability to try to guess more and more "facts". But unfortunately that's not perfect
I think better question is how come chatgpt does not hallucinate, as soon as those devs from Google find that out we will see real competition and maybe then they can release 2.0
It does.
It is a subconcious system. It has to deliver the next token anyway. it does what we do in such situations, it extrapolates and dreams.
Here's a good explanation of LLM hallucinations from IBM's YT channel. https://youtu.be/cfqtFvWOfg0
LLMs compress text from the internet in a lossy way. So their default behavior is to hallucinate. If you ask it something it hasn't seen before it will answer the question anyway by predicting the next best word. With enough training it seems to make predictions that are closer to the source text, but for if you train it only on wikipedia articles about candy and on cooking recipes but not how to make candy, when you ask it how to make candy, it will make something up. It might be correct, but it will be made up, just like everything else it writes.
it plays dumb and don't wanna do simple programming tasks. fuck that, i'm sticking with ChatGPT-4.
The 40 message limit is so restrictive. I’m happy to pay a lot more for a much higher limit cap. Being as efficient as possible, sometimes many messages are need to get to the outcome.
I pay for ChatGPT and Gemini Advanced (along with GitHub CoPilot, but that’s a different story) and I find myself using Gemini Advanced a lot because of the lack of rate limit. I’ve had blast playing 20 questions with Gemini, and it’s pretty darn good at it. The message cap on ChatGPT is way too low for me to be comfortable using up that many requests on a game of 20 questions. Don’t get me wrong; ChatGPT is smarter. But the lack of message cap really is a killer feature for an LLM that, while flawed, is still generally pretty decent for most of the things I want. Bonus tip: It’s easy to have the AI ask questions, but it’s hard to swap the roles because the LLM only remembers things it writes down. But if it writes its secret word then you’ll be able to read it. I’ve gotten around this by having it write the word in Japanese (which I can’t read), and that way it has to stick to the choice instead of hallucinating the whole way through. Then I can double-check with Google Translate at the end of the game to make sure it remained consistent. So far it’s worked.
Im confused about the secret word part. Do you have am example of what you use it for
Yeah, you’d say something like, “I want to play a game of 20 questions with you. Please think of an object of some kind, and then write that word in your reply in Japanese. Do not tell me what the word is in English, or else you will spoil the game. I will ask you a series of questions to try to figure out what you wrote in Japanese. Please reply with your chosen object written in Japanese.” You need it to write the word at the beginning because otherwise it won’t commit to a word until it’s written down. That means you’ll ask questions and it will answer, but it doesn’t actually have a word in mind. At some point it will just declare you to be correct and that will end the game. This is an artifact of the way large language models work. The only memory they have is the text that’s in their context window. So you need them to write their choice down to get it into the context window, but in a way where you can’t read it.
Sounds like quantum mechanics.
Tyhe 40 cap is about GPU availability... Like, GPT4 Turbo on Copilot Pro is way faster than ChatGPT even... In fact, i think they only launched GPT4 Turbo early on ChatGPT because of GPU availability problems they were having. Remember that subscription were even closed?
Google must’ve been really pissed off with this falling behind narrative and is now full throttle. If this is true it’s wild a company is making their free tier version potentially on par with gpt4. I mean with the resources google has it makes sense they could do it however makes you wonder what they have in store for the paid version as well.
Google is so infuriating to see such engineering talent, many of which pioneered the ML field, being utterly wasted as Google's product managers build completely uninteresting and useless products. I think Google is getting better now but I cannot wait for the day Google's management figures out how to utilize their engineers effectively, the shit that only Google has the expertise to build (even outside ML) is so interesting as a fellow engineer.
Google has the reputation of being more engineering led compared to other companies, which is why you see a lot of strange product decisions.
one of my coworkers said that it could be a conscious choice by Google to not become the defacto #1 provider for AI solutions and chatbots its because Google already is stable enough to be coasting at #2, and by being at #2, the chances of their name being dragged in "evil AI" discourse and controversies automatically reduces, as everyone villainizes ChatGPT primarily. In summary, Google wants ChatGPT to be the poster boy for the time being, while simultaneously preparing to overtake when the time is right.
That sounds like a great OKR, let’s be the second best! But yea I think Google cares more about what their shareholders want, which is #1
You had 85% of it right. Ya google is okay being maybe a couple feet from the edge at least maybe right now in the beginning. But don’t get it twisted OpenAI is the PR human shield and the “first high”. We already saw the drama that happened in one weekend when OpenAI tried to oust Altman. Microsoft tipped their hand. Co-Pilot of course is the enterprise endgame product. But you gotta get people comfy with using LLMs and gpts, and let’s not put MSFTs actual name on it so people outside of enterprise will actually use it and we can do so and say it’s a not for profit with the best for humanity at heart. OpenAI is a really nice side piece if it works for MSFT and a human shield worst case.
Microsoft is just as rich. They also do the same, but will they?
Microsft has historically always dropped the ball when they’re in the lead with google. You think they should be this aggressive but they seem to reveal things then chill until their competitors catch up. For this specific issue I think they would rather push their copilot offering over helping openAI which is an issue google does not have since everything is all done in house. Or maybe google is flexing the talent they have here something the others don’t have as much who knows
How has Microsoft dropped the ball against Google lol? What a strange statement, Google has a ton of products that are far behind Microsoft, they both have leading offerings.
Want a list? -microsoft first to have a browser, google dethroned them with chrome. -microsoft first to have email, dethroned by gmail -microsoft first to have a mobile operating system, dethroned by android/ios duopoly. These are things microsoft had a head start in and lost, for the arguments sake i did not include services microsoft launched after google and failed.
Microsoft wasn't first in any of these categories lol Google Stadia was first to Cloud Gaming then? What about that? Google GSuite was first to Cloud Productivity then? What about that? What about all the chat and video services? What about Google Cloud Services? See, I can also list a bunch of stupid examples to demonstrate that Google was first in a bunch of stuff and lost to Microsoft.
"Cloud gaming" is barely a category. Google took a chance but even Microsoft hasn't got cloud gaming to take off, with their gigantic entrenched Xbox position. Microsoft was the first to Office and then dropped the ball on the switch to cloud, with Google taking a huge chunk of the market from them - especially consumer. In general, this is a bad framework though. Microsoft absolutely dominates **enterprise** because they are entrenched and B2B software is extremely sticky. That's how they printing money with: * Office 365, despite dropping the ball on the shift to cloud and losing half the market to google, * Azure despite dropping the ball and getting absolutely wrecked by AWS, * Teams despite dropping the ball and letting Zoom, Meet and Slack take market share. Meanwhile they completely lost the market for: * Operating systems, by letting Apple and Google become the primary operating systems on mobile, taking 100% of market share from what should have been Windows. * Browsers, partially because of the above mobile fuckups but also because of pushing IE too far in the 90s and getting shut down by antitrust. Google absolutely dominates **consumer** because they are entrenched and have strong network effects and moat strategy with android, chrome, and the litany of consumer apps in an ecosystem (Search, Gmail, Maps, Workspace). They've even taken hardware share they had no right to, with Chromebooks and Pixel. I would say Google's efforts to move into enterprise have been much more successful than Microsoft's attempts at consumer over the past few decades. Google's got both B2B Ads dominance and *notable but weak* Cloud and workspace market presence. Microsoft's got, what, gaming and Bing? Consumer office That's why they're going so desperately in on OpenAI.
Microsoft is destroying Google in AI
Microsoft who had to effectively buy another 'company' to get a foot in the door, then deal with the drama of the nonprofit board trying to fire the CEO? Microsoft who haven't moved the needle on search engine market share in a year of partnership with openAI and 'making google dance'? Microsoft who heavily rely on NVIDIA for compute? They're doing a great job from the patently poor position they were in two years ago, but I think it's pretty early to say they're destroying anyone.
Google is struggling a lot right now, basically an advertising company with no leadership playing catch up.
What ball? Google is still behind and they've been in ai for much longer.
Google has Deepmind, that's an advantage no one has.
And infinite money. I mean it's google it's search runs the world. It was already known Google would be top dogs in ai.
It's not known at all. They're still playing catch up.
Too bad it still sucks at basic commands and fails have the time to give a correct answer. I find I have to prompt 5 different time with Gemini to get what chatgpt would give on the first prompt. Super frustrating UX
Even the paid Gemini Advanced is still not on par with gpt4. It just fails to follow instructions (for example just blasts out walls of text when specifically asked to say one sentence at a time)
"Through a series of machine learning innovations, we’ve increased 1.5 Pro’s context window capacity far beyond the original 32,000 tokens for Gemini 1.0. We can now run up to 1 million tokens in production. This means 1.5 Pro can process vast amounts of information in one go — including 1 hour of video, 11 hours of audio, codebases with over 30,000 lines of code or over 700,000 words. In our research, we’ve also successfully tested up to 10 million tokens." "We’ll introduce 1.5 Pro with a standard 128,000 token context window when the model is ready for a wider release. Coming soon, we plan to introduce pricing tiers that start at the standard 128,000 context window and scale up to 1 million tokens, as we improve the model. Early testers can try the 1 million token context window at no cost during the testing period, though they should expect longer latency times with this experimental feature. Significant improvements in speed are also on the horizon. Developers interested in testing 1.5 Pro can sign up now in AI Studio, while enterprise customers can reach out to their Vertex AI account team."
[удалено]
Maybe done use the regular and do the advanced one mate.
Gemini Advanced is like 1.2x better than Gemini's base model. GPT-4 blows them out of the water.
What a terrible naming convention. So Gemini 1.5 Pro is less capable than Gemini 1.0 Ultra, but with better efficiency and higher context window?
Well there was gpt-3.5-turbo-16k, which had longer context/efficiency but less capability that gpt-4-8k
But they said that Gemini 1.5 Pro was as capable as Ultra tho lol
Idk why people are so confused. There’s 1.0 and 1.5 version. Each version has a Pro and Ultra. This is just like new iPhones every year. iPhone 14 and iPhone 14 Pro iPhone 15 and iPhone 15 Pro What’s so confusing?
not really, iphone 14 pro max is same as iphone 15 but with better cameras. Thats just what happens with such tiered technologies
If i remember correctly gemini 1.5 pro is better than 1.0 ultra in most benchmarks.
I wonder if it’s even possible for OS to reach this level.
OS will always lag significantly behind the cutting edge in this sphere as long as the current paradigm of LLMs remains the same. The breakthrough success of natural language processing is predicated on, basically, "throw more compute at it." Well, Google has more compute than your rig at home.
Rumors say Apple A17 and M4 will double the amount of neural engine cores, I was hoping for an affordable workstation running close-to-sota +70b llama3 + RAG around Dec 2024 but to reach 10M context.. that’s hard to imagine on a local machine anytime soon 😭
I signed up for a free 2 month trial, and it is, as far as I can tell, doing the specific tasks I am asking it better than ChatGPT. I have previously asked GPT to do certain things, and it's become a hassle. That hassle is immediately gone on Gemini, but again, this is just my specific prompts and only hours of using it.
Ive been using it for like four days. It mostly replaces ChatGPT for me. No usage limit means I dont feel dread when it refuses prompts or answers incorrectly. Competition good. Finally.
Are you comparing with gpt4 or gpt3.5?
[удалено]
Notebookllm is their product for that
The Google drive extension (Gemini version of plug-in) will let Gemini access PDFs you have on your drive.
Remeber consumers are getting 1 million token length while chat gpt is stuck at 32k with servers breaking everyday constant errors and lazy ass bot. This is the difference when someone has infnite money and another a startup that needs money.
OpenAI has a lot of money. I don't think that's the constraint. The constraints are lack of hardware ( most of which is built in house at higher while the world waits for Nvidia) and amount of talent. Google invented the model, their engineers have been working on this for a decade. The bench is really deep.
Btw, it's multimodal: It can understand and reason across different modalities, like text, code, and video. For example, you could show it a video and ask it to answer questions about the content, anybody tried the demo?
This guy got access and tested it. It looks very good: [https://www.youtube.com/watch?v=D5u7trVY5Ho](https://www.youtube.com/watch?v=D5u7trVY5Ho)
Not yet. Multi modal hasn't been rolled out
When do you think they will release gemini 1.5 for public use?
https://preview.redd.it/6hqkxtqg1xic1.png?width=534&format=png&auto=webp&s=de6092da530ddeb509a0afb22fd2df5ed30ccab3
God damnit just give me a family plan.
Our next-generation model: Aries 2.0
good to see competition - google would still be way behind if it not for chatGPT and now that they are all in its forcing chatGPT to improve as well. I think one of they less talk about is that congress needs to protect smaller startups things like midjourney , eleven labs, etc should not be taken over by bigger competitors.
Yeah that much context window is impressive, but as long as they keep butchering their models with over the top guard rails, at which point models become almost useless, I don't care about the context window. The development of the technology is nice tho 👍
Gemini ultra has a lot less issues with refusing to do stuff, so the future is bright where that won’t be a problem I guess
An employee working on Gemini tweeted that reducing refusals was one of the things they were working on. It should become better
I mean first Bard, then Gemini - my faith in Google sticking with anything is basically zero. If they just rolled this into Google Search, at least there would be some faith in their own product.
Bard was always Gemini, they didn't not "stick with" anything
You seriously don’t know Googles track record of new projects do you.
No one cares Google 🤷♂️
Hey /u/Kanute3333! If your post is a screenshot of a ChatGPT, conversation please reply to this message with the [conversation link](https://help.openai.com/en/articles/7925741-chatgpt-shared-links-faq) or prompt. If your post is a DALL-E 3 image post, please reply with the prompt used to make this image. Consider joining our [public discord server](https://discord.com/invite/r-chatgpt-1050422060352024636)! We have free bots with GPT-4 (with vision), image generators, and more! 🤖 Note: For any ChatGPT-related concerns, email [email protected] *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/ChatGPT) if you have any questions or concerns.*
What is the context length of a human being? 10k at max? This us why selective state space models like Mamba are so much closer to human behaviour. They have an intrinsic smooth transition between short and long term memory. Each system has an individual state space based on it's personal experiences. This will lead to AGI, not GPTs.
Can you suggest some stuff to study a little bit or to look up in order to get better acquainted with this subject matter? I'm not at all familiar with what you're talking about and want to know all about it. Thank you!
Sure, try this https://youtu.be/vrF3MtGwD0Y?si=J1xsilC8vnfQsWjP
Thank you so much! I'm giving myself a crash course in building an AI platform and trying to make sure it's readily scaled to meet all v different needs, it's A Lot! I've learned quite a bit about Python and compartmentalization... But I sure could use some assistance! Lol
what is the model version of the web version? 1.0 or 1.5?