I heavily use Chat GPT's API in my day job, as it is the core of our business (Ai-powered startup). When Gemini Pro launched, me and my team tested it in the same day for our product, but we where disappointed as it is was a bit worse than gpt 3.5 (at least in the same prompts that we already had). I really hope that Gemini Ultra surpass gpt4, it is always exciting to see and use new advanced tech, but I'm still a little skeptical about it, since Pro wasn't that great...
> it is was a bit worse than gpt 3.5 (at least in the same prompts that we already had)
I'm willing to believe that Gemini isn't as good, but my impression was that you expect a new model to not perform as well on your existing prompts because the training set and training methodology is different. That's why one of the major risks of an AI business is vendor lock in, because you spend so much time optimizing a prompt for a specific model and you'll have to redo much of that work in order to switch vendors.
That you gave up so quickly when trialing a new model suggests the problem is even worse than I thought—you're locked in to OpenAI because every other model will always look worse to you, even if it would be better if you took the time to tune a new prompt.
Indeed, I believe Gemini is equally capable, but its response style significantly differs from ChatGPT 4's. My expectations for how chatbots should communicate have been largely shaped by ChatGPT's approach.
Specifically, Gemini's tendency to structure responses with bullet points and headings from the start doesn't appeal to me. It feels more like receiving an essay in response to a simple question, whereas ChatGPT's responses facilitate a flowing conversation. Gemini aims to provide exhaustive answers, which can be overwhelming when I'm interested in a more open-ended dialogue that smoothly transitions between topics. In essence, Gemini Advanced focuses on delivering detailed answers without allowing much time for reflection.
Therefore, when it comes to preference in RLHF (Reinforcement Learning from Human Feedback) chatbots, ChatGPT-4 is my favorite for its ability to better engage in the type of conversational dynamics I prefer.
It turns out iterating on and incorporating a large volume of user feedback is more important than having the most and most talented AI researchers, at least in the short term.
They have the most prestigious researchers and engineers that can jump through interview hoops. That doesnt translate to world changing products, and they arent "the best"
I'm sure there are multiple layers of meaning behind the name, but Jeff Dean once mentioned the name had something to do with the latin translation being twins. That is, Gemini is a product of Alphabet's "twin" AI orgs, Google Brain and DeepMind, working closely together, and eventually fusing into GDM.
Anybody really surprised at this point? G has had DeepMind in their pockets since ‘12-‘14 and made little advancements. OpenAI changed the game in half the time.
OpenAI is built on top of Google advancements and research. It didn't change the game, more like took a shortcut and landed on a gold mine.
The fact that many products and models including open source have catched up on such a short notice and now compete with OpenAI, in what should be their self-proclaimed backyard, suggest it's just a one-trick pony.
It's doubly strange because Google had an implicit reputation of being the unbeatable giant in computing research and resources.. many expected them to compete and smoke chatGPT in a few weeks. It's been months and nothing came up except fumblings and confusion.
Sundar has zero vision and has created a culture that stifles new developments in bureaucratic morass while threatening to kill them shortly after birth.
Google may have more scientists and some of the best minds in the business, but ChatGPT has nearly 200 million users that are feeding it back data for RLHF, and data is a much more important moat than better tech (which mostly ends up being published and disseminated anyhow).
AI is a game between OpenAI and Meta. ChatGPT has a ton of users creating highly relevant data, but Meta has the incredible walled trove of facebook/instagram/whatsapp/+ data that dwarfs pretty much anyone else on the planet, and with Mark's recent push to build up their compute their only competitors in that space are microsoft and google. People discounted Meta because of that horrible metaverse move, but Mark is being pretty canny now, they're very well positioned to choke the life out of specialty chatbot products while integrating SOTA AI into all of their products to slowly crank up the time people are on platform.
