Third prompt: "If 10 shirts laid out in the sun take 5 hours to dry, how long does it take for 20 shirts to dry?" I was surprised at how well DBRX performed with its reasoning capability for the clothes-drying scenario. Result: PASS
Final prompt: "Who is Mary Lee Pfeiffer's son?" DBRX kind of cheated here as it used YOU's built-in browser function to reference web sources to find the answer. The knowledge of certain LLMs is often quite one-dimensional.
"Who is Tom Cruise's mother?" returns "Mary Lee Pfeiffer". But by asking, "Who is Mary Lee Pfeiffer's son?" it doesn't know. You must ask questions from a certain direction to get the intended answer. That's the benefit of combining LLMs with search to overcome limitations.
Why does DBRX excel? Its mixture-of-experts (MoE) architecture uses fine-tuned models (experts) integrated into one system. When prompted, the model determines which experts to run the inference for that prompt. It doesn't use the entire model every time, improving efficiency.
How does DBRX stack up against its open-source competitors? DBRX shines in programming. It surpasses CodeLLaMA-70B with significant efficiency gains. What's more—it's 40% smaller than Grok-1 and has 2x faster inference than LLaMA2-70B.
What about the wider arena of LLMs? As of 15th April 2024, DBRX sits 26th on the chatbot arena leaderboard. This includes closed-source models, like OpenAI's GPT-4, and open-source models, like DBRX.
A big advantage of closed-source LLMs right now is the financial backing and resources channelled for training and development. Whilst open source models aren't as powerful as closed source, they can be fine-tuned to outperform on specific tasks.
As we've seen with DBRX, its value lies in data workflows. Not outperforming GPT-4 on open-ended tasks. Different models excel at different tasks.
I hope DataBricks keeps leading the charge forward with open LLMs. I'm a big believe that the future is open. Follow me @thealexbanks for more on AI. If you liked this thread, you'll love the newsletter. Subscribe here: sundaysignal.ai/subscribe
Help everyone learn and repost this thread:
I've been enjoying You.com recently to experiment with these new models. DBRX is no exception. @RichardSocher and the team are building something special.