Waypoint AI @waypointai

Streamlining knowledge work with LlamaParse, Fireworks, and MongoDB! Check out this neat project and blog post from Team CLAB, winners of our recent hackathon, that creates a full-stack documentation bot that uses ✅ LlamaIndex to parse and orchestrate ✅ Nomic embeddings via…

Thanks to @nextjs app router & RSC, @vercel dashboard (e.g.: new Usage tab) has been getting faster notably, product surface is larger & there are more cooks in the kitchen than ever best part: will get even faster in the coming weeks 🫡

Check out our first-party guide to building advanced RAG with @llama_index + the AWS ecosystem ⭐️ ✅ integration with Bedrock LLMs/Knowledge Base/Agents ✅ Use S3 and Step Functions with LlamaParse and LlamaCloud ✅ Build Agentic RAG with Bedrock Agents + Lambda functions +…

LlamaIndex 🦙 @llama_index

5 days ago

We’re excited to feature @llama_index + AWS workshop materials featuring 3+ patterns for building LLM apps on AWS 💫 These include: 1️⃣ Using S3 as a data source for ingestion (with LlamaParse and LlamaCloud) 2️⃣ Use @llama_index with AWS Bedrock LLMs and embeddings 3️⃣ Using…

1 26 154 39K 107

Download Image

Llama-3 70b QLoRA finetuning is 1.83x faster & uses 63% less VRAM than HF+FA2 1. Llama-3 70b + Unsloth can fit 48K context lengths on bsz=1 on A100 80GB (6x longer than FA2) with +1.9% overhead 2. Llama-3 8b QLoRA fits in a 8GB card & is 2x faster, uses 68% less VRAM. Can fit…

Unsloth AI @UnslothAI

7 days ago

Long-context Llama 3 finetuning is here! 🦙 Unsloth supports 48K context lengths for Llama-3 70b on a 80GB GPU - 6x longer than HF+FA2 QLoRA finetuning Llama-3 70b is 1.8x faster, uses 68% less VRAM & Llama-3 8b is 2x faster and fits in a 8GB GPU! Blog: unsloth.ai/blog/llama3

3 46 300 58K 146

Download Image

Llama 3 🤜🤛 Phi 3 There's been some exciting advancements in the open-source LLM space, and @akshay_pachaar provides a fantastic reference architecture to compare the two models in RAG capabilities. Check out the detailed architecture diagram and link to the open-source code…

Akshay 🚀 @akshay_pachaar

7 days ago

Let's compare Llama-3 & Phi-3 using RAG:

35 173 1K 732K 2K

Coming out to SF with my @waypointai starting next month for some accelerator fun time. Would love to connect with product & engineering leaders while I'm out there to talk about using AI to protect engineering time. Any recommendations?

Microsoft presents Multi-Head Mixture-of-Experts Achieves notable improvements over the baseline MoE by using multiple MoE heads repo: github.com/yushuiwx/MH-MoE abs: arxiv.org/abs/2404.15045

Corrective RAG == retrieval with reflection 🔎🪞 A way to make RAG more advanced is to expand the decision making capabilities in the middle of the RAG process. - Pre-retrieval: use the LLM to do query decomposition, API parameter inference, etc. - Post-retrieval (as shown…

LlamaIndex 🦙 @llama_index

a week ago

A way to fix bad retrieval issues in RAG is to add a “reflection” layer on the retrieved context 💡 Corrective RAG (CRAG, Yan et al.) adds a retrieval eval module that categorizes gathered information into “Correct”, “Incorrect”, and “Ambiguous”. In the event that context is…

2 74 326 68K 325

Download Image

There aren't nearly enough thoughtful papers on evals so I'm excited to go through @sh_reya's paper in detail

Shreya Shankar @sh_reya

a week ago

Evals are arguably the hardest part of LLMOps. LLMs mess up, so we check them w/ other LLMs, but this feels icky. Who validates the validators?? We built an interface to align LLM-based evals with user preferences, learning a lot about why this is hard: arxiv.org/abs/2404.12272

8 72 382 70K 345

Fine-tuning Embedding Models for RAG with LoRA 💡 This is a great tutorial (by Mariboo) showcasing how to cleverly subclass @llama_index finetuning abstractions to finetune any @huggingface embedding model with LoRA weights, which you can then plug in as a crucial step of any…

Inspired by @AndrewYNg’s threads on agents in the past few weeks, I’m excited to share my talk from the @weights_biases conference that outlines how to build a general context-augmented research assistant 🧑‍🔬 Naive RAG is mostly good for simple questions in a single-shot setting.…

⚡️🔬📣Excited to share our new @NatureMedicine article, examining disparities in pathology AI models, assessing how modeling choices impact disparities, and evaluating the potential of self-supervised foundation models in mitigating these disparities. nature.com/articles/s4159… See…

This project is sick 👇 - learn how to stream a knowledge graph step-by-step by hooking into LLM streaming, partial JSON parsing in @llama_index, and rendering with the @vercel AI SDK 🔥 Streaming text in the UI is standard for any chatbot experience, but streaming visual…

Rohan @clusteredbytes

a week ago

Open source AI Diagram Generator 🔥 Uses @llama_index Pydantic program with partial JSON parsing and @vercel AI SDK to send intermediate diagrams during generation for improved UX 🚀 Repo: github.com/rsrohan99/ai-d… Full tutorial under 2.5 minutes 👇

3 40 308 189K 456

Download Video

This stack (DREAM) is a must-reference resource for any AI engineer looking to setup principled experimentation/eval infrastructure for productionizing LLM apps. Use Ray, Ragas, @llama_index, and @MLflow to easily experiment with all the different parameter settings in a RAG…

LlamaIndex 🦙 @llama_index

2 weeks ago

DREAM - a Distributed RAG Experimentation Framework 🧑‍🔬 Building RAG comes with a lot of knobs that you need to tune, and it’s important that you setup the right experimentation infrastructure to build production RAG. This project by Aishwarya Prabhat provides a comprehensive…

11 117 535 101K 635

Download Image

🔥llm.c update: Our single file of 2,000 ~clean lines of C/CUDA code now trains GPT-2 (124M) on GPU at speeds ~matching PyTorch (fp32, no flash attention) github.com/karpathy/llm.c… On my A100 I'm seeing 78ms/iter for llm.c and 80ms/iter for PyTorch. Keeping in mind this is fp32,…

We present LLaMADuo 🦙 It's a collection of Python scripts and configurations that implement how to improve a small language (typically open) with a bigger one (typically gated). Hop in for a 🧶 1/7

HISTORIC MOMENT! Llama-3 70B numbers are INSANE! At 82.0 MMLU, it's FAR AND AWAY the best OSS Model. GSM-8K, Math, and Human Eval are MIND BLOWING as well. The OSS community is definitely going to beat GPT-4 in a matter of weeks!! Xmas came very very early

🥁 Llama3 is out 🥁 8B and 70B models available today. 8k context length. Trained with 15 trillion tokens on a custom-built 24k GPU cluster. Great performance on various benchmarks, with Llam3-8B doing better than Llama2-70B in some cases. More versions are coming over the next…

Slides are done! See you tomorrow at 4:30pm PST for the whole story. From alpaca, openassistant, qlora, zephyr, tulu, to today. Slides: docs.google.com/presentation/d… Course link (with zoom): web.stanford.edu/class/cs25/

Nathan Lambert @natolambert

3 weeks ago

Watch my @Stanford CS 25 lecture next week, "aligning open language models," it'll be good, v excited

25 96 629 139K 388

Download Image

Today, we are pleased to announce the availability of Stable Diffusion 3 and Stable Diffusion 3 Turbo on the Stability AI Developer Platform API. We have partnered with @FireworksAI_HQ , the fastest and most reliable API platform in the market, to deliver these models. In…