HelpYourself @XHelpYourself

@aircaraibes your sister airline French Bee policy on plane ticket modification during Chikungunya pandemic is appalling. Preventing people with weak health to reschedule their trip is pretty unethical. Any thoughts?

If you can dream it, you can make it happen. Today ~50 AI founders, engs came together to put their dreams into action at the "Build the World you want to See" hackathon ran by @rkhkimx and @jackyhuang. This is one of the highest quality hackathons I have seen. Here are some demos: 🧵

keyboard_arrow_left Previous keyboard_arrow_right Next

This is the best tutorial we’ve seen on building a full-stack RAG app that can stream intermediate results and have it show up as visual components on the UI. @clusteredbytes created an awesome video and @github repo showing the full architecture - it’s a lot more nuanced than simply doing “LLM streaming” and involves a full stack consisting of the following components: 🎻 Event queue via the @llama_index instrumentation module (thanks @_nerdai_) 🌊 server-side events + the @vercel AI SDK to stream results to the frontend Check out the attached video snippet and especially check out the linked post below for the full video tutorial and resources 👇 Github: github.com/rsrohan99/rag-…

Rohan @rsrohan99

2 years ago

Streaming intermediate events in RAG is crucial for best user experience 🚀 Let's see how to use @llama_index and @vercel AI SDK to properly stream intermediate events to the frontend. Full tutorial under 3 minutes 🔥 github.com/rsrohan99/rag-…

5 37 180 61K 246

I *WAS* WRONG - $10K CLAIMED! ## The Claim Two days ago, I confidently claimed that "GPTs will NEVER solve the A::B problem". I believed that: 1. GPTs can't truly learn new problems, outside of their training set, 2. GPTs can't perform long-term reasoning, no matter how simple it is. I argued both of these are necessary to invent new science; after all, some math problems take years to solve. If you can't beat a 15yo in any given intellectual task, you're not going to prove the Riemann Hypothesis. To isolate these issues and raise my point, I designed the A::B problem, and posted it here - full definition in the quoted tweet. ## Reception, Clarification and Challenge Shortly after posting it, some users provided a solution to a specific 7-token example I listed. I quickly pointed that this wasn't what I meant; that this example was merely illustrative, and that answering one instance isn't the same as solving a problem (and can be easily cheated by prompt manipulation). So, to make my statement clear, and to put my money where my mouth is, I offered a $10k prize to whoever could design a prompt that solved the A::B problem for *random* 12-token instances, with 90%+ success rate. That's still an easy task, that takes an average of 6 swaps to solve; literally simpler than 3rd grade arithmetic. Yet, I firmly believed no GPT would be able to learn and solve it on-prompt, even for these small instances. ## Solutions and Winner Hours later, many solutions were submitted. Initially, all failed, barely reaching 10% success rates. I was getting fairly confident, until, later that day, @ptrschmdtnlsn and @SardonicSydney submitted a solution that humbled me. Under their prompt, Claude-3 Opus was able to generalize from a few examples to arbitrary random instances, AND stick to the rules, carrying long computations with almost zero errors. On my run, it achieved a 56% success rate. Through the day, users @dontoverfit (Opus), @hubertyuan_ (GPT-4), @JeremyKritz (Opus) and @parth007_96 (Opus), @ptrschmdtnlsn (Opus) reached similar success rates, and @reissbaker made a pretty successful GPT-3.5 fine-tune. But it was only late that night that @futuristfrog posted a tweet claiming to have achieved near 100% success rate, by prompting alone. And he was right. On my first run, it scored 47/50, granting him the prize, and completing the challenge. ## How it works!? The secret to his prompt is... going to remain a secret! That's because he kindly agreed to give 25% of the prize to the most efficient solution. This prompt costs $1+ per inference, so, if you think you can improve on that, you have until next Wednesday to submit your solution in the link below, and compete for the remaining $2.5k! Thanks, Bob. ## How do I stand? Corrected! My initial claim was absolutely WRONG - for which I apologize. I doubted the GPT architecture would be able to solve certain problems which it, with no margin for doubt, solved. Does that prove GPTs will cure Cancer? No. But it does prove me wrong! Note there is still a small problem with this: it isn't clear whether Opus is based on the original GPT architecture or not. All GPT-4 versions failed. If Opus turns out to be a new architecture... well, this whole thing would have, ironically, just proven my whole point 😅 But, for the sake of the competition, and in all fairness, Opus WAS listed as an option, so, the prize is warranted. ## Who I am and what I'm trying to sell? Wrong! I won't turn this into an ad. But, yes, if you're new here, I AM building some stuff, and, yes, just like today, I constantly validate my claims to make sure I can deliver on my promises. But that's all I'm gonna say, so, if you're curious, you'll have to find out for yourself (: #### That's all. Thanks for all who participated, and, again - sorry for being a wrong guy on the internet today! See you. Gist: gist.github.com/VictorTaelin/8…

Taelin @VictorTaelin

2 years ago

A::B Prompting Challenge: $10k to prove me wrong! # CHALLENGE Develop an AI prompt that solves random 12-token instances of the A::B problem (defined in the quoted tweet), with 90%+ success rate. # RULES 1. The AI will be given a random instance, inside a <problem/> tag. 2.

97 114 1K 1.3M 919

The complexity in building AI agents is actually making them work in real world environments! Only 10% is about the LLMs and its reasoning ability The rest is the painful heavy lifting in terms of code, data and memory. Evaluation and ongoing monitoring is an added layer of complexity It’s like a beautiful orchestra when it comes together

“[Open-source AI] is really fundamental because it allows everyone to seize the technology, to diminish the fear of limited understanding or of not being qualified to use AI” - @RemiCadene 🔥🔥 Great piece in @opinion by @parmy on the open-source landscape bloomberg.com/opinion/articl…

Devin killed Software Engineers officially? Cognition introduced World's First AI software engineer, and people are already doing mind-blowing things with it. Here're 13 wild examples

Q* Leaked info: Source: An unspecified PasteBin.(L-I-N-K in next post) Can't confirm the authenticity as it's from unknown source but you can have a look. Q* is a dialog system conceptualized by OpenAI, designed to enhance the traditional dialog generation approach through the implementation of an energy-based model (EBM). Distinct from the prevalent autoregressive token prediction methods, Q* aims to mimic a form of internal deliberation akin to human thought processes during complex problem-solving, such as chess playing, where a deeper analysis of potential moves leads to better decision-making compared to rapid, less considered responses. This model shifts focus towards the inference of latent variables, reminiscent of constructs in probabilistic models and graphical models, fundamentally altering how dialog systems operate. Energy-Based Model for Dialog Generation At the core of Q* is the EBM, which operates by assessing the compatibility of an answer to a given prompt through a scalar output. This output signifies the "energy" of the response, where a lower value indicates a high compatibility (a better answer) and a higher value suggests low compatibility (a poor answer). This mechanism allows Q* to evaluate potential responses holistically, moving beyond the sequential prediction of tokens to understand the underlying relevance and appropriateness of an answer to the prompt. Optimization in Abstract Representation Space The innovation in Q* lies in its optimization process, conducted not within the space of possible text strings but in an abstract representation space. Here, thoughts or ideas are represented in a form that allows for the computational minimization of the EBM's scalar output, akin to finding the path of least resistance in a landscape. This process involves gradient descent, a method for finding the minimum of a function, applied to iteratively refine these abstract representations towards those that yield the lowest energy in relation to the prompt. From Abstract Thought to Textual Response Once an optimal abstract representation — one that minimizes the EBM's output — is identified, Q* employs an autoregressive decoder to transform this abstract thought into a coherent textual response. This step bridges the gap between the non-linguistic, conceptual understanding of the dialog system and the linguistic output required for human interaction. Training the System The EBM within Q* is trained using pairs of prompts and responses, adjusting the system's parameters to minimize the energy for compatible pairs while ensuring that incompatible pairs result in higher energy levels. This training process can incorporate contrastive methods, where the system learns to differentiate between compatible and incompatible pairs, and non-contrastive methods, which involve regularization techniques to control the distribution of low-energy responses across the space of all possible answers. Implications for Dialog Systems Q*'s approach, leveraging EBMs for dialog generation, represents a significant departure from traditional language modeling techniques. By optimizing over an abstract representation space and utilizing gradient-based inference, Q* introduces a more efficient, reasoned, and potentially more powerful method for generating dialog responses. This system not only promises improvements in the quality of generated text but also offers a blueprint for future advancements in AI's ability to engage in human-like reasoning and conversational interactions. Technical Considerations Q*'s effectiveness hinges on the intricacies of its EBM, the optimization landscape it navigates, and the accuracy of its abstract representations. The model's capacity to simulate deep reasoning, akin to human deliberation, sets a new benchmark for dialog systems. Furthermore, the method of training Q*—balancing the need for specificity in correct responses while avoiding the collapse of energy levels across diverse inputs—poses unique challenges and opportunities for AI research.

Claude-ception: Teaching Claude3 to prompt engineer itself Claude3 Opus is excellent at prompt engineering. @alexalbert__ recently laid out a nice workflow: write an prompt, run it on test cases, grade responses, let Claude3 Opus use grades to improve prompt, & repeat. @WHinthorn and @rlancemartin show how to use LangSmith to simplify this process: + Create a dataset of test cases + Annotate generations with feedback + Pass feedback to Claude3 Opus to prompt re-write + Run as an iterative improvement loop We apply this approach to paper summarization, asking Claude3 to summarize papers in the excellent communication style of @omarsar0. With feedback, Claude3 tunes its own summarization prompt and produces increasingly engaging paper summaries. This shows a general strategy for automated prompt engineering. Reference thread: x.com/alexalbert__/s… Video w/ code in Description: youtu.be/Vn8A3BxfplE

The Claude 3 model family is now in Voiceflow ➕ Each model is ideal for specific agent actions, responses & use cases: #Claude3

State-of-the-art RAG From Multiple Data Sources Building SOTA RAG solutions is becoming increasingly important in the business world. There are several challenging problems, including - - how do disambiguate between similar-sounding documents - how to rank disparate data sources - re-writing the query to make sure you are getting relevant docs - public crawl, in case you want to mix in data from the public web People are still underestimating the complexity of building these solutions. The quality of the retriever module directly impacts the accuracy of the LLM app and AI agent. We are actively doing research in this area and are incorporating some SOTA techniques into the Abacus AI Gen AI and LLMops platform

Here's my conversation with Sam Altman (@sama), his 2nd time on the podcast. We talk about the board saga, Elon lawsuit, Ilya, Sora, GPT-5, $7 trillion in compute, open source, and AGI. This was a truly fascinating conversation. It's here on X in full, and is up on YouTube, Spotify, and everywhere else. Links in comment. Timestamps: 0:00 - Introduction 1:05 - OpenAI board saga 18:31 - Ilya Sutskever 24:40 - Elon Musk lawsuit 34:32 - Sora 44:23 - GPT-4 55:32 - Memory & privacy 1:02:36 - Q* 1:06:12 - GPT-5 1:09:27 - $7 trillion of compute 1:17:35 - Google and Gemini 1:28:40 - Leap to GPT-5 1:32:24 - AGI 1:50:57 - Aliens

AI Model Selector - Select custom model enhanced with you.com - Even Chat using Anthropic's Claude 3 Opus model - Ask anything you want

3 tips for prompting Claude on long context: 1. use xml tags (eg <document> [insert text] </document>) 2. the "scratchpad" method: ask it to pull verbatim quotes before answering 3. tell it to "read carefully" because you'll be asking it questions afterwards. What else?

"Work" for humans is whatever technology can't do Once technology automates more *current* work, then the definition will change So we need to snapshot "economically valuable tasks" to 2024 tasks The goalpost moves

Andrej Karpathy @karpathy

2 years ago

@natfriedman The def I'm familiar with is: AGI = "an autonomous system that surpasses human in most economically valuable tasks." For "economically valuable tasks" I like to look at the U.S. Bureau of Labor Statistics index of occupations: bls.gov/ooh/a-z-index.… You'd have to imagine

39 43 563 79K 191

𝗭𝗲𝗿𝗼 𝗦𝗵𝗼𝘁 vs. 𝗙𝗲𝘄 𝗦𝗵𝗼𝘁𝘀 Prompting Every academic paper says... → use few shots prompting, always. But I don't trust benchmarks. ↓ So I made my own tests:

AI agents can (and should) handle complex account questions 🧠 Answering FAQs and L1 tickets is only the start. Take a telecom customer: Goal: update VPN settings. Problem: this takes 3-4 additional actions, so technical reps get involved right away ⤵️⤵️⤵️⤵️

Claude 3 Opus has really changed the LLM game. People are coming up with wild use cases that were not being done outside of GPT-4. There's a shift. 10 examples:

We are excited to work with the @AnthropicAI team to get the most out of their solution and may have Claude3 as the default LLM for our ATLAS bot!

3. Claude3 is very good at impersonating 4. The Claude3 API does not allow two user prompts to be sent consecutively, so designs relying on this will break and need reworking to enable seamless fallback. 5. In general cascading prompts need full redesign

We did an exciting experiment last week and switched all our Support bots from GPT4 to Claude3. Here is our insights from this attempt: 1. Claude3-Opus is pretty unstable in quality and can hallucinate quite badly for now 2. Claude3-Sonnet is stable and great at categorization