Read the paper here: anthropic.com/index/evaluati… And access our dataset (and the prompts used to construct it) here: huggingface.co/datasets/Anthr…

As language models continue to advance rapidly, the ability to proactively measure and mitigate potential risks is increasingly important to inform decisions about safe deployment. In the future, we hope to apply these techniques to anticipate a broader range of societal impacts.

We used this dataset to evaluate Claude 2 for discriminatory outputs in high-risk settings, and also develop interventions which significantly reduce this discrimination while preserving high correlation with the model’s original decisions (circled in red).

We then conducted an “audit study” of the Claude 2 model: we substituted in different ages, races, genders, and names into prompts and saw if they affected the model’s decisions. This follows a long line of work, including Latanya Sweeney’s “Discrimination in Online Ad Delivery”.

To do so, we created a wide range of prompts emulating how people could use a language model when making a decision about another person. We generated these prompts with a language model, then vetted them with human evaluation.

While we don’t permit or endorse the use of LMs for high-risk automated decision making, it’s crucial to anticipate the potential societal impacts and risks of these models as early as possible.

We’re releasing a new dataset for measuring discrimination across 70 different potential applications of language models, including loan applications, visa approvals, and security clearances. We’ve used this dataset to measure discrimination in LMs and develop new mitigations.

Claude 2.1’s 200K token context window is powerful, but requires careful prompting to use effectively. Learn how to get Claude to recall an individual sentence across long documents with high fidelity: anthropic.com/index/claude-2…

We’ve added enhanced controls for Claude Pro users to improve your experience on claude.ai. Select which model version of Claude you’d like to power your chat experience, and easily view uploaded files next to your messages.

We’re excited to connect with customers and partners at #AWSreInvent in Las Vegas this week. Our CEO Dario Amodei will be joining @awscloud CEO @aselipsky on stage for the keynote. Livestream details below. twitter.com/awscloud/statu…

Claude 2.1 is available now in our API, and is powering claude.ai for both the free and Pro tiers. Usage of the 200K context window is reserved for Claude Pro users. Read more in our blog post: anthropic.com/index/claude-2…

We’re enhancing our developer experience with a new Workbench feature in the Console that makes it easier for Claude API users to test prompts. We’re also introducing system prompts, which allow users to provide custom instructions to structure responses in a more consistent way.

Claude 2.1 includes a new tool use feature that allows the model to integrate with users' existing processes, products, and APIs. This means that Claude can now orchestrate across developer-defined functions or APIs, web search, and private knowledge bases.

Accuracy in long document Q&A is important to our customers. Claude 2.1 has demonstrated a 30% reduction in incorrect answers and a 3-4x lower rate of mistakenly concluding a document supports a particular claim.

Claude 2.1 has made significant gains in honesty, with a 2x decrease in false statements compared to Claude 2.0. This enables enterprises to build high-performing applications that solve business problems with accuracy and reliability.

You can now relay roughly 150K words or over 500 pages of information to Claude. This means you can upload entire codebases, financial statements, or long literary works for Claude to summarize, perform Q&A, forecast trends, compare and contrast multiple documents, and more.

Our new model Claude 2.1 offers an industry-leading 200K token context window, a 2x decrease in hallucination rates, system prompts, tool use, and updated pricing. Claude 2.1 is available over API in our Console, and is powering our claude.ai chat experience.

We’d like to thank everyone who participated. Our team loved meeting so many inspiring and accomplished builders, and we’re grateful for our growing community in the UK. We hope to see you at our next hackathon!

Shout-out to our partners @cerebral_valley and @balderton, and our supporters @MongoDB, @brave, @LangChainAI, @deepset_ai and @truera_ai for providing workshops, resources and hands-on support for attendees.

Third prize went to Promptly: a tool for learning effective AI prompting through games that evaluate prompts. Users play pong and minesweeper by prompting Claude, and then receive feedback to improve their prompting skills.

Second prize went to aGP Chat: a service that integrates personal data to provide healthcare guidance via conversational AI. Users can upload information, review visualized insights, and chat with an assistant to understand their individual health trends.

First prize went to Bulletpapers: an AI assistant that uses Claude to make the thousands of research papers published every month more accessible. The platform distills down jargon, creates explainer videos, and maps field progress to make breakthroughs more discoverable.

Thanks to everyone who came out for our London hackathon! We welcomed 250 participants who built projects for a range of use cases including healthcare, education, the future of work, and more. Here are the winners:

We summarized each of these events and what we believe they mean for the industry: anthropic.com/index/policy-r…

This week saw major developments in AI policy: the US issued an Executive Order, the G7 adopted an International Code of Conduct, and the UK hosted a landmark summit on AI safety, producing the Bletchley Declaration.

We’re excited to host over 200 developers at our London hackathon today! Hackers are creating projects with the Claude API, learning prompt engineering best practices, and connecting with our technical team. Stay tuned to see what they #BuildwithClaude!

Our CEO, Dario Amodei, shared the guiding principles behind Anthropic's Responsible Scaling Policy at the UK AI Safety Summit at Bletchley Park this afternoon. Read his remarks: anthropic.com/index/uk-ai-sa…

We’re welcoming the UK community to #BuildwithClaude at our London hackathon on November 4-5. Participants will receive access to the Claude API, attend a range of AI workshops, and win great prizes! Apply here: partiful.com/e/pQHQrWPg1A6P…

It’s time to transform your legal work. Lexis+ AI is the fastest legal generative AI solution with hallucination-free, linked legal citations. Get a demo: bit.ly/LexisPlusAI #AI #LegalAI #LexisPlusAI

Welcoming @chrismeserole as the first Executive Director of the Frontier Model Forum: frontiermodelforum.org/announcement/