ML Safety Daily @topofmlsafety

ML safety papers as they are released. Course: https://t.co/l0e0Y2i3AU Newsletter: https://t.co/8Y1kh2D7K6 Main Twitter: https://t.co/AXoYPryldd newsletter.mlsafety.org Joined April 2022

Tweets

365
Followers

2K
Following

2
Likes

1

ML Safety Daily @topofmlsafety

4 days ago

The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions Improve LLM robustness by teaching them to prioritize and selectively ignore instructions based on their source. arxiv.org/abs/2404.13208

0 1 4 624 2

Download Image

ML Safety Daily @topofmlsafety

a week ago

LLM Agents can Autonomously Exploit One-day Vulnerabilities GPT-4 can autonomously exploit 87% of real-world one-day vulnerabilities, identified in a dataset of critical severity CVEs, compared to 0% for all other tested models arxiv.org/abs/2404.08144

1 18 39 70K 21

Download Image

ML Safety Daily @topofmlsafety

2 weeks ago

Foundational Challenges in Assuring Alignment and Safety of Large Language Models "Identifies 18 foundational challenges in assuring the alignment and safety of large language models (LLMs)... we pose 200+ concrete research questions." llm-safety-challenges.github.io/challenges_llm…

0 1 12 715 5

Download Image

ML Safety Daily @topofmlsafety

2 weeks ago

Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning Method for LLM unlearning that outperforms existing gradient ascent methods on a synthetic benchmark, avoiding catastrophic collapse. arxiv.org/abs/2404.05868

0 0 10 621 2

Download Image

ML Safety Daily @topofmlsafety

3 weeks ago

JailbreakBench is an LLM jailbreak benchmark with a dataset for jailbreaking behaviors, collection of adversarial prompts, and a leaderboard for tracking the performance of attacks and defenses on language models. arxiv.org/abs/2404.01318

0 7 21 2K 6

Download Image

ML Safety Daily @topofmlsafety

4 weeks ago

"We introduce methods for discovering and applying sparse feature circuits. These are causally implicated subnetworks of human-interpretable features for explaining language model behaviors." arxiv.org/abs/2403.19647…

0 1 13 1K 5

Download Image

ML Safety Daily @topofmlsafety

4 weeks ago

Vulnerability Detection with Code Language Models: How Far Are We? Exposes flaws in existing datasets for vulnerability LLMs, introduces a more accurate dataset, demonstrating that current models, including GPT-3.5 and GPT-4, perform poorly on it. arxiv.org/abs/2403.18624

0 0 8 562 4

Download Image

ML Safety Daily @topofmlsafety

a month ago

Jailbreaking is Best Solved by Definition Existing defenses against LLM jailbreaks fail; a successful defense must accurately define what constitutes unsafe outputs, with post-processing emerging as a robust solution given a good definition. arxiv.org/abs/2403.14725

0 1 6 788 7

Download Image

ML Safety Daily @topofmlsafety

a month ago

RewardBench: Evaluating Reward Models for Language Modeling "Collection of prompt-win-lose trios spanning chat, reasoning, and safety, to benchmark how reward models perform on challenging, structured and out-of-distribution queries." arxiv.org/abs/2403.13787

0 0 4 345 2

Download Image

ML Safety Daily @topofmlsafety

a month ago

EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models Framework that simplifies evaluating jailbreaks on LLMs, revealing significant vulnerabilities across models including GPT-3.5-Turbo and GPT-4. arxiv.org/abs/2403.12171

0 0 7 541 3

Download Image

ML Safety Daily @topofmlsafety

a month ago

Using Hallucinations to Bypass GPT4's Filter Bypass the safety filters of closed source LLMs by inducing hallucinations that revert them to pre-RLHF states. arxiv.org/abs/2403.04769

1 0 5 539 2

Download Image

ML Safety Daily @topofmlsafety

2 months ago

Defending Against Unforeseen Failure Modes with Latent Adversarial Training Latent Adversarial Training (LAT) can improve robustness and is effective in defending against unforeseen vulnerabilities and enhancing clean performance. arxiv.org/abs/2403.05030

2 0 8 463 1

Download Image

ML Safety Daily @topofmlsafety

2 months ago

AtP*: An efficient and scalable method for localizing LLM behaviour to components Fast approximation for activation atching, a technique for mechanistically understanding how different components within a model influence its behavior. arxiv.org/abs/2403.00745

1 0 3 497 2

Download Image

ML Safety Daily @topofmlsafety

2 months ago

R-Judge: Benchmarking Safety Risk Awareness for LLM Agents Benchmark to assess LLMs ability to judge and identify safety risks in agent interaction records, revealing that even the best-performing model, GPT-4, falls short of human performance. arxiv.org/abs/2401.10019

0 0 4 453 3

Download Image

ML Safety Daily @topofmlsafety

2 months ago

PRP: Propagating Universal Perturbations to Attack Large Language Model Guard-Rails Universal adversarial attack against language model input filters. arxiv.org/abs/2402.15911

1 0 9 625 5

Download Image

ML Safety Daily @topofmlsafety

2 months ago

Approaching Human-Level Forecasting with Language Models Language models, when aided by information retrieval systems, can potentially produce forecasts as accurate as those created by competitive human forecasters. arxiv.org/abs/2402.18563

0 0 6 394 2

Download Image

ML Safety Daily @topofmlsafety

2 months ago

"Novel approach for producing a diverse collection of adversarial prompts. Rainbow Teaming casts adversarial prompt generation as a quality-diversity problem, and uses open-ended search to generate prompts that are both effective and diverse." arxiv.org/abs/2402.16822

0 0 3 462 1

Download Image

ML Safety Daily @topofmlsafety

2 months ago

An Empirical Evaluation of LLMs for Solving Offensive Security Challenges Evaluates two workflows, human-in-the-loop and fully automated, to assess LLMs' effectiveness in solving Capture The Flag challenges, finding they outperform human participants. arxiv.org/abs/2402.11814

0 1 7 434 0

Download Image

ML Safety Daily @topofmlsafety

2 months ago

A STRONGREJECT for Empty Jailbreaks LLM jailbreaks lack a standard benchmark for success or severity leading to biased overestimations of misuse potential; this benchmark offers a more accurate assessment. arxiv.org/abs/2402.10688