Research Scientist @google, PhD from @penn, BA from @columbia
Building efficient multi-{lingual,modal,agent,objective} models for searchmanestay.github.io New YorkJoined May 2020
Do LLMs' reasoning abilities come from training on code🤔? Many think so, but how does this hold across languages🌐?
We study the interplay of code and reasoning in our recent work (#acl2024).
📃arxiv.org/abs/2403.02567
🗃️github.com/amazon-science…
1/6 🧵
I'm in Vienna this week to present our poster on the robustness of RAG systems to multilingual contexts at #ACL2025NLP!
🗓️ Poster Session | Wednesday, July 30, 16:00 - 17:30
📍 Hall 4/5
@aclmeeting
In a world of geopolitical conflicts, how can AI help us navigate? Our #ACL2025-F work studies RAG robustness across 49 languages.
TL;DR: 📈 boost robustness w/ multilingual RAG, 🤔 take care w/ low-resource citations
📜arxiv.org/abs/2410.01171
🤗huggingface.co/datasets/borde…
1/4 🧵
This is the final paper of my PhD! Thanks to my many @upennnlp collaborators: @samarhdr, Chris, and the 7 wonderful students who I was fortunate to mentor. Please look out for our poster at ACL 2025 in Vienna.
4/4 🧵
We study cross-lingual robustness over 4 LLMs and 2 IR models. We find A) multilingual RAG performs best; B) LLM’s citations varies widely across langs. Our further experiments investigate aspects of cross-lingual RAG from IR to LLM explanations.
3/4 🧵
@yong_zhengxin Really thorough work on multilingual reasoning! A quick self-promotion of our xSTREET dataset arxiv.org/abs/2403.02567… (ACL 2024), which has annotations for the intermediate reasoning steps for STEM problems.
🚀 How well can LLMs know you and personalize your response? Turns out, not so much!
Introducing the PersonaMem Benchmark --
👩🏻💻Evaluate LLM's ability to understand evolving persona from 180+ multi-session user-chatbot conversation history
🎯Latest models (GPT-4.1, GPT-4.5, o4-mini, Llama-4, Gemini 2.0, Deepseek-R1, Claude-3.7) all struggle in personalization!
🎨7 personalization skills tested in 15 scenarios
🌟Realistic long-context evaluation up to 1M tokens
👇 Check out what we discovered… (1/6)
TL;DR - translation pairs > bilingual terminologies, generation especially boosts translations for small LLMs
Our ablations highlight the need for more challenging domain-adapted MT datasets with modern LLMs. Thanks to collaborators Jiaming, @ebriakou & @ColinCherry!
Externally retrieving knowledge empowers LLMs for domain-adapted MT ⚖️🩺. But how is knowledge best represented, and how viable is generating it from an LLM itself? Our @GoogleAI paper investigates these questions through a careful experimental setup 📜. arxiv.org/abs/2503.05010
@_reachsumit Great work! Nice to see a pipeline approach to multilingual QA generation in 2025. Reminds me of our EMNLP 2023 work arxiv.org/abs/2304.12206 (my last paper without LLMs 😅)
🚨 LLMs must grasp implied language to reason about emotions, social cues, etc.
Our @GoogleDeepMind paper presents the Implied NLI dataset. Targeting social norms 🌎 and conversational dynamics 💬, we enhance LLM understanding of real-world implication!
arxiv.org/abs/2501.07719
We'll be presenting this at the NLP for Wikipedia workshop @emnlpmeeting. This is ongoing work, and we'd love to hear feedback from the community!
A shout-out to my collaborators Fiona and Adwait for their amazing first paper efforts, @samarhdr, and Chris.
4/4 🧵
Using cross-lingually aligned queries, we analyze responses in a RAG setting. Responses can be "flipped" by varying passages' linguistic composition. We thus find these systems to be far from cross-lingually robust, as certain viewpoints can be amplified over others.
3/4 🧵
RAG enables LLMs to access external info 📖. But when this info is multiple languages 🌐, can LLMs reconcile differing viewpoints 🧐? We introduce BordIRlines, a dataset to study the robustness of cross-lingual RAG.
📃arxiv.org/abs/2410.01171
🗃️ huggingface.co/datasets/borde…
1/4 🧵
185 Followers 326 FollowingPhD student @CarnegieMellon and @MozillaAI, making things for thinking with // once upon a time @GoogleAI @Waymo @BKCHarvard, cs+phil @UChicago
2K Followers 7K FollowingStudent,Love to solve hardest math problem.LLM's,Mathematical Research(Geometric Topology,Differential Geometry,Algebraic Geometry).Lord Krishna is God Of Math
224 Followers 778 FollowingPostdoc @ University of Pennsylvania | The World Bank | PhD, University of Delhi. #CulturalNLP #NormativeAlignment #AI4Health
3K Followers 13 FollowingUnderstanding Intelligence.
Measurement. Explanation. Application. That's how we're tackling AI interpretability: the greatest scientific problem of our age.
265K Followers 182 FollowingCo-founder of Thinking Machines Lab @thinkymachines; Ex-VP, AI Safety & robotics, applied research @OpenAI; Author of Lil'Log
11K Followers 3K FollowingAssoc. prof. of CS at University of Maryland. Researcher in #PhysicalAI, #TrustworthyML, #EthicalAI, AI #Alignment, #AI for ALL.
185 Followers 326 FollowingPhD student @CarnegieMellon and @MozillaAI, making things for thinking with // once upon a time @GoogleAI @Waymo @BKCHarvard, cs+phil @UChicago
1K Followers 605 FollowingPhD student at @LTIatCMU / @SCSatCMU she/her, prev. @UVA and intern @ai2_allennlp
@/clara on https://t.co/GHxXbrRHSB and @/clarana on https://t.co/47UIhMGaRd
28K Followers 298 FollowingProfessor of linguistics and professor of computer science at Stanford and author of the James Beard award finalist "The Language of Food"
206K Followers 0 FollowingOG of: P and T in ChatGPT, 100x deeper learning, meta learning and RSI, neural distillation, GAN/World Model... Co-authored most-cited AI paper of 20th century
2K Followers 804 FollowingMTS @adaption_ai | Prev: Research Engineer @Cohere_Labs @cohere | “Research means that you don't know, but are willing to find out” ✨
578 Followers 46 FollowingA @PennEngineers research center devoted to science and tools for ensuring AI-enabled systems are safe, explainable, and trustworthy