Aditya Sarawgi @_truncs_

@patrick_oshag “You just put a radiator on the dark side of the satellite” thermal engineers in absolute shambles rn

Update on this (it somehow gets even more fun) We wrote a rebuttal highlighting how these citations don't exist. Reviewer replies with "i sincerely apologise for this oversight", score changes to 6. Faith in academia restored? Wait for it... The reviewer then writes a completely new review changing score to 4. The new review asks for 4 new datasets that don't exist... Pangram says it's fully AI generated.

Shashwat Goel @ShashwatGoel7

7 months ago

I wasted 3 hours of my life today 😃 An @iclr_conf review, gave 2/10 only because "these 7 papers already showed what you showed" 3/7 don't exist (also confirmed w an author) 3/7 don't do anything like what we do. 1/7 is similar, but far from the same takeaway. So productive.

9 8 322 55K 18

We did it! We tested 300 Bay Area foods for plastic chemicals. We found some interesting surprises. Top 5 findings in our test results: 1. Our tests found plastic chemicals in 86% of all foods, with phthalates in 73% of the tested products and bisphenols in 22%. It's everywhere. 2. We detected phthalates in most baby foods and prenatal vitamins. 3. Hot foods which spend 45 minutes in takeout containers have 34% higher levels of plastic chemicals than the same dishes tested directly from the restaurant. 4. The 1950s Army rations we tested contained surprisingly high levels of plastic chemicals. 5. Almost every single one of the foods we tested are within both US FDA and EU EFSA regulations. Check out our full results below.

Nat Friedman @natfriedman

2 years ago

I'm going to re-run all these tests on food we eat in California. Also going to test for other plastic chemicals. Let me know what foods we should test and suggestions for methodology.

368 189 3K 3.5M 972

James Cook is an extraordinary artist who uses vintage typewriters to create detailed and stunning artworks. Based in the UK, Cook has produced nearly 300 pieces, ranging from portraits to intricate cityscapes. His unique method involves meticulously arranging letters, numbers, and symbols to form i images, often incorporating relevant words or phrases to add depth and context to his pieces. Artist: jamescookartwork

What's that black spot on Jupiter? No one is sure. During one pass of NASA's Juno over Jupiter, the robotic spacecraft imaged an usually dark cloud feature informally dubbed the Abyss. Surrounding cloud patterns show the Abyss to be at the center of a vortex. Since dark features on Jupiter's atmosphere tend to run deeper than light features, the Abyss may really be the deep hole that it appears -- but without more evidence that remains conjecture. The Abyss is surrounded by a complex of meandering clouds and other swirling storm systems, some of which are topped by light colored, high-altitude clouds. The featured image was captured in 2019 while Juno passed only about 15,000 kilometers above Jupiter's cloud tops. The next close pass of Juno near Jupiter will be in about three weeks. Image Credit: NASA, Juno, SwRI, MSSS; Processing & License: Gerald Eichstädt & Sean Doran

My first draft of a reaction to @beenwrekt's blog post on the Higgs discovery. TL;DR, yes it did. theoryandpractice.org/2024/10/Yes,%2…

Ben Recht @beenwrekt

2 years ago

The Higgs Discovery did not take place. argmin.net/p/the-higgs-di…

33 17 128 200K 88

2¹³⁶²⁷⁹⁸⁴¹−1, discovered today, is the largest known prime. It's a Mersenne prime (2ᵖ-1), which are easier to find. It took nearly 6 years for the GIMPS software to find it after the previous largest known prime. It was also the first Mersenne prime found using GPUs.

With David and the Baker Lab in the spotlight today, I wanted to share some insights into the @UWproteindesign and how it operates, a glimpse behind the curtain. I had planned to write this post-graduation, but now seems as good a time as any. (Got twitter blue free trial so this could all fit in less tweets!) First, the lab is enormous. ~60 grad students, ~60 postdocs, a handful of visitors, undergrads, and a surrounding institution of another 150 or so. Collaboration is strongly encouraged (even mandated) by David, who sets up pro-collaboration incentives. Notably, he's fine with grad students graduating without a sole first-author paper—it's acceptable to "only" have worked as a co-first author. This is a key ingredient in the secret sauce: the tight collaboration between wet lab and dry lab. It ensures that all our work is ultimately grounded in strong wet-lab validation—our "oracle" is the real world, not another computational model. While we have regular meetings for different subgroups and the entire group, much information travels through the lab via informal one-on-one interactions. In some ways, it reminds me of a classic "tribe of humans in the state of nature"—100-200 people with no clear hierarchy, passing information via "gossip". It’s maybe not the most complete way of ensuring everyone is on the same page, but saves time as we aren’t drowning in endless meetings. Does David stay in touch with all these grad students and post-docs? Remarkably, yes. Unlike some very large labs known for being run entirely by post-docs, he knows exactly what everyone is working on and the stage of their projects. Each member has monthly one-on-ones with him, and monthly subgroup meetings that David attends. If he suggests you try something at your previous one-on-one, you'd better have it done by the next. Does he actually contribute research ideas, or is he more of a detached big-picture project manager? Definitely the former. He understands the intricacies of a shocking range of topics. I'll be discussing some arcane deep learning concept with him, and then he'll turn around and talk to someone about the details of a catalytic mechanism. He's actually the most hands-on PI I've ever had—if anything, he verges on over-managing rather than being too detached. How does he keep track of everything? Partly, he's just a brilliant person with exceptional recall. But he has also built infrastructure above and below him in the lab to handle many of the details, bureaucracy, big picture, and management tasks. This allows him to spend most of his day doing what he's most passionate about and skilled at: walking around talking to people about science. He also lives very much in the moment and in his own words, “never thinks very far ahead". To keep up with tools, methods, and wet lab techniques, he does the occasional project and design campaign himself on the side when time allows. It's still a tremendous cognitive load to keep all this in his head, but as much as possible, he has offloaded non-scientific cognitive burdens. It helps that he’s in the lab in person most days of the year, rarely traveling for conferences or talks, instead doing them over Zoom or not attending. (1/2).

The Nobel Prize @NobelPrize

2 years ago

BREAKING NEWS The Royal Swedish Academy of Sciences has decided to award the 2024 #NobelPrize in Chemistry with one half to David Baker “for computational protein design” and the other half jointly to Demis Hassabis and John M. Jumper “for protein structure prediction.”

510 8K 19K 9.1M 1K

can AI do research-level mathematics? make conjectures? prove theorems? there’s a moving frontier between what can and cannot be done with LLMs. that boundary just shifted a little. this is my experience with AI proving a new theorem. 1/

A student reached out asking for advice on research directions in optimization, so I wrote a long response with pointers to interesting papers. I thought it'd be worth sharing it here too: 1. Adaptive optimization. There has been a lot going on in the last year, below are some papers I personally found interesting. First of all, this paper by Li and Lan on Nesterov's acceleration of adaptive gradient descent: arxiv.org/abs/2310.10082 Check Corollary 1 for a simple description of their method. There is one thing I don't like about it: the amount by which we can increase the stepsize at each iteration decreases as t grows. That being said, I don't know if this restriction can be lifted, and perhaps it's the best thing we can get. Yura Malitsky and I also did some work on adaptive gradient descent, making the stepsizes a bit larger, roughly sqrt(2) improvement over our previous result: arxiv.org/abs/2308.02261 We still don't know if that's the best we can do or if a tighter analysis can give us better methods. I should also mention that there is more push in the literature on Polyak stepsize, see for instance these two papers: arxiv.org/abs/2407.04358 (a stepsize very similar to Polyak) arxiv.org/abs/2406.04142 (Polyak stepsize with momentum) 2. Adagrad-like methods still can be studied, I believe it's an underexplored direction. I wish there was more papers on studying the importance of coordinate-wise stepsizes. One paper on the topic I really liked is this study of when Adam is more useful than SGD: arxiv.org/abs/2402.19449 There is also some research on new practical methods, for instance, acceleration of DoG is interesting: arxiv.org/abs/2404.00666 And I also enjoyed reading this paper by Rodomanov et al. on line-search-inspired stochastic methods: arxiv.org/abs/2402.03210 3. I also like the direction of getting better assumptions for optimization theory and studying the implications. A good example is the gradient clipping literature: arxiv.org/abs/1905.11881 ((L₀, L₁)-smoothness) arxiv.org/abs/2305.01588 (same revisited) arxiv.org/abs/2406.04443 (on heavy-tailed noise) We need to bridge optimization assumptions with what we know about neural networks, so read about properties of neural networks themselves like this: arxiv.org/abs/2405.14813 (on scales of layers and how their type affects Lipschitz constants) 4. These days, people are using deep networks of all scales for their tasks, and they have discovered a lot of tricks that haven't been studied thoroughly in optimization literature: quantization, Straight-Through Estimator, (arxiv.org/abs/1903.05662), low-rank techniques such as LoRA, learning-rate warm-up, etc. You should expose yourself to those tricks to get a better understanding of what the current theory is lacking. If you're considering choosing optimization as the topic for your PhD, here are some extra thoughts. Right now there is less activity than about 5 years ago, most low-hanging fruits seem to have been taken, and the remaining questions seem quite challenging. So if you're looking for a field where it is easy to get publications, it might not be perfect. However, it's still a good field to produce meaningful theory. It's also important who you would work with, i.e. if you can find a good advisor, that often affects one's satisfaction to a larger degree than the topic itself, so make your decision carefully. As my last word of advice, I definitely encourage testing new methods on neural networks (and preferably not on CIFAR10/CIFAR100, because they give misleading results), at least something like nanoGPT (github.com/karpathy/nanoG…). When I was a PhD student, I did a lot of theoretical research testing my methods on logistic regression and that was useful to understand the theory, but I also had the wrong impression about what works and what doesn't because of that. If you can, do both, understand the theory as much as you can, but also learn its limits and failure modes.

Congrats to my friend and colleague Guosong Hong for his stunning and original discovery, published today in Science, on clearing tissues *in living animals* with a common food dye! The dye is tartrazine, used in Doritos! science.org/doi/10.1126/sc…

AI is replacing LSD dealers

very caveat emptor newcomer.co/p/my-conversat…

I had a chat last night with a billion-dollar SaaS founder/CEO. He's raised $200M at a $2B valuation in 2021. Living the high life in SF with a crazy house and a Porsche 911. On the surface, it seems like he has it all and living the dream. But he called me out of the blue at 11p last night to vent. It’s been 9 years since he started the company although people think it's overnight success. Last two years have been brutal. He’s sprinting to profitability. Layoffs. Now navigating the AI landscape. He goes: “I hope we can compete in an OpenAI world” I could practically hear his uncertainty crackling through the phone. Four years ago, he had a full head of blonde hair. Today, it’s mostly gray. He looks like Obama post-presidency. Then he dropped the bombshell, “I’m running a zombie company. I don’t know if we’ll ever IPO. Hopefully, we sell, and I get a 1-3x return for investors. I feel stuck.” This is the dirty secret of Silicon Valley. So many founders are trapped in the hype cycle, drowning in VC money but feeling like teenage toddlers trying to figure it out. I've been stuck before. I've been on this treadmill before. Sometimes you get unstuck, find that killer product and before you know it you're Silicon Valley's darling again. But most, time you're not. You're just treading water. And then you wake-up, you're 45, no hair, no kids, only yearly trips to visits family and realized you could have made a more profitable career (and more fruitful life?) just doing your own thing. Maybe you're happy. Because you gave it your best shot at "building something big". You wanted to become the next Mark Zuckerberg. But maybe you aren't. Maybe you realize that you can build something big on your own. No investors. Or join something early that isnt on the VC-train. He called because he’s desperate for a new direction. Profitable, indie startups. I keep getting these calls. The air is thick with anxiety. The unicorn dream is cracking, and people are waking up.

Five years ago we gambled on an entirely new way of entering a vehicle from orbit (flaps, not wings) and until Flight 4 there was still a chance that this method, and the ship that was designed around it, would fundamentally not work (2/3)

Lars Blackmore @larsblackmore

2 years ago

Somewhat belated on account of my appendix getting deleted…but landing Starship on the ocean feels as significant as Falcon 9’s first landing (1/3)

10 37 876 210K 34

From Kyle, Dianna’s husband: Two years ago, I married my smart, adventurous, curious best friend. I loved her curiosity and so did her 3M+ followers on her Science YouTube channel. Now, she’s been 100% bedbound with long Covid for 1.5 years. She lays in a dark room, earplugs in, with only her mind and simple thoughts to keep her company. No, she can’t talk, watch TV, read books, listen to podcasts or even music. She can’t tolerate visitors (even the faint smell of laundry detergent makes her sicker). It’s a kind of locked-in hell and there’s currently no cure for this disease. We desperately need more funding for research to find the cure for #longcovid and #MECFS . @SenDuckworth @SenMarkey @timkaine @BernieSanders

keyboard_arrow_left Previous keyboard_arrow_right Next

The only perspective on AI that matters

@lemire @BrendanEich @ylecun Where is the evidence to the contrary?

when TSMC engineers work from home: