Maximilian Böther @MaxiBoether

making data loaders go brr | mts @datologyai | Ph.D. student @ETH_EN @SystemsGroupETH @anaklimovic | previous gigs at @HPI_DE @apple @google mboether.com Zurich, Switzerland Joined October 2012

Tweets

854
Followers

397
Following

1K
Likes

3K

Charles 🎉 Frye @charles_irl

2 weeks ago

wild to me that people vibe-generate slides for conference talks they are ugly (for now). they are low info densiry (thanks rlhf) but worse, they don't represent your thoughts, so your presentation of them will be terrible, unless you put in a ton of work (so just write them!)

16 7 112 11K 8

View Details

Maximilian Böther @MaxiBoether

4 weeks ago

@HKydlicek At @datologyai we are currently building a new dataloader - currently in private beta. Just DMd you!

0 0 3 90 0

View Details

Markus Wulfmeier @m_wulfmeier

2 months ago

A key reason for Zürich's recent robotics and AI boom. There's just incredible talent hidden in ETH, EPFL, UZH and many other places!

Harry Stebbings @HarryStebbings

2 months ago

DeepMind stayed in London because it is better for talent than Silicon Valley. "I saw London and the UK as having incredible talent from top universities like Cambridge, Oxford, Imperial and UCL. There is a deep heritage of scientific breakthroughs and world-class thinkers.

55 190 2K 719K 654

6 17 219 33K 51

View Details

Cody Blakeney @code_star

3 months ago

Poor one out for the homies

xlr8harder @xlr8harder

3 months ago

A tragedy is unfolding at AllenAI. Fs in the chat

29 54 678 181K 181

8 33 479 52K 48

View Details

Christina Baek @_christinabaek

3 months ago

Models are typically specialized to new domains by finetuning on small, high-quality datasets. We find that repeating the same dataset 10–50× starting from pretraining leads to substantially better downstream performance, in some cases outperforming larger models. 🧵

19 80 618 95K 521

View Details

Ricardo Monti @RicardoMonti9

4 months ago

1/ People often think better multilingual models must come at the cost of English performance. Not true. The constraint isn’t capacity, it’s data quality, and we can fix it. Today @datologyai shares ÜberWeb: a year of multilingual curation lessons, scaled to 20T+ tokens.

7 30 153 39K 67

View Details

Maximilian Böther @MaxiBoether

4 months ago

@xeophon @RicardoMonti9 @datologyai > You even got one in your name, how fun Until you start applying for a US visa/bank account/SSN, this is where the fun stops 😂 > ß/ẞ into the data as well As part of my PhD contract at ETH Zurich I had to agree to not use those anymore :( Swiss rules and whatnot 🇨🇭

1 0 3 61 0

View Details

Maximilian Böther @MaxiBoether

4 months ago

@RicardoMonti9 @xeophon @datologyai Intern-driven data curation (TM) He keeps telling me we „look at the data“, „thats your job“

1 0 9 107 0

View Details

Maximilian Böther @MaxiBoether

4 months ago

@RicardoMonti9 @xeophon @datologyai ricardo made me manually review each and every of them😩

1 0 10 573 0

View Details

Maximilian Böther @MaxiBoether

4 months ago

@RicardoMonti9 @chipro ack

0 0 2 93 0

View Details

Squirtle @squirtle_says

4 months ago

the world you remember no longer exists

496 5K 76K 2.5M 5K

View Details

Maximilian Böther @MaxiBoether

4 months ago

@StasBekman @datologyai @josh_wills I sent over a mail to your Snowflake address, probably easier than a tweet. In any case, glad the OVERLORD ref was already helpful!

0 0 1 40 0

View Details

Maximilian Böther @MaxiBoether

6 months ago

@leavittron Would this be the right time and place to re-negotiate my intern stipend?

1 0 3 49 0

View Details

Maximilian Böther @MaxiBoether

6 months ago

@leavittron I forgive you

1 0 1 55 0

View Details

Maximilian Böther @MaxiBoether

6 months ago

@josh_wills @leavittron @datologyai @sjoshi804 @HaoliYin interns dont get tagged😤

0 0 4 965 0

View Details

Maximilian Böther @MaxiBoether

6 months ago

I'll be at NeurIPS in San Diego this week! If you want to talk about data loading/mixing/curricula and what we are currently cooking at @datologyai and @SystemsGroupETH, hit me up!

0 2 7 862 1

View Details

Maximilian Böther @MaxiBoether

8 months ago

@joemelko @josh_wills Totally agreed! There is a spectrum: if you do n-phase training and pre compute everything, no need for online mixing. But are your mixtures for each phase truly optimal? It requires a lot of iteration, and algorithms like ADO rly help here.

1 0 3 72 0

View Details

JosH100 @josh_wills

8 months ago

4/ Mixing + Curricula I want the dataloader equivalent of torch.optim.lr_scheduler so I can schedule data mixtures over time with the same flexibility that we have for learning rates. Mixtera, from the systems group at ETH, is the most interesting thing I’ve seen along these lines (and has the bonus for me of being built using our lord and savior @duckdb), and I’m excited to be hosting @MaxiBoether over at @datologyai to explore this space with us!

1 1 21 3K 3

View Details

Maximilian Böther @MaxiBoether

8 months ago

@josh_wills with lots of important thoughts on data loaders for foundation model training. We are cooking something up at @datologyai 👀 But a dish needs to be tailored to taste, so if you have any thoughts on data loaders, reach out! ✉️

JosH100 @josh_wills

8 months ago

1/ Really looking forward to #PytorchConf this week in SF-- I've spent the last couple of months at @datologyai immersed in the DataLoader ecosystem (especially for our VLM stack) and I have a few topics I would love to discuss with folks (DMs are open, say hi if you see me, etc.