malteos @XYOU

Berlin, Germany Joined June 2009

Tweets

241
Followers

750
Following

2K
Likes

2K

malteos @XYOU

a month ago

@RishiBommasani @percyliang The analogy for cloud vs local would be restaurant vs takeout. At the restaurant you better behave otherwise you get kicked out. At home you eat your food however you want.

0 0 0 21 0

View Details

malteos @XYOU

a year ago

@MatthewBerman Sure about this? Given the current reproducibility crisis in ML research, I doubt that humans would achieve a much higher replication score.

0 0 0 16 0

View Details

malteos @XYOU

a year ago

4/ In academia, the work is very different. PhD students or even undergraduates are the ones doing most the actual research work. But as a PhD student, you need to decide whether you prioritize the project work over your own PhD work (papers and thesis).

0 0 0 204 0

View Details

malteos @XYOU

a year ago

3/ LLMs and other foundation models are no longer research artifacts but products. Frontier models are developed by dedicated teams of +100 people specialized across the whole stack (from low level hardware optimization over data to ML and UX topics).

1 0 0 235 0

View Details

malteos @XYOU

2 years ago

@hu_yifei Did you already try Grobid? github.com/kermitt2/grobid

0 0 1 96 1

View Details

malteos @XYOU

2 years ago

@gui_penedo @pjox13 That’s even better. I will share the data with you as soon it’s ready!

0 0 2 21 0

View Details

malteos @XYOU

2 years ago

@gui_penedo @pjox13 We will release a filtered version of Colossal OSCAR soon. Is your training and evaluation script somewhere available? I would love to do the comparison with that version.

1 0 1 34 0

View Details

malteos @XYOU

2 years ago

@mark_cummins For Germany, we have ~50B tokens of court decisions but that are only the publicly available ones and that represent ~1% of all court decisions. However, you won't need all for LLM training due to high duplicate ratio. @mlissner might have the US numbers.

1 0 1 43 0

View Details

malteos @XYOU

2 years ago

@gui_penedo Awesome work. Will the remaining models also be released? And from your experience what model and data size do you need to see a significant difference in performance?

0 0 0 505 0

View Details

malteos @XYOU

2 years ago

@yoavgo "collected" 😎

0 0 2 307 0

View Details

malteos @XYOU

2 years ago

@saattrupdan @SebastianB929 @occiglot Do you have the whole eval setup in containers? If so, I could help with compute.

0 0 0 36 0

View Details

malteos @XYOU

2 years ago

@SebastianB929 @occiglot Pinging @saattrupdan who did the evals.

1 0 1 31 0

View Details

malteos @XYOU

2 years ago

@qinzytech @OpenAI @Meta Great work! Will the pretraining code be open source?

0 0 2 847 0

View Details

malteos @XYOU

2 years ago

@BramVanroy @VSC_HPC If your cluster uses slurm you can catch the kill signal and save a checkpoint before that. See this script for an example. Line 14 and 293-300 do the magic. gist.github.com/malteos/71635c…

0 0 2 102 2

View Details

malteos @XYOU

2 years ago

@SebastianB929 Opengptx is an official government funded research project. Occiglot is a loose group of individuals from different organizations without any formal ties. We call it a research collective. You may also call it simply a discord server. And yes, the website needs to be improved.

0 0 2 41 0

View Details

malteos @XYOU

2 years ago

@ZedDou1 @occiglot As mentioned in the readme, we suspect that this is due to the benchmarks being machine translated from English and based on English prompts.