Model Size
Select Models
1.3B
2.7B
Training Budget
Select Training Budget(s)
20B
60B
180B
Training Data
Select Training Data
DatologyAI 540B -> 180B
DatologyAI 600B -> 60B
DatologyAI 360B -> 60B
DatologyAI 180B -> 60B
DatologyAI 200B -> 20B
DatologyAI 120B -> 20B
DatologyAI 60B -> 20B
DataComp-LM
FineWeb-Edu
FineWeb
RefinedWeb
RedPajama-V1
C4
Task
Global Average
Commonsense Reasoning
Language Understanding
Reading Comprehension
World Knowledge
ARC-Easy
ARC-Challenge
BoolQ
CommonsenseQA
COPA
HellaSwag
LAMBADA
MMLU
OpenBookQA
PIQA
RACE-High
RACE-Middle
SciQ
SIQA
Winogrande
Number of Shots
0-shot
5-shot
X-Axis
Number of Tokens
FLOPS
Hover Type
Iso-Compute (x-axis)
Iso-Accuracy (y-axis)