DATA
Pretraining Data Infrastructure
Pretraining data acquisition and quality at Apodex.
PRETRAINING DATA
Data acquisition & quality assessment
Building the data pipeline for pretraining.
How can we acquire useful web data at scale and assess its quality before training?
My work
Built pretraining data infrastructure from scratch and scaled it to web scale, with data quality assessment guiding what is collected.
Outcome
An acquisition pipeline covering raw HTML, PDFs, and other information-rich documents, with quality scoring to guide collection.
Context and my contribution
Since May 2026, my focus at Apodex has been pretraining data acquisition and quality. I built the acquisition and processing infrastructure from scratch, scaled it to web scale, and developed quality assessment to decide what is worth collecting and training on.