How can we synthesize high-quality pretraining data? A systematic study of prompt design, generator model, and source data

This is the latest version of this eprint.

Niklaus, J., Yamaguchi, A. orcid.org/0000-0001-8327-7598, Štefánik, M. et al. (9 more authors) (Accepted: 2026) How can we synthesize high-quality pretraining data? A systematic study of prompt design, generator model, and source data. In: Conference On Language Models 3 proceedings. The Third Annual Conference on Language Modeling (COLM 2026), 06-09 Oct 2026, San Francisco, CA, United States. COLM. (In Press)

Abstract

Metadata

Item Type: Proceedings Paper
Authors/Creators:
Copyright, Publisher and Additional Information:

© 2026 The Author(s).

Dates:
  • Accepted: 30 July 2026
Institution: The University of Sheffield
Academic Units: The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield)
Date Deposited: 18 Nov 2026 13:00
Last Modified: 18 Nov 2026 13:00
Status: In Press
Publisher: COLM
Refereed: Yes
Related URLs:
Open Archives Initiative ID (OAI ID):

Available Versions of this Item

Export

Statistics