How can we synthesize high-quality pretraining data? A systematic study of prompt design, generator model, and source data

This is a preprint and may not have undergone formal peer review

Niklaus, J., Yamaguchi, A. orcid.org/0000-0001-8327-7598, Štefánik, M. et al. (9 more authors) (Submitted: 2026) How can we synthesize high-quality pretraining data? A systematic study of prompt design, generator model, and source data. [Preprint - arXiv] (Submitted)

Abstract

Metadata

Item Type: Preprint
Authors/Creators:
Copyright, Publisher and Additional Information:

© 2026 The Author(s). This preprint is made available under a Creative Commons Attribution 4.0 International License. (https://creativecommons.org/licenses/by/4.0/)

Keywords: Data Management and Data Science; Information and Computing Sciences
Dates:
  • Submitted: 15 April 2026
Institution: The University of Sheffield
Academic Units: The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield)
Date Deposited: 12 Nov 2026 15:58
Last Modified: 12 Nov 2026 15:58
Status: Submitted
Identification Number: 10.48550/arxiv.2604.13977
Related URLs:
Open Archives Initiative ID (OAI ID):

Download

Export

Statistics