When Does Generating More Help? Disentangling Fixed-Source Synthesis from Source Expansion in Synthetic Data Scaling

arXiv cs.CL4w4 min read

arXiv:2607.01727v1 Announce Type: new Abstract: Synthetic data can be scaled along two routes: Source Expansion (SE), which enlarges the source by adding seed materials or generators, and Fixed-Source Synthesis (FSS), which holds the source fixed and scales the generation budget. Existing scaling studies typically expand the source as the data grows, conflating SE with FSS and leaving FSS underexplored. We isolate FSS by holding the seed-question pool and teacher model fixed, varying only the per-question response budget under Rejection Sampling (RS). We adapt the rectified scaling law to FSS,

Read the full story at arXiv cs.CL →

When Does Generating More Help? Disentangling Fixed-Source Synthesis from Source Expansion in Synthetic Data Scaling

More in Machine Learning

Reducing Human Annotation with ML Active Learning

Learning What Matters: Supervising Sparse Attention Routing with Causal Evidence Sets

On the Depth Scalability of Logic Gate Networks

Pretraining EHR Foundation Models with Patient-Aware Sampling

From Perturbation Correction to Geometry-Aware Sampling: Sharpness-Guided Equilibrium Sampling for Balanced Flat Minima in Long-Tailed Learning

Parameter-free Adaptive Sparse Attention via Compression-Based Content Selection