Rosetta: Composable Native Multimodal Pretraining

arXiv cs.CV1mo4 min read

arXiv:2607.00293v1 Announce Type: new Abstract: Achieving true artificial general intelligence requires foundation models capable of integrating new modalities without forgetting prior knowledge. However, accommodating continuous generative objectives alongside discrete understanding tasks causes severe gradient conflicts. Existing architectures, including standard Mixture-of-Experts (MoE), are highly susceptible to representation overwriting. Even structurally partitioned paradigms like Mixture-of-Transformers (MoT) remain vulnerable to catastrophic forgetting, severely impeding multimodal sc

Read the full story at arXiv cs.CV →

Rosetta: Composable Native Multimodal Pretraining

More in Machine Learning

Reducing Human Annotation with ML Active Learning

Learning What Matters: Supervising Sparse Attention Routing with Causal Evidence Sets

On the Depth Scalability of Logic Gate Networks

Pretraining EHR Foundation Models with Patient-Aware Sampling

From Perturbation Correction to Geometry-Aware Sampling: Sharpness-Guided Equilibrium Sampling for Balanced Flat Minima in Long-Tailed Learning

Parameter-free Adaptive Sparse Attention via Compression-Based Content Selection