SciJEPA Framework for Asymmetric Within-Document Scientific Representation Learning

SciJEPA utilizes a citation-free, asymmetric predictive pretraining method where titles and abstracts predict method representations, which in turn predict conclusions. Adding Sliced Isotropic Gaussian Regularization (SIGReg) improves performance across RELISH and SciDocs benchmarks, narrowing the gap with contrastive baselines.