LG AI Research's compact tabular foundation model family: classification and regression via in-context learning with no dataset-specific gradient updates, pretrained exclusively on a synthetic structural-causal-model prior. The core contribution is architectural — instead of compressing features into a fixed row embedding for a separate row-level learner, it interleaves feature-axis attention within each item with support-conditioned item-axis attention within each feature at every Transformer layer, mediated by item-summary and feature-summary tokens.

Reports strong performance-per-parameter on four public benchmarks (20.81M-parameter model on TabArena). Extends the tracked tabular-FM line (cf. Google's TabFM); no public weights at filing.

Paper

sciencetabularfoundation-modelresearch

Related