DSAIEngineering @dsaiengineering

The six-part miniseries on the architecture of TabICLv2 is finished.

keyboard_arrow_left Previous keyboard_arrow_right Next

A short quiz at the end lets you check whether the architecture and label-flow details are clear.

In this post, we start from the target-aware tensor E2, pass it through column-wise and row-wise transformer blocks, and end with dataset-wise ICL, where test row representations attend to labeled training row representations.

New post: [P28] Architecture of TabICLv2: compression-then-ICL. How TabICLv2 compresses target-aware feature tokens into row representations, then uses dataset-wise in-context learning to predict test rows. This post covers the next architectural step: compression-then-ICL.

The next post covers how TabICLv2 compresses these tokens into row representations, adds target information again at the row-token level for labeled rows, and then performs dataset-wise in-context learning.

The result is E2: a target-aware feature-token tensor ready for the compression-then-ICL pipeline.

[P27] Architecture of TabICLv2: target-aware embedding. How TabICLv2 uses target-aware embedding to add training labels to tabular in-context learning tokens while preventing label leakage in test rows.

This post covers repeated feature grouping. Later posts will cover target-aware embedding, column/row transformers, QASSMax, and the prediction heads.

With this post, I am starting a six-part miniseries on the architecture of TabICLv2. The goal is to cover the architecture one subsection at a time, so each post can focus on the details needed to understand that component without making a single article too long.

[P26] Architecture of TabICLv2: repeated feature grouping. A technical guide to TabICLv2 repeated feature grouping: why similar columns confuse encoders, how circular shifts add context, with NanoTabICL implementation.

This GitHub repository (soda-inria/nanotabicl) provides a short (~170 lines of code) self-contained implementation of the TabICLv2 architecture for educational and experimental purposes. It's a good point to start before diving into the full model's code.

Tabular foundation models TabPFN and TabICL are pretrained on synthetic data. The data generation mechanism is termed prior. The picture shows the high-level structure of the synthetic dataset generation prior of TabICL v2. Read more: arxiv.org/pdf/2602.11139.

keyboard_arrow_left Previous keyboard_arrow_right Next

A brief summary of the evolution of the architecture of TabPFN and TabICL. Read more: arxiv.org/pdf/2602.11139.

keyboard_arrow_left Previous keyboard_arrow_right Next

Are tabular foundation models the same as large language models? Picture 1: the answer. Picture 2: adaptions of LLMs to tabular data (source: TabICL v2 paper arxiv.org/pdf/2602.11139).

keyboard_arrow_left Previous keyboard_arrow_right Next