Anti-collapse penalties transfer to real data, and a second collapse mode at the decoder in slot-based object discovery

Summary

Three positive sub-findings from an object-emergence investigation in slot-based representation learning that survived under scale and adversarial checking. They are reported separately from a companion negative result because they stand on their own and are reusable. These are mechanism and ablation findings, not a claim of object-emergence success.

1. An anti-collapse penalty transfers from synthetic micro-worlds to real data

A slot-diversity penalty — designed to stop an encoder's per-slot representations from collapsing onto a single collinear direction — was developed on hand-authored synthetic micro-worlds. The same penalty transfers to real images (COCO): across a long training run (on the order of 10^5 steps) the maximum pairwise slot cosine similarity is driven to a near-zero level and all slots remain distinct throughout, with no collapse observed. The mechanism is therefore not an artifact of the synthetic micro-world setting.

2. A second, distinct collapse mode — and a fix at a different layer

Pure feature-reconstruction with slot attention exhibits a second degeneracy that the slot-vector penalty does not address: the decoder's per-slot masks collapse to a single-slot-per-image regime (one slot reconstructs the entire image while the others go unused at the arg-max assignment). A per-image mask-entropy balance term breaks this mode — decoder assignment entropy rises from near zero to near its maximum, and masks spread across slots.

Two collapse modes living at two layers (encoder slot vectors; decoder alpha masks), each requiring its own discipline term, supports a general methodological claim worth stating: anti-collapse is a required component at every layer that can degenerate, not a single optional regularizer. This rhymes with discipline mechanisms elsewhere in representation learning (e.g. a discriminator that blocks degenerate outputs, or a normalizer that corrects them): degeneracy can appear per-layer, and each susceptible layer needs its own gate.

3. Frozen self-supervised features already carry real object structure

On COCO, zero-training k-means over frozen self-supervised ViT patch features reaches a non-trivial object-alignment score well above chance, and k-NN object purity of those features is approximately 0.70–0.80. The target structure already exists in the observation encoder's features; the open problem is a binding mechanism that exceeds naive clustering of those features (addressed in the companion negative-result report).

Honest framing

These are mechanism and ablation findings, not a demonstration of object emergence. Findings (1) and (2) make training stable and non-degenerate; (3) shows the raw material is present. None of them establishes that the trained binding beats a no-training (naive clustering) baseline — that remains the open negative result.

Setup (non-sensitive)

Slot-attention encoder with a spatial-broadcast decoder, reconstructing frozen self-supervised ViT-B/16 patch features; evaluation on COCO. Reported numbers are relative/qualitative trends (cosine similarity, assignment entropy, k-NN purity) rather than tuned production configurations.