ArXiv

Lesion Detection in CT with Frozen Self-Distilled Features: SALT, a Spatially Adaptive Label-Guided Temperature

Authors
Mahmut S. Gokmen, Evan W. Damron, Mitchell A. Klusty...
Categories
cs.CV
arXiv
https://arxiv.org/abs/2608.05100v1
PDF
https://arxiv.org/pdf/2608.05100v1

Brief

The paper introduces SALT, a pretraining modification that uses weak box-derived spatial annotations to sharpen teacher softmax temperature and up-weight masked-patch loss inside annotated patches, producing label-free frozen encoders for downstream use. The authors freeze the encoder, train only a lightweight CenterNet-style head, and report improved patch separability, cohort- and size-stratified 3D lesion detection, box quality, and a detector-free re-identification probe. Summary is based on the abstract; full text was not reviewed.

Why it matters

SALT (Spatially Adaptive Label-guided Temperature) conditions self-distillation targets with weak box-derived labels during pretraining: inside annotated patch regions the teacher softmax temperature is sharpened and the masked-patch loss is up-weighted, while masking policy and other statistics remain unchanged.

Key details

  • Evaluation freezes the encoder and trains a lightweight multi-depth CenterNet-style head to detect lesions in 3D on four CT cohorts, isolating the effect against an identical backbone without target conditioning; additionally a detector-free probe shows a single frozen patch embedding can re-identify a lesion in a follow-up scan without registration or fine-tuning.
Source evidence

Abstract

Self-supervised pretraining objectives are spatially uniform: the teacher temperature and the per-patch loss weight are identical everywhere in the image, so a lesion a few patches wide contributes no more to the training signal than the surrounding parenchyma. Prior work biases the views toward annotated regions, which changes what the model sees but adds no pressure on the objective. We instead condition the targets of self-distillation, a method we call SALT (Spatially Adaptive Label-guided Temperature). Weak, box-derived labels, available only during pretraining, define a compact region on the encoder's patch grid, inside which the teacher's softmax temperature is sharpened and the masked-patch loss is up-weighted. The objectives, the masking policy and the centering statistics are otherwise unchanged, and at every downstream use the encoder is a plain feature extractor with no labels and no conditioning. We evaluate by freezing the encoder and training only a lightweight multi-depth CenterNet-style head, detecting lesions in 3D on four CT cohorts, and we isolate the mechanism against a backbone identical in architecture, pretraining data, schedule and label-guided cropping but with no target conditioning. We report patch-level separability, 3D detection stratified by cohort and by lesion size, box quality, and a detector-free probe in which a single frozen patch embedding re-identifies a lesion in a follow-up scan without registration, masks or fine-tuning. Because the conditioning is expressed through a spatial indicator rather than through label semantics, the formulation admits any weak spatial annotation; we instantiate and validate it for lesions.