A layman’s look at hyperspectral denoising

The paper on S2TDM, a spatial-spectral transformer-based diffusion model for hyperspectral image denoising, fascinated me. I am a layman to the topic, so this is only my attempt to understand why the idea is interesting.

A hyperspectral image is essentially a 3D dataset. Each pixel does not just record a colour. It carries information across a wide range of wavelengths. An image like this contains spatial structure as well as spectral information, which makes it much richer than an ordinary photograph.

Processing signals in 3D is something deep learning has been grappling with for a long time. S2TDM combines two very trendy ideas: transformers and diffusion. The transformer is useful for understanding relationships between signals that may be far apart, helping the model preserve the overall structure. Diffusion provides a way to restore a corrupted signal gradually, conditioned on the information that remains.

The training setup is fairly intuitive. The model is given clean hyperspectral images and artificially corrupted versions of them. It learns to recover the clean image, then applies that ability through several denoising steps.

I find the idea interesting because it borrows from parallel lines of research. The paper is not simply applying a transformer or diffusion model in isolation. It is asking what each idea can contribute to a specialised problem.

This reminds me of when I was first introduced to word2vec as a way to find relationships in high-dimensional signals, such as product information in a retail store. The domains are very different, but the attraction is similar. A technique developed for one kind of structure can reveal something useful when it is carried into another.