This is an automated archive made by the Lemmit Bot.

The original was posted on /r/machinelearning by /u/SwaroopMeher on 2024-08-28 23:57:38+00:00.

I’ve been studying Variational Autoencoders (VAEs) and I keep coming across the term “reparameterization trick.” From what I understand, the trick involves using the formula ( X = mean + standard dev * Z ) to sample from a normal distribution, where Z is drawn from a standard normal distribution. This formula seems to be a standard method for sampling from a normal distribution

Here’s my confusion:

Why is it a trick?

The reparameterization “trick” is often highlighted as a clever trick, but to me, it appears to be a straightforward application of the transformation formula. If ( X = mean + standard dev * Z ) is the only way to sample from a normal distribution, why is the reparameterization trick considered particularly innovative?

I understand that the trick allows backpropagation through the sampling process. However, it seems like using ( X = mean + standard dev * Z ) is the only way to generate samples from a normal distribution given ( mean ) and ( standard deviation ). What makes this trick special beyond ensuring differentiability?

Here’s my thought process: We get mean and standard deviation from the encoder, and to sample from them, the only and most obvious way is `X = mean + standard deviation * Z’.

Could someone help clarify why the reparameterization trick is called a “trick”?

Thanks in advance for your insights!