Skip to content

Fix factor of 2 on gradients of models with real parameters

Vicentini Filippo requested to merge pv/fix-real-grad into master

Created by: PhilipVinc

I think the gradient for real-valued models we are using is wrong

I think we forgot to account for a factor of 2 that arises from taking the real part when parameters are real. When weights are complex, I think the gradient is correct.

Proving that this gradient was wrong, you can see that in the test we were dividing by two the finite-difference gradient for no reason when weights were real.

(Noticed because when doing the dynamics of some model with real parameter my timescale was off by a factor of 2 wrt the exact dynamics, but I'll double check...)

THIS IS BREAKING because some models will now have a gradient doubled (so learning rate must be changed) but I think it's a bug fix

Merge request reports