Skip to content

Fix Selu Activation Function

Vicentini Filippo requested to merge github/fork/chrisrothUT/GCNN into master

Created by: chrisrothUT

@attila-i-szabo noticed that jax.nn.selu loses it's self normalizing property with complex numbers. Namely, it applies selu(x) when it should be applying selu(Re(x)) + 1j*selu(Im(x)) . Making this small change improves performance tremendously. I introduce the correct non-linearity as piecewise_selu

Merge request reports