Unverified Commit 029d7c75 authored by AngelBottomless's avatar AngelBottomless Committed by GitHub
Browse files

Revert unresolved changes in Bias initialization 

it should be zeros_ or parameterized in future properly.
parent cc56df99

modules/hypernetworks/hypernetwork.py

+1 −1
Original line number Diff line number Diff line
@@ -75,7 +75,7 @@ class HypernetworkModule(torch.nn.Module): 
                    w, b = layer.weight.data, layer.bias.data

                    if weight_init == "Normal" or type(layer) == torch.nn.LayerNorm:

                        normal_(w, mean=0.0, std=0.01)

                        normal_(b, mean=0.0, std=0.005)

                        normal_(b, mean=0.0, std=0)

                    elif weight_init == 'XavierUniform':

                        xavier_uniform_(w)

                        zeros_(b)

CRA Git | Maintained and supported by SUSTech CRA and CCSE