I just discovered a stylegan surprise. In stylegan2, the discriminator learning rate can't be set. G_lrate is used for both the generator *and* the discriminator's rate. Everyone's been fooling themselves. It's currently impossible to set the discriminator's learning rate.
My tips are, try the two-time update rule - set the generator learning rate to half of the discriminator learning rate - try increasing the minibatch size - and, yes, if all else fails, also try lowering the learning rate.
6:54 AM · Feb 19, 2020