L1Decay
L1Decay实现L1权重衰减正则化,用于模型训练,使得权重矩阵稀疏。
具体实现中,L1权重衰减正则化的计算公式如下:
参数:
- regularization_coeff (float) – L1正则化系数,默认值为0.0。
代码示例2
# 在 ParamAttr 和 optimizer 中同时设置正则化
import paddle.fluid as fluid
l2 = fluid.regularizer.L2Decay(regularization_coeff=0.1)
x = fluid.layers.uniform_random([3,4])
# 在ParamAttr中设置L1正则化
w_param = fluid.ParamAttr(regularizer=l1)
hidden1 = fluid.layers.fc(x, 8, param_attr=w_param) # fc_0.w_0(L1), fc_0.b_0
hidden2 = fluid.layers.fc(hidden1, 16, param_attr=w_param) # fc_1.w_0(L1), fc_1.b_0
avg_loss = fluid.layers.mean(predict)
# 在optimizer中设置L2正则化
optimizer = fluid.optimizer.SGD(learning_rate=1e-4, regularization=l2)
optimizer.minimize(avg_loss)
# 将会打印出提示信息: