有关Tensorflow梯度下降常用的优化方法分享

1.tf.train.exponential_decay() 指数衰减学习率：

#tf.train.exponential_decay(learning_rate, global_steps, decay_steps, decay_rate, staircase=True/False)：#指数衰减学习率#learning_rate-学习率#global_steps-训练轮数

#decay_steps-完整的使⽤⼀遍训练数据所需的迭代轮数；=总训练样本数/batch#decay_rate-衰减速度

#staircase-衰减⽅式；=True，那就表明每decay_steps次计算学习速率变化，更新原始学习速率；=alse，那就是每⼀步都更新学习速率。learning_rate = tf.train.exponential_decay(initial_learning_rate = 0.001

global_step = tf.Variable(0, trainable=False)decay_steps = 100decay_rate = 0.95

total_loss = slim.losses.get_total_loss()

learning_rate = tf.train.exponential_decay(initial_learning_rate, global_step, decay_steps, decay_rate, True, name='learning_rate')optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(total_loss, global_step)

2.tf.train.ExponentialMovingAverage(decay, steps) 滑动平均更新参数：

initial_learning_rate = 0.001

global_step = tf.Variable(0, trainable=False)decay_steps = 100decay_rate = 0.95

total_loss = slim.losses.get_total_loss()

learning_rate = tf.train.exponential_decay(initial_learning_rate, global_step, decay_steps, decay_rate, True, name='learning_rate')optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(total_loss, global_step)ema = tf.train.ExponentialMovingAverage(decay=0.9999)#tf.trainable_variables--返回的是需要训练的变量列表averages_op = ema.apply(tf.trainable_variables())with tf.control_dependencies([optimizer]): train_op = tf.group(averages_op)

以上这篇有关Tensorflow梯度下降常⽤的优化⽅法分享就是⼩编分享给⼤家的全部内容了，希望能给⼤家⼀个参考，也希望⼤家多多⽀持。