批量化f1is trainingruetf nn tensorflow使用bnbatch normalization-csdn博客.pdfVIP

你要的答案或许都在这里：小鹏的博客

注意：不要随便加BN，有些问题加了后会导致loss变大。

上一篇是

BatchNormalization的原理介绍，看一下tf的实现，加到卷积后面和全连接层后面都可：

（1）

训练的时候：is_training为True。

[python]

01.importtensorflowastf

02.importnumpyasnp

03.fromtensorflow.python.opsimportcontrol_flow_ops

04.fromtensorflow.python.trainingimportmoving_averages

05.

06.defbn(x,is_training):

07.x_shape=x.get_shape()

08.params_shape=x_shape[-1:]

09.

10.axis=list(range(len(x_shape)-1))

11.

12.beta=_get_variabla,params_shape,initializer=tf.zeros_initializer())

13.gamma=_get_variable(gamma,params_shape,initializer=tf.ones_initializer())

14.

15.moving_mean=_get_variable(moving_mean,params_shape,initializer=tf.zeros_initializer(),trainab

16.moving_variance=_get_variable(moving_variance,params_shape,initializer=tf.ones_initializer(),

17.

18.#Theseopswillonlybepreformedwhentraining.#Theseopswillonlybepreformedwhentraining.#Theseopswillonlybepreformedwhentraining.#Theseopswillonlybepreformedwhentraining.#Theseopswillonlybepreformedwhentraining.

19.mean,variance=tf.nn.moments(x,axis)mean,variance=tf.nn.moments(x,axis)mean,variance=tf.nn.moments(x,axis)mean,variance=tf.nn.moments(x,axis)

20.update_moving_mean=moving_averages.assign_moving_average(moving_mean,mean,BN_DECAY)

21.update_moving_variance=moving_averages.assign_moving_average(moving_variance,variance,BN_DECAY)

22.tf.add_to_collection(UPDATE_OPS_COLLECTION,update_moving_mean)

23.tf.add_to_collection(UPDATE_OPS_COLLECTION,update_moving_variance)

24.

25.mean,variance=control_flow_ops.cond(

26.is_training,lambda:(mean,