title: Keras backend mean function: " 'float' object has no attribute 'dtype' "?
body: <p>I am trying to introduce a new kernel regularize for a network using Keras. But, it gives me the error:
   'float' object has no attribute 'dtype'
  How can I fix it?</p>
  
  <p>I found the code here: <a href="https://stackoverflow.com/questions/43599082/kl-divergence-in-kerastensorflow-backend">KL divergence in keras(tensorflow backend)</a></p>
  
  <p>Here is my code:</p>
  
  <pre><code>    from keras import backend as K
  
      kullback_leibler_divergence = keras.losses.kullback_leibler_divergence
  
      def kl_divergence_regularizer(inputs):
          means = K.mean((inputs))
          return 0.5 *(0.01 * (kullback_leibler_divergence(0.05, means)
                        + kullback_leibler_divergence(1 - 0.05, 1 - means)))
  
  
      model = Sequential([
          Dense(100, input_shape=(x_train_s.shape[1],),kernel_initializer='random_uniform'),
          Activation('elu'),
          Dense(x_train_s.shape[1],kernel_initializer='random_uniform', kernel_regularizer=kl_divergence_regularizer),
          Activation('tanh')
      ])
  
      model.compile(optimizer='adam',loss='mean_squared_error')
  
      model.fit(x_train_s, x_train_s, epochs=5,validation_split=0.1, shuffle=True, verbose=1,batch_size=np.uint(x_train_s.shape[0]/100))
  </code></pre>
  
  <p>The full error trace back is presented here:</p>
  
  <pre><code>AttributeError                            Traceback (most recent call last)
  &lt;ipython-input-39-59bc90c687de&gt; in &lt;module&gt;
       39     Activation('elu'),
       40     Dense(x_train_s.shape[1],kernel_initializer='random_uniform', kernel_regularizer=kl_divergence_regularizer),
  ---&gt; 41     Activation('tanh')
       42 ])
       43 
  
  C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\sequential.py in __init__(self, layers, name)
       91         if layers:
       92             for layer in layers:
  ---&gt; 93                 self.add(layer)
       94 
       95     @property
  
  C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\sequential.py in add(self, layer)
      179                 self.inputs = network.get_source_inputs(self.outputs[0])
      180         elif self.outputs:
  --&gt; 181             output_tensor = layer(self.outputs[0])
      182             if isinstance(output_tensor, list):
      183                 raise TypeError('All layers in a Sequential model '
  
  C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\base_layer.py in __call__(self, inputs, **kwargs)
      429                                          'You can build it manually via: '
      430                                          '`layer.build(batch_input_shape)`')
  --&gt; 431                 self.build(unpack_singleton(input_shapes))
      432                 self.built = True
      433 
  
  C:\ProgramData\Anaconda3\lib\site-packages\keras\layers\core.py in build(self, input_shape)
      864                                       name='kernel',
      865                                       regularizer=self.kernel_regularizer,
  --&gt; 866                                       constraint=self.kernel_constraint)
      867         if self.use_bias:
      868             self.bias = self.add_weight(shape=(self.units,),
  
  C:\ProgramData\Anaconda3\lib\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs)
       89                 warnings.warn('Update your `' + object_name + '` call to the ' +
       90                               'Keras 2 API: ' + signature, stacklevel=2)
  ---&gt; 91             return func(*args, **kwargs)
       92         wrapper._original_function = func
       93         return wrapper
  
  C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\base_layer.py in add_weight(self, name, shape, dtype, initializer, regularizer, trainable, constraint)
      253         if regularizer is not None:
      254             with K.name_scope('weight_regularizer'):
  --&gt; 255                 self.add_loss(regularizer(weight))
      256         if trainable:
      257             self._trainable_weights.append(weight)
  
  &lt;ipython-input-39-59bc90c687de&gt; in kl_divergence_regularizer(inputs)
       14     means = K.mean((inputs))
       15 #     means=1e-6
  ---&gt; 16     return 0.5 *(0.01 * (kullback_leibler_divergence(0.05, means)
       17                   + kullback_leibler_divergence(1 - 0.05, 1 - means)))
       18 #     return 1e-10
  
  C:\ProgramData\Anaconda3\lib\site-packages\keras\losses.py in kullback_leibler_divergence(y_true, y_pred)
       79 
       80 def kullback_leibler_divergence(y_true, y_pred):
  ---&gt; 81     y_true = K.clip(y_true, K.epsilon(), 1)
       82     y_pred = K.clip(y_pred, K.epsilon(), 1)
       83     return K.sum(y_true * K.log(y_true / y_pred), axis=-1)
  
  C:\ProgramData\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py in clip(x, min_value, max_value)
     1599     if max_value is None:
     1600         max_value = np.inf
  -&gt; 1601     min_value = _to_tensor(min_value, x.dtype.base_dtype)
     1602     max_value = _to_tensor(max_value, x.dtype.base_dtype)
     1603     return tf.clip_by_value(x, min_value, max_value)
  
  AttributeError: 'float' object has no attribute 'dtype'
  </code></pre>
  
  <p>I need the KL divergence between 0.05 and mean calculate the following sum over i:</p>
  
  <p>KL=sum(0.05*\log(0.05/mean[i])) </p>