Loading...

TensorFlow base operations

TensorFlow base operations

NeuralNetwork tensorflow layers examples and usage

1. TensorFlow conv2d (tf.nn.conv2d)

Computes a 2-D convolution given 4-D input and filter tensors.

tf.nn.conv2d(
    input,
    filter=None,
    strides=None,
    padding=None,
    use_cudnn_on_gpu=True,
    data_format='NHWC',
    dilations=[1, 1, 1, 1],
    name=None,
    filters=None
)

Given an input tensor of shape [batch, in_height, in_width, in_channels] and a filter / kernel tensor of shape [filter_height, filter_width, in_channels, out_channels], this op performs the following:

Flattens the filter to a 2-D matrix with shape [filter_height * filter_width * in_channels, output_channels]. Extracts image patches from the input tensor to form a virtual tensor of shape [batch, out_height, out_width, filter_height * filter_width * in_channels]. For each patch, right-multiplies the filter matrix and the image patch vector. In detail, with the default NHWC format

output[b, i, j, k] =
    sum_{di, dj, q} input[b, strides[1] * i + di, strides[2] * j + dj, q]
                    * filter[di, dj, q, k]
Must have strides[0] = strides[3] = 1. 
For the most common case of the same horizontal and vertices strides, strides = [1, stride, stride, 1]

Arguments:

- input: A Tensor. Must be one of the following types: half, bfloat16, float32, float64. A 4-D tensor. The dimension order is interpreted according to the value of data_format, see below for details.

- filter: A Tensor. Must have the same type as input. A 4-D tensor of shape [filter_height, filter_width, in_channels, out_channels]

- strides: An int or list of ints that has length 1, 2 or 4. The stride of the sliding window for each dimension of input. If a single value is given it is replicated in the H and W dimension. By default the N and C dimensions are set to 1. The dimension order is determined by the value of data_format, see below for details.

- padding: Either the string "SAME" or "VALID" indicating the type of padding algorithm to use, or a list indicating the explicit paddings at the start and end of each dimension. When explicit padding is used and data_format is "NHWC", this should be in the form [[0, 0], [pad_top, pad_bottom], [pad_left, pad_right], [0, 0]]. When explicit padding used and data_format is "NCHW", this should be in the form [[0, 0], [0, 0], [pad_top, pad_bottom], [pad_left, pad_right]].

- data_format: An optional string from: "NHWC", "NCHW". Defaults to "NHWC". Specify the data format of the input and output data. With the default format "NHWC", the data is stored in the order of: [batch, height, width, channels]. Alternatively, the format could be "NCHW", the data storage order of: [batch, channels, height, width].

- dilations: An int or list of ints that has length 1, 2 or 4, defaults to 1. The dilation factor for each dimension ofinput. If a single value is given it is replicated in the H and W dimension. By default the N and C dimensions are set to 1. If set to k > 1, there will be k-1 skipped cells between each filter element on that dimension. The dimension order is determined by the value of data_format, see above for details. Dilations in the batch and depth dimensions if a 4-d tensor must be 1.

- name: A name for the operation (optional).

- filters: Alias for filter.

Returns variable:

A Tensor. Has the same type as input.

2. TensorFlow bias_add (tf.nn.bias_add)

Adds bias to value.

Aliases: tf.compat.v1.nn.bias_add, tf.compat.v2.nn.bias_add

tf.nn.bias_add(
    value,
    bias,
    data_format=None,
    name=None
)

This is (mostly) a special case of tf.add where bias is restricted to 1-D. Broadcasting is supported, so value may have any number of dimensions. Unlike tf.add, the type of bias is allowed to differ from value in the case where both types are quantized.

Arguments:

- value: A Tensor with type float, double, int64, int32, uint8, int16, int8, complex64, or complex128.

- bias: A 1-D Tensor with size matching the channel dimension of value. Must be the same type as value unless value is a quantized type, in which case a different quantized type may be used.

- data_format: A string. 'N...C' and 'NC...' are supported.

- name: A name for the operation (optional).

Returns variable:

A Tensor with the same type as value.

3. TensorFlow max_pool (tf.nn.max_pool)

Performs the max pooling on the input.

tf.nn.max_pool(
    value,
    ksize,
    strides,
    padding,
    data_format='NHWC',
    name=None,
    input=None
)

Arguments:

- value: A 4-D Tensor of the format specified by data_format.

- ksize: An int or list of ints that has length 1, 2 or 4. The size of the window for each dimension of the input tensor.

- strides: An int or list of ints that has length 1, 2 or 4. The stride of the sliding window for each dimension of the input tensor.

- padding: A string, either 'VALID' or 'SAME'. The padding algorithm. See the "returns" section of tf.nn.convolution for details.

- name: Optional name for the operation.

- input: Alias for value.

4. TensorFlow reduce_mean (tf.math.reduce_mean)

Aliases: tf.compat.v1.math.reduce_mean, tf.compat.v1.reduce_mean, tf.reduce_mean

Computes the mean of elements across dimensions of a tensor.

tf.math.reduce_mean(
    input_tensor,
    axis=None,
    keepdims=None,
    name=None,
    reduction_indices=None,
    keep_dims=None
)

Reduces input_tensor along the dimensions given in axis. Unless keepdims is true, the rank of the tensor is reduced by 1 for each entry in axis. If keepdims is true, the reduced dimensions are retained with length 1.

If axis is None, all dimensions are reduced, and a tensor with a single element is returned.

Example:

x = tf.constant([[1., 1.], [2., 2.]])
tf.reduce_mean(x)  # 1.5
tf.reduce_mean(x, 0)  # [1.5, 1.5]
tf.reduce_mean(x, 1)  # [1.,  2.]

Returns:

The reduced tensor.

Numpy compatibility:

Equivalent to np.mean

Please note that np.mean has a dtype parameter that could be used to specify the output type. By default this is dtype=float64. On the other hand, tf.reduce_mean has an aggressive type inference from input_tensor.

Example:

x = tf.constant([1, 0, 1, 0])
tf.reduce_mean(x)  # 0
y = tf.constant([1., 0., 1., 0.])
tf.reduce_mean(y)  # 0.5

5. TensorFlow reduce_sum (tf.math.reduce_sum)

Aliases: tf.compat.v1.math.reduce_sum, tf.compat.v1.reduce_sum, tf.reduce_sum

Computes the sum of elements across dimensions of a tensor. (deprecated arguments)

tf.math.reduce_sum(
    input_tensor,
    axis=None,
    keepdims=None,
    name=None,
    reduction_indices=None,
    keep_dims=None
)

Reduces input_tensor along the dimensions given in axis. Unless keepdims is true, the rank of the tensor is reduced by 1 for each entry in axis. If keepdims is true, the reduced dimensions are retained with length 1.

If axis is None, all dimensions are reduced, and a tensor with a single element is returned.

Example:

x = tf.constant([[1, 1, 1], [1, 1, 1]])
tf.reduce_sum(x)  # 6
tf.reduce_sum(x, 0)  # [2, 2, 2]
tf.reduce_sum(x, 1)  # [3, 3]
tf.reduce_sum(x, 1, keepdims=True)  # [[3], [3]]
tf.reduce_sum(x, [0, 1])  # 6

Arguments:

- input: The tensor to reduce. Should have numeric type.

- axis: The dimensions to reduce. If None (the default), reduces all dimensions. Must be in the range [-rank(input_tensor), rank(input_tensor)).

- keepdims: If true, retains reduced dimensions with length 1.

- name: A name for the operation (optional).

- reduction_indices: The old (deprecated) name for axis.

- keep_dims: Deprecated alias for keepdims.

6. TensorFlow softmax_cross_entropy_with_logits (tf.nn.softmax_cross_entropy_with_logits)

Distributed training with TensorFlow

Measures the probability error in discrete classification tasks in which the classes are mutually exclusive (each entry is in exactly one class). For example, each CIFAR-10 image is labeled with one and only one label: an image can be a dog or a truck, but not both.

NOTE: While the classes are mutually exclusive, their probabilities need not be. All that is required is that each row of labels is a valid probability distribution. If they are not, the computation of the gradient will be incorrect.

If using exclusive labels (wherein one and only one class is true at a time), see sparse_softmax_cross_entropy_with_logits.

Usage:

logits = [[4.0, 2.0, 1.0], [0.0, 5.0, 1.0]] labels = [[1.0, 0.0, 0.0], [0.0, 0.8, 0.2]] 
tf.nn.softmax_cross_entropy_with_logits(labels=labels, logits=logits)

WARNING: This op expects unscaled logits, since it performs a softmax on logits internally for efficiency. Do not call this op with the output of softmax, as it will produce incorrect results.

A common use case is to have logits and labels of shape [batch_size, num_classes], but higher dimensions are supported, with the axis argument specifying the class dimension.

Logits and labels must have the same dtype (either float16, float32, or float64).

Backpropagation will happen into both logits and labels. To disallow backpropagation into labels, pass label tensors through tf.stop_gradient before feeding it to this function.

Note that to avoid confusion, it is required to pass only named arguments to this function.

Arguments:

- labels: Each vector along the class dimension should hold a valid probability distribution e.g. for the case in which labels are of shape [batch_size, num_classes], each row of labels[i] must be a valid probability distribution.

- logits: Per-label activations, typically a linear output. These activation energies are interpreted as unnormalized log probabilities.

- axis: The class dimension. Defaulted to -1 which is the last dimension.

- name: A name for the operation (optional).

Returns:

A Tensor that contains the softmax cross entropy loss. Its type is the same as logits and its shape is the same as labels except that it does not have the last dimension of labels.

7. TensorFlow matmul (tf.matmul)

Aliases: tf.linalg.matmul

Multiplies matrix a by matrix b, producing a * b.

tf.linalg.matmul(
    a,
    b,
    transpose_a=False,
    transpose_b=False,
    adjoint_a=False,
    adjoint_b=False,
    a_is_sparse=False,
    b_is_sparse=False,
    name=None
)

Argumentss:

- a: tf.Tensor of type float16, float32, float64, int32, complex64, complex128 and rank > 1.

- b: tf.Tensor with same type and rank as a.

Returns:

A tf.Tensor of the same type as a and b where each inner-most matrix is the product of the corresponding matrices in a and b, e.g. if all transpose or adjoint attributes are False:

output[..., i, j] = sum_k (a[..., i, k] * b[..., k, j]), for all indices i, j.

Note: This is matrix product, not element-wise product.

Raises:

- ValueError: If transpose_a and adjoint_a, or transpose_b and adjoint_b are both set to True.

Example:

# Create some tensors
a = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
c = tf.matmul(a, b)

print(c)

Executing operation MatMul:

tf.Tensor(
[[22. 28.]
 [49. 64.]], shape=(2, 2), dtype=float32)

8. TensorFlow softmax_cross_entropy_with_logits (tf.nn.softmax_cross_entropy_with_logits)

Computes softmax cross entropy between logits and labels.

tf.nn.softmax_cross_entropy_with_logits(
    labels, logits, axis=-1, name=None
)

Measures the probability error in discrete classification tasks in which the classes are mutually exclusive (each entry is in exactly one class). For example, each CIFAR-10 image is labeled with one and only one label: an image can be a dog or a truck, but not both.

Note: While the classes are mutually exclusive, their probabilities need not be. All that is required is that each row of labels is a valid probability distribution. If they are not, the computation of the gradient will be incorrect.

If using exclusive labels (wherein one and only one class is true at a time), see sparse_softmax_cross_entropy_with_logits.

Usage:

# Create some tensors
logits = [[4.0, 2.0, 1.0], [0.0, 5.0, 1.0]]
labels = [[1.0, 0.0, 0.0], [0.0, 0.8, 0.2]]
c = tf.nn.softmax_cross_entropy_with_logits(labels=labels, logits=logits)
print(c)

Executing operation of softmax_cross_entropy_with_logits:

<tf.Tensor: shape=(2,), dtype=float32,
numpy=array([0.16984604, 0.82474494], dtype=float32)>
Warning: This op expects unscaled logits, since it performs a softmax on logits internally for efficiency. Do not call this op with the output of softmax, as it will produce incorrect results.

A common use case is to have logits and labels of shape [batch_size, num_classes], but higher dimensions are supported, with the axis argument specifying the class dimension.

logits and labels must have the same dtype (either float16, float32, or float64).

Backpropagation will happen into both logits and labels. To disallow backpropagation into labels, pass label tensors through tf.stop_gradient before feeding it to this function.

Arguments:

- labels Each vector along the class dimension should hold a valid probability distribution e.g. for the case in which labels are of shape [batch_size, num_classes], each row of labels[i] must be a valid probability distribution.

- logits Per-label activations, typically a linear output. These activation energies are interpreted as unnormalized log probabilities.

- axis The class dimension. Defaulted to -1 which is the last dimension.

- name A name for the operation (optional).

Returns:

A Tensor that contains the softmax cross entropy loss. Its type is the same as logits and its shape is the same as labels except that it does not have the last dimension of labels.