# Tensorflow - part 1: Creating and manipulating tensors (continue)

In the previous part, we have shown how to create a tensor, convert a tensor to numpy array, some attributes of a tensor, math operations, concatenating and stacking, reshaping. Now, we will tell a litte more things about what we can do with tensors.

First, supposing that you have installed Tensorflow, let's import it.

``````import numpy as np # Numpy array is often used together with tensors,
# so you need to include it as well.
import tensorflow as tf
print(tf.__version__) # You can also use this line to check its installed version``````

## Calculating mean, standard deviation

In `tf.math`, there are functions that are used to compute the mean, sum, or standard deviation of values in a tensor. We can choose which axis to compute along. You should investigate the code below to see how `axis` affects the shape and values of the result tensor.

``````tensor_x = tf.random.uniform(shape=(5, 3), minval=-1.0, maxval=1.0) # create a tensor to work with
tensor_mean = tf.math.reduce_mean(tensor_x, axis=0) # mean
tensor_sum = tf.math.reduce_sum(tensor_x, axis=1) # sum
tensor_std = tf.math.reduce_std(tensor_x, axis=0) # standard deviation``````

Output

``````tf.Tensor(
[[ 0.47  -0.716 -0.873]
[-0.622  0.596  0.408]
[-0.182  0.029 -0.856]
[-0.309  0.501 -0.693]
[-0.473  0.25   0.309]], shape=(5, 3), dtype=float32)
tf.Tensor([-0.223  0.132 -0.341], shape=(3,), dtype=float32)
tf.Tensor([-1.119  0.383 -1.009 -0.502  0.087], shape=(5,), dtype=float32)
tf.Tensor([0.377 0.468 0.576], shape=(3,), dtype=float32)``````

When `axis=0`, the calculations happen along the vertical direction (column) of the `tensor_x`. There are 3 columns in the `tensor_x`, so the result tensor (`tensor_mean` and `tensor_std`) will have 3 values, each of which is the result of one column. When `axis=1`, in contrast, the calculations happen along the horizontal direction (row) of the `tensor_x`.

## Slicing

If wanting to extract sub tensors from a tensor, we need to use slicing. Supposing we have a tensor

``tensor = tf.range(0, 8)``

Output

``tf.Tensor([0 1 2 3 4 5 6 7], shape=(8,), dtype=int32)``

Extract from the second element to the third element. It is also the same as in python slicing that the last element (at position 4) is excluded.

``tensor[1:4]``

Output

``tf.Tensor([1 2 3], shape=(3,), dtype=int32)``

Extract 3 last elements

``tensor[-3:]``

Output

``tf.Tensor([5 6 7], shape=(3,), dtype=int32)``

You can also use tf.slice for this with one difference. For the second argument `size`, it does not use the end index like above, but size instead. Get a tensor from previous post as example.

``stack_axis_1_tensor # from previous post``

Output

``````tf.Tensor(
[[[  0   1   1   2   3   5   8  13  21  34]
[ 22  40  42  31  10 -13  17  19 -24  46]]

[[  5   6   9   7  11  10   0   4   3   2]
[ 17 -31  -3 -43  17   7  46  19  12  11]]

[[ 20  30  40  50  60 100  90  80  70  10]
[-27  -3 -20  34 -50   6  23 -12 -26 -50]]], shape=(3, 2, 10), dtype=int32)``````

Use tf.slice:

``tf.slice(stack_axis_1_tensor, begin=[1, 0, 3], size=[2, 1, 4])``

Output

``````tf.Tensor(
[[[  7  11  10   0]]

[[ 50  60 100  90]]], shape=(2, 1, 4), dtype=int32)``````

We can also use step for strided slice. For example, if you want to extract one element in every two elements (step=2), you can use slicing as above. `tf.gather` is an another way to do this, but with this you need need to specify correctly which positions to get the value.

``````tensor[::2]
tf.gather(tensor, indices=[0, 2, 4, 6]) # tf.gather to get elements at even indices: 0, 2, 4, 6``````

Output

``````tf.Tensor([0 2 4 6], shape=(4,), dtype=int32)
tf.Tensor([0 2 4 6], shape=(4,), dtype=int32)``````

tf.gather can even get elements at indices that are not equally spaces and the new tensor can also be shuffled

``````string_tensor = tf.constant(list('Hello world'))
tf.gather(string_tensor, indices=[8, 1, 4]) # Get at positions 8, 1, 4 respectively``````

Output

``tf.Tensor([b'r' b'e' b'o'], shape=(3,), dtype=string)``

Here, the `indices` list does not contain positions in the increasing or decreasing order, but they are shuffled in an unordered style.

## Expand dims

Sometimes you need to add a new axis to a tensor so it can be computed (or concatenated) together with another tensor. Some operations required the involved tensors have the same number of dimensions.

``````tensor_x = tf.zeros([50, 50, 4])
tf.expand_dims(tensor_x, axis=0).shape
tf.expand_dims(tensor_x, axis=1).shape``````

Output

``````(1, 50, 50, 4)
(50, 1, 50, 4)``````

The new axis has been added at the specified position (0 and 1).

## Tiling

Repeat elements in tensor_a by a factor as specified for each dim in tensor_b

``````tensor_a = tf.constant([[1, 2], [3, 4]], tf.int32)
tensor_b = tf.constant([1, 3], tf.int32) # repeat 1 time in dimension 0, repeat 3 times in dimension 1
print(tf.tile(tensor_a, tensor_b))
tensor_c = tf.constant([3, 1], tf.int32) # repeat 3 times in dimension 0, repeat 1 time in dimension 1
print(tf.tile(tensor_a, tensor_c))``````

Output

``````tf.Tensor(
[[1 2 1 2 1 2]
[3 4 3 4 3 4]], shape=(2, 6), dtype=int32)

tf.Tensor(
[[1 2]
[3 4]
[1 2]
[3 4]
[1 2]
[3 4]], shape=(6, 2), dtype=int32)``````

In the example above, the `tensor_b` is [1, 3]. This means the `tensor_a` is replicated one time along the 0-th dimension (row) and 3 times along the 1-th dimension (column).

## Squeeze

Removing the unnecessary dimensions (dimensions that have size 1)

``````tensor_x = tf.zeros((1, 2, 1, 4, 1))
tensor_sqz = tf.squeeze(tensor_x)
print('old shape:', tensor_x.shape, '-->', 'new shape:', tensor_sqz.shape)
tensor_sqz = tf.squeeze(tensor_x, axis=(2, 4)) # We can also choose what axis to remove
print('old shape:', tensor_x.shape, '-->', 'new shape:', tensor_sqz.shape)``````

Output

``````old shape: (1, 2, 1, 4, 1) --> new shape: (2, 4)
old shape: (1, 2, 1, 4, 1) --> new shape: (1, 2, 4)``````

## Split

There are 2 types of splits:

• Split a tensor into multiple tensors of equal size
``````tf.random.set_seed(1)
tensor = tf.random.uniform((9,))
print(tensor.numpy())

tensor_splits = tf.split(tensor, num_or_size_splits=3)
[item.numpy() for item in tensor_splits]``````

Output

``````[0.165 0.901 0.631 0.435 0.292 0.643 0.976 0.435 0.66 ]

[array([0.165, 0.901, 0.631], dtype=float32),
array([0.435, 0.292, 0.643], dtype=float32),
array([0.976, 0.435, 0.66 ], dtype=float32)]``````
• Split a tensor into multiple tensors of specified size for each new tensor
``````tensor = tf.random.uniform((7,))
print(tensor.numpy())

tensor_splits = tf.split(tensor, num_or_size_splits=[3, 4]) # size
[item.numpy() for item in tensor_splits]``````

Output

``````[0.829 0.634 0.515 0.391 0.581 0.048 0.178]

[array([0.829, 0.634, 0.515], dtype=float32),
array([0.391, 0.581, 0.048, 0.178], dtype=float32)]``````

First tensor has size 3 and second tensor has size 4. That is because the numbers that we give to the paramerter `num_or_size_splits`.

## Unique

Let's have a small exercise here. We have 2 flat tensors and want to check if all of their elements are equal.

Supposing we have 2 tensors: `reshape_0` and `reshape_1`:

``````print(reshape_0)
print(reshape_1)``````

Output

``````tf.Tensor(
[  0   1   1   2   3   5   8  13  21  34   5   6   9   7  11  10   0   4
3   2  20  30  40  50  60 100  90  80  70  10  22  40  42  31  10 -13
17  19 -24  46  17 -31  -3 -43  17   7  46  19  12  11 -27  -3 -20  34
-50   6  23 -12 -26 -50], shape=(60,), dtype=int32)

tf.Tensor(
[  0   1   1   2   3   5   8  13  21  34  22  40  42  31  10 -13  17  19
-24  46   5   6   9   7  11  10   0   4   3   2  17 -31  -3 -43  17   7
46  19  12  11  20  30  40  50  60 100  90  80  70  10 -27  -3 -20  34
-50   6  23 -12 -26 -50], shape=(60,), dtype=int32)``````

To check if they are equal or not, use tf.math.equal: this method checks in an element-wise way and return a list of True, False values. If the list are all True values, it means the 2 tensors are equal.

``print('Use tf.math.equal: ', tf.math.equal(reshape_0, reshape_1))``

Output

``````Use tf.math.equal:  tf.Tensor(
[ True  True  True  True  True  True  True  True  True  True False False
False False False False False False False False False False False False
False False False False False False False False False False False False
False  True False False False False False False False False False False
False False  True  True  True  True  True  True  True  True  True  True], shape=(60,), dtype=bool)``````

In the case when there are too many elements, it would be better if we only show the unique elements.

``print('Use tf.unique: ', tf.unique(tf.math.equal(reshape_0, reshape_1)))``

Output

``````Use tf.unique:  Unique(y=<tf.Tensor: shape=(2,), dtype=bool, numpy=array([ True, False])>, idx=<tf.Tensor: shape=(60,), dtype=int32, numpy=
array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32)>)``````

Moreover, you can count the number of repetitions of each unique element.

``print('Use tf.unique_with_count', tf.unique_with_counts(tf.math.equal(reshape_0, reshape_1)))``

Output

``````Use tf.unique_with_count: UniqueWithCounts(y=<tf.Tensor: shape=(2,), dtype=bool, numpy=array([ True, False])>, idx=<tf.Tensor: shape=(60,), dtype=int32, numpy=
array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32)>, count=<tf.Tensor: shape=(2,), dtype=int32, numpy=array([21, 39], dtype=int32)>)``````

You can see in the end of the output there is `[21, 39]` which means there are 21 values 0 (True) and 39 values 1 (False). Therefore, not all elements are equal.

When running computation on 2 tensors having different sizes, the smaller tensor will be automatically expanded to be matched to the larger one.

``````smaller = tf.constant(2) # is automatically expanded into [2, 2, 2]
larger = tf.constant([1, 2, 3])
print(smaller * larger)
print(tf.multiply(smaller, larger))``````

Output

``````tf.Tensor([2 4 6], shape=(3,), dtype=int32)
tf.Tensor([2 4 6], shape=(3,), dtype=int32)``````

Broadcasting is also useful when we consider the case of outer product of 2 tensors

``````x = tf.reshape(larger, [3, 1])
print(x)
y = tf.range(1, 6)
print(y)
print(tf.multiply(x, y)) # like outer product``````

Output

``````tf.Tensor(
[

], shape=(3, 1), dtype=int32)

tf.Tensor([1 2 3 4 5], shape=(5,), dtype=int32)

tf.Tensor(
[[ 1  2  3  4  5]
[ 2  4  6  8 10]
[ 3  6  9 12 15]], shape=(3, 5), dtype=int32)``````