In this tutorial, we will cover:
Numpy is the core library for scientific computing in Python. It provides a high-performance multidimensional array object, and tools for working with these arrays.
To use Numpy, we first need to import the numpy package:
import numpy as np
A numpy array is a grid of values, all of the same type, and is indexed by a tuple of nonnegative integers. The number of dimensions is the rank of the array; the shape of an array is a tuple of integers giving the size of the array along each dimension.
We can initialize numpy arrays from nested Python lists. We can create a 1D(rank 1) array as follows.
a = np.array([1,2,3,4]) # rank 1 array (1D array)
print(a.shape)
print(a)
We'll work exclusively with 2D(rank 2) arrays in this course.
b = np.array([[1,2,3],[4,5,6]])
print(b)
print(b.shape)
Numpy provides a way to generate a random array of any shape whose values are drawn from the standard normal distribution with mean $0$ and standard deviation $1$.
d = np.random.randn(2,3) # standard normal distribution
print(d)
We'll deal exclusively with greyscale images(2D arrays, or matrices) in this course. We begin by reviewing some basic matrices and their operations.
An $m\times n$ matrix $A$ is a rectangular array of real numbers arranged in $m$ rows and $n$ columns and has the form:
$$\left[ \begin{array}{cccc} a_{11} & a_{12} & \ldots & a_{1n} \\ a_{21} & a_{22} & \ldots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \ldots & a_{mn} \end{array} \right] $$
We will denote an $m\times n$ matrix $A$ as $[a_{ij}]$.
In Numpy, such a 2D array has shape $(m,n)$.
Given two matrices $A=[a_{ij}]$ and $B=[b_{ij}]$, of dimension $m\times n$. The sum $A+B$ and difference $A-B$ are calculated elementwise.
For example, $$\left[\begin{array}{rr} 1 & 2 \\ 3 & 4 \end{array}\right]+\left[\begin{array}{rr} 5 & 6 \\ 7 & 8 \end{array}\right]=\left[\begin{array}{rr} 6 & 8 \\ 10 & 12 \end{array}\right]$$
x = np.array([[1,2],[3,4]])
y = np.array([[5,6],[7,8]])
# elementwise sum
x + y
# Elementwise difference
x - y
The scalar product $cA$ of a real number(scalar) $c$ and a matrix $A$ is computed by multiplying every entry of $\textbf{A}$ by $c$. Thus $cA=[ca_{ij}].$
For example, $$3\left[\begin{array}{rr} 1 & 2 \\ 3 & 4 \end{array}\right]=\left[\begin{array}{rr} 3 & 6 \\ 9 & 12 \end{array}\right]$$
# scalar product
3*x
The elementwise product of two matrices $A=[a_{ij}]$ and $B=[b_{ij}]$, of dimension $m\times n$, also known as the Hadamard product is $A*B=[a_{ij}*b_{ij}]$.
For example, $$\left[\begin{array}{rr} 1 & 2 \\ 3 & 4 \end{array}\right]*\left[\begin{array}{rr} 5 & 6 \\ 7 & 8 \end{array}\right]=\left[\begin{array}{rr} 5 & 12 \\ 21 & 32 \end{array}\right]$$
# Elementwise product
x*y
Note that * is elementwise multiplication, not matrix multiplication. We instead use the dot function to multiply matrices. Matrix multiplication is VERY important in machine learning. You need to be very comfortable with it. Please review it, if necessary.
We'll review matrix multiplication through some examples.
$$\left[\begin{array}{rr} 1 & 2 \\ 3 & 4 \end{array}\right]\left[\begin{array}{r} 1 \\ 2 \end{array}\right]=\left[\begin{array}{r} 5 \\ 11 \end{array}\right]$$
x = np.array([[1,2],[3,4]]) # shape (2,2)
v = np.array([[1],[2]]) # shape (2,1)
print(np.dot(x,v))
$$\left[\begin{array}{rr} 1 & 0 & 3 \\ 4 & -1 & 2 \end{array}\right]\left[\begin{array}{rr} 0 & 1 \\ 2 & -1 \\ 3 & 0 \end{array}\right]=\left[\begin{array}{r} 9 & 1 \\ 4 & 5 \end{array}\right]$$
Note that a (2,3) array times (3,2) array yields a (2,2) array.
x = np.array([[1,0,3],[4,-1,2]])
y = np.array([[0,1],[2,-1],[3,0]])
print(np.dot(x,y))
The following matrix multiplication is undefined since the dimensions are not aligned correctly.
$$\left[\begin{array}{rr} 1 & 2 \\ 3 & 4 \end{array}\right]\left[\begin{array}{rr} 0 & 1 \end{array}\right]$$
x = np.array([[1,2],[3,4]])
y = np.array([[0,1]])
np.dot(x,y)
Given a matrix $A$ of dimension $m\times n$, define the transpose of $A$ denoted by $A^T$ as the matrix whose rows are the columns of $A$. Thus the dimension of $A^T$ is $n\times m$.
In Numpy, to transpose a matrix, simply use the T attribute of an array object:
v = np.array([[1,2,3]])
print(v) # shape (1,3)
print(v.T) # shape (3,1)
Numpy provides many useful functions for performing computations on arrays; one of the most useful is sum:
# .sum(axis = n) dimension n is collapsed
# and all values in the new matrix equal to the sum of the corresponding collapsed values
x = np.array([[1,2,3],[4,5,6]])
print(x)
# Compute sum of all elements;
print(np.sum(x))
# all the rows(axis=0) are collapsed, summing the columns
print(np.sum(x, axis = 0))
print(np.sum(x, axis = 1)) # all the columns(axis=1) are collapsed,
# summing the rows
Another useful numpy function that we will use is argmax which returns the index of the largest value.
x = np.array([1,2,5,8,4,5])
print(np.argmax(x))
You can find the full list of mathematical functions provided by numpy in the documentation.
Broadcasting allows operations on arrays whose shapes are not compatible. This notebook's treatment of broadcasting is VERY minimal. We only cover what is necessary for the rest of the course.
You can find more information about broadcasting in the documentation.
Consider the following matrices
$A = \left[\begin{array}{rr}
1 & 2 \\
3 & 4
\end{array}\right]$ and $B=\left[\begin{array}{rr}
0 & 1
\end{array}\right]$. Note that $A+B$ is undefined because the shapes (2,2) and (2,1) are not compatible.
Numpy, however, will broadcast the shape of the second matrix to shape (2,2) to be compatible with the first so that the addition can be done. It does this without making copies of the data and wasting memory.
$$\left[\begin{array}{rr} 1 & 2 \\ 3 & 4 \end{array}\right]+\left[\begin{array}{rr} 0 & 1\end{array}\right]\rightarrow\left[\begin{array}{rr} 1 & 2 \\ 3 & 4 \end{array}\right]+\left[\begin{array}{rr} 0 & 1 \\ 0 & 1 \end{array}\right]=\left[\begin{array}{rr} 1 & 3 \\ 3 & 5 \end{array}\right]$$
Similarly,
$$\left[\begin{array}{rr} 1 & 2 \\ 3 & 4 \end{array}\right]+\left[\begin{array}{rr} 0 \\ 1 \end{array}\right]\rightarrow\left[\begin{array}{rr} 1 & 2 \\ 3 & 4 \end{array}\right]+\left[\begin{array}{rr} 0 & 0 \\ 1 & 1 \end{array}\right]=\left[\begin{array}{rr} 1 & 2 \\ 4 & 5 \end{array}\right]$$
A = np.array([[1,2],[3,4]])
B = np.array([[0,1]])
A + B
A = np.array([[1,2],[3,4]])
B = np.array([[0],[1]])
A + B