python-data-2-numpy.ipynb

    "\n",
    "For example `maximum()` selects the biggest values from two input arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.random.randn(10)\n",
    "y = np.random.randn(10)\n",
    "np.maximum(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is a list of useful *ufuncs* in NumyPy:\n",
    "\n",
    "*Again, you don't need to memorise them. This is just a reference*\n",
    "### Unary functions (accept one argument)\n",
    "\n",
    "| Function | Description |\n",
    "|----|----|\n",
    "| abs, fabs | Compute the absolute value element-wise for integer, floating point, or complex values.<br>Use fabs as a faster alternative for non-complex-valued data |\n",
    "| sqrt | Compute the square root of each element. Equivalent to arr ** 0.5 |\n",
    "| exp | Compute the exponent ex of each element |\n",
    "| log, log10, log2, log1p | Natural logarithm (base e), log base 10, log base 2, and log(1 + x), respectively |\n",
    "| cos, cosh, sin, sinh, tan, tanh | Regular and hyperbolic trigonometric functions |\n",
    "\n",
    "### Binary functions (accept 2 arguments)\n",
    "| Functions | Description |\n",
    "| ---- | ---- |\n",
    "| add | Add corresponding elements in arrays |\n",
    "| subtract | Subtract elements in second array from first array |\n",
    "| multiply | Multiply array elements |\n",
    "| divide, floor_divide | Divide or floor divide (truncating the remainder) |\n",
    "| mod | Element-wise modulus (remainder of division) |\n",
    "| power | Raise elements in first array to powers indicated in second array |\n",
    "| maximum | Element-wise maximum. fmax ignores NaN |\n",
    "| minimum | Element-wise minimum. fmin ignores NaN |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Other useful operations <a name=\"other\"></a>\n",
    "NumPy offers a set of mathematical functions that compute statistics about an entire array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "B = np.random.randn(5, 4)\n",
    "B"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "B.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.mean(B)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "B.sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "B.mean(axis=1) # Compute mean in column (axis 1) direction (i.e. the mean of each row)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here `mean(axis=1)` means compute the mean across the columns (axis 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is a set of other similar functions:\n",
    "\n",
    "| Function | Description|\n",
    "| --- | --- |\n",
    "|sum | Sum of all the elements in the array or along an axis. Zero-length arrays have sum 0. |\n",
    "| mean | Arithmetic mean. Zero-length arrays have NaN mean. |\n",
    "| std, var | Standard deviation and variance, respectively, with optional<br>degrees of freedom adjustment (default denominator n). |\n",
    "|min, max | Minimum and maximum. |\n",
    "| argmin, argmax | Indices of minimum and maximum elements, respectively. |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There also are some boolean operations. `any` tests whether one or more values in an array is `True`, and `all` tests whether all values are `True`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = np.random.randn(100)\n",
    "A_bool = A > 0\n",
    "A_bool"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A_bool.any()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A_bool.all()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 8\n",
    "Generate and normalise a random 5x5x5 matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 9\n",
    "Create a random vector of size 30 and find its mean value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 10\n",
    "\n",
    "Subtract the mean of each row of a randomly generated matrix:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sorting <a name=\"sorting\"></a>\n",
    "Similar to Python's built-in list type, NumyPy arrays can be sorted in place:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = np.random.randn(10)\n",
    "A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A.sort()\n",
    "A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another option is `unique()` which returns the sorted unique values in an array."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Linear Algebra <a name=\"linear\"></a>\n",
    "Similar to other languages like MATLAB, NumyPy offers a set of standard linear algebra operations, like matrix multiplication, decompositions, determinants and etc.. Unlike some other languages though, the default operations like `*` peform element-wise operations. To perform matrix-wise operartions we need to use special functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "temp = np.arange(16)\n",
    "A = temp[:8]\n",
    "B = temp[8:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A.dot(B)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also extend this with the `numpy.linalg` package:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from numpy.linalg import inv, qr\n",
    "A = np.random.randn(5, 5)\n",
    "mat = A.T.dot(A)\n",
    "mat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "inv(mat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mat.dot(inv(mat))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is a set of commonly used `numpy.linalg` functions\n",
    "\n",
    "| Function | Description |\n",
    "| --- | --- |\n",
    "| diag | Return the diagonal (or off-diagonal) elements of a square matrix as a 1D array,<br>or convert a 1D array into a square matrix with zeros on the off-diagonal |\n",
    "| dot | Matrix multiplication |\n",
    "| trace | Compute the sum of the diagonal elements |\n",
    "| det | Compute the matrix determinant |\n",
    "| eig | Compute the eigenvalues and eigenvectors of a square matrix |\n",
    "| inv | Compute the inverse of a square matrix |\n",
    "| pinv | Compute the Moore-Penrose pseudo-inverse inverse of a square matrix |\n",
    "| qr | Compute the QR decomposition |\n",
    "| svd | Compute the singular value decomposition (SVD) |\n",
    "| solve | Solve the linear system Ax = b for x, where A is a square matrix |\n",
    "| lstsq | Compute the least-squares solution to y = Xb |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 11\n",
    "Obtain the diagonal of a dot product of 2 random matrices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## File IO <a name=\"file\"></a>\n",
    "NumPy offers its own set of File IO functions.\n",
    "\n",
    "The most common one is `genfromtxt()` which can load the common `.csv` and `.tsv` files.\n",
    "\n",
    "Now let us analyse temperature data from Stockholm over the years.\n",
    "\n",
    "First we have to load the file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = np.genfromtxt(\"./data/stockholm_td_adj.dat\")\n",
    "data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first column of this array gives years, and the 6th gives temperature readings. We can extract these."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "yrs = data[:, 0]\n",
    "temps = data[:, 5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Having read in our data, we can now work with it - for example, we could produce a plot.\n",
    "We will cover plotting in more depth in notebook 4, so there's no need to get too caught up in the details right now - this is just an examle of something we might do having read in some data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(16, 6)) # Create a 16x6 figure\n",
    "plt.plot(yrs, temps) # Plot temps vs yrs\n",
    "\n",
    "#Set some labels\n",
    "plt.title(\"Temperatures in Stockholm\")\n",
    "plt.xlabel(\"year\")\n",
    "plt.ylabel(\"Temperature (C)\")\n",
    "\n",
    "plt.show() # Show the plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 12\n",
    "Read in the file `daily_gas_price.csv`, which lists the daily price of natural gas since 1997. Each row contains a date and a price, separated by a comma. Find the minimum, maximum, and mean gas price over the dataset.\n",
    "\n",
    "(Hint: you will need to use the delimiter option in `np.genfromtxt` to specify that data is separated by commas. Also, NumPy will interpret the data in float format by default - we may need to set the dtype to a string format at first, then discard the dates, before turning the gas prices back into floats to process them! Otherwise, NumPy may find it confusing to try and interpret dates formatted as YYYY-MM-DD as floats and will probably complain.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}