python-data-1-recap-sol.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Notebook 1 - Programming with Python recap\n",
    "\n",
    "This course assumes that you have knowledge of basic Python (i.e. that you have attended the Introduction to Python course ran by Information Services).\n",
    "\n",
    "Just to make sure we are all on the same page, this notebook will serve as a recap of basic Python functionality (and also allow you to warm up)!\n",
    "\n",
    "## Table of Contents\n",
    "- [Types](#types)\n",
    "- [Arithmetic operations](#arith)\n",
    "- [Boolean logic](#boolean)\n",
    "- [Printing](#print)\n",
    "- [Lists](#lists)\n",
    "- [If-Else](#if)\n",
    "- [Loops](#loops)\n",
    "- [Functions](#func)\n",
    "- [Text Analysis](#text)\n",
    "\n",
    "You are intended to skim over this notebook and do the exercises at the end. Furthermore, at the end of the notebook there is a big exercise with textual analysis\n",
    "\n",
    "Estimated duration: 60 min\n",
    "\n",
    "You can use this notebook as a reference for the rest of the course."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Types <a name=\"types\"></a>\n",
    "| Type | Declaration | Example | Usage |\n",
    "|----|----|----|----|\n",
    "| Integer  | int | `x = 124` | Numbers without decimal point |\n",
    "| Float  | float | `x = 124.56` | Numbers with decimcal point |\n",
    "| String  | str | `x = \"Hello world\"` | Used for text |\n",
    "| Boolean  | bool | `x = True` or `x = False` | Used for conditional statements |\n",
    "| NoneType  | None | `x = None` | Whenever you want an empty variable |\n",
    "\n",
    "You can convert from one type to another:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 10    # This is an integer\n",
    "y = \"20\"  # This is a string\n",
    "x + int(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can find the type of a variable using the `type()` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "var = 13.4\n",
    "type(var)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Arithmetical operators <a name=\"arith\"></a>\n",
    "\n",
    "Python supports a range of different arithmetical operators:\n",
    "\n",
    "| Symbol | Task Performed | Example | Result\n",
    "|----|---|---|---|\n",
    "| +  | Addition | 4 + 3 | 7 |\n",
    "| -  | Subtraction | 4 - 3 | 1 |\n",
    "| /  | Division | 7 / 2 | 3.5 |\n",
    "| %  | Mod | 7 % 2 | 1 |\n",
    "| *  | Multiplication | 4 * 3 | 12 |\n",
    "| //  | Floor division | 7 // 2 | 3 |\n",
    "| **  | Power of | 7 ** 2 | 49 |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "16 ** 2 / 4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Boolean logic <a name=\"boolean\"></a>\n",
    "| Operator | Output | \n",
    "|----|---|\n",
    "| x == y | True if x and y have the same value |\n",
    "| x != y | True if x and y don't have the same value |\n",
    "| x < y | True if x is less than y |\n",
    "| x > y | True if x is more than y |\n",
    "| x <= y | True if x is less than or equal to y |\n",
    "| x >= y | True if x is more than or equal to y |\n",
    "\n",
    "Make note that these operators return Boolean values (ie. `True` or `False`). Naturally, if the operations don't return `True`, they will return `False`. Let's try some of them out:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 5       # assign 5 to the variable x\n",
    "x == 5      # check if value of x is 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x > 7"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That is nice! How can we extend this to link multiple combinations like that? Luckily, Python offers the usual set of **logical operations**:\n",
    "\n",
    "| Operation | Result | \n",
    "|----|---|\n",
    "| x or y | True if at least on is True |\n",
    "| x and y  | True only if both are True |\n",
    "| not x | True only if x is False |\n",
    "\n",
    "\n",
    "\n",
    "With this, we can now chain different boolean operations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# check if x is divisible by both 2 and 3\n",
    "x = 42\n",
    "( x % 2 == 0 ) and ( x % 3 == 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Printing <a name=\"print\"></a>\n",
    "Output text to the user of the program"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Python is powerful!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = \"Python is powerful\"\n",
    "y = \" and versatile!\"\n",
    "print(x + y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "str1 = \"The string has\"\n",
    "str2 = 76\n",
    "str3 = \"methods!\"\n",
    "print(str1, str2, str3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Furthermore, there is also a way to insert variables inbetween text using placeholders like `%d` which stands for decimal. As seen from the code cell below, this is an extremely compact and easy way to print. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "num = 12\n",
    "print(\"You can also include a number like %d like this.\" %num)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = \"\"\"To include\n",
    "multiple lines\n",
    "you have to do this\"\"\"\n",
    "y =\"or you can also\\ninclude the special\\ncharacter `\\\\n` between lines\"\n",
    "print(x)\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lists <a name=\"lists\"></a>\n",
    "Probably the most commonly used data structure in Python. Lists simply group multiple variables together."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fruits = [\"apple\", \"orange\", \"tomato\", \"banana\"]  # a list of strings\n",
    "print(type(fruits))\n",
    "print(fruits)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can access each individual item within the list by **indexing** it.\n",
    "\n",
    "Recall that indices **start from 0**.\n",
    "\n",
    "![List Indexing](https://gdurl.com/ebW1)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(fruits[0])\n",
    "print(fruits[2])\n",
    "print(fruits[-1])\n",
    "print(len(fruits))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can modify existing lists as:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fruits.append(\"lime\")    # add new item to list\n",
    "print(fruits)\n",
    "fruits.append(\"orange\")\n",
    "print(fruits)\n",
    "fruits.remove(\"orange\")  # remove orange from list\n",
    "print(fruits)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That seems useful! Can we do the same with integers? Python actually offers various functions for generating numerical lists. The most useful out of which is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nums = list(range(10))\n",
    "print(nums)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also get only a part of a list you have already created. This is called **slicing** and can be done in various different ways:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(nums[0:3])\n",
    "print(nums[4:])\n",
    "print(nums[-1])\n",
    "print(nums[::-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# If Else <a name=\"if\"></a>\n",
    "The statement after `if` must evaluate to a Boolean value (`True` or `False`), if it is `True` then we execute the code between the `if` and `else` statements and skip the code after the else statement. If the Boolean expression after `if` evaluates to `False` then we skip the code between `if` and `else` and execute only the code after `else`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = True\n",
    "if x:\n",
    "    print(\"Executing if\")\n",
    "else:\n",
    "    print(\"Executing else\")\n",
    "print(\"Prints regardless of the outcome of the if-else block\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try changing the value of x to `False`.\n",
    "\n",
    "if-else statements can also be extended with `elif` which as implied combines both else + if = elif. This is useful if you want to have multiple conditions for example:\n",
    "\n",
    "Let us illustrate this in an elaborate example. Imagine you have purchased a stock and are looking to make a profit out of it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "purchasePrice = float(input(\"Price at which you have purchased bitcoins: \"))\n",
    "currentPrice = float(input(\"Current price of the bitcoins: \"))\n",
    "\n",
    "if currentPrice < purchasePrice*0.9:\n",
    "    print(\"Not a good idea to sell your bitcoins now.\")\n",
    "    print(\"You will lose\", purchasePrice - currentPrice, \"£ per bitcoin.\")\n",
    "elif currentPrice > purchasePrice*1.2:\n",
    "    print(\"You will make\", currentPrice - purchasePrice, \"£ per bitcoin.\")\n",
    "else:\n",
    "    print(\"Not worth selling right now.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Loops <a name=\"loops\"></a>\n",
    "\n",
    "## for\n",
    "Generally useful whenever you want to iterate over a list (or other data structure) of items and apply the same operation to all items within it. In general, `for` loops look like this and have indentation just like if-else statements:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fruitList = [\"apple\", \"orange\", \"tomate\", \"banana\"]\n",
    "for fruit in fruitList:\n",
    "    print(\"The fruit\", fruit, \"has index\", fruitList.index(fruit))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "numbers = range(10)\n",
    "for num in numbers:\n",
    "    squared = num ** 2\n",
    "    print(num, \"squared is\", squared)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## while\n",
    "Another useful loop which is a bit less controllable. It executes over and over until its condition becomes false. For example, we can make a loop that executes 5 times and then stops."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n = 0\n",
    "while n < 5:\n",
    "    print(\"Executing while loop\")\n",
    "    n = n + 1\n",
    "\n",
    "print(\"Finished while loop\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Functions <a name=\"func\"></a>\n",
    "Also referred to as methods, functions are effectively small programs that take in arguments(ie. inputs) and return values(outputs).\n",
    "\n",
    "Functions are an incredibly useful concept since they allow us to package functionality in a convenient and easy to read manner and reproduce the same result without having to write it again and again. A quick example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def printNum(num):\n",
    "    print(\"My favourite number is\", num)\n",
    "    \n",
    "printNum(7)\n",
    "printNum(14)\n",
    "printNum(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Rule of thumb** - if you are planning on using very similar code more than once, it may be wortwhile writing it as a reusable function. \n",
    "\n",
    "Here's also an example of a function that rounds a number:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def roundNum(num):\n",
    "    remainder = num % 1\n",
    "    if remainder < 0.5:\n",
    "        return num - remainder\n",
    "    else:\n",
    "        return num + (1 - remainder)\n",
    "\n",
    "# Will it work?\n",
    "x = roundNum(3.4)\n",
    "print (x)\n",
    "\n",
    "y = roundNum(7.7)\n",
    "print(y)\n",
    "\n",
    "z = roundNum(9.2)\n",
    "print(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1\n",
    "Create a function `fibonacci()` which takes an integer `num` as an input and returns a list of the first `num` fibonacci numbers.\n",
    "\n",
    "Eg.\n",
    "\n",
    "Input: `8`\n",
    "\n",
    "Output: `[1, 1, 2, 3, 5, 8, 13, 21]`\n",
    "\n",
    "*Hint: You might want to recall [fibonacci numbers](https://en.wikipedia.org/wiki/Fibonacci_number)*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fibonacci(num):\n",
    "    # [ WRITE YOUR CODE HERE ]\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check your answer by running the code cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "################ Checking code ########################\n",
    "# Please don't edit this code\n",
    "newList = fibonacci(10)\n",
    "if newList == [1, 1, 2, 3, 5, 8, 13, 21, 34, 55]:\n",
    "    print(\"Success!\")\n",
    "else:\n",
    "    print(\"Error! Your function returned\")\n",
    "    print(newList)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text analysis <a name=\"text\"></a>\n",
    "\n",
    "Here we will do basic text analysis with a practical purpose.\n",
    "\n",
    "For this we will be using the text [Humanistic Nursing by Josephine G. Paterson and Loretta T. Zderad](http://www.gutenberg.org/ebooks/25020). You already have this downloaded in your workspace.\n",
    "\n",
    "To open up the file, Python gives us a very handy function, we just have to give it the path to the file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "file = open(\"data/humanistic_nursing.txt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An easy way to deal with text files is reading it line by line within a for loop:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "for line in file:\n",
    "    print(line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Well, that was a lot of text. Can we turn it into something useful?\n",
    "\n",
    "For example, we can split up each line into the words making it and then count the occurances of the word \"and\". Here's a function that does that. Try it out!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define functino\n",
    "def countAnd(file_path):\n",
    "    counter = 0\n",
    "    file = open(file_path)\n",
    "    \n",
    "    for line in file:\n",
    "        for word in line.split():\n",
    "            if word == \"and\":\n",
    "                counter += 1\n",
    "                \n",
    "    return counter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# try function\n",
    "countAnd(\"./data/humanistic_nursing.txt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2: Count any word\n",
    "Based on the function above, now write your own function which counts the occurances of any word. For example:\n",
    "```python\n",
    "countAny(filename, \"medicine\")\n",
    "```\n",
    "will return the occurences of the word \"medicine\" in the file filename."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def countAny(file_path, des_word):\n",
    "    # [ WRITE YOUR CODE HERE ]\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Verify your function\n",
    "countAny(\"./data/humanistic_nursing.txt\", \"patient\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3: Count multiple words\n",
    "Before this exercise you should be familiar with Python dictionaries. If you're not, please see [here](https://docs.python.org/3/tutorial/datastructures.html#dictionaries).\n",
    "\n",
    "Write a function which takes a file path and a list of words, and returns a dictionary mapping each word to its frequency in the given file.\n",
    "\n",
    "Intuitively, we can first fill in the dictionary keys with the words in our list. Afterwards we can count the occurrences of each word and and fill in the appropriate dictionary value.\n",
    "\n",
    "*Hint: Can we use `countAny()` for this?*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def countAll(file_path, words):\n",
    "    # [ WRITE YOUR CODE HERE ]\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Verify your function\n",
    "countAll(\"./data/humanistic_nursing.txt\", [\"patient\", \"and\", \"the\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should expect `{'patient': 125, 'and': 1922, 'the': 2604}`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 4: Cleaning up \n",
    "Unless you have already accounted for it, your counter was thrown off by some words. Consider this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(countAny(filename, \"work\"))\n",
    "print(countAny(filename, \"work.\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Although the word is the same, you have different values for the 2 keys.\n",
    "\n",
    "In order to fix this we have to clean up the words before they are counted.\n",
    "\n",
    "There are multiple ways to do this. A good approach will be:\n",
    "- take all wards, one by one\n",
    "- use the `.strip()` method to clean up bad charecters\n",
    "- convert all words to lowercase\n",
    "\n",
    "You can get some ideas from the [String methods page](https://docs.python.org/3/library/stdtypes.html#string-methods)\n",
    "\n",
    "A good way to \n",
    "\n",
    "Now write a function that opens up the text, cleans all of the words and returns a big long list of words:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def cleanText(file_path):\n",
    "    # [ WRITE YOUR CODE HERE ]\n",
    "             "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Verify your function\n",
    "cleanText(filename)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}