"> *This is extra `Python` material, to introduce the concepts of more complex list processing. This will be covered in greater detail next year, but it is useful to for you to work through this if you have completed the checkpoints.*\n",
"\n",
"The real power of `Python` becomes evident when we start performing operations on whole `lists` rather than variables. We will consider some simple example here; you will come back to this in detail in future year’s courses.\n",
"\n",
"\n",
"## The `map` Function\n",
"\n",
"Consider a problem were you have a `list` of angle values and you want to form another `list` being the `sin()` of each. Clearly you could use a `Loop` to do this, but this is exactly what the `map()` function is for. Looking at the code:\n",
"\n",
"```python\n",
"..... \n",
"thetadata = [0.0,0.1,0.2,0.3,.....] # List of floats \n",
"sindata = map(math.sin, thetadata) \n",
".....\n",
"```\n",
"if we have a list of angle values in list `thetadata` then `map()` forms a new list with the supplied function (in this case `math.sin()`) applied to each element of `thetadata`.\n",
"\n",
"The real power of this comes when we combine it with the `Lambda Function` which allows you to create fast and efficent *one-line* functions, so if we take the same problem as above, except that the values in `thetadata` are in *degrees*, then we can write\n",
"Here the `lambda` function is applied to each element, so it\n",
"\n",
"- converts degree $\\rightarrow$ radius with `math.radians()`\n",
"\n",
"- calculates the sin of the angle with `math.sin()`\n",
"\n",
"where the `map()` function applies this to *each element* of the supplied `list`.\n",
"\n",
"So `map()` functions start to allow us to process *whole lists* with a single statement without the need to consider the details of `loops`. This means our programs are shorter, more efficient and, as a consquence, more *likely* to be correct! (provided they can be put in this form).\n",
"\n",
"> ### Key Point\n",
"In addition the use of these high level functions mirrors the\n",
"concepts used in the efficient numerical `Python` libraries and also the vector stucture of the high performance parallel systems, see for example\n",
"- Numpy\n",
"- Scipy\n",
"\n",
"> These are in practice what is used in scientific `Python` coding and you will see these in future courses.\n",
"\n",
"> Both of these modules along with `Matplotlib` are included in all (useful) `Python` kits.\n",
"\n",
"\n",
"## The `filter` function\n",
"\n",
"The `filter()` function now adds `Conditionals` processing of lists. It applies a conditional test to each element of a list and returns the elements where the conditional test is `True`. Looking at the example below,\n",
"\n",
"```python\n",
"..... \n",
"vals = [-3,2,-4,6,8,10] \n",
"posvals = filter(lambda x : x \\> 0, vals) \n",
".....\n",
"```\n",
"\n",
"we have a list `vals` that contains both positive and negative numbers; the `filter()` function will apply the `lambda` function to each element and return the *positive* ones in the new list, so $posvals=[2,6,8,10]$\n",
"\n",
"> ### Functional Programming \n",
"More technically this style of programming is known as *functional programming* which is part of a higher level programming methodology.\n",
"> Scientific `Python` users will not typically need to know the formal\n",
"computational theory behind these methods, but will need to understand\n",
"these calls and what they do.\n",
"\n",
"\n",
"## The `zip` function\n",
"\n",
"The `map()` function above allows us to process *one list at a time* which is rather limited, so to overcome this there is the associated `zip()` function. To see this at work, consider we have two `list`s, being `realdata` and `imagdata`, that hold the real and imaginary parts of a complex signal. They can be *zipped* together with\n",
"\n",
"```python\n",
"...... \n",
"realdata = [0,1,2,3,.....] \n",
"imagdata = [-3,-2,0,1,2,3...] \n",
"comb = zip(realdata, imagdata) \n",
"......\n",
"```\n",
"which will produces a combined list `comb` with elements `[[0,-3], [ 1,-2], [2,0], [1,1]…]`, i.e. each element of `comb` will be a `list` which will contain the corresponding *real* and *imaginary*\n",
"parts from `realdata` and `imagdata` which have been *zipped* together.\n",
"\n",
"Now we combine this with the `map()` function above to process the output of `zip()`, for example using\n",
"- the `zip()` combines the two lists to forms `[re,im]` pairs,\n",
"\n",
"- the `complex()` function combines the `[re,im]` pairs into a `complex` number,\n",
"\n",
"- the `map()` applies `complex()` to all the elements that are output by `zip()`\n",
"\n",
"so this *one-line* piece of code will take the two lists of real and\n",
"imaginary parts and combine them into a single list of `complex` numbers.\n",
"\n",
"Clearly you can then replace the `complex` function with a more complicated `lambda` function that can perform more complex processing.\n",
"\n",
"> ### Where next \n",
"This is just the start of *functional programming* and is just a little taster of what you can start to do with `Python`.\n",
"\n",
"> This is the way that `Python` is typically used *in anger* in a scientific context. Processing individual `float`s in `Python` is computationally slow, but if we can arrange the code so that `Python` only handles `lists` the high level and the low\n",
"level processing is done in function calls that are implemented in a\n",
"*fast* language, then we can get the *best of both worlds*, the\n",
"convenient, user friendly `Python` syntax, but still retain the performance of low level languages and parallel systems.\n",
"\n",
"> More of this in future courses…\n",
"\n",
"\n",
"## List Comprehension\n",
"\n",
"Another unique syntax in `Python` is called *list comprehension*, which is a fast and efficient route to building lists which at first sight looks confusing. Consider the problem of wanting to form a list of squares, so `[0,1,4,9,16,…]` 30 elements long. The standard method with loops is:\n",
"\n",
"```python\n",
"vals = [] \n",
"for x in range(0,30): \n",
" vals.append(x**2)\n",
"```\n",
"\n",
"where the single line, *list comprehension* route is:\n",
"\n",
"```python\n",
"vals = [x**2 for x in range(0,30)]\n",
"```\n",
"which will go round a `for` loop the required 30 times for values of `x` in the $0\\rightarrow29$ range , set the elements of the list to $x**2$ and return a `lsit` called `vals`.\n",
"\n",
"If you combine this syntax with the `map()`, `filter()` and `reduce()` functions you have scope to produce very efficient, but terse and possibly *cryptic* code, so beware.\n",
"\n",
"> ### The web \n",
"If you search of the web for `Python` code it will very often have been\n",
"written using list comprehension and/or using the functional\n",
"programming methods which can make it hard to follow. Advice for novices\n",
"is that they should start off with a long-winded (and obvious) route\n",
"before you try and be too cryptic; also if you do use these techniques\n",
"make sure you document your code with *lots of comments* so *you*\n",
"remember what you did !"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.10"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
%% Cell type:markdown id: tags:
# Advanced List Processing
> *This is extra `Python` material, to introduce the concepts of more complex list processing. This will be covered in greater detail next year, but it is useful to for you to work through this if you have completed the checkpoints.*
The real power of `Python` becomes evident when we start performing operations on whole `lists` rather than variables. We will consider some simple example here; you will come back to this in detail in future year’s courses.
## The `map` Function
Consider a problem were you have a `list` of angle values and you want to form another `list` being the `sin()` of each. Clearly you could use a `Loop` to do this, but this is exactly what the `map()` function is for. Looking at the code:
```python
.....
thetadata=[0.0,0.1,0.2,0.3,.....]# List of floats
sindata=map(math.sin,thetadata)
.....
```
if we have a list of angle values in list `thetadata` then `map()` forms a new list with the supplied function (in this case `math.sin()`) applied to each element of `thetadata`.
The real power of this comes when we combine it with the `Lambda Function` which allows you to create fast and efficent *one-line* functions, so if we take the same problem as above, except that the values in `thetadata` are in *degrees*, then we can write
Here the `lambda` function is applied to each element, so it
- converts degree $\rightarrow$ radius with `math.radians()`
- calculates the sin of the angle with `math.sin()`
where the `map()` function applies this to *each element* of the supplied `list`.
So `map()` functions start to allow us to process *whole lists* with a single statement without the need to consider the details of `loops`. This means our programs are shorter, more efficient and, as a consquence, more *likely* to be correct! (provided they can be put in this form).
> ### Key Point
In addition the use of these high level functions mirrors the
concepts used in the efficient numerical `Python` libraries and also the vector stucture of the high performance parallel systems, see for example
- Numpy
- Scipy
> These are in practice what is used in scientific `Python` coding and you will see these in future courses.
> Both of these modules along with `Matplotlib` are included in all (useful) `Python` kits.
## The `filter` function
The `filter()` function now adds `Conditionals` processing of lists. It applies a conditional test to each element of a list and returns the elements where the conditional test is `True`. Looking at the example below,
```python
.....
vals=[-3,2,-4,6,8,10]
posvals=filter(lambdax:x \>0,vals)
.....
```
we have a list `vals` that contains both positive and negative numbers; the `filter()` function will apply the `lambda` function to each element and return the *positive* ones in the new list, so $posvals=[2,6,8,10]$
> ### Functional Programming
More technically this style of programming is known as *functional programming* which is part of a higher level programming methodology.
> Scientific `Python` users will not typically need to know the formal
computational theory behind these methods, but will need to understand
these calls and what they do.
## The `zip` function
The `map()` function above allows us to process *one list at a time* which is rather limited, so to overcome this there is the associated `zip()` function. To see this at work, consider we have two `list`s, being `realdata` and `imagdata`, that hold the real and imaginary parts of a complex signal. They can be *zipped* together with
```python
......
realdata=[0,1,2,3,.....]
imagdata=[-3,-2,0,1,2,3...]
comb=zip(realdata,imagdata)
......
```
which will produces a combined list `comb` with elements `[[0,-3], [ 1,-2], [2,0], [1,1]…]`, i.e. each element of `comb` will be a `list` which will contain the corresponding *real* and *imaginary*
parts from `realdata` and `imagdata` which have been *zipped* together.
Now we combine this with the `map()` function above to process the output of `zip()`, for example using
```python
.....
realdata=[0,1,2,3,.....]
imagdata=[-3,-2,0,1,2,3...]
complexdata=map(complex,zip(realdata,imagdata))
......
```
- the `zip()` combines the two lists to forms `[re,im]` pairs,
- the `complex()` function combines the `[re,im]` pairs into a `complex` number,
- the `map()` applies `complex()` to all the elements that are output by `zip()`
so this *one-line* piece of code will take the two lists of real and
imaginary parts and combine them into a single list of `complex` numbers.
Clearly you can then replace the `complex` function with a more complicated `lambda` function that can perform more complex processing.
> ### Where next
This is just the start of *functional programming* and is just a little taster of what you can start to do with `Python`.
> This is the way that `Python` is typically used *in anger* in a scientific context. Processing individual `float`s in `Python` is computationally slow, but if we can arrange the code so that `Python` only handles `lists` the high level and the low
level processing is done in function calls that are implemented in a
*fast* language, then we can get the *best of both worlds*, the
convenient, user friendly `Python` syntax, but still retain the performance of low level languages and parallel systems.
> More of this in future courses…
## List Comprehension
Another unique syntax in `Python` is called *list comprehension*, which is a fast and efficient route to building lists which at first sight looks confusing. Consider the problem of wanting to form a list of squares, so `[0,1,4,9,16,…]` 30 elements long. The standard method with loops is:
```python
vals=[]
forxinrange(0,30):
vals.append(x**2)
```
where the single line, *list comprehension* route is:
```python
vals=[x**2forxinrange(0,30)]
```
which will go round a `for` loop the required 30 times for values of `x` in the $0\rightarrow29$ range , set the elements of the list to $x**2$ and return a `lsit` called `vals`.
If you combine this syntax with the `map()`, `filter()` and `reduce()` functions you have scope to produce very efficient, but terse and possibly *cryptic* code, so beware.
> ### The web
If you search of the web for `Python` code it will very often have been
written using list comprehension and/or using the functional
programming methods which can make it hard to follow. Advice for novices
is that they should start off with a long-winded (and obvious) route
before you try and be too cryptic; also if you do use these techniques
make sure you document your code with *lots of comments* so *you*
