Summary and Schedule
Welcome to Python Optimisation and Performance Profiling Training!
The training curriculum for this course is designed for researchers that are writing Python and lack formal computer science training. The curriculum covers how to assess where time is being spent during execution of a Python program, it also provides a high level understanding of how code executes and how this maps to the limiting factors of performance and good practice.
If you are now comfortable using Python, this course may be of interest to supplement and advance your programming knowledge. This course is particularly relevant if you are writing from scratch or re-using and existing research code and would desire a greater confidence that your code is both performant and suitable for publication. This is an all-day course, however it normally finishes by early afternoon.
If you would like to register to take the course, check the registration information.
Learning Objectives
After attending this training, participants will be able to:
- recognise and implement optimisations for common limiting factors of performance.
- identify the most expensive functions and lines of code using
cprofileandline_profiler. - evaluate code to determine the limiting factors of its performance.
Prerequisites
Before joining Python Optimisation and Performance Profiling Training, participants should be able to:
- implement basic algorithms in Python.
- follow the control flow of Python code, and dry run the execution in their head or on paper.
See the Python novice carpentry for another course to help with learning these skills.
| Setup Instructions | Download files required for the lesson | |
| Duration: 00h 00m | 1. Introduction to Optimisation | Why could optimisation of code be harmful? |
| Duration: 00h 10m | 2. Data Structures & Algorithms |
What’s the most efficient way to construct a list? When should tuples be used? When are sets appropriate? What is the best way to search a list? |
| Duration: 00h 45m | 3. Understanding Python (NumPy/Pandas) |
Why are Python loops slow? Why is NumPy often faster than raw Python? How can processing rows of a Pandas data table be made faster? |
| Duration: 01h 15m | 4. Keep Python & Packages up to Date |
Why would a newer version of Python or a package be faster? Are there any risks to updating Python and packages? How can reproducibility be ensured through package upgrades? |
| Duration: 01h 25m | 5. Understanding Memory |
How does a CPU look for a variable it requires? What impact do cache lines have on memory accesses? Why is it faster to read/write a single 100mb file, than 100 1mb files? |
| Duration: 01h 55m | 6. Optimisation Conclusion | What has been learnt about writing performant Python? |
| Duration: 02h 00m | 7. Lunch Break | |
| Duration: 03h 00m | 8. Introduction to Profiling |
Why should you profile your code? How should you choose which type of profiler to use? Which test case should be profiled? |
| Duration: 03h 25m | 9. Function Level Profiling |
When is function level profiling appropriate? How can cProfile and snakeviz be used to profile a
Python program?How are the outputs from function level profiling interpreted? |
| Duration: 04h 05m | 10. Line Level Profiling |
When is line level profiling appropriate? What adjustments are required to Python code to profile with line_profiler?How can kernprof be used to
profile a Python program?
|
| Duration: 04h 55m | 11. Profiling Conclusion | What has been learnt about profiling? |
| Duration: 05h 00m | Finish |
The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor.
Software Setup
Details
This course uses Python and was developed using Python 3.11, therefore it is recommended that you have a Python 3.11 or newer environment.
You may want to create a new Python virtual environment for the
course, this can be done with your preferred Python environment manager
(e.g. conda, pipenv), the required packages
can all be installed via pip.
To create a new Anaconda environment named py311_env
with Python 3.11, use the following command for conda:
The non-core Python packages required by the course are
pytest, snakeviz, line_profiler,
numpy, pandas and matplotlib
which can be installed via pip.
To complete some of the exercises you will need to use a text-editor or Python IDE, so make sure you have your favourite available.