Cours 11 : Performance Python

1. Profiling

1.1 cProfile

cProfile est le profileur standard de Python :

python -m cProfile -o output.prof script.py
python -m pstats output.prof

import cProfile
import pstats

profiler = cProfile.Profile()
profiler.enable()
# Code a profiler
profiler.disable()
stats = pstats.Stats(profiler).sort_stats("cumtime")
stats.print_stats(20)

1.2 py-spy

Profileur pour processus en production sans modification du code :

pip install py-spy
py-spy record -o profile.svg -- python script.py
py-spy top --pid 1234

Avantage : ne necessite pas de modifier le code source, peut profiler des processus en production.

1.3 pyinstrument

from pyinstrument import Profiler

profiler = Profiler()
profiler.start()
# Code a profiler
profiler.stop()
print(profiler.output_text(unicode=True, color=True))

1.4 snakeviz

Visualisation interactive des profils cProfile :

pip install snakeviz
snakeviz output.prof  # Ouvre un graphique dans le navigateur

2. Memoire

2.1 memory_profiler

from memory_profiler import profile

@profile
def my_func():
    data = [i for i in range(100000)]
    return data

python -m memory_profiler script.py

2.2 tracemalloc

import tracemalloc

tracemalloc.start()
snapshot1 = tracemalloc.take_snapshot()
# Code a analyser
snapshot2 = tracemalloc.take_snapshot()
stats = snapshot2.compare_to(snapshot1, "lineno")
for stat in stats[:10]:
    print(stat)

2.3 objgraph

import objgraph

objgraph.show_most_common_types(limit=20)
objgraph.show_growth(limit=10)

3. Cython

3.1 Types

# example.pyx
def compute(int n):
    cdef int i
    cdef double total = 0
    for i in range(n):
        total += i ** 2
    return total

3.2 Compilation

from setuptools import setup
from Cython.Build import cythonize

setup(ext_modules=cythonize("example.pyx"))

4. Numba

4.1 JIT

from numba import jit
import numpy as np

@jit(nopython=True)
def sum_array(arr):
    total = 0
    for i in range(arr.shape[0]):
        total += arr[i]
    return total

4.2 Vectorize

from numba import vectorize

@vectorize(["float64(float64, float64)"])
def add(x, y):
    return x + y

4.3 CUDA

from numba import cuda

@cuda.jit
def kernel(arr):
    i = cuda.grid(1)
    if i < arr.size:
        arr[i] *= 2

5. Rust avec PyO3

use pyo3::prelude::*;

#[pyfunction]
fn fibonacci(n: u64) -> u64 {
    if n <= 1 { return n; }
    let (mut a, mut b) = (0, 1);
    for _ in 2..=n { let c = a + b; a = b; b = c; }
    b
}

#[pymodule]
fn fast_math(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_function(wrap_pyfunction!(fibonacci, m)?)?;
    Ok(())
}

6. Caching

from functools import lru_cache
import redis

@lru_cache(maxsize=128)
def expensive(n):
    return n ** n

# Redis
r = redis.Redis(host="localhost", port=6379)
r.setex("key", 3600, "value")
val = r.get("key")

7. GIL et Parallelisme

from multiprocessing import Pool

def cpu_bound(n):
    return sum(i * i for i in range(n))

with Pool(4) as p:
    results = p.map(cpu_bound, [1000000] * 4)

Diagramme en cours de génération...

8. Bonnes pratiques

1. Toujours profiler avant d'optimiser
2. Ameliorer d'abord les algorithmes (O(n) vs O(n2))
3. Utiliser les structures de donnees adaptees
4. Paralleliser intelligemment (multiprocessing pour CPU, asyncio pour IO)
5. Cachez les resultats couteux
6. Ecrire du Python pur d'abord, puis optimiser les hotspots