Profile a Python script

This task describes how to profile a Python script. This feature is useful when profiling a mixed C, C++, Fortran, and Python program.

About this task

Python profiling replaces main thread stack frames originating from the Python interpreter with Python stack frames of the profiled Python script. To disable this feature, set FORGE_SAMPLER_DISABLE_PYTHON_PROFILING=1.

Linaro MAP supports Python profiling with the following features:

  • Profiles Python scripts running under the CPython interpreter.

  • Profiles Python scripts running under virtual environments.

  • Profiles Python scripts that import modules which perform MPI on the main thread, such as mpi4py.

  • Profiles Python scripts that import modules which use OpenMP.

  • Profiles Python scripts that use the threading module.

Note

Linaro MAP will output warnings if the threading model of the MPI module is MPI_THREAD_MULTIPLE, such as in mpi4py. To prevent these warnings, change the default settings in mpi4py with the following: mpi4py.rc.threaded = False or mpi4py.rc.thread_level = "funneled".

Note

If you are profiling on a system using ALPS or SLURM and the Python script does not use MPI, you can set environment variables (Starting a program) or you can import the mpi4py module.

Procedure

  1. Check that the Python script runs successfully:

    $ python myscript.py

  2. To profile the Python script with Linaro MAP, prepend the run command with map:

    $ map python myscript.py
    Profiling a Python script

  3. Click Run and wait for Linaro MAP to finish profiling the Python script.

  4. View the profiling results in Linaro MAP.

Results

When Linaro MAP finishes profiling the Python script, it saves a .map file in the current working directory and opens it for viewing in the user interface (unless you are using the offline feature).

Example: Profiling a simple Python script

This section demonstrates how to profile the Python example script python-profiling.py located in the examples directory.

  1. Change into the examples directory and run the makefile to compile the example.

    $ make -f python-profiling.makefile

  2. Start Linaro MAP.

    $ ../bin/map python ./python-profiling.py --seconds 15

  3. Click Run.

  4. Wait for Linaro MAP to finish analyzing samples after the Python script has completed.

    Note

    The Linaro MAP user interface launches showing the Python script and the line in the script where the most time was spent is selected.

    Viewing Python profiling results

  5. Locate the first logistic_map_c stack frame in the Main Thread Stacks view. The callout to the C function is appended under main Python stack frame.

Next steps

  • Examine the Main Thread Activity graph (Metrics view) for an overview of time spent in Python code compared with non-Python code.

  • View source code lines (Source code (MAP)) on which time was spent executing Python code and non-Python code.

  • Compare time spent on the selected line executing Python code with non-Python code in the Selected lines view (Selected lines view).

  • View a breakdown of time spent in different code paths in the Main Thread Stacks view (Stacks view).