Attach to running programs
You can attach to running processes on any machine you can access, whether they are from MPI or scalar jobs, even if they have different executables and source pathnames.
Click Attach to a Running Program on the Welcome page to open the Attach window.
There are two ways to select the processes you want to attach to. You can either choose from a list of automatically detected MPI jobs (for supported MPI implementations), or manually select from a list of processes.
Automatically-detected jobs
Linaro DDT can automatically detect MPI jobs started on the local host for selected MPI implementations. This also applies to other hosts you can access if an Attach Hosts File is configured. See System on Optional configuration for more details.
The list of detected MPI jobs is shown on the Automatically-detected jobs tab of the Attach window. Click the header for a particular job to see more information about that job. When you have found the job you want to attach to, simply click the Attach button to attach to it.
Note
Non-MPI programs that were started using MPI may not appear in this window. For example mpirun -np 2 sleep 1000
If you only want to attach to a subset of ranks from your MPI job, you can choose this subset using Attach to ranks on the Automatically-detected jobs tab. You can change the subset later by selecting . The menu item is only available for jobs that were attached to, and not for jobs that were launched using DDT.
List of all processes
You can manually select which processes to attach to from the list of processes on the List of all processes tab of the Attach window.
If you want to attach to a process on a remote host, see Connecting to compute nodes and remote programs (remote-exec) first.
Initially the list of processes is blank while DDT scans the nodes, provided in your node list file, for running processes. When all the nodes have been scanned (or have timed out) the window appears as shown above. Use Filter for process names containing to find the processes you want to attach to. On non-Linux platforms you also need to select the application executable you want to attach to. Ensure that the list shows all the processes you wish to debug in your job, and no extra/unnecessary processes. You can modify the list by selecting and removing unwanted processes, or alternatively selecting the processes you want to attach to and clicking Attach to selected processes. If no processes are selected, DDT attaches to all processes in the list.
On Linux you can use DDT to attach to multiple processes running different executables. When you select processes with different executables the application box changes to read Multiple applications selected. Linaro DDT creates a process group for each distinct executable.
With some supported MPI implementations (for example, Open MPI) Linaro DDT shows MPI processes as children of the (or equivalent) command.
If you click the command, this automatically selects all the MPI child processes.
When you click Attach to selected/listed processes, Linaro DDT
uses remote-exec to attach a debugger to each process you selected and
proceeds to debug your application as if you had started it with Linaro DDT.
When you end the debug session, Linaro DDT detaches from the processes rather
than terminating them. This means you can attach to them again later if you
want.
Linaro DDT examines the processes it attaches to and tries to discover the
MPI_COMM_WORLD rank of each process. If you have attached to two MPI
programs, or a non-MPI program, you might see this message:
If there is no rank, for example, if you have attached to a non-MPI program, you can ignore this message and use Linaro DDT as normal. If there is a rank, you can easily tell Linaro DDT the correct rank for each process via Use as MPI Rank in the Cross-Process Comparison View. See Assign MPI ranks for details.
Note that stdin, stderr, and stdout (standard input, standard error and
standard output) are not captured by Linaro DDT if used in attaching mode. Any
input/output continues to work as it did before Linaro DDT attached to the
program, for example, from the terminal or perhaps from a file.
Choose hosts
To attach to remote hosts, click Choose Hosts in the Attach window. The Hosts window displays the list of hosts that you can use for attaching.
In the Hosts windows you can add and remove hosts, and uncheck hosts that you want to temporarily exclude.
To import a list of hosts from a file, click Import.
The hosts list populates using the Attach Hosts File. To configure the hosts, click ( on Mac OS X) to open the Options window.
Each remote host is scanned for processes, and the result is displayed in the Attach window. If you have trouble connecting to remote hosts, see Connecting to compute nodes and remote programs (remote-exec).
Use command-line arguments
As an alternative to starting Linaro DDT and using the Welcome page, Linaro DDT can instead be instructed to attach to running processes from the command-line.
To do so, you need to specify a list of hostnames and process identifiers (PIDs). If a hostname is omitted then localhost is assumed.
The list of hostnames and PIDs can be given on the command-line using
the --attach option:
user@holly:~$ ddt --attach=11057,node5:11352
Another command-line possibility is to specify the list of hostnames and
PIDs in a file and use the --attach-file option:
user@holly:~$ cat /home/user/ddt/examples/hello.list
node1:11057
node1:11094
node2:11352
node2:11362
node3:12357
user@holly:~$ ddt --attach-file=/home/user/ddt/examples/hello.list
In both cases, if just a number is specified for a hostname:PID
pair, then localhost: is assumed.
These command-line options work for both single- and multi-process attaching.