HPE Cray PALS

Arm® Forge supports the HPE Cray parallel application launch service (PALS) on HPE Cray Shasta and HPCM systems. The Arm® Forge startup method for PALS interacts with the HPE Cray common tools interface (CTI) API.

Note

HPE Cray PALS version >= 1.1.6 and CTI version >= 2.15.10 is required.

Note

Ensure that the cray-cti and cray-pals modules are loaded.

Launching a PALS job

Arm® Forge supports the Express Launch (DDT) method for PALS jobs, by prepending the product name to your launch command.

$ ddt mpiexec -n 128 examples/hello

Arm® Forge supports the Compatibility Launch mode for PALS jobs.

$ map --np=256 examples/wave_c 20

Attaching to a PALS job

To attach Arm® DDT to a running PALS job, retrieve the associated PBS job ID by running qstat and specify it when you launch Arm® DDT from the command line.

$ ddt --attach-mpi=1234.login

Arm® Forge also supports attaching to a running job by specifying the mpiexec job launcher PID, and you can additionally list a specific subset of ranks:

$ ddt --attach-mpi=1234 --subset=23,100-112,782,1199

Arm® DDT automatically detects and lists running jobs in the Attach dialog, described in Attach to running programs. Selecting processes to attach to from the list of processes on the List of all processes tab of the Attach window is not supported.

Known issues

  • There is no support for debugging or profiling single-process, non-MPI programs. Convert the program to an MPI program by adding MPI_Init and MPI_Finalize statements, and run it as a single-process MPI job.
  • Jobs might not terminate cleanly when exiting a session due to an issue in CTI. If you are running the Arm® Forge GUI, manually terminate the forge-cti process when the option is available. This is due to an issue in CTI.
  • Arm® MAP and Arm® Performance Reports cannot use the LD_PRELOAD mechanism to link the profiler libraries to your program due to an issue in CTI. Instead, manually link your program to the profiler libraries, as described in Prepare a program for profiling. Set ALLINEA_SKIP_PRELOADS to 1 if you are using Express Launch.