User Tools

Site Tools


cluster:234

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
cluster:234 [2026/10/08 20:25] – hmeij07cluster:234 [2026/10/09 13:09] (current) – hmeij07
Line 6: Line 6:
   * source and user manual at   * source and user manual at
   * https://github.com/NVIDIA/cuda-checkpoint   * https://github.com/NVIDIA/cuda-checkpoint
 +  * https://developer.nvidia.com/blog/checkpointing-cuda-applications-with-criu/
  
   * located in n91:/projects/cuda-checkpoint-main/580.65.06   * located in n91:/projects/cuda-checkpoint-main/580.65.06
Line 58: Line 59:
 You use cuda-checkpoint to freeze and extract  You use cuda-checkpoint to freeze and extract 
 the GPU memory layout into host memory space the GPU memory layout into host memory space
 +...this terminates the counter on GPU
  
 cuda-checkpoint --toggle --pid $PID cuda-checkpoint --toggle --pid $PID
Line 64: Line 66:
 You use CRIU to write the combined state out  You use CRIU to write the combined state out 
 into your output directory into your output directory
 +...this terminates the counter on CPU
  
-criu dump --shell-job --images-dir demo --tree $PID+criu dump --shell-job --images-dir demo --tree $PID --unprivileged
  
 Once completed, the target counter process is terminated,  Once completed, the target counter process is terminated, 
Line 71: Line 74:
 will be packed with the binary image (.img) files representing  will be packed with the binary image (.img) files representing 
 the exact frozen state of your application. the exact frozen state of your application.
- 
- 
- 
-[hmeij@n91 src]$ criu dump --shell-job --images-dir demo --tree $PID 
-CRIU needs to have the CAP_SYS_ADMIN or the CAP_CHECKPOINT_RESTORE capability: 
  
 # as root set # as root set
Line 122: Line 120:
  
 [hmeij@n91 src]$ criu restore --shell-job --restore-detached --images-dir demo --unprivileged [hmeij@n91 src]$ criu restore --shell-job --restore-detached --images-dir demo --unprivileged
- +2262128: Error (criu/tty.c:831): tty: Can't set tty params on 0xf: Operation not permitted 
-Warn  (criu/kerndat.c:1103): $XDG_RUNTIME_DIR not set. Cannot find location for kerndat file +2262128: Error (criu/files.c:1213): Unable to open fd=0 id=0xf
-Warn  (criu/kerndat.c:1421): Can't get pidfd +
-Warn  (criu/kerndat.c:1103): $XDG_RUNTIME_DIR not set. Cannot find location for kerndat file +
-1583155: Error (criu/tty.c:872): tty: Can't setup uid 0 gid 5 on 0xf: Operation not permitted +
-1583155: Error (criu/files.c:1213): Unable to open fd=0 id=0xf+
 Error (criu/cr-restore.c:2547): Restoring FAILED. Error (criu/cr-restore.c:2547): Restoring FAILED.
-Error (criu/cr-restore.c:1514): 1583155 killed by signal 9: Killed+
  
 BUMMER! BUMMER!
cluster/234.1791491114.txt.gz · Last modified: by hmeij07