• source and user manual at
• https://github.com/NVIDIA/cuda-checkpoint

• located in n91:/projects/cuda-checkpoint-main/580.65.06
• utility linked in n91:/usr/local/bin/cuda-checkpoint

• do not find a way to download 550 features version
• for our mwgpu256 gpu compute nodes

Put your checkpoints in /sanscratch/checkpoints/$SLURM_JOB_ID directory. You need to make that directory in your submit script but do not have to clean it up. Checkpoints older than 120 days are automatically removed.

[root@n91 ~]# rpm -qa criu
criu-3.18-5.module+el8.10.0+2001+6a33db9f.x86_64

[root@n91 ~]# rpm -ql criu
/run/criu
/usr/lib/.build-id
/usr/lib/.build-id/9b
/usr/lib/.build-id/9b/603c4fff3830838f46feaf4bd1204adc49421e
/usr/lib/tmpfiles.d/criu.conf
/usr/sbin/criu
/usr/share/doc/criu
/usr/share/doc/criu/COPYING
/usr/share/doc/criu/README.md
/usr/share/man/man8/criu.8.gz

An example of performing a checkpoint/restore is
/projects/cuda-checkpoint-main/580.65.06/src/example.sh
the counter does not spit out output so hard to test….

Back