GCP-2026-062: Testing Slurm Exposure in Cluster Toolkit Blueprints

TL;DR

Google's GCP-2026-062 bulletin, published 2026-09-11, lists eight High-severity Slurm CVEs — CVE-2026-65107, CVE-2026-65108, CVE-2026-65109, CVE-2026-65138, CVE-2026-65139, CVE-2026-65140, CVE-2026-65165, and CVE-2026-65168 — affecting Cluster Toolkit blueprints that reference specific image versions, spanning the slurmstepd daemon, RPC request handling, and the accounting database. A blueprint pinned to an old image is a lead, not a finding; the boot disk of each running controller, login node, and compute node is the ground truth. Test it by resolving what image each instance actually runs, then checking whether an ordinary Slurm user can reach the affected component. Impact is proven when the request and its effect appear in the same evidence package.

What this is

Cluster Toolkit is Google's Terraform-based toolkit for deploying HPC environments on GCP. Its Slurm blueprints create a controller, one or more login nodes, and one or more nodesets, and each of those modules references a boot image — by project, by image family, or by an explicit image name.

The vulnerability lives in the image, not in the Terraform. That reframes the question. "Are we affected by GCP-2026-062" is not a question about your repository; it is a question about eight CVEs, an inventory of running images, and which of three components each CVE touches. The deliverable is a mapping: instance → image version → affected component → who can reach it. Anything short of that is a version-guess dressed up as a finding.

Why it matters

Slurm is a multi-tenant scheduler. Unprivileged users are supposed to run arbitrary code on shared compute nodes — that is the product working as designed. slurmstepd is the per-node daemon that holds the boundary between one job and the next, so defects there sit on a host where untrusted code already executes and where the blast radius is every job and every tenant on that node. The accounting database is the audit record of who ran what, and it is exactly the artifact you need after an incident.

The second reason is propagation. Blueprints get forked and copied. An image pin carried into a research cluster two years ago keeps getting carried into new clusters, and nothing in the deployment pipeline tells you that the pin has crossed a security bulletin. The login node compounds it: it is the part of an HPC cluster most likely to hold an external address, and it shares a control-plane network with the controller and slurmd RPC endpoints.

How it works

The bulletin attributes the issues to three areas: the slurmstepd daemon (the per-node job step supervisor), RPC request handling (the controller/slurmd control channel), and the accounting database (slurmdbd's store). Affected image versions are enumerated in the Cluster Toolkit security bulletin referenced by GCP-2026-062 — read that page before you claim anything. The remediation path matters as much as the version list: these are fixed by moving an image pin to a fixed image, not by upgrading packages inside a running node.

Step 1 — resolve ground truth per instance. Not the blueprint; the instance. On the controller, login, and each nodeset member:

gcloud compute instances describe NODE --zone=ZONE \
  --format='value(disks[0].source.basename())'

gcloud compute images describe IMAGE \
  --format='value(name,family,creationTimestamp)'

If the nodes are managed by a regional or zonal MIG, the instance inherits its image from the instance template — describe the template's source image, not just one sample VM, or you will miss a nodeset that still points at the old pin.

Step 2 — interpret the pin against the bulletin. Three outcomes. Explicitly pinned to an affected image: that is your candidate. Family-based (source_image_family): the node moves with the family, so compare the family's current image creation timestamp against the bulletin's fix date instead of assuming either way. Custom image built from an affected base: the hardest case, because no bulletin will name your image — you have to map the base image and the Slurm build inside it (slurmd -V, plus the package version).

Step 3 — do not stop at the version. Reachability decides whether the lead becomes a finding. Use two accounts: A, an ordinary Slurm user with a job allocation; B, a cluster operator. Under A, exercise the affected component minimally — submit a job step, issue the RPC path, read the accounting record — and capture both sides: on the node, ps -o pid,ppid,user,cmd -C slurmstepd plus the timestamped slurmd log lines; on the controller, the matching sacct rows for that job ID. Then look at B. If the operator's view shows nothing that distinguishes A's request from routine scheduler traffic, you have two findings: the exposure, and the detection gap.

Keep the proof minimal on a production cluster. You do not need to complete an exploit chain to demonstrate that an unprivileged account reaches the affected code path and that the daemon acted on the request — a captured log line and a matching accounting record are checkable by a second person.

Detection and response

• Daemon logs. SlurmdLogFile and SlurmctldLogFile on every node, not just the controller. In a MIG-scaled nodeset, ephemeral nodes take their logs with them when they are replaced — ship them off-node or you will have no evidence after remediation.
• Authentication noise. Munge failures and unexpected slurm user activity around the daemon; correlate slurmstepd children with the job IDs that spawned them.
• Accounting integrity. Compare sacct job records against what actually ran on the node. A step that executed without a corresponding accounting entry is the signal worth escalating.
• Network reachability. Which hosts can reach the controller RPC and slurmdbd ports. If an ordinary compute node or login node can talk to slurmdbd directly, the accounting surface is wider than the design intended.
• Inventory drift. A scheduled comparison of instance and template boot images against the current bulletin has caught more real exposure than any one-off scan.

What to do

1. Pull the Cluster Toolkit security bulletin referenced by GCP-2026-062 and write down the fixed image versions. Do not rely on the CVE descriptions alone.
2. Enumerate every Slurm deployment: controllers, login nodes, nodesets, instance templates, and any blueprint fork outside the main repo.
3. Resolve the actual image per instance and per template; record name, family, and creation timestamp.
4. Classify each against the bulletin: affected, family-tracked, or custom build.
5. Map reachability: who can submit a job, who can reach the RPC channel, who can reach slurmdbd.
6. Run the two-account test on one affected cluster and keep the evidence package: job ID, submitted command, node-side daemon log excerpt, controller-side accounting row, operator view.
7. Remediate by image pin, not by in-place package upgrades; treat blueprint-deployed nodes as disposable and replace them.
8. Retest after the pin change with the same evidence package so the closure is verifiable, not asserted.

FAQ

Does GCP-2026-062 mean my cluster is compromised? No. It means the images your blueprints reference may contain the flaws. Exploitation requires an actor who can reach the affected component — which, for the slurmstepd issues, means anyone holding a job allocation on the cluster.

Should I upgrade Slurm inside the running node instead of changing the image? No. In-place upgrades drift from the blueprint and are lost the next time the node is replaced. Move the pin to a fixed image and let the deployment recreate the hosts.

The bulletin lists image versions, but we build our own images. How do we know? You have to answer it yourself: identify the base image, then the Slurm build inside it. A custom image based on an affected source will never appear in a vendor advisory.

Do I need to test all eight CVEs? No. Test by trust boundary, not by CVE count. Start with the daemon issues reachable by any user with a job allocation, then the RPC surface reachable from hosts inside the cluster network, then the accounting database.

Is the login node in scope? Yes. It typically holds the external address, it shares the cluster network with the controller, and it is where unprivileged users land first.

Where this fits in a program

This is an inventory problem before it is a testing problem. An asset and technology inventory that tracks which image each node boots, plus a validation gate that separates a scanner's version match from an exploitable, reachable finding, is what turns a bulletin into a queue of hosts someone can actually check. Trusteed's CTEM workspace is built around that order: surface first, investigation second, evidence last.

Sources

• Google Cloud security bulletin GCP-2026-062 (primary): https://docs.cloud.google.com/support/bulletins/index#gcp-2026-062
• NVD entry for the lead CVE: https://nvd.nist.gov/vuln/detail/CVE-2026-65107
• Cluster Toolkit source and blueprint modules: https://github.com/GoogleCloudPlatform/cluster-toolkit
• Slurm slurmd daemon documentation: https://slurm.schedmd.com/slurmd.html
• Slurm configuration reference (SlurmdLogFile, accounting): https://slurm.schedmd.com/slurm.conf.html
• Trusteed CTEM: https://trusteed.io