AstraLog-HPC
Loading...
Searching...
No Matches
JobScheduler.hpp File Reference

SLURM job creation and HPC cluster submission for the AstraLog Collector. More...

#include <memory>
#include <string>
#include <vector>
Include dependency graph for JobScheduler.hpp:
This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes

class  JobWriter
 Fills a SLURM job template ({{TOKEN}} placeholders) and writes it to disk. More...
 
struct  JobWriter::EnvVars
 Env-var values cached at construction time. More...
 
class  ClusterConnection
 SSH/SCP connection to a single remote host via the system ssh/scp. More...
 
struct  ClusterConnection::Secrets
 SSH credentials; never exposed outside this class. More...
 
class  JobScheduler
 Orchestrates job writing, file transfer, and SLURM submission. More...
 

Detailed Description

SLURM job creation and HPC cluster submission for the AstraLog Collector.

Three cooperating classes handle the full lifecycle of a remote Engine job:

  • JobWriter — instantiates a SLURM template from env-var tokens and writes the .slurm script to disk.
  • ClusterConnection — thin wrapper over the system ssh/scp binaries for SCP upload/download and remote command execution. Uses the operator's SSH certificate (agent or file) and multiplexes calls over a single master connection. Knows nothing about env vars or business logic.
  • JobScheduler — orchestrates the two above. On each schedule_job() call it writes the script, uploads all inputs via the data-mover node (CLUSTER_TRANSFER_HOST), and submits via the login node (CLUSTER_SUBMIT_HOST). On retrieve_results() it pulls output files back.

Required environment variables

Variable Owner Description
JOB_TEMPLATE_PATH JobWriter Local path to the .slurm template
REMOTE_WORKDIR JobWriter / JobScheduler Absolute path on cluster (no $SCRATCH expansion on data mover)
REMOTE_OUTPUT_DIR JobWriter / JobScheduler Absolute path where Engine writes valid_data.csv and alarms.log
CONTAINER_SIF_PATH JobWriter Absolute path to container.sif on the cluster
CLUSTER_TRANSFER_HOST JobScheduler Data-mover node, e.g. data.g100.cineca.it
CLUSTER_SUBMIT_HOST JobScheduler Login node, e.g. login.g100.cineca.it
CLUSTER_USER JobScheduler CINECA username
CLUSTER_KEY_PATH *(optional)* JobScheduler Path to SSH private key (public key = same path + .pub); unset for agent-based auth
CLUSTER_KNOWN_HOSTS *(optional)* ClusterConnection known_hosts pinning the load-balanced pool keys; falls back to ~/.ssh/known_hosts
OUTPUT_DIR *(optional)* JobScheduler Local dir for downloaded results; falls back to the log_path directory
MAIL_USER JobWriter Email for SLURM --mail-user