AstraLog-HPC
Loading...
Searching...
No Matches
JobScheduler Class Reference

Orchestrates job writing, file transfer, and SLURM submission. More...

#include <JobScheduler.hpp>

Collaboration diagram for JobScheduler:
Collaboration graph

Public Member Functions

 JobScheduler ()=default
 
 ~JobScheduler ()=default
 
bool preflight ()
 Verifies that the cluster is reachable and the container image is in place.
 
bool schedule_job (const std::string &batch_path, int num_proc)
 Writes a SLURM script, uploads all inputs, and submits the job.
 
bool retrieve_results ()
 Downloads valid_data.csv and alarms.log from the cluster.
 
void set_paths (const std::string &sensors_path, const std::string &rules_path, const std::string &log_path)
 Sets the local file paths used by subsequent schedule_job() calls.
 

Private Attributes

std::string log_path
 
std::string sensors_path
 
std::string rules_path
 
std::unique_ptr< JobWriter > job_writer
 Constructed in set_paths() once all paths are known.
 
std::unique_ptr< ClusterConnection > submit_conn_
 
std::string last_job_id_
 

Detailed Description

Orchestrates job writing, file transfer, and SLURM submission.

Typical call sequence from Collector:

js.set_paths(sensors, rules, log);
// called once per collected batch:
js.schedule_job(batch_path);
// called once when collection stops:
Orchestrates job writing, file transfer, and SLURM submission.
Definition JobScheduler.hpp:244
bool retrieve_results()
Downloads valid_data.csv and alarms.log from the cluster.
Definition JobScheduler.cpp:591
bool schedule_job(const std::string &batch_path, int num_proc)
Writes a SLURM script, uploads all inputs, and submits the job.
Definition JobScheduler.cpp:485
void set_paths(const std::string &sensors_path, const std::string &rules_path, const std::string &log_path)
Sets the local file paths used by subsequent schedule_job() calls.
Definition JobScheduler.cpp:477

schedule_job() uploads inputs over a per-call connection to the data-mover node (CLUSTER_TRANSFER_HOST, which can't multiplex) and submits over a persistent SSH master to the login node (CLUSTER_SUBMIT_HOST) that is opened on the first call and reused thereafter, so the handshake is paid once.

Constructor & Destructor Documentation

◆ JobScheduler()

JobScheduler::JobScheduler ( )
default

◆ ~JobScheduler()

JobScheduler::~JobScheduler ( )
default

Member Function Documentation

◆ preflight()

bool JobScheduler::preflight ( )

Verifies that the cluster is reachable and the container image is in place.

Connects to CLUSTER_SUBMIT_HOST with the configured credentials and checks that CONTAINER_SIF_PATH exists on the cluster filesystem. Must be called before the Collector starts accepting packets so that delivery failures are detected immediately rather than after batches have accumulated.

Returns
true if the login node is reachable and container.sif is present; false with a diagnostic on std::clog otherwise.
Here is the call graph for this function:

◆ retrieve_results()

bool JobScheduler::retrieve_results ( )

Downloads valid_data.csv and alarms.log from the cluster.

Uses CLUSTER_TRANSFER_HOST (data-mover node) for the SCP download. Files are written into OUTPUT_DIR when that env var is set (created if missing), otherwise into the same directory as log_path.

The per-job SLURM logs (slurm_*.out / slurm_*.err) are additionally pulled into a debug/ subfolder of that directory on a best-effort basis, so engine failures (written to slurm_j.err) are visible locally. A failure to fetch them does not affect the return value.

Returns
true if both result files were retrieved successfully.
Note
Call set_paths() before this method; returns false otherwise.
Here is the call graph for this function:

◆ schedule_job()

bool JobScheduler::schedule_job ( const std::string &  batch_path,
int  num_proc 
)

Writes a SLURM script, uploads all inputs, and submits the job.

Steps:

  1. JobWriter::write_job() — instantiates the template to job_<id>.slurm.
  2. SCP upload of batch, sensors, rules, and the script to CLUSTER_TRANSFER_HOST.
  3. sbatch job_<id>.slurm executed on CLUSTER_SUBMIT_HOST.
  4. The sbatch output is appended to the log file (if log_path is set).
Parameters
batch_pathLocal path to the batch input file to submit.
num_procNumber of MPI tasks to request; forwarded to JobWriter::write_job to fill the script's {{NTASKS}} token.
Returns
true if sbatch reported "Submitted batch job"; false if any env var is missing, a transfer fails, the SSH connection can't be established, or sbatch did not confirm submission. On any failure the staged batch file is restored to its original name.
Here is the call graph for this function:

◆ set_paths()

void JobScheduler::set_paths ( const std::string &  sensors_path,
const std::string &  rules_path,
const std::string &  log_path 
)

Sets the local file paths used by subsequent schedule_job() calls.

Also constructs the internal JobWriter. Must be called before schedule_job() or retrieve_results().

Parameters
sensors_pathLocal path to sensors.yaml.
rules_pathLocal path to rules.json.
log_pathLocal path for the job submission log (append mode). Its parent directory is used as the download destination in retrieve_results().

Member Data Documentation

◆ job_writer

std::unique_ptr<JobWriter> JobScheduler::job_writer
private

Constructed in set_paths() once all paths are known.

◆ last_job_id_

std::string JobScheduler::last_job_id_
private

SLURM id of the previously submitted job in this run, parsed from the sbatch output. The next schedule_job() submits with --dependency=afterany:<last_job_id_> so jobs run one at a time in submission order (they share output/state files). Empty until the first successful submission, so the first job of a run has no dependency.

◆ log_path

std::string JobScheduler::log_path
private

◆ rules_path

std::string JobScheduler::rules_path
private

◆ sensors_path

std::string JobScheduler::sensors_path
private

◆ submit_conn_

std::unique_ptr<ClusterConnection> JobScheduler::submit_conn_
private

Persistent SSH master to the login node, reused across schedule_job() calls so the cold handshake is paid only on the first submission. Opened lazily on first use; closed when this JobScheduler is destroyed. The data-mover connection stays per-call (it can't multiplex, so nothing to keep alive).


The documentation for this class was generated from the following files: