Orchestrates job writing, file transfer, and SLURM submission.
More...
#include <JobScheduler.hpp>
Orchestrates job writing, file transfer, and SLURM submission.
Typical call sequence from Collector:
Orchestrates job writing, file transfer, and SLURM submission.
Definition JobScheduler.hpp:244
bool retrieve_results()
Downloads valid_data.csv and alarms.log from the cluster.
Definition JobScheduler.cpp:591
bool schedule_job(const std::string &batch_path, int num_proc)
Writes a SLURM script, uploads all inputs, and submits the job.
Definition JobScheduler.cpp:485
void set_paths(const std::string &sensors_path, const std::string &rules_path, const std::string &log_path)
Sets the local file paths used by subsequent schedule_job() calls.
Definition JobScheduler.cpp:477
schedule_job() uploads inputs over a per-call connection to the data-mover node (CLUSTER_TRANSFER_HOST, which can't multiplex) and submits over a persistent SSH master to the login node (CLUSTER_SUBMIT_HOST) that is opened on the first call and reused thereafter, so the handshake is paid once.
◆ JobScheduler()
| JobScheduler::JobScheduler |
( |
| ) |
|
|
default |
◆ ~JobScheduler()
| JobScheduler::~JobScheduler |
( |
| ) |
|
|
default |
◆ preflight()
| bool JobScheduler::preflight |
( |
| ) |
|
Verifies that the cluster is reachable and the container image is in place.
Connects to CLUSTER_SUBMIT_HOST with the configured credentials and checks that CONTAINER_SIF_PATH exists on the cluster filesystem. Must be called before the Collector starts accepting packets so that delivery failures are detected immediately rather than after batches have accumulated.
- Returns
true if the login node is reachable and container.sif is present; false with a diagnostic on std::clog otherwise.
◆ retrieve_results()
| bool JobScheduler::retrieve_results |
( |
| ) |
|
Downloads valid_data.csv and alarms.log from the cluster.
Uses CLUSTER_TRANSFER_HOST (data-mover node) for the SCP download. Files are written into OUTPUT_DIR when that env var is set (created if missing), otherwise into the same directory as log_path.
The per-job SLURM logs (slurm_*.out / slurm_*.err) are additionally pulled into a debug/ subfolder of that directory on a best-effort basis, so engine failures (written to slurm_j.err) are visible locally. A failure to fetch them does not affect the return value.
- Returns
true if both result files were retrieved successfully.
- Note
- Call
set_paths() before this method; returns false otherwise.
◆ schedule_job()
| bool JobScheduler::schedule_job |
( |
const std::string & |
batch_path, |
|
|
int |
num_proc |
|
) |
| |
Writes a SLURM script, uploads all inputs, and submits the job.
Steps:
JobWriter::write_job() — instantiates the template to job_<id>.slurm.
- SCP upload of batch, sensors, rules, and the script to
CLUSTER_TRANSFER_HOST.
sbatch job_<id>.slurm executed on CLUSTER_SUBMIT_HOST.
- The
sbatch output is appended to the log file (if log_path is set).
- Parameters
-
| batch_path | Local path to the batch input file to submit. |
| num_proc | Number of MPI tasks to request; forwarded to JobWriter::write_job to fill the script's {{NTASKS}} token. |
- Returns
true if sbatch reported "Submitted batch job"; false if any env var is missing, a transfer fails, the SSH connection can't be established, or sbatch did not confirm submission. On any failure the staged batch file is restored to its original name.
◆ set_paths()
| void JobScheduler::set_paths |
( |
const std::string & |
sensors_path, |
|
|
const std::string & |
rules_path, |
|
|
const std::string & |
log_path |
|
) |
| |
Sets the local file paths used by subsequent schedule_job() calls.
Also constructs the internal JobWriter. Must be called before schedule_job() or retrieve_results().
- Parameters
-
| sensors_path | Local path to sensors.yaml. |
| rules_path | Local path to rules.json. |
| log_path | Local path for the job submission log (append mode). Its parent directory is used as the download destination in retrieve_results(). |
◆ job_writer
| std::unique_ptr<JobWriter> JobScheduler::job_writer |
|
private |
◆ last_job_id_
| std::string JobScheduler::last_job_id_ |
|
private |
SLURM id of the previously submitted job in this run, parsed from the sbatch output. The next schedule_job() submits with --dependency=afterany:<last_job_id_> so jobs run one at a time in submission order (they share output/state files). Empty until the first successful submission, so the first job of a run has no dependency.
◆ log_path
| std::string JobScheduler::log_path |
|
private |
◆ rules_path
| std::string JobScheduler::rules_path |
|
private |
◆ sensors_path
| std::string JobScheduler::sensors_path |
|
private |
◆ submit_conn_
Persistent SSH master to the login node, reused across schedule_job() calls so the cold handshake is paid only on the first submission. Opened lazily on first use; closed when this JobScheduler is destroyed. The data-mover connection stays per-call (it can't multiplex, so nothing to keep alive).
The documentation for this class was generated from the following files: