Class: OodCore::Job::Adapters::PBSPro::Batch Private

Inherits:
Object
  • Object
show all
Defined in:
lib/ood_core/job/adapters/pbspro.rb

Overview

This class is part of a private API. You should avoid using this class if possible, as it may be removed or be changed in the future.

Object used for simplified communication with a PBS Pro batch server

Defined Under Namespace

Classes: Error

Constant Summary collapse

UNAVAILABLE_NODE_STATES =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future.

Node state tokens that mark a node as unavailable for scheduling. PBS Pro may report compound states (e.g. "offline,down", "state-unknown"), so states are tokenized before lookup.

Set.new(
  %w[down offline drain drained maint maintenance unknown]
).freeze

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(host: nil, submit_host: "", strict_host_checking: true, pbs_exec: nil, bin_overrides: {}) ⇒ Batch

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of Batch.

Parameters:

  • host (#to_s, nil) (defaults to: nil) —

    the batch server host

  • submit_host (#to_s, nil) (defaults to: "") —

    the login node to ssh to

  • strict_host_checking (bool, true) (defaults to: true) —

    wheter to use strict host checking when ssh to submit_host

  • exec (#to_s, nil) —

    path to pbs executables



89
90
91
92
93
94
95
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 89

def initialize(host: nil, submit_host: "", strict_host_checking: true, pbs_exec: nil, bin_overrides: {})
  @host                 = host && host.to_s
  @submit_host          = submit_host && submit_host.to_s
  @strict_host_checking = strict_host_checking
  @pbs_exec             = pbs_exec && Pathname.new(pbs_exec.to_s)
  @bin_overrides        = bin_overrides
end

Instance Attribute Details

#bin_overrides ⇒ Object (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Optional overrides for PBS Pro client executables

Examples:

{'qsub' => '/usr/local/bin/qsub'}


79
80
81
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 79

def bin_overrides
  @bin_overrides
end

#host ⇒ String? (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

The host of the PBS Pro batch server

Examples:

my_batch.host #=> "my_batch.server.edu"

Returns:

  • (String, nil) —

    the batch server host



55
56
57
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 55

def host
  @host
end

#pbs_exec ⇒ Pathname? (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

The path containing the PBS executables

Examples:

my_batch.pbs_exec.to_s #=> "/usr/local/pbspro/10.0.0

Returns:

  • (Pathname, nil) —

    path to pbs executables



73
74
75
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 73

def pbs_exec
  @pbs_exec
end

#strict_host_checking ⇒ Bool, true (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Whether to use strict host checking when ssh to submit_host

Examples:

my_batch.strict_host_checking #=> "false"

Returns:

  • (Bool, true) —

    the login node; true if not present



67
68
69
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 67

def strict_host_checking
  @strict_host_checking
end

#submit_host ⇒ String? (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

The login node to submit the job via ssh

Examples:

my_batch.submit_host #=> "my_batch.server.edu"

Returns:

  • (String, nil) —

    the login node



61
62
63
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 61

def submit_host
  @submit_host
end

Instance Method Details

#delete_job(id) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Delete a specified job from batch server

Examples:

Delete job "1234"

my_batch.delete_job("1234")

Parameters:

  • id (#to_s) —

    the id of the job

Raises:

  • (Error) —

    if qdel command exited unsuccessfully



203
204
205
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 203

def delete_job(id)
  call("qdel", id.to_s)
end

#get_cluster_info ⇒ ClusterInfo

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Get a ClusterInfo object containing information about the given cluster

Returns:



99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 99

def get_cluster_info
  data = JSON.parse(call("pbsnodes", "-aSj", "-F", "json"))
  nodes_h = data.fetch("nodes", {})

  stats = nodes_h.values.each_with_object(Hash.new(0)) do |info, acc|
    state = info.fetch("state", "").to_s.downcase
    next if state.split(/\W+/).any? { |token| UNAVAILABLE_NODE_STATES.include?(token) }

    f_c, t_c = parse_free_total(info["ncpus f/t"])
    f_g, t_g = parse_free_total(info["ngpus f/t"])

    acc[:available_nodes] += 1
    acc[:busy_nodes] += 1 if f_c == 0 || (f_g == 0 && t_g > 0)

    acc[:free_ncpus] += f_c
    acc[:total_ncpus] += t_c
    acc[:free_gpus] += f_g
    acc[:total_gpus] += t_g
  end

  ClusterInfo.new(
    active_nodes: stats[:busy_nodes],
    total_nodes: stats[:available_nodes],
    active_processors: stats[:total_ncpus] - stats[:free_ncpus],
    total_processors: stats[:total_ncpus],
    active_gpus: stats[:total_gpus] - stats[:free_gpus],
    total_gpus: stats[:total_gpus]
  )
end

#get_jobs(id: "") ⇒ Array<Hash>

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Get a list of hashes detailing each of the jobs on the batch server

Examples:

Status info for all jobs

my_batch.get_jobs
#=>
#[
#  {
#    :account => "account",
#    :job_id => "my_job",
#    ...
#  },
#  {
#    :account => "account",
#    :job_id => "my_other_job",
#    ...
#  },
#  ...
#]

Parameters:

  • id (#to_s) (defaults to: "") —

    the id of the job

Returns:

  • (Array<Hash>) —

    list of details for jobs

Raises:

  • (Error) —

    if qstat command exited unsuccessfully



149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 149

def get_jobs(id: "")
  args = ["-f", "-t"]   # display all information
  args.concat [id.to_s] unless id.to_s.empty?
  lines = call("qstat", *args).gsub("\n\t", "").split("\n").map(&:strip)

  jobs = []
  lines.each do |line|
    if /^Job Id: (?<job_id>.+)$/ =~ line
      jobs << { job_id: job_id }
    elsif /^(?<key>[^\s]+) = (?<value>.+)$/ =~ line
      hsh = jobs.last
      k1, k2 = key.split(".").map(&:to_sym)
      k2 ? ( hsh[k1] ||= {} and hsh[k1][k2] = value ) : ( hsh[k1] = value )
    end
  end

  jobs
end

#hold_job(id) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Put a specified job on hold

Examples:

Put job "1234" on hold

my_batch.hold_job("1234")

Parameters:

  • id (#to_s) —

    the id of the job

Raises:

  • (Error) —

    if qhold command exited unsuccessfully



183
184
185
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 183

def hold_job(id)
  call("qhold", id.to_s)
end

#release_job(id) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Release a specified job that is on hold

Examples:

Release job "1234" from on hold

my_batch.release_job("1234")

Parameters:

  • id (#to_s) —

    the id of the job

Raises:

  • (Error) —

    if qrls command exited unsuccessfully



193
194
195
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 193

def release_job(id)
  call("qrls", id.to_s)
end

#select_jobs(args: []) ⇒ Array<String>

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Select batch jobs from the batch server

Parameters:

  • args (Array<#to_s>) (defaults to: []) —

    arguments passed to qselect command

Returns:

  • (Array<String>) —

    list of job ids that match selection criteria

Raises:

  • (Error) —

    if qselect command exited unsuccessfully



173
174
175
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 173

def select_jobs(args: [])
  call("qselect", *args).split("\n").map(&:strip)
end

#submit_string(str, args: [], chdir: nil) ⇒ String

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Submit a script expanded as a string to the batch server

Parameters:

  • str (#to_s) —

    script as a string

  • args (Array<#to_s>) (defaults to: []) —

    arguments passed to qsub command

  • chdir (#to_s, nil) (defaults to: nil) —

    working directory where qsub is called

Returns:

  • (String) —

    the id of the job that was created

Raises:

  • (Error) —

    if qsub command exited unsuccessfully



213
214
215
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 213

def submit_string(str, args: [], chdir: nil)
  call("qsub", *args, stdin: str.to_s, chdir: chdir).strip
end