Class: OodCore::Job::Adapter Abstract

Inherits:
Object
  • Object
show all
Defined in:
lib/ood_core/job/adapter.rb

Overview

This class is abstract.

A class that handles the communication with a resource manager for submitting/statusing/holding/deleting jobs

Instance Method Summary collapse

Instance Method Details

#accounts ⇒ Array<AccountInfo>

Retrieve the accounts available to use for the current user. The same account might appear muiltiple times if it has access to multiple clusters.

Subclasses that do not implement this will return empty arrays.

Returns:

  • (Array<AccountInfo>) —

    the accounts available to the user.



226
227
228
 
# File 'lib/ood_core/job/adapter.rb', line 226

def accounts
  []
end

#cluster_info ⇒ ClusterInfo

This method is abstract.

Subclass is expected to implement #cluster_stats

Retrieve the number of active and total cpus, nodes, and gpus

Returns:

  • (ClusterInfo) —

    Object containing quantified statistics about the cluster's active/total cpus, nodes, and gpus

Raises:

  • (NotImplementedError) —

    if subclass did not define #cluster_stats



41
42
43
 
# File 'lib/ood_core/job/adapter.rb', line 41

def cluster_info
  raise NotImplementedError, "subclass did not define #cluster_stats"
end

#delete(id) ⇒ void

This method is abstract.

Subclass is expected to implement #delete

This method returns an undefined value.

Delete the submitted job

Parameters:

  • id (#to_s) —

    the id of the job

Raises:

  • (NotImplementedError) —

    if subclass did not define #delete



186
187
188
 
# File 'lib/ood_core/job/adapter.rb', line 186

def delete(id)
  raise NotImplementedError, "subclass did not define #delete"
end

#directive_prefix ⇒ String

This method is abstract.

Subclass is expected to implement #directive_prefix

Return the scheduler-specific directive prefix

Examples of directive prefixes include #QSUB, #BSUB and allow placing what would otherwise be command line options inside the job launch script.

The method should return nil if the adapter does not support prefixes

Returns:

  • (String)

Raises:



200
201
202
 
# File 'lib/ood_core/job/adapter.rb', line 200

def directive_prefix
  raise NotImplementedError, "subclass did not define #directive_prefix"
end

#hold(id) ⇒ void

This method is abstract.

Subclass is expected to implement #hold

This method returns an undefined value.

Put the submitted job on hold

Parameters:

  • id (#to_s) —

    the id of the job

Raises:

  • (NotImplementedError) —

    if subclass did not define #hold



168
169
170
 
# File 'lib/ood_core/job/adapter.rb', line 168

def hold(id)
  raise NotImplementedError, "subclass did not define #hold"
end

#info(id) ⇒ Info

This method is abstract.

Subclass is expected to implement #info

Retrieve job info from the resource manager

Parameters:

  • id (#to_s) —

    the id of the job

Returns:

  • (Info) —

    information describing submitted job

Raises:

  • (NotImplementedError) —

    if subclass did not define #info



149
150
151
 
# File 'lib/ood_core/job/adapter.rb', line 149

def info(id)
  raise NotImplementedError, "subclass did not define #info"
end

#info_all(attrs: nil) ⇒ Array<Info>

This method is abstract.

Subclass is expected to implement #info_all

Retrieve info for all jobs from the resource manager

Parameters:

  • attrs (Array<symbol>) (defaults to: nil) —

    defaults to nil (and all attrs are provided) This array specifies only attrs you want, in addition to id and status. If an array, the Info object that is returned to you is not guarenteed to have a value for any attr besides the ones specified and id and status.

    For certain adapters this may speed up the response since adapters can get by without populating the entire Info object

Returns:

  • (Array<Info>) —

    information describing submitted jobs

Raises:

  • (NotImplementedError) —

    if subclass did not define #info_all



56
57
58
 
# File 'lib/ood_core/job/adapter.rb', line 56

def info_all(attrs: nil)
  raise NotImplementedError, "subclass did not define #info_all"
end

#info_all_each(attrs: nil) {|Info| ... } ⇒ Enumerator

Iterate over each job Info object

Parameters:

  • attrs (Array<symbol>) (defaults to: nil) —

    defaults to nil (and all attrs are provided) This array specifies only attrs you want, in addition to id and status. If an array, the Info object that is returned to you is not guarenteed to have a value for any attr besides the ones specified and id and status.

    For certain adapters this may speed up the response since adapters can get by without populating the entire Info object

Yields:

  • (Info) —

    of each job to block

Returns:

  • (Enumerator) —

    if no block given



102
103
104
105
106
107
108
 
# File 'lib/ood_core/job/adapter.rb', line 102

def info_all_each(attrs: nil)
  return to_enum(:info_all_each, attrs: attrs) unless block_given?

  info_all(attrs: attrs).each do |job|
    yield job
  end
end

#info_historic(opts: {}) ⇒ Array<Info>

This method is abstract.

Subclass is expected to implement #info_historic

Retrieve historic info for all completed jobs from the resource manager. This depends on the data retention configuration of the resource manager.

Parameters:

  • opts (#to_h) (defaults to: {}) —

    options to filter jobs in the resource manager.

Returns:

  • (Array<Info>) —

    information describing the jobs

Raises:



68
69
70
 
# File 'lib/ood_core/job/adapter.rb', line 68

def info_historic(opts: {})
  raise NotImplementedError, "subclass did not define #info_historic"
end

#info_where_owner(owner, attrs: nil) ⇒ Array<Info>

Retrieve info for all jobs for a given owner or owners from the resource manager

Parameters:

  • owner (#to_s, Array<#to_s>) —

    the owner(s) of the jobs

  • attrs (Array<symbol>) (defaults to: nil) —

    defaults to nil (and all attrs are provided) This array specifies only attrs you want, in addition to id and status. If an array, the Info object that is returned to you is not guarenteed to have a value for any attr besides the ones specified and id and status.

    For certain adapters this may speed up the response since adapters can get by without populating the entire Info object

Returns:

  • (Array<Info>) —

    information describing submitted jobs



83
84
85
86
87
88
89
90
 
# File 'lib/ood_core/job/adapter.rb', line 83

def info_where_owner(owner, attrs: nil)
  owner = Array.wrap(owner).map(&:to_s)

  # must at least have job_owner to filter by job_owner
  attrs = Array.wrap(attrs) | [:job_owner] unless attrs.nil?

  info_all(attrs: attrs).select { |info| owner.include? info.job_owner }
end

#info_where_owner_each(owner, attrs: nil) {|Info| ... } ⇒ Enumerator

Iterate over each job Info object

Parameters:

  • owner (#to_s, Array<#to_s>) —

    the owner(s) of the jobs

  • attrs (Array<symbol>) (defaults to: nil) —

    defaults to nil (and all attrs are provided) This array specifies only attrs you want, in addition to id and status. If an array, the Info object that is returned to you is not guarenteed to have a value for any attr besides the ones specified and id and status.

    For certain adapters this may speed up the response since adapters can get by without populating the entire Info object

Yields:

  • (Info) —

    of each job to block

Returns:

  • (Enumerator) —

    if no block given



121
122
123
124
125
126
127
 
# File 'lib/ood_core/job/adapter.rb', line 121

def info_where_owner_each(owner, attrs: nil)
  return to_enum(:info_where_owner_each, owner, attrs: attrs) unless block_given?

  info_where_owner(owner, attrs: attrs).each do |job|
    yield job
  end
end

#job_name_illegal_chars ⇒ String

Illegal chars that should not be used in a job name A dash is assumed to be legal in job names in all batch schedulers

Returns:

  • (String) —

    string of chars



217
218
219
 
# File 'lib/ood_core/job/adapter.rb', line 217

def job_name_illegal_chars
  ENV["OOD_JOB_NAME_ILLEGAL_CHARS"].to_s
end

#nodes ⇒ Array<NodeInfo>

Return the list of nodes for this scheduler.

Subclasses that do not implement this will return empty arrays.

Returns:



242
243
244
 
# File 'lib/ood_core/job/adapter.rb', line 242

def nodes
  []
end

#queues ⇒ Array<QueueInfo>

Return the list of queues for this scheduler.

Subclasses that do not implement this will return empty arrays.

Returns:



234
235
236
 
# File 'lib/ood_core/job/adapter.rb', line 234

def queues
  []
end

#release(id) ⇒ void

This method is abstract.

Subclass is expected to implement #release

This method returns an undefined value.

Release the job that is on hold

Parameters:

  • id (#to_s) —

    the id of the job

Raises:

  • (NotImplementedError) —

    if subclass did not define #release



177
178
179
 
# File 'lib/ood_core/job/adapter.rb', line 177

def release(id)
  raise NotImplementedError, "subclass did not define #release"
end

#sanitize_job_name(job_name) ⇒ String

Replace illegal chars in job name with a dash

Returns:

  • (String) —

    job name with dashes replacing illegal chars



207
208
209
210
211
 
# File 'lib/ood_core/job/adapter.rb', line 207

def sanitize_job_name(job_name)
  # escape ^ and omit -
  chars = job_name_illegal_chars.to_s.gsub("^", "\\^").gsub("-", "")
  job_name.tr(chars, "-")
end

#status(id) ⇒ Status

This method is abstract.

Subclass is expected to implement #status

Note:

Optimized slightly over retrieving complete job information from server

Retrieve job status from resource manager

Parameters:

  • id (#to_s) —

    the id of the job

Returns:

Raises:

  • (NotImplementedError) —

    if subclass did not define #status



159
160
161
 
# File 'lib/ood_core/job/adapter.rb', line 159

def status(id)
  raise NotImplementedError, "subclass did not define #status"
end

#submit(script, after: [], afterok: [], afternotok: [], afterany: []) ⇒ String

This method is abstract.

Subclass is expected to implement #submit

Submit a job with the attributes defined in the job template instance

Examples:

Submit job template to cluster

solver_id = job_adapter.submit(solver_script)
#=> "1234.server"

Submit job that depends on previous job

post_id = job_adapter.submit(
  post_script,
  afterok: solver_id
)
#=> "1235.server"

Parameters:

  • script (Script) —

    script object that describes the script and attributes for the submitted job

  • after (#to_s, Array<#to_s>) (defaults to: []) —

    this job may be scheduled for execution at any point after dependent jobs have started execution

  • afterok (#to_s, Array<#to_s>) (defaults to: []) —

    this job may be scheduled for execution only after dependent jobs have terminated with no errors

  • afternotok (#to_s, Array<#to_s>) (defaults to: []) —

    this job may be scheduled for execution only after dependent jobs have terminated with errors

  • afterany (#to_s, Array<#to_s>) (defaults to: []) —

    this job may be scheduled for execution after dependent jobs have terminated

Returns:

  • (String) —

    the job id returned after successfully submitting a job

Raises:

  • (NotImplementedError) —

    if subclass did not define #submit



32
33
34
 
# File 'lib/ood_core/job/adapter.rb', line 32

def submit(script, after: [], afterok: [], afternotok: [], afterany: [])
  raise NotImplementedError, "subclass did not define #submit"
end

#supports_job_arrays? ⇒ Boolean

Whether the adapter supports job arrays

Returns:

  • (Boolean) —
    • assumes true; but can be overridden by adapters that explicitly do not


132
133
134
 
# File 'lib/ood_core/job/adapter.rb', line 132

def supports_job_arrays?
  true
end

#supports_job_dependencies? ⇒ Boolean

Whether the adapter supports job dependencies via after, afterok, afternotok, and afterany

Returns:

  • (Boolean) —
    • assumes false; adapters that support dependencies should override this


140
141
142
 
# File 'lib/ood_core/job/adapter.rb', line 140

def supports_job_dependencies?
  false
end