Class: OodCore::Job::Adapters::Kubernetes::Helper

Inherits:
Object
  • Object
show all
Defined in:
lib/ood_core/job/adapters/kubernetes/helper.rb

Overview

Utility class for the Kubernetes adapter to parse json data into Ruby objects.

Defined Under Namespace

Classes: K8sDataError

Constant Summary collapse

DURATION_UNITS =

Units of a Go duration string, in seconds. Rationals keep fractions like "0.7h" exact.

{
  'ns' => Rational(1, 1_000_000_000),
  'us' => Rational(1, 1_000_000),
  'µs' => Rational(1, 1_000_000),
  'ms' => Rational(1, 1_000),
  's' => 1,
  'm' => 60,
  'h' => 3600
}.freeze

Instance Method Summary collapse

Instance Method Details

#configmap_from_native(native, id, script_content) ⇒ OodCore::Job::Adapters::Kubernetes::Resources::ConfigMap

Turn a configmap hash into a Kubernetes::Resources::ConfigMap that can be used in templates. Needs an id so that the resulting configmap has a known name.

Parameters:

  • native (#to_h) —

    the input configmap hash

  • id (#to_s) —

    the id to use for giving the configmap a name

  • script_content (#to_s) —

    the batch script content

Returns:



96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
 
# File 'lib/ood_core/job/adapters/kubernetes/helper.rb', line 96

def configmap_from_native(native, id, script_content)
  configmap = native.fetch(:configmap, {})
  configmap[:files] ||= []
  configmap[:files] << {
    filename: 'script.sh',
    data: script_content,
    mount_path: '/ood/script.sh',
    sub_path: 'script.sh',
  } unless configmap[:files].any? { |f| f[:filename] == 'script.sh' }

  OodCore::Job::Adapters::Kubernetes::Resources::ConfigMap.new(
    configmap_name(id),
    (configmap[:files] || [])
  )
end

#configmap_name(id) ⇒ Object 



140
141
142
 
# File 'lib/ood_core/job/adapters/kubernetes/helper.rb', line 140

def configmap_name(id)
  id + '-configmap'
end

#container_from_native(container, default_env) ⇒ OodCore::Job::Adapters::Kubernetes::Resources::Container

Turn a container hash into a Kubernetes::Resources::Container

Parameters:

  • container (#to_h) —

    the input container hash

  • default_env (#to_h) —

    Default env to merge with defined env

Returns:



48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
 
# File 'lib/ood_core/job/adapters/kubernetes/helper.rb', line 48

def container_from_native(container, default_env)
  env = container.fetch(:env, {}).to_h.symbolize_keys
  OodCore::Job::Adapters::Kubernetes::Resources::Container.new(
    container[:name],
    container[:image],
    command: parse_command(container[:command]),
    port: container[:port],
    env: default_env.merge(env),
    memory_limit: container[:memory_limit] || container[:memory],
    memory_request: container[:memory_request] || container[:memory],
    cpu_limit: container[:cpu_limit] || container[:cpu],
    cpu_request: container[:cpu_request] || container[:cpu],
    working_dir: container[:working_dir],
    restart_policy: container[:restart_policy],
    image_pull_policy: container[:image_pull_policy],
    image_pull_secret: container[:image_pull_secret],
    supplemental_groups: container[:supplemental_groups],
    startup_probe: container[:startup_probe],
    labels: container[:labels],
  )
end

#duration_to_seconds(duration) ⇒ Integer?

Convert a Go duration string ("24h", "1h30m", "01h00m00s") to whole seconds; the inverse of seconds_to_duration. The pod lifetime annotation holds one of these, and anything that reads it (job-pod-reaper uses Go's time.ParseDuration) accepts the general format, not just ours.

Returns:

  • (Integer, nil) —

    seconds, or nil if the text isn't a duration



166
167
168
169
170
171
172
173
174
175
176
 
# File 'lib/ood_core/job/adapters/kubernetes/helper.rb', line 166

def duration_to_seconds(duration)
  text = duration.to_s.strip.delete_prefix('+')
  return nil if text.empty?
  return 0 if text == '0'

  parts = text.scan(/(\d+(?:\.\d*)?|\.\d+)(ns|us|µs|ms|s|m|h)/)
  return nil unless parts.map(&:join).join == text

  seconds = parts.sum { |number, unit| number.to_r * DURATION_UNITS.fetch(unit) }
  seconds.floor
end

#info_from_json(pod_json: nil, service_json: nil, secret_json: nil, ns_prefix: nil) ⇒ OodCore::Job::Adapters::Kubernetes::K8sJobInfo

Extract info from json data. The data is expected to be from the kubectl command and conform to kubernetes' datatype structures.

Returns K8sJobInfo in the in lieu of writing a connection.yml

Parameters:

  • pod_json (#to_h) (defaults to: nil) —

    the pod data returned from 'kubectl get pod abc-123'

  • service_json (#to_h) (defaults to: nil) —

    the service data returned from 'kubectl get service abc-123-service'

  • secret_json (#to_h) (defaults to: nil) —

    the secret data returned from 'kubectl get secret abc-123-secret'

  • ns_prefix (#to_s) (defaults to: nil) —

    the namespace prefix so that namespaces can be converted back to usernames

Returns:



29
30
31
32
33
34
35
36
37
38
39
 
# File 'lib/ood_core/job/adapters/kubernetes/helper.rb', line 29

def info_from_json(pod_json: nil, service_json: nil, secret_json: nil, ns_prefix: nil)
  pod_hash = pod_info_from_json(pod_json, ns_prefix: ns_prefix)
  service_hash = service_info_from_json(service_json)
  secret_hash = secret_info_from_json(secret_json)

  pod_hash.deep_merge!(service_hash)
  pod_hash.deep_merge!(secret_hash)
  OodCore::Job::Adapters::Kubernetes::K8sJobInfo.new(**pod_hash)
rescue NoMethodError
  raise K8sDataError, "unable to read data correctly from json"
end

#init_ctrs_from_native(ctrs, default_env) ⇒ Array<OodCore::Job::Adapters::Kubernetes::Resources::Container>

parse initialization containers from native data

Parameters:

  • native_data (#to_h) —

    the native data to parse. Expected key init_ctrs and for that key to be an array of hashes.

  • default_env (#to_h) —

    Default env to merge with defined env

Returns:



121
122
123
124
125
126
127
128
129
130
 
# File 'lib/ood_core/job/adapters/kubernetes/helper.rb', line 121

def init_ctrs_from_native(ctrs, default_env)
  init_ctrs = []

  ctrs&.each do |ctr_raw|
    ctr = container_from_native(ctr_raw, default_env)
    init_ctrs.push(ctr)
  end

  init_ctrs
end

#parse_command(cmd) ⇒ Array<#to_s>

Parse a command string given from a user and return an array. If given an array, the input is simply returned back.

Parameters:

  • cmd (#to_s) —

    the command to parse

Returns:

  • (Array<#to_s>) —

    the command parsed into an array of arguements



77
78
79
80
81
82
83
 
# File 'lib/ood_core/job/adapters/kubernetes/helper.rb', line 77

def parse_command(cmd)
  if cmd&.is_a?(Array)
    cmd
  else
    Shellwords.split(cmd.to_s)
  end
end

#pod_info_from_json(json_data, ns_prefix: nil) ⇒ #to_h

Extract pod info from json data. The data is expected to be from the kubectl command and conform to kubernetes' datatype structures.

Parameters:

  • json_data (#to_h) —

    the pod data returned from 'kubectl get pod abc-123'

  • ns_prefix (#to_s) (defaults to: nil) —

    the namespace prefix so that namespaces can be converted back to usernames

Returns:

  • (#to_h) —

    the hash of info expected from adapters



187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
 
# File 'lib/ood_core/job/adapters/kubernetes/helper.rb', line 187

def pod_info_from_json(json_data, ns_prefix: nil)
  {
    id: json_data.dig(:metadata, :name).to_s,
    job_name: name_from_metadata(json_data.dig(:metadata)),
    status: OodCore::Job::Status.new(state: pod_status_from_json(json_data)),
    job_owner: job_owner_from_json(json_data, ns_prefix),
    submission_time: submission_time(json_data),
    dispatch_time: dispatch_time(json_data),
    wallclock_time: wallclock_time(json_data),
    wallclock_limit: wallclock_limit(json_data),
    ood_connection_info: { host: get_host(json_data.dig(:status, :hostIP)) },
    procs: procs_from_json(json_data)
  }
rescue NoMethodError
  # gotta raise an error because Info.new will throw an error if id is undefined
  raise K8sDataError, "unable to read data correctly from json"
end

#seconds_to_duration(s) ⇒ Object 



144
145
146
 
# File 'lib/ood_core/job/adapters/kubernetes/helper.rb', line 144

def seconds_to_duration(s)
  "%02dh%02dm%02ds" % [s / 3600, s / 60 % 60, s % 60]
end

#secret_name(id) ⇒ Object 



136
137
138
 
# File 'lib/ood_core/job/adapters/kubernetes/helper.rb', line 136

def secret_name(id)
  id + '-secret'
end

#service_name(id) ⇒ Object 



132
133
134
 
# File 'lib/ood_core/job/adapters/kubernetes/helper.rb', line 132

def service_name(id)
  id + '-service'
end