Class: Wurk::Swarm::MemoryWatch

Inherits:
Object
  • Object
show all
Defined in:
lib/wurk/swarm/memory_watch.rb

Overview

MAXMEM recycling (Ent §7.5): every CHECK_INTERVAL the supervisor reads each child's memory and hands one over the limit to the restart machine (recycle), which replaces it gracefully.

A child already over the limit while still FRESH is never recycled: its replacement would boot to the same size and be recycled in turn, a fork loop that replaces a slot every few seconds and drains nothing useful. Such a child is marked for the rest of its life and the limit reported once per slot index at ERROR — the setting is below what a booted child needs. A child that crosses the limit only after FRESH seconds is recycled as before; if its replacement is over within its own fresh window, the replacement is the one marked, so a slot recycles at most once per cause.

The metric is memory_limit_metric: :rss from /proc//statm (Ent's meaning; counts pages still shared with the parent), :pss/:uss from /proc//smaps_rollup. Without /proc nothing is read and nothing is recycled; warn_if_unreadable says so at boot.

Constant Summary collapse

CHECK_INTERVAL = 
10
FRESH = 
Backoff::RESET_AFTER
SMAPS_FIELDS = 
{ pss: %w[Pss], uss: %w[Private_Clean Private_Dirty] }.freeze

Instance Method Summary collapse

Constructor Details

#initialize(limit_kb:, metric:, logger:, recycle:, interval: CHECK_INTERVAL, fresh: FRESH, proc_root: '/proc', now: -> { ::Process.clock_gettime(::Process::CLOCK_MONOTONIC) }) ⇒ MemoryWatch

Returns a new instance of MemoryWatch.



27
28
29
30
31
32
33
34
35
36
37
38
39
40
 
# File 'lib/wurk/swarm/memory_watch.rb', line 27

def initialize(limit_kb:, metric:, logger:, recycle:, interval: CHECK_INTERVAL, fresh: FRESH,
               proc_root: '/proc', now: -> { ::Process.clock_gettime(::Process::CLOCK_MONOTONIC) })
  @limit_kb = limit_kb
  @metric = metric
  @logger = logger
  @recycle = recycle
  @interval = interval
  @fresh = fresh
  @now = now
  @proc_root = proc_root
  @last_check = nil
  @marked = {}
  @reported_slots = {}
end

Instance Method Details

#forget(pid) ⇒ Object 



51
52
53
 
# File 'lib/wurk/swarm/memory_watch.rb', line 51

def forget(pid)
  @marked.delete(pid)
end

#tick(children) ⇒ Object

children is { pid => [seconds since fork, slot index] }.



43
44
45
46
47
48
49
 
# File 'lib/wurk/swarm/memory_watch.rb', line 43

def tick(children)
  now = @now.call
  return if @last_check && now - @last_check < @interval

  @last_check = now
  children.each { |pid, (age, slot)| check(pid, age, slot) }
end

#warn_if_unreadable ⇒ Object 



55
56
57
58
59
60
61
62
 
# File 'lib/wurk/swarm/memory_watch.rb', line 55

def warn_if_unreadable
  return if read_kb(::Process.pid)

  @logger.warn do
    "swarm: memory_limit_mb is set but this host has no /proc #{source} to read #{@metric} from; " \
      'children will not be recycled'
  end
end