Module: Wurk::RailsBoot

Defined in:
lib/wurk/rails_boot.rb

Overview

Coordinates how Wurk boots inside a Rails host: whether this process should boot at all, whether it may fork the swarm, and the boot itself. The Railtie owns only the Rails hooks (config namespace + server_mode initializer + after_initialize) and delegates every decision and side effect here, so the boot policy stays pure and unit-testable without the Railtie DSL. See docs/idea/03-process-model.md for the exact ordering.

Constant Summary collapse

DRAIN_JOIN_SLACK =

What at_exit waits for the supervise thread on top of the swarm's own drain budget: one supervise tick to notice the request, plus slack for the final reap.

1
BOOT_INITIALIZER = 
:wurk_boot

Class Method Summary collapse

Class Method Details

.boot(app = ::Rails.application) ⇒ Object

Invoked last of all initializers, once the host app has fully initialized.



63
64
65
66
67
68
69
70
71
72
 
# File 'lib/wurk/rails_boot.rb', line 63

def boot(app = ::Rails.application)
  reason = skip_reason
  return log_client_only(reason) if reason

  case boot_action(app)
  when :fork   then boot_swarm
  when :embed  then boot_embedded
  when :refuse then refuse_preforking_boot
  end
end

.boot_action(app = ::Rails.application) ⇒ Object

What boot should do once skip_boot? is false. Pure — reads env / loaded constants / host config, no side effects — so the boot decision is unit-testable without forking:

:fork   — safe to fork the swarm in the background (the default).
:refuse — a preforking web server owns process forking here; don't.
:embed  — host opted into in-process threads-only via embed_in_web.


156
157
158
159
160
 
# File 'lib/wurk/rails_boot.rb', line 156

def boot_action(app = ::Rails.application)
  return :fork unless preforking_web_server?

  embed_in_web?(app) ? :embed : :refuse
end

.boot_embedded(embedded = Wurk::Embedded) ⇒ Object

Sidekiq-embedded parity: a threads-only worker inside the web process, no fork. Redis validation failure keeps the host serving HTTP (log + carry on). at_exit drains on the graceful shutdown the web server runs on TERM.



275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
 
# File 'lib/wurk/rails_boot.rb', line 275

def boot_embedded(embedded = Wurk::Embedded)
  instance = embedded.new(Wurk.configuration)
  # Registered BEFORE run, for the reason boot_swarm registers before the
  # fork: run brings the heartbeat, pollers, managers and health listener up
  # one at a time, and a raise partway through would otherwise leave the ones
  # already running with nothing to drain them on host exit.
  at_exit { instance.stop }
  instance.run
  check_embedded_boot(Wurk.configuration)
  instance
rescue StandardError => e
  logger.error { "wurk: embedded boot failed: #{e.class}: #{e.message}" }
  instance&.stop
  nil
end

.boot_scheduled=(value) ⇒ Object 



58
59
60
 
# File 'lib/wurk/rails_boot.rb', line 58

def boot_scheduled=(value)
  @boot_scheduled = value
end

.boot_scheduled? ⇒ Boolean

Set by the railtie's first initializer. Rails builds the initializer list once, so if that one ran the Finisher entry is in the list too. A railtie required from a host's own initializer missed it, and the after_initialize hook boots instead.

Returns:

  • (Boolean) 


56
 
# File 'lib/wurk/rails_boot.rb', line 56

def boot_scheduled? = @boot_scheduled == true

.boot_swarm ⇒ Object

rubocop:disable Metrics/AbcSize



208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
 
# File 'lib/wurk/rails_boot.rb', line 208

def boot_swarm # rubocop:disable Metrics/AbcSize
  warn_on_production_fork
  Health.options_from_env
  config = Wurk.configuration
  timeout = config[:timeout] || Swarm::DEFAULT_SHUTDOWN_TIMEOUT
  BootChecks.log_swarm_role(config, "swarm parent in #{server_name}")
  BootChecks.run(config, nil)
  swarm = Wurk::Swarm.new(topology: config.topology, shutdown_timeout: timeout)
  supervisor = nil
  # Registered BEFORE boot, which forks the children one slot at a time: a
  # fork that raises partway through would otherwise leave the children it
  # already spawned with nothing to drain them on host exit. Children
  # inherit the hook — stop_swarm no-ops off the process that forked them.
  at_exit { stop_swarm(swarm, supervisor, timeout + Swarm::SHUTDOWN_GRACE + DRAIN_JOIN_SLACK) }
  # Co-hosted in the web process (e.g. Puma single mode): the host owns the
  # process-wide TERM/INT traps. Installing the swarm's own would hijack
  # them — a deploy TERM would drain the swarm but never stop the HTTP
  # server. Let the host keep signal ownership and drain the swarm on its
  # graceful exit (same contract as boot_embedded).
  swarm.boot(install_signals: false)
  # supervise must still run somewhere or crashed children never respawn and
  # memory checks never fire. A background thread keeps the host's main
  # thread free to serve HTTP.
  supervisor = Thread.new do
    swarm.supervise
  rescue StandardError => e
    logger.error { "wurk supervisor thread died: #{e.class}: #{e.message}" }
  end
end

.building? ⇒ Boolean

A build/precompile step must never fork the swarm (#247). The default Rails Dockerfile runs SECRET_KEY_BASE_DUMMY=1 ./bin/rails assets:precompile; that loads :environment → fires after_initialize, but there's no Redis during docker build, so a fork would hang/fail the build. Same for other env-loading rake tasks (db:prepare, db:migrate). The real server path is unaffected: rails server / puma boot through Rails::Command, not Rake, and don't set the dummy secret.

Returns:

  • (Boolean) 


320
321
322
323
324
325
326
 
# File 'lib/wurk/rails_boot.rb', line 320

def building?
  return true if ENV.key?('SECRET_KEY_BASE_DUMMY')

  defined?(::Rake) && ::Rake.application.top_level_tasks.any?
rescue StandardError
  false
end

.check_embedded_boot(config) ⇒ Object 



266
267
268
269
270
 
# File 'lib/wurk/rails_boot.rb', line 266

def check_embedded_boot(config)
  BootChecks.log_process_role(config, "embedded in #{server_name}, threads only")
  BootChecks.run(config, nil)
  BootChecks.warn_on_ignored_topology(config, "a worker embedded in #{server_name}")
end

.embed_in_web?(app = ::Rails.application) ⇒ Boolean

Host opt-in (config/application.rb): run workers as in-process threads instead of forking, like Sidekiq embedded. Set it in application.rb, not an initializer — server mode is decided before initializers load.

Returns:

  • (Boolean) 


202
203
204
205
206
 
# File 'lib/wurk/rails_boot.rb', line 202

def embed_in_web?(app = ::Rails.application)
  app.config.wurk&.embed_in_web == true
rescue StandardError
  false
end

.enter_server_mode_if_serving(app = ::Rails.application) ⇒ Object

Invoked from the wurk.server_mode initializer, before config/initializers load. This Rails process forks the workers, so it IS the server. Enter server mode now — otherwise the app's Sidekiq.configure_server blocks gate on config.server? (still false) and wait for server mode, logging the skip at DEBUG (configure_server / server? is false). A process that won't run workers (skip_boot?, or one that refuses to boot under a preforking web server) is not a server.

It is a client as well: its web requests enqueue, so the app's Sidekiq.configure_client blocks (client middleware, the client's Redis) must apply here too instead of being skipped because server? is true.



34
35
36
37
38
39
40
 
# File 'lib/wurk/rails_boot.rb', line 34

def enter_server_mode_if_serving(app = ::Rails.application)
  return if skip_boot?
  return if boot_action(app) == :refuse

  Wurk.enter_server_mode
  Wurk.configuration[:client_in_server] = true
end

.install_reloader(app = ::Rails.application, config = Wurk.configuration) ⇒ Object

A host that already replaced the reloader keeps its own. A config already frozen, by workers embedded from an initializer, can't take it: their Processors resolved the default themselves (Processor#resolve_reloader).



45
46
47
48
49
50
 
# File 'lib/wurk/rails_boot.rb', line 45

def install_reloader(app = ::Rails.application, config = Wurk.configuration)
  return if config.frozen?
  return unless config[:reloader].equal?(Configuration::DEFAULTS[:reloader])

  config[:reloader] = RailsReloader.new(app)
end

.log_client_only(reason) ⇒ Object

The runner announces its own role; a second line from here would describe the same process twice.



99
100
101
102
103
 
# File 'lib/wurk/rails_boot.rb', line 99

def log_client_only(reason)
  return if Wurk.worker_boot_claimed?

  BootChecks.log_client_only(logger, reason)
end

.logger ⇒ Object 



309
310
311
 
# File 'lib/wurk/rails_boot.rb', line 309

def logger
  Wurk.configuration.logger
end

.preforking_web_server? ⇒ Boolean

Preforking / clustered web servers (Puma cluster, Unicorn, Passenger) fork their own worker processes. Forking the swarm from after_initialize in one of them is the highest-risk boot path: without app preloading every server-worker re-runs the hook and forks its own full swarm (N× oversubscription); with preloading the swarm supervisor ends up entangled with the server's own fork/signal supervision. Detect the common three.

Returns:

  • (Boolean) 


168
169
170
171
172
173
 
# File 'lib/wurk/rails_boot.rb', line 168

def preforking_web_server?
  return true if defined?(::PhusionPassenger)
  return true if defined?(::Unicorn)

  puma_cluster?
end

.puma_cluster? ⇒ Boolean

Puma only preforks in cluster mode (workers > 0); single mode is threaded and safe to co-host the swarm. Best-effort: a missed cluster falls through to the historical fork path; an over-eager match is escapable via embed_in_web / WURK_DISABLED / running the swarm as its own process.

Returns:

  • (Boolean) 


179
180
181
182
183
184
 
# File 'lib/wurk/rails_boot.rb', line 179

def puma_cluster?
  return false unless defined?(::Puma)

  count = puma_worker_count
  count.is_a?(Integer) && count.positive?
end

.puma_launcher_loaded? ⇒ Boolean

Bundler.require of the puma gem defines Puma in every process, and puma >= 6 registers autoload :Launcher in lib/puma.rb — defined? reports an autoload as "constant" without loading it, so the bare check was true in rails runner / a console and forked a swarm there. Only a running server has actually loaded the launcher.

Returns:

  • (Boolean) 


136
137
138
139
140
 
# File 'lib/wurk/rails_boot.rb', line 136

def puma_launcher_loaded?
  return false unless defined?(::Puma::Launcher)

  !::Puma.autoload?(:Launcher)
end

.puma_worker_count ⇒ Object

Worker count from Puma's parsed CLI config when the server booted it, else from WEB_CONCURRENCY (the near-universal convention for Puma workers). Guarded — the accessor and options shape vary across Puma versions.



189
190
191
192
193
194
195
196
197
 
# File 'lib/wurk/rails_boot.rb', line 189

def puma_worker_count
  cfg = ::Puma.cli_config if ::Puma.respond_to?(:cli_config)
  workers = cfg&.options&.[](:workers)
  return workers.to_i if workers

  ENV['WEB_CONCURRENCY']&.to_i
rescue StandardError
  nil
end

.refuse_preforking_boot ⇒ Object 



291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
 
# File 'lib/wurk/rails_boot.rb', line 291

def refuse_preforking_boot
  logger.warn { <<~MSG }
    wurk: preforking web server detected (Puma cluster / Unicorn / Passenger).
    Refusing to fork the worker swarm from a process that forks its own workers.
    Run the swarm as its own process instead:

        bundle exec wurkswarm    # forked swarm, real parallelism
        bundle exec wurk         # single process, thread pool

    Or run workers inside this web process (threads only, no fork):

        # config/application.rb
        config.wurk.embed_in_web = true

    Already running workers elsewhere? Set WURK_DISABLED=1 here to silence this.
  MSG
end

.server_name ⇒ Object

Named the way #serving? recognised it: a bundled gem defines its constant in processes that are not that server.



107
108
109
110
111
112
113
114
 
# File 'lib/wurk/rails_boot.rb', line 107

def server_name
  return 'Puma' if puma_launcher_loaded?
  return 'Passenger' if defined?(::PhusionPassenger)
  return 'Unicorn' if unicorn_server?
  return 'rails server' if defined?(::Rails::Server)

  'this process (WURK_EMBED=1)'
end

.serving? ⇒ Boolean

Boot only inside a web server, or where the host opted in. Every Rails process runs after_initialize — rails runner, rails generate, a custom script that requires config/environment — and a denylist can never name them all: each one forked a full swarm that fetched jobs, and its at_exit drain then cut those jobs off when the one-off command finished. The markers are constants only a running server defines. Any other server (Falcon, Thin, Pitchfork, a bespoke rackup) opts in with WURK_EMBED=1.

Returns:

  • (Boolean) 


124
125
126
127
128
129
 
# File 'lib/wurk/rails_boot.rb', line 124

def serving?
  return true if ENV['WURK_EMBED'] == '1'
  return true if defined?(::Rails::Server) || puma_launcher_loaded? || defined?(::PhusionPassenger)

  unicorn_server?
end

.skip_boot? ⇒ Boolean

A process that won't run workers isn't a server: skip both server mode and the swarm boot. Console mode is detected reliably here — the console command file defines ::Rails::Console before initializers run.

Wurk.worker_boot_claimed? covers the CLI: bundle exec wurk boots the host app itself and then runs a Launcher in this very process, so forking a swarm here as well would put two independent workers on one queue and run every job — every cron tick included — twice.

Everything above is a denylist; serving? is the allowlist that backs it.

Returns:

  • (Boolean) 


84
 
# File 'lib/wurk/rails_boot.rb', line 84

def skip_boot? = !skip_reason.nil?

.skip_reason ⇒ Object

Why this process runs no jobs, or nil when it boots them.



87
88
89
90
91
92
93
94
95
 
# File 'lib/wurk/rails_boot.rb', line 87

def skip_reason
  return 'WURK_DISABLED=1' if ENV['WURK_DISABLED'] == '1'
  return 'the wurk runner owns this process' if Wurk.worker_boot_claimed?
  return 'a build or rake task' if building?
  return 'Rails console' if defined?(::Rails::Console)
  return 'test environment' if ::Rails.env.test?

  'not a web server; WURK_EMBED=1 runs jobs here' unless serving?
end

.stop_swarm(swarm, supervisor, drain_wait) ⇒ Object

at_exit fires on the host's main thread, but the supervise thread owns the child table and two threads inside shutdown race on it — so request the drain and wait for the supervisor to run it. Draining here is the fallback for when no live supervisor will: it never started (boot raised) or it died early; after one that drained it finds no children and no-ops. A supervisor still alive past the join is wedged mid-drain — leave it be rather than race it; its children self-terminate (OrphanGuard) once this process is gone.



258
259
260
261
262
263
264
 
# File 'lib/wurk/rails_boot.rb', line 258

def stop_swarm(swarm, supervisor, drain_wait)
  return unless swarm.owner?

  swarm.request_shutdown
  supervisor&.join(drain_wait)
  swarm.shutdown unless supervisor&.alive?
end

.unicorn_server? ⇒ Boolean

require 'unicorn' (a Gemfile entry) defines the constant in every process, so the program name is what tells the server apart.

Returns:

  • (Boolean) 


144
145
146
147
148
 
# File 'lib/wurk/rails_boot.rb', line 144

def unicorn_server?
  return false unless defined?(::Unicorn)

  ::File.basename($PROGRAM_NAME.to_s).start_with?('unicorn')
end

.warn_on_production_fork ⇒ Object

A deliberate default, but a surprise after a gem swap: stock Sidekiq never runs jobs in the web process, and a separate worker on top of this runs everything against two pools. WURK_EMBED=1 says it is intended.



241
242
243
244
245
246
247
248
 
# File 'lib/wurk/rails_boot.rb', line 241

def warn_on_production_fork
  return unless ::Rails.env.production? && ENV['WURK_EMBED'] != '1'

  logger.warn do
    'wurk: this web process is forking a worker swarm. If workers run elsewhere, set WURK_DISABLED=1 here; ' \
      'if this is intended, set WURK_EMBED=1 to silence this.'
  end
end