Class: Wurk::Configuration
- Inherits:
-
Object
- Object
- Wurk::Configuration
- Defined in:
- lib/wurk/configuration.rb
Overview
Owns runtime knobs (concurrency, queues, timeouts, lifecycle events, error/death handlers) and the registry of Capsules. Single source of truth for everything the swarm / managers / processors need to boot.
Spec: docs/target/sidekiq-free.md §4 (Sidekiq::Config).
Constant Summary collapse
- DEFAULTS =
Mirrors Sidekiq::Config::DEFAULTS. Order and keys are part of the drop-in contract — third-party gems read @options via [] / fetch / dig.
{ labels: Set.new, require: '.', environment: nil, concurrency: 5, timeout: 25, poll_interval_average: nil, average_scheduled_poll_interval: 5, on_complex_arguments: :raise, max_iteration_runtime: nil, error_handlers: [], death_handlers: [], lifecycle_events: { startup: [], fork: [], quiet: [], shutdown: [], exit: [], heartbeat: [], beat: [], leader: [] }, dead_max_jobs: 10_000, dead_timeout_in_seconds: 180 * 24 * 60 * 60, reloader: proc { |&b| b.call }, backtrace_cleaner: ->(bt) { bt }, logged_job_attributes: %w[bid tags], redis_idle_timeout: nil, # seconds; RedisPool closes connections idle this long redis_error_handlers: [], # Wurk extras, appended after the mirrored keys so the Sidekiq prefix # above stays byte-for-byte what a gem reading @options expects. # # Lifetime of a `status:<jid>` row (Wurk::Status), re-stamped on every # write, and the separate lifetime a `complete` row gets once the job # succeeded. Retention is off by default — nil means a finished job's # row expires on the same clock as a running one. Set it to seconds to # keep succeeded jobs around longer than they ran, or to 0 for Sidekiq's # own behavior, where a job that succeeds leaves nothing behind. status_ttl: Keys::STATUS_TTL, status_retention: nil, # TTL in seconds of the 10-minute `j|` metrics buckets (Metrics::History); # nil keeps MID_TERM, 30 days. See #metrics_retention=. metrics_retention: nil, # Distributed tracing (Wurk::Telemetry). Seeded here, not written on # demand, so the read side answers on a config that `freeze!` has already # closed — and so a gem inspecting @options sees the same key set whether # or not the host traces. Flipping it is only half the gate; the # opentelemetry-api gem has to be present too (Wurk::Telemetry.enabled?). telemetry: false, # Bearer tokens for the machine-facing HTTP API (Wurk::API), as # `token => [scopes]`. Empty means the API does not exist: the engine # skips the mount and a directly mounted app answers 404, so a host that # never registers a token carries no new HTTP surface at all. Seeded here # rather than written on demand so the read side answers on a config that # `freeze!` has already closed. api_tokens: {}, # Boundary caps for the HTTP API's produce plane. A body larger than # `api_max_body_bytes` is refused before it is parsed; a job whose args # serialize larger than `api_max_args_bytes` before it is pushed — the # second is the one that bounds what a bulk request lands in Redis, since # one request becomes many payloads. `api_idempotency_ttl` is how long an # `Idempotency-Key` is remembered: a replay window for a producer # retrying a lost response, not an audit log. api_max_body_bytes: 1_048_576, api_max_args_bytes: 65_536, api_idempotency_ttl: 3_600, # Allow-list of classes the HTTP API may enqueue. nil (the default) means # only an `admin` token may enqueue at all — see #api_enqueue_classes=. api_enqueue_classes: nil, # Three-state on purpose — see #api_read_only=. nil is "inherit", which # only mount mode 1 has anything to inherit from. api_read_only: nil, # Per-token request ceiling. nil is off: the API adds no Redis work to a # request until a host asks for a limit, and no number picked here would # be right for both a cron relay and a fan-out producer. api_rate_limit: nil, api_rate_limit_interval: :minute, # Cluster-wide ceiling per queue (Wurk::QueueSlot), as # `{ 'critical' => 20 }`. Empty is the whole point of seeding it here: # the fetch path reads this once at boot, so an app that caps nothing # resolves to "no capped queues" before the first fetch and never asks # again. global_concurrency: {}, # See #swarm_crash_exit_after=. Long enough to outlast a slow rollout or a # dependency coming back, short enough that a bad release does not sit # looking Running for an afternoon. swarm_crash_exit_after: 600 }.freeze
- LIFECYCLE_EVENTS =
:fork fires only inside swarm children, after fork + internal AR/Redis reconnect — apps reopen sockets / non-fork-safe libs there (Ent §7.4).
%i[startup fork quiet shutdown exit heartbeat beat leader].freeze
- DEFAULT_THREAD_PRIORITY =
-1
- REDIS_ERROR_CLASSES =
Redis client / pool errors that the pool wrapper already retried before re-raising. Logged one level up (WARN, not INFO) so a transient blip surfaces in ops dashboards without drowning steady-state noise (#101). RedisClient + ConnectionPool are always loaded before this file (capsule → redis_pool requires both), so referencing them here is safe.
[RedisClient::Error, ConnectionPool::TimeoutError].freeze
- ERROR_HANDLER =
Default error handler. Wraps the report in the thread-local Wurk::Context so logger formatters/JSON layouts can pick up jid/bid/tags.
full_message(with backtrace) in dev/debug,detailed_messagein prod — mirrors the Sidekiq behavior so log scrapers built for one work for both.Spec: docs/target/sidekiq-free.md §4.3.
lambda do |ex, ctx, cfg = Wurk.configuration| safe_ctx = ctx || {} safe_ctx = cfg.redacted_context(safe_ctx) if cfg.respond_to?(:redacted_context) Wurk::Context.with(safe_ctx) do dev = $DEBUG || ENV['WURK_DEBUG'] || cfg.logger.debug? msg = dev ? ex. : ex. level = REDIS_ERROR_CLASSES.any? { |k| ex.is_a?(k) } ? :warn : :info cfg.logger.public_send(level) { msg } end end
- WEB_POOL_DEFAULT_SIZE =
Default connection count for the dedicated web pool (#web_redis_pool).
5- WEB_POOL_TIMEOUT =
Deliberately short checkout wait for the web pool: a saturated dashboard should fail fast rather than tie up a web-server thread queuing for a slot.
1.0- SWARM_HEARTBEAT_TIMEOUT_FLOOR =
Seconds a swarm child may go without a heartbeat before the parent replaces it (Swarm::Liveness). false or 0 turns the check off. The floor sits above the heartbeat's own cadence so a child that is merely busy is never read as hung.
20- HISTORY_DEFAULT_INTERVAL =
--- Historical metrics snapshotter (Ent §5) -------------------------
30- MEMORY_LIMIT_METRICS =
What
memory_limit_mbis compared against.:rss(the default, and Enterprise's) counts pages still shared copy-on-write with the parent, so every child carries the whole preloaded app;:psssplits shared pages across their sharers and:usscounts only the child's own. Wurk-only; WURK_MAXMEM_METRIC sets it, an explicit value wins. %i[rss pss uss].freeze
- METRICS_RETENTION_FLOOR =
Upstream's own TTL for the 10-minute buckets;
hours:windows read up to 72 hours of them, so nothing shorter is accepted. 3 * 24 * 60 * 60
Instance Attribute Summary collapse
-
#capsules ⇒ Object
readonly
Returns the value of attribute capsules.
-
#directory ⇒ Object
readonly
Returns the value of attribute directory.
-
#dogstatsd ⇒ Object
Pro parity: callable that builds the statsd / dogstatsd client.
-
#logger ⇒ Object
--- Logger -----------------------------------------------------------.
-
#periodic_manager ⇒ Object
readonly
The periodic manager if
config.periodicwas ever called, else nil: unlike #periodic, never creates one (Swarm::Prefork reads it). -
#redis_config ⇒ Object
readonly
Returns the value of attribute redis_config.
-
#super_fetch_callback ⇒ Object
readonly
Returns the value of attribute super_fetch_callback.
-
#thread_priority ⇒ Object
Returns the value of attribute thread_priority.
Instance Method Summary collapse
-
#[](key) ⇒ Object
--- Hash-like options access -----------------------------------------.
- #[]=(key, val) ⇒ Object
-
#api_enabled? ⇒ Boolean
Whether the HTTP API exists at all.
-
#api_enqueue_classes ⇒ Array<String>, ...
Classes the HTTP API may enqueue.
-
#api_enqueue_classes=(classes) ⇒ Object
The allow-list
JobUtil::TRANSIENT_ATTRIBUTESdeliberately is not — that is a strip-list, right for a producer already running your code:. -
#api_idempotency_ttl ⇒ Integer
Seconds an
Idempotency-Keystays replayable. - #api_idempotency_ttl=(seconds) ⇒ Object
-
#api_max_args_bytes ⇒ Integer
Largest
argsany single job may serialize to. - #api_max_args_bytes=(bytes) ⇒ Object
-
#api_max_body_bytes ⇒ Integer
Largest request body the produce plane will read.
- #api_max_body_bytes=(bytes) ⇒ Object
-
#api_rate_limit ⇒ Integer?
Requests one token may make per interval.
-
#api_rate_limit=(count) ⇒ Object
Requests per
api_rate_limit_interval, per token, across the fleet — enforced by aWurk::Limitersliding window rather than a second limiter, so an API ceiling shows up on the Limiters page beside every other one and resets on the same Redis clock. -
#api_rate_limit_interval ⇒ Symbol, Integer
The window the limit is counted over.
-
#api_rate_limit_interval=(interval) ⇒ Object
:second :minute :hour :dayor a raw Integer of seconds — whateverLimiter.windowaccepts, validated here so a typo raises in the initializer that wrote it rather than on the first throttled request. -
#api_read_only ⇒ Boolean?
True/false when the host said so, nil to inherit.
-
#api_read_only=(value) ⇒ Object
Freezes the API's writes — enqueue included, not only the destructive
adminroutes. -
#api_read_only? ⇒ Boolean
The resolved verdict for a mount with nothing to inherit; App layers mode 1's inherited flag over it.
-
#api_token(token, scopes:) ⇒ Object
Registers a bearer token for the HTTP API and the scopes it grants — any of
:enqueue,:read,:admin(which grants the other two):. -
#api_tokens ⇒ Hash{String => Array<Symbol>}
Registered tokens and their scopes.
-
#apply_periodic! ⇒ Object
Writes the
config.periodicregistrations to Redis. - #average_scheduled_poll_interval=(interval) ⇒ Object
- #capsule(name) {|cap| ... } ⇒ Object
-
#client_config_only? ⇒ Boolean
True once a configure_client block was skipped and no configure_server block ran: the server got none of the settings the app made for a role.
-
#client_middleware {|@client_chain| ... } ⇒ Object
--- Middleware -------------------------------------------------------.
-
#concurrency ⇒ Integer
Threads per worker process for the default capsule (default 5).
- #concurrency=(val) ⇒ Object
-
#configure_client ⇒ Object
A web process that also boots the swarm (Wurk::RailsBoot) is flagged server, yet every enqueue it makes from a request is client work, so
client_in_serverlets its configure_client blocks run too. -
#configure_server(&block) ⇒ Object
The gate is evaluated per call: an allowed block yields the configuration and returns the block's value.
- #death_handlers ⇒ Object
- #default_capsule ⇒ Object
- #dig(*keys) ⇒ Object
- #enter_server_mode ⇒ Object
-
#error_handlers ⇒ Object
--- Handlers ---------------------------------------------------------.
- #fetch ⇒ Object
- #fetch_poll_interval ⇒ Object
-
#fetch_poll_interval=(seconds) ⇒ Object
Reliable-fetch empty-poll backoff: the BLMOVE block timeout (seconds) used when every served queue is empty.
-
#freeze! ⇒ Object
Guarded on the capsule table, not
@frozen: a swarm child reaches this withprepare_for_fork!already run in its parent, so@frozenis true while the capsules it has just configured are still open. - #frozen? ⇒ Boolean
-
#global_concurrency ⇒ Hash{String => Integer}
Queue name → cluster-wide ceiling.
-
#global_concurrency=(caps) ⇒ Object
Caps how many jobs from a queue may run at once across the whole cluster, whatever the worker count:.
-
#global_concurrency? ⇒ Boolean
Whether any queue is capped.
- #handle_exception(ex, ctx = {}) ⇒ Object
-
#health_check(port:, bind: '0.0.0.0', ready_window: 30, metrics: true, min_ready: nil) ⇒ Object
Opt-in thin HTTP listener inside the worker process for k8s probes.
- #history_collector ⇒ Object
- #history_enabled? ⇒ Boolean
- #history_interval ⇒ Object
-
#initialize(options = {}) ⇒ Configuration
constructor
rubocop:disable Metrics/AbcSize.
- #inspect ⇒ Object
- #key?(key) ⇒ Boolean (also: #has_key?)
-
#lookup(name, default_class = nil) ⇒ Object
Memoizes on a miss, and the first miss can land long after boot (an extension resolved on its first tick), which is why
freeze!leaves@directorywritable: frozen, that lookup raised FrozenError instead of building the default. -
#memory_limit_kb ⇒ Object
Threshold in KB, the unit the swarm reads child memory in.
-
#memory_limit_mb ⇒ Object
Memory-based child recycling (Sidekiq Ent §7.5): the swarm parent TERMs (and respawns) any child whose RSS exceeds this many MB.
- #memory_limit_mb=(value) ⇒ Object
- #memory_limit_metric ⇒ Object
- #memory_limit_metric=(value) ⇒ Object
- #merge!(other) ⇒ Object
-
#metrics_retention=(seconds) ⇒ Object
How long the 10-minute
j|metrics buckets live. - #new_redis_pool(size, name = 'custom') ⇒ Object
-
#on(event, &block) ⇒ Object
--- Lifecycle hooks --------------------------------------------------.
-
#on_redis_error(&block) ⇒ Object
Telemetry hook fired by RedisPool on every transient-error retry and final give-up.
-
#periodic {|mgr| ... } ⇒ Wurk::Cron::Manager
Yields a Wurk::Cron::Manager so the host app can register periodic jobs at boot.
-
#prepare_for_fork! ⇒ Object
The pre-fork half of
freeze!, and the only half a forking parent can run: capsules stay writable until each child has applied its slot (ChildBoot#apply_slot_to_config) and opened its own Redis pools. - #queues ⇒ Object
- #queues=(val) ⇒ Object
-
#reap_idle_redis_connections(timeout = 60) ⇒ Object
Sidekiq's setter for
:redis_idle_timeout; read when a pool is built, so call it in the initializer. - #redact_args ⇒ Object
-
#redact_args=(hook) ⇒ Object
Display-side argument redaction (Wurk::Redact): a callable taking a deep copy of the job Hash and returning the args to show in logs, the dashboard and the fallback error log.
-
#redacted_context(ctx) ⇒ Object
The error context with the job's args passed through the
redact_argshook, for the two reporters that print or tag the whole context: the fallback log line and the default ERROR_HANDLER (whose Wurk::Context a JSON log formatter emits). - #redis(idempotent: false) ⇒ Object
-
#redis=(hash) ⇒ Object
Validated here, in the process running the initializer, rather than later in whichever process first builds a pool.
- #redis_error_handlers ⇒ Object
-
#redis_info ⇒ Object
Parsed INFO, without the pool's own counters RedisPool#info merges in.
- #redis_pool ⇒ Object
-
#register(name, instance) ⇒ Object
--- Service locator (extension registry) ----------------------------.
-
#reliable_scheduler! ⇒ Object
Pro reliable scheduler (§4): promote due jobs from retry/schedule onto their target queue in a single atomic Lua (ZRANGEBYSCORE+ZREM+LPUSH), much faster than the default poller but skipping client middleware on promotion.
-
#reload_redis_pools! ⇒ Object
Non-terminal counterpart, for a fork in a parent whose threads are live.
-
#reopen_in_forked_child ⇒ Object
A swarm child inherits the parent's prepare_for_fork! freeze, but its copy is its own, and Launcher#run freezes it again.
-
#report_fetch_error(exception) ⇒ Object
What the fetch loop reports through.
-
#reset_redis_pools! ⇒ Object
Disconnect and drop every capsule's cached pools (main + fetch) plus the web pool.
-
#retain_history(seconds = HISTORY_DEFAULT_INTERVAL, &block) ⇒ Object
Enables the Ent Historical Metrics snapshotter: every
secondsthe cluster leader emits a statsd-shaped snapshot to the configureddogstatsdclient. - #server? ⇒ Boolean
- #server_middleware {|@server_chain| ... } ⇒ Object
-
#super_fetch!(&block) ⇒ Object
Sidekiq Pro's opt-in toggle for reliable fetch.
- #swarm_crash_exit_after ⇒ Object
-
#swarm_crash_exit_after=(seconds) ⇒ Object
Seconds every swarm slot may spend crash-looping before the standalone runner reports it and exits non-zero.
- #swarm_heartbeat_timeout ⇒ Object
- #swarm_heartbeat_timeout=(seconds) ⇒ Object
-
#telemetry=(enabled) ⇒ Object
Opt into Wurk::Telemetry — a producer span carrying W3C trace context on the job hash at enqueue, a linked consumer span on execute:.
-
#telemetry? ⇒ Boolean
Whether the host asked for tracing.
- #to_json ⇒ Object
-
#topology ⇒ Object
Worker topology for the swarm.
- #topology=(value) ⇒ Object
-
#topology_assigned? ⇒ Boolean
Whether the app declared a topology, as opposed to the default one #topology builds.
- #total_concurrency ⇒ Object
-
#web ⇒ Object
Web UI configuration: the authorization hook and read-only mode.
-
#web_pool_size ⇒ Object
Connections in the dedicated web pool.
- #web_pool_size=(size) ⇒ Object
-
#web_redis_pool ⇒ Object
Dedicated Redis pool for the dashboard / JSON API / SSE, disjoint from every worker capsule's pool.
Constructor Details
#initialize(options = {}) ⇒ Configuration
rubocop:disable Metrics/AbcSize
162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 |
# File 'lib/wurk/configuration.rb', line 162 def initialize( = {}) # rubocop:disable Metrics/AbcSize @options = deep_dup_defaults.merge() # Through the same door `global_concurrency=` uses, so a cap passed to # `Configuration.new` is validated rather than trusted, and the default # Hash arrives frozen: the fetch path resolves caps once at boot, so a # Hash still mutable here is one a caller can add a queue to and have # nothing read it. FrozenError is the honest answer to that. @options[:error_handlers] << ERROR_HANDLER if @options[:error_handlers].empty? @capsules = {} @directory = {} @client_chain = Middleware::Chain.new(self) @server_chain = Middleware::Chain.new(self, on_registration: method(:warn_on_late_server_registration)) @redis_config = { url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/0') } @web_redis_pool = nil @logger = nil @thread_priority = DEFAULT_THREAD_PRIORITY @telemetry_installed = false @frozen = false @deferred_server_blocks = [] @fetch_error_coalescer = ErrorCoalescer.new end |
Instance Attribute Details
#capsules ⇒ Object (readonly)
Returns the value of attribute capsules.
141 142 143 |
# File 'lib/wurk/configuration.rb', line 141 def capsules @capsules end |
#directory ⇒ Object (readonly)
Returns the value of attribute directory.
141 142 143 |
# File 'lib/wurk/configuration.rb', line 141 def directory @directory end |
#dogstatsd ⇒ Object
Pro parity: callable that builds the statsd / dogstatsd client. Invoked once per process AFTER fork; see Wurk::Metrics::Statsd.client. Assignable as a Proc, lambda, or any object responding to #call:
config.dogstatsd = -> { Datadog::Statsd.new('host', 8125) }
Spec: docs/target/sidekiq-pro.md §9.1.
151 152 153 |
# File 'lib/wurk/configuration.rb', line 151 def dogstatsd @dogstatsd end |
#logger ⇒ Object
--- Logger -----------------------------------------------------------
812 813 814 |
# File 'lib/wurk/configuration.rb', line 812 def logger @logger ||= default_logger end |
#periodic_manager ⇒ Object (readonly)
The periodic manager if config.periodic was ever called, else nil:
unlike #periodic, never creates one (Swarm::Prefork reads it).
502 503 504 |
# File 'lib/wurk/configuration.rb', line 502 def periodic_manager @periodic_manager end |
#redis_config ⇒ Object (readonly)
Returns the value of attribute redis_config.
141 142 143 |
# File 'lib/wurk/configuration.rb', line 141 def redis_config @redis_config end |
#super_fetch_callback ⇒ Object (readonly)
Returns the value of attribute super_fetch_callback.
141 142 143 |
# File 'lib/wurk/configuration.rb', line 141 def super_fetch_callback @super_fetch_callback end |
#thread_priority ⇒ Object
Returns the value of attribute thread_priority.
142 143 144 |
# File 'lib/wurk/configuration.rb', line 142 def thread_priority @thread_priority end |
Instance Method Details
#[](key) ⇒ Object
--- Hash-like options access -----------------------------------------
187 |
# File 'lib/wurk/configuration.rb', line 187 def [](key) = @options.[](key) |
#[]=(key, val) ⇒ Object
189 190 191 192 193 194 |
# File 'lib/wurk/configuration.rb', line 189 def []=(key, val) guard_frozen! RedisPool.validate_idle_timeout(val) if key == :redis_idle_timeout val = parse_metrics_retention(val) if key == :metrics_retention && !val.nil? @options[key] = val end |
#api_enabled? ⇒ Boolean
Returns whether the HTTP API exists at all. The engine's mount is conditional on this.
605 |
# File 'lib/wurk/configuration.rb', line 605 def api_enabled? = !@options[:api_tokens].empty? |
#api_enqueue_classes ⇒ Array<String>, ...
Returns classes the HTTP API may enqueue.
629 |
# File 'lib/wurk/configuration.rb', line 629 def api_enqueue_classes = @options[:api_enqueue_classes] |
#api_enqueue_classes=(classes) ⇒ Object
The allow-list JobUtil::TRANSIENT_ATTRIBUTES deliberately is not — that
is a strip-list, right for a producer already running your code:
config.api_enqueue_classes = %w[Billing::Charge ReportJob]
Unset, only an admin token may enqueue at all. class is a code
selector — a producer token that can name any constant in the host is
choosing which of its jobs to run — so an enqueue-scoped credential has
to be told exactly what it is for. Once a list exists it binds every
token, admin included: a host that writes one means it.
:any turns the check off, which is what an enqueue-scoped relay that
legitimately pushes arbitrary classes needs — the alternative is handing
it admin and the destructive routes that come with it.
645 646 647 648 649 |
# File 'lib/wurk/configuration.rb', line 645 def api_enqueue_classes=(classes) guard_frozen! require_relative 'api/validation' @options[:api_enqueue_classes] = Wurk::API::Validation.enqueueable_classes!(classes) end |
#api_idempotency_ttl ⇒ Integer
Returns seconds an Idempotency-Key stays replayable.
622 |
# File 'lib/wurk/configuration.rb', line 622 def api_idempotency_ttl = @options[:api_idempotency_ttl] |
#api_idempotency_ttl=(seconds) ⇒ Object
624 625 626 |
# File 'lib/wurk/configuration.rb', line 624 def api_idempotency_ttl=(seconds) @options[:api_idempotency_ttl] = api_limit!(:api_idempotency_ttl, seconds) end |
#api_max_args_bytes ⇒ Integer
Returns largest args any single job may serialize to.
615 |
# File 'lib/wurk/configuration.rb', line 615 def api_max_args_bytes = @options[:api_max_args_bytes] |
#api_max_args_bytes=(bytes) ⇒ Object
617 618 619 |
# File 'lib/wurk/configuration.rb', line 617 def api_max_args_bytes=(bytes) @options[:api_max_args_bytes] = api_limit!(:api_max_args_bytes, bytes) end |
#api_max_body_bytes ⇒ Integer
Returns largest request body the produce plane will read.
608 |
# File 'lib/wurk/configuration.rb', line 608 def api_max_body_bytes = @options[:api_max_body_bytes] |
#api_max_body_bytes=(bytes) ⇒ Object
610 611 612 |
# File 'lib/wurk/configuration.rb', line 610 def api_max_body_bytes=(bytes) @options[:api_max_body_bytes] = api_limit!(:api_max_body_bytes, bytes) end |
#api_rate_limit ⇒ Integer?
Returns requests one token may make per interval.
685 |
# File 'lib/wurk/configuration.rb', line 685 def api_rate_limit = @options[:api_rate_limit] |
#api_rate_limit=(count) ⇒ Object
Requests per api_rate_limit_interval, per token, across the fleet —
enforced by a Wurk::Limiter sliding window rather than a second
limiter, so an API ceiling shows up on the Limiters page beside every
other one and resets on the same Redis clock. nil turns it off.
691 692 693 694 |
# File 'lib/wurk/configuration.rb', line 691 def api_rate_limit=(count) guard_frozen! @options[:api_rate_limit] = count.nil? ? nil : api_limit!(:api_rate_limit, count) end |
#api_rate_limit_interval ⇒ Symbol, Integer
Returns the window the limit is counted over.
697 |
# File 'lib/wurk/configuration.rb', line 697 def api_rate_limit_interval = @options[:api_rate_limit_interval] |
#api_rate_limit_interval=(interval) ⇒ Object
:second :minute :hour :day or a raw Integer of seconds — whatever
Limiter.window accepts, validated here so a typo raises in the
initializer that wrote it rather than on the first throttled request.
702 703 704 705 706 |
# File 'lib/wurk/configuration.rb', line 702 def api_rate_limit_interval=(interval) guard_frozen! Wurk::Limiter.interval_seconds(interval, allow_integer: true) @options[:api_rate_limit_interval] = interval end |
#api_read_only ⇒ Boolean?
Returns true/false when the host said so, nil to inherit.
652 |
# File 'lib/wurk/configuration.rb', line 652 def api_read_only = @options[:api_read_only] |
#api_read_only=(value) ⇒ Object
Freezes the API's writes — enqueue included, not only the destructive
admin routes. Same rule as the dashboard's Wurk::Web.config.read_only
and deliberately the same sentence: a read-only deployment answers reads.
Carving enqueue out would mean a viewer-only deploy could still be made
to run arbitrary work, and would leave an operator holding two different
definitions of the same word.
Three states, because the two planes are not always one deployment:
nil (default) inherit. Mounted inside the engine, the API is part of
the dashboard's deployment and `WURK_WEB_READ_ONLY=1`
freezes both. Mounted on its own path or run standalone
there is nothing to inherit from and this means off.
true frozen wherever it is mounted. `WURK_API_READ_ONLY=1`
sets it, which is the only door mode 3 has.
false live even inside a read-only dashboard — the host that
wants a viewer-only UI and a working producer on the
same mount, said out loud.
672 673 674 675 |
# File 'lib/wurk/configuration.rb', line 672 def api_read_only=(value) guard_frozen! @options[:api_read_only] = value.nil? ? nil : truthy_setting?(value) end |
#api_read_only? ⇒ Boolean
Returns the resolved verdict for a mount with nothing to inherit; App layers mode 1's inherited flag over it.
679 680 681 682 |
# File 'lib/wurk/configuration.rb', line 679 def api_read_only? value = @options[:api_read_only] value.nil? ? ENV['WURK_API_READ_ONLY'] == '1' : value end |
#api_token(token, scopes:) ⇒ Object
Registers a bearer token for the HTTP API and the scopes it grants —
any of :enqueue, :read, :admin (which grants the other two):
# config/initializers/wurk.rb
Wurk.configuration.api_token ENV.fetch('WURK_API_TOKEN'), scopes: %i[enqueue read]
Registered unconditionally, not inside configure_server/configure_client
— the process that serves the API has to be the one holding the credential,
and neither block runs in every such process. A Puma-cluster web process
never enters server mode (Wurk::RailsBoot refuses to fork a swarm from
one), so a token registered in configure_server would be absent from the
exact process serving the engine-nested mount.
Registering the first token is what brings the API into existence; with
none the app is never mounted. Re-registering a token replaces its
scopes. Loading wurk/api/auth is deferred to this call so a host that
serves no HTTP API never pays for openssl/digest in every swarm child.
592 593 594 595 596 597 598 |
# File 'lib/wurk/configuration.rb', line 592 def api_token(token, scopes:) guard_frozen! require_relative 'api/auth' value, granted = Wurk::API::Auth.credential!(token, scopes) @options[:api_tokens][value] = granted nil end |
#api_tokens ⇒ Hash{String => Array<Symbol>}
Returns registered tokens and their scopes.
601 |
# File 'lib/wurk/configuration.rb', line 601 def api_tokens = @options[:api_tokens] |
#apply_periodic! ⇒ Object
Writes the config.periodic registrations to Redis. Launcher#boot calls
it, so the loops are registered by a booting server, against the Redis
settings the app ended up with.
507 508 509 510 |
# File 'lib/wurk/configuration.rb', line 507 def apply_periodic! @periodic_manager&.apply! nil end |
#average_scheduled_poll_interval=(interval) ⇒ Object
380 381 382 |
# File 'lib/wurk/configuration.rb', line 380 def average_scheduled_poll_interval=(interval) @options[:average_scheduled_poll_interval] = interval end |
#capsule(name) {|cap| ... } ⇒ Object
237 238 239 240 241 242 |
# File 'lib/wurk/configuration.rb', line 237 def capsule(name) name = name.to_s cap = @capsules[name] ||= Capsule.new(name, self) yield cap if block_given? cap end |
#client_config_only? ⇒ Boolean
True once a configure_client block was skipped and no configure_server block ran: the server got none of the settings the app made for a role.
928 929 930 |
# File 'lib/wurk/configuration.rb', line 928 def client_config_only? @configure_client_skip_warned == true && !@configure_server_ran end |
#client_middleware {|@client_chain| ... } ⇒ Object
--- Middleware -------------------------------------------------------
246 247 248 249 |
# File 'lib/wurk/configuration.rb', line 246 def client_middleware yield @client_chain if block_given? @client_chain end |
#concurrency ⇒ Integer
Returns threads per worker process for the default capsule
(default 5). The process count is separate — set via WURK_COUNT
(defaults to the CPU count). With a single capsule, total in-flight
jobs = WURK_COUNT × concurrency; with multiple capsules see
#total_concurrency for the cluster aggregate.
216 |
# File 'lib/wurk/configuration.rb', line 216 def concurrency = default_capsule.concurrency |
#concurrency=(val) ⇒ Object
219 220 221 |
# File 'lib/wurk/configuration.rb', line 219 def concurrency=(val) default_capsule.concurrency = val end |
#configure_client ⇒ Object
A web process that also boots the swarm (Wurk::RailsBoot) is flagged
server, yet every enqueue it makes from a request is client work, so
client_in_server lets its configure_client blocks run too.
In a server, skipping the client block is the Sidekiq contract and the usual case: an app configures both roles. So the skip itself is DEBUG, and BootChecks warns at boot only for the case worth one, a server process in which no configure_server block ran at all (#816).
915 916 917 918 919 920 921 922 923 924 |
# File 'lib/wurk/configuration.rb', line 915 def configure_client return unless block_given? return yield self if !server? || @options[:client_in_server] return if @configure_client_skip_warned @configure_client_skip_warned = true logger.debug { 'configure_client block skipped: server? is true and client_in_server is false or nil' } nil end |
#configure_server(&block) ⇒ Object
The gate is evaluated per call: an allowed block yields the configuration and returns the block's value. A skip reports itself once per gate per instance — the flags below decide only whether the skip is reported again, and they live outside @options so a frozen configuration can still warn.
A skipped block is kept for #enter_server_mode. Gems call this at load
time, and under the railtie that is Bundler.require, before the
wurk.server_mode initializer: dropping it left sidekiq-cron, sentry-sidekiq
and the like silently absent from a process that does run workers.
So the skip is DEBUG, as in stock Sidekiq, which skips it silently: every
worker entry point enters server mode and runs the kept blocks, and a
process that never does (a test run, rails runner, a web process) is
meant to skip them. A WARN here was noise in each of those.
890 891 892 893 894 895 896 897 898 899 900 |
# File 'lib/wurk/configuration.rb', line 890 def configure_server(&block) return unless block return run_server_block(block) if server? @deferred_server_blocks << block return if @configure_server_skip_warned @configure_server_skip_warned = true logger.debug { 'configure_server block skipped: server? is false' } nil end |
#death_handlers ⇒ Object
363 364 365 |
# File 'lib/wurk/configuration.rb', line 363 def death_handlers @options[:death_handlers] end |
#default_capsule ⇒ Object
233 234 235 |
# File 'lib/wurk/configuration.rb', line 233 def default_capsule(&) capsule('default', &) end |
#dig(*keys) ⇒ Object
206 |
# File 'lib/wurk/configuration.rb', line 206 def dig(*keys) = @options.dig(*keys) |
#enter_server_mode ⇒ Object
902 903 904 905 |
# File 'lib/wurk/configuration.rb', line 902 def enter_server_mode self[:server] = true run_server_block(@deferred_server_blocks.shift) until @deferred_server_blocks.empty? end |
#error_handlers ⇒ Object
--- Handlers ---------------------------------------------------------
359 360 361 |
# File 'lib/wurk/configuration.rb', line 359 def error_handlers @options[:error_handlers] end |
#fetch ⇒ Object
196 |
# File 'lib/wurk/configuration.rb', line 196 def fetch(*, &) = @options.fetch(*, &) |
#fetch_poll_interval ⇒ Object
393 394 395 |
# File 'lib/wurk/configuration.rb', line 393 def fetch_poll_interval @options[:fetch_poll_interval] end |
#fetch_poll_interval=(seconds) ⇒ Object
Reliable-fetch empty-poll backoff: the BLMOVE block timeout (seconds)
used when every served queue is empty. Pro super_fetch §3.3's
fetch_poll_interval knob. Unset (nil) → the fetcher's default
(Wurk::Fetcher::Reliable::TIMEOUT, 2s). Also readable as
config[:fetch_poll_interval].
389 390 391 |
# File 'lib/wurk/configuration.rb', line 389 def fetch_poll_interval=(seconds) @options[:fetch_poll_interval] = seconds end |
#freeze! ⇒ Object
Guarded on the capsule table, not @frozen: a swarm child reaches this
with prepare_for_fork! already run in its parent, so @frozen is true
while the capsules it has just configured are still open.
1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 |
# File 'lib/wurk/configuration.rb', line 1039 def freeze! return self if @capsules.frozen? prepare_for_fork! @capsules.each_value(&:freeze) @capsules.freeze warn_on_missing_server_middleware warn_on_small_db_pool self end |
#frozen? ⇒ Boolean
1060 1061 1062 |
# File 'lib/wurk/configuration.rb', line 1060 def frozen? @frozen end |
#global_concurrency ⇒ Hash{String => Integer}
Returns queue name → cluster-wide ceiling. Always frozen — the default included, normalized in #initialize; see #global_concurrency=.
713 |
# File 'lib/wurk/configuration.rb', line 713 def global_concurrency = @options[:global_concurrency] |
#global_concurrency=(caps) ⇒ Object
Caps how many jobs from a queue may run at once across the whole cluster, whatever the worker count:
Wurk.configure_server { |config| config.global_concurrency = { critical: 20 } }
Not the same thing as concurrency, which is threads in one process, nor
as a Limiter, which is keyed per lock and gates inside the job
(by which point the job is already claimed off the queue). This is a
ceiling on the queue itself, enforced where the work is picked up.
Names are normalized to Strings and the result is frozen, because the fetch path resolves it once at boot: a cap added by mutating this Hash afterwards would be read by nothing, so it raises rather than doing nothing. Assign a new Hash to change one.
729 730 731 732 |
# File 'lib/wurk/configuration.rb', line 729 def global_concurrency=(caps) guard_frozen! @options[:global_concurrency] = normalize_global_concurrency(caps) end |
#global_concurrency? ⇒ Boolean
Returns whether any queue is capped. The one question the fetcher asks at boot to decide whether it has a gate to run at all.
736 |
# File 'lib/wurk/configuration.rb', line 736 def global_concurrency? = !@options[:global_concurrency].empty? |
#handle_exception(ex, ctx = {}) ⇒ Object
818 819 820 821 822 823 824 825 826 827 828 829 |
# File 'lib/wurk/configuration.rb', line 818 def handle_exception(ex, ctx = {}) ctx = redact_jobstr(ctx) if error_handlers.empty? logger.error("#{redacted_context(ctx)} #{ex.class}: #{ex.}") else error_handlers.each do |handler| handler.call(ex, ctx, self) rescue StandardError => e logger.error("error_handler raised: #{e.class}: #{e.}") end end end |
#health_check(port:, bind: '0.0.0.0', ready_window: 30, metrics: true, min_ready: nil) ⇒ Object
Opt-in thin HTTP listener inside the worker process for k8s probes.
When called, the Launcher will start a TCP server on port bound to
bind exposing GET /live (200 while not stopping) and GET /ready
(200 only when Redis is reachable AND heartbeat fired within
ready_window seconds).
Off by default. Put this in Rails config/initializers/wurk.rb loaded
by the worker boot, or in the Ruby boot file loaded via -r by wurk,
wurkswarm, or sidekiqswarm:
Wurk.configure_server do |config|
config.health_check(port: 7433)
end
configure_server executes its block only when the current process is
already in server mode. Configuration is process-local: configuring a
separate web/client process cannot enable the worker's probes.
For explicit embedding, configure the embedded worker directly before
calling run; configure_embed does not enter server mode:
= Wurk. do |config|
config.health_check(port: 7433)
end
.run
Spec: docs/target/sidekiq-ent.md §7.1.2.
metrics: serves the Prometheus scrape on the same listener (on by
default). min_ready: — children with a fresh heartbeat required for
/ready inside a swarm; nil means half the swarm, rounded up.
784 785 786 787 788 789 790 791 792 793 794 795 796 797 |
# File 'lib/wurk/configuration.rb', line 784 def health_check(port:, bind: '0.0.0.0', ready_window: 30, metrics: true, min_ready: nil) guard_frozen! p = Integer(port) rw = Integer(ready_window) raise ArgumentError, 'port must be between 0 and 65535' unless (0..65_535).cover?(p) raise ArgumentError, 'ready_window must be > 0' unless rw.positive? b = bind.to_s raise ArgumentError, 'bind must be a non-empty string' if b.empty? @options[:health_check_options] = { port: p, bind: b, ready_window: rw, metrics: health_check_metrics(metrics), min_ready: health_check_min_ready(min_ready) } end |
#history_collector ⇒ Object
536 |
# File 'lib/wurk/configuration.rb', line 536 def history_collector = @options[:history_collector] |
#history_enabled? ⇒ Boolean
534 |
# File 'lib/wurk/configuration.rb', line 534 def history_enabled? = @options.key?(:history_interval) |
#history_interval ⇒ Object
535 |
# File 'lib/wurk/configuration.rb', line 535 def history_interval = @options.fetch(:history_interval, HISTORY_DEFAULT_INTERVAL) |
#inspect ⇒ Object
1064 1065 1066 |
# File 'lib/wurk/configuration.rb', line 1064 def inspect "#<#{self.class} capsules=#{@capsules.keys} concurrency=#{total_concurrency}>" end |
#key?(key) ⇒ Boolean Also known as: has_key?
197 |
# File 'lib/wurk/configuration.rb', line 197 def key?(key) = @options.key?(key) |
#lookup(name, default_class = nil) ⇒ Object
Memoizes on a miss, and the first miss can land long after boot (an
extension resolved on its first tick), which is why freeze! leaves
@directory writable: frozen, that lookup raised FrozenError instead of
building the default. register — the host-facing half — still refuses
writes past the freeze, so the closed surface is unchanged.
353 354 355 |
# File 'lib/wurk/configuration.rb', line 353 def lookup(name, default_class = nil) @directory[name] ||= default_class&.new end |
#memory_limit_kb ⇒ Object
Threshold in KB, the unit the swarm reads child memory in. nil when recycling is disabled.
969 970 971 972 |
# File 'lib/wurk/configuration.rb', line 969 def memory_limit_kb mb = memory_limit_mb mb&.positive? ? mb * 1024 : nil end |
#memory_limit_mb ⇒ Object
Memory-based child recycling (Sidekiq Ent §7.5): the swarm parent TERMs (and respawns) any child whose RSS exceeds this many MB. Set in code or via SIDEKIQ_MAXMEM_MB (WURK_MAXMEM_MB is the native alias); an explicit value wins over the env. nil/0 disables recycling (the default).
958 959 960 |
# File 'lib/wurk/configuration.rb', line 958 def memory_limit_mb @memory_limit_mb || env_memory_limit_mb end |
#memory_limit_mb=(value) ⇒ Object
962 963 964 965 |
# File 'lib/wurk/configuration.rb', line 962 def memory_limit_mb=(value) guard_frozen! @memory_limit_mb = value.nil? ? nil : Integer(value) end |
#memory_limit_metric ⇒ Object
1003 1004 1005 |
# File 'lib/wurk/configuration.rb', line 1003 def memory_limit_metric @memory_limit_metric || env_memory_limit_metric || :rss end |
#memory_limit_metric=(value) ⇒ Object
1007 1008 1009 1010 |
# File 'lib/wurk/configuration.rb', line 1007 def memory_limit_metric=(value) guard_frozen! @memory_limit_metric = value.nil? ? nil : parse_memory_limit_metric(value) end |
#merge!(other) ⇒ Object
199 200 201 202 203 204 |
# File 'lib/wurk/configuration.rb', line 199 def merge!(other) guard_frozen! RedisPool.validate_idle_timeout(other[:redis_idle_timeout]) if other.key?(:redis_idle_timeout) parse_metrics_retention(other[:metrics_retention]) unless other[:metrics_retention].nil? @options.merge!(other) end |
#metrics_retention=(seconds) ⇒ Object
How long the 10-minute j| metrics buckets live. A TTL only: the keys,
fields and scores stay Sidekiq's. Each live job class costs 3 fields per
bucket, one bucket per 10 minutes, so retention scales those keys
linearly (docs/metrics.md, "Redis footprint and retention"). nil restores
the 30-day default.
990 991 992 993 |
# File 'lib/wurk/configuration.rb', line 990 def metrics_retention=(seconds) guard_frozen! @options[:metrics_retention] = seconds.nil? ? nil : parse_metrics_retention(seconds) end |
#new_redis_pool(size, name = 'custom') ⇒ Object
300 301 302 |
# File 'lib/wurk/configuration.rb', line 300 def new_redis_pool(size, name = 'custom') build_redis_pool(size: size, name: name) end |
#on(event, &block) ⇒ Object
--- Lifecycle hooks --------------------------------------------------
801 802 803 804 805 806 807 808 |
# File 'lib/wurk/configuration.rb', line 801 def on(event, &block) raise ArgumentError, "block required for on(#{event.inspect})" unless block unless LIFECYCLE_EVENTS.include?(event) raise ArgumentError, "invalid event #{event.inspect}, must be one of #{LIFECYCLE_EVENTS.inspect}" end @options[:lifecycle_events][event] << block end |
#on_redis_error(&block) ⇒ Object
Telemetry hook fired by RedisPool on every transient-error retry and final give-up. The block receives one Hash: { error:, attempt:, retried:, pool: }. Opt-in — pools stay silent until a handler is registered.
370 371 372 373 374 |
# File 'lib/wurk/configuration.rb', line 370 def on_redis_error(&block) raise ArgumentError, 'block required for on_redis_error' unless block @options[:redis_error_handlers] << block end |
#periodic {|mgr| ... } ⇒ Wurk::Cron::Manager
Yields a Wurk::Cron::Manager so the host app can register periodic
jobs at boot. Manager state is shared per-process so multiple
config.periodic blocks accumulate (matches Sidekiq Ent §2.1). This is
the native replacement for the sidekiq-cron gem. Nothing reaches
Redis until the server boots (#apply_periodic!).
Spec: docs/target/sidekiq-ent.md §2.
494 495 496 497 498 |
# File 'lib/wurk/configuration.rb', line 494 def periodic @periodic_manager ||= Wurk::Cron::Manager.new(self, deferred: true) yield @periodic_manager if block_given? @periodic_manager end |
#prepare_for_fork! ⇒ Object
The pre-fork half of freeze!, and the only half a forking parent can
run: capsules stay writable until each child has applied its slot
(ChildBoot#apply_slot_to_config) and opened its own Redis pools. What is
left is slot-independent — the options Hash and every capsule's middleware
chains — so the swarm parent settles it once and every child inherits the
result copy-on-write instead of allocating and dirtying its own copy.
Freezing the options here also makes a post-fork option write raise in the
child that wrote it, rather than silently diverging from its siblings.
@directory is deliberately left out — see #lookup.
1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 |
# File 'lib/wurk/configuration.rb', line 1022 def prepare_for_fork! # The capsule every swarm child configures (ChildBoot reaches for it by # name), and the one a client-only config builds on its first enqueue. # Materialized here so its chains are shared rather than rebuilt N times # — and so `freeze!` can't close `@capsules` around a name that is only # ever resolved later, which turned that first resolution into a # FrozenError on the frozen Hash. default_capsule @capsules.each_value(&:prepare_shared!) @options.freeze @frozen = true self end |
#queues ⇒ Object
223 |
# File 'lib/wurk/configuration.rb', line 223 def queues = default_capsule.queues |
#queues=(val) ⇒ Object
225 226 227 |
# File 'lib/wurk/configuration.rb', line 225 def queues=(val) default_capsule.queues = val end |
#reap_idle_redis_connections(timeout = 60) ⇒ Object
Sidekiq's setter for :redis_idle_timeout; read when a pool is built, so
call it in the initializer.
272 273 274 |
# File 'lib/wurk/configuration.rb', line 272 def reap_idle_redis_connections(timeout = 60) self[:redis_idle_timeout] = timeout end |
#redact_args ⇒ Object
406 407 408 |
# File 'lib/wurk/configuration.rb', line 406 def redact_args @options[:redact_args] end |
#redact_args=(hook) ⇒ Object
Display-side argument redaction (Wurk::Redact): a callable taking a deep copy of the job Hash and returning the args to show in logs, the dashboard and the fallback error log. nil (default) shows args as-is.
400 401 402 403 404 |
# File 'lib/wurk/configuration.rb', line 400 def redact_args=(hook) raise ArgumentError, 'redact_args must be nil or respond to #call' unless hook.nil? || hook.respond_to?(:call) @options[:redact_args] = hook end |
#redacted_context(ctx) ⇒ Object
The error context with the job's args passed through the redact_args
hook, for the two reporters that print or tag the whole context: the
fallback log line and the default ERROR_HANDLER (whose Wurk::Context a
JSON log formatter emits). Host-registered error handlers still receive
the real job — they are the host's own code (jobstr is another matter:
see #redact_jobstr).
851 852 853 854 855 856 857 858 |
# File 'lib/wurk/configuration.rb', line 851 def redacted_context(ctx) hook = @options[:redact_args] job = ctx.is_a?(Hash) ? ctx[:job] : nil return ctx unless hook && job.is_a?(Hash) && job.key?('args') require_relative 'redact' ctx.merge(job: job.merge('args' => Wurk::Redact.args(job, hook))) end |
#redis(idempotent: false) ⇒ Object
304 305 306 |
# File 'lib/wurk/configuration.rb', line 304 def redis(idempotent: false, &) PoolCheckout.trusted(redis_pool, idempotent, &) end |
#redis=(hash) ⇒ Object
Validated here, in the process running the initializer, rather than later in whichever process first builds a pool. The swarm's children are the ones that construct pools, so a bad key used to kill every child on boot while the parent stayed up and healthy — Running pod, passing probe, zero jobs processed (#283).
263 264 265 266 267 268 |
# File 'lib/wurk/configuration.rb', line 263 def redis=(hash) guard_frozen! RedisOptions.validate!(hash) warn_on_late_redis_assignment @redis_config = @redis_config.merge(hash.transform_keys(&:to_sym)) end |
#redis_error_handlers ⇒ Object
376 377 378 |
# File 'lib/wurk/configuration.rb', line 376 def redis_error_handlers @options[:redis_error_handlers] end |
#redis_info ⇒ Object
Parsed INFO, without the pool's own counters RedisPool#info merges in.
281 282 283 |
# File 'lib/wurk/configuration.rb', line 281 def redis_info redis_pool.info.except('size', 'available') end |
#redis_pool ⇒ Object
276 277 278 |
# File 'lib/wurk/configuration.rb', line 276 def redis_pool default_capsule.redis_pool end |
#register(name, instance) ⇒ Object
--- Service locator (extension registry) ----------------------------
343 344 345 346 |
# File 'lib/wurk/configuration.rb', line 343 def register(name, instance) guard_frozen! @directory[name] = instance end |
#reliable_scheduler! ⇒ Object
Pro reliable scheduler (§4): promote due jobs from retry/schedule onto
their target queue in a single atomic Lua (ZRANGEBYSCORE+ZREM+LPUSH),
much faster than the default poller but skipping client middleware on
promotion. Swaps the pluggable scheduled_enq for the atomic promoter;
idempotent.
470 471 472 473 |
# File 'lib/wurk/configuration.rb', line 470 def reliable_scheduler!(*) self[:scheduled_enq] = Wurk::Scheduled::ReliableEnq nil end |
#reload_redis_pools! ⇒ Object
Non-terminal counterpart, for a fork in a parent whose threads are live.
295 296 297 298 |
# File 'lib/wurk/configuration.rb', line 295 def reload_redis_pools! @capsules.each_value(&:reload_redis_pools!) @web_redis_pool&.reload! end |
#reopen_in_forked_child ⇒ Object
A swarm child inherits the parent's prepare_for_fork! freeze, but its copy
is its own, and Launcher#run freezes it again. In between it is what a
non-swarm process is: open to :fork / :startup hooks and to gems that
prepend Launcher#initialize and write an option, as sidekiq-cron does.
1054 1055 1056 1057 1058 |
# File 'lib/wurk/configuration.rb', line 1054 def reopen_in_forked_child @options = @options.dup @frozen = false self end |
#report_fetch_error(exception) ⇒ Object
What the fetch loop reports through. It sleeps 1s and retries on every thread of every worker, so an outage would otherwise call each handler once per thread per second. The count lands in the metric, the handlers see the first of each exception class and one per window after that.
835 836 837 838 839 840 841 842 843 |
# File 'lib/wurk/configuration.rb', line 835 def report_fetch_error(exception) Metrics::ProcessCounters.increment(:fetch_errors) suppressed = @fetch_error_coalescer.admit(exception.class) return unless suppressed context = { context: 'Error fetching job' } context[:suppressed] = suppressed if suppressed.positive? handle_exception(exception, context) end |
#reset_redis_pools! ⇒ Object
Disconnect and drop every capsule's cached pools (main + fetch) plus the web pool. Used by Wurk::Swarm so the parent never leaks sockets into forks and each child can build fresh ones.
288 289 290 291 292 |
# File 'lib/wurk/configuration.rb', line 288 def reset_redis_pools! @capsules.each_value(&:reset_redis_pools!) @web_redis_pool&.disconnect! @web_redis_pool = nil end |
#retain_history(seconds = HISTORY_DEFAULT_INTERVAL, &block) ⇒ Object
Enables the Ent Historical Metrics snapshotter: every seconds the
cluster leader emits a statsd-shaped snapshot to the configured
dogstatsd client. With no block the default §5.2 gauge set is
published; a block receives the dogstatsd client s and collects
custom metrics instead. The Launcher starts the snapshotter only when
this has been called.
Spec: docs/target/sidekiq-ent.md §5.1.
524 525 526 527 528 529 530 531 532 |
# File 'lib/wurk/configuration.rb', line 524 def retain_history(seconds = HISTORY_DEFAULT_INTERVAL, &block) guard_frozen! interval = Float(seconds) raise ArgumentError, 'retain_history interval must be > 0' unless interval.positive? @options[:history_interval] = interval @options[:history_collector] = block nil end |
#server? ⇒ Boolean
932 933 934 |
# File 'lib/wurk/configuration.rb', line 932 def server? @options[:server] == true end |
#server_middleware {|@server_chain| ... } ⇒ Object
251 252 253 254 |
# File 'lib/wurk/configuration.rb', line 251 def server_middleware yield @server_chain if block_given? @server_chain end |
#super_fetch!(&block) ⇒ Object
Sidekiq Pro's opt-in toggle for reliable fetch. Already the default in Wurk — the fetcher is always the reliable BLMOVE fetcher with orphan reclamation — so the toggle is a no-op beyond capturing the recovery callback. It exists so a Pro initializer drops in unchanged instead of raising NoMethodError.
NOTE: reliable_scheduler! below is NOT a no-op: it swaps in the atomic
promoter, much faster but skipping client middleware on promotion.
The optional block is Pro's recovery callback: |jobstr, pill|, fired
once per orphan recovery (pill nil) and once on a poison kill (pill
responds to .jid/.klass/.count/.queue). The reaper drives it via
Wurk::Middleware::PoisonPill.recovered!. Spec: docs/target/sidekiq-pro.md §3.1.
460 461 462 463 |
# File 'lib/wurk/configuration.rb', line 460 def super_fetch!(*, &block) @super_fetch_callback = block if block nil end |
#swarm_crash_exit_after ⇒ Object
441 442 443 |
# File 'lib/wurk/configuration.rb', line 441 def swarm_crash_exit_after @options[:swarm_crash_exit_after] end |
#swarm_crash_exit_after=(seconds) ⇒ Object
Seconds every swarm slot may spend crash-looping before the standalone runner reports it and exits non-zero. nil or false keeps respawning forever.
433 434 435 436 437 438 439 |
# File 'lib/wurk/configuration.rb', line 433 def swarm_crash_exit_after=(seconds) unless seconds.nil? || seconds == false || (seconds.is_a?(Numeric) && seconds.positive?) raise ArgumentError, "swarm_crash_exit_after must be nil, false, or a positive number (got #{seconds.inspect})" end @options[:swarm_crash_exit_after] = seconds || nil end |
#swarm_heartbeat_timeout ⇒ Object
427 428 429 |
# File 'lib/wurk/configuration.rb', line 427 def swarm_heartbeat_timeout @options[:swarm_heartbeat_timeout] end |
#swarm_heartbeat_timeout=(seconds) ⇒ Object
416 417 418 419 420 421 422 423 424 425 |
# File 'lib/wurk/configuration.rb', line 416 def swarm_heartbeat_timeout=(seconds) unless seconds == false || seconds == 0 || # rubocop:disable Style/NumericPredicate ((seconds.is_a?(Integer) || seconds.is_a?(Float)) && seconds >= SWARM_HEARTBEAT_TIMEOUT_FLOOR) raise ArgumentError, "swarm_heartbeat_timeout must be false, 0, or a number >= #{SWARM_HEARTBEAT_TIMEOUT_FLOOR} " \ "(got #{seconds.inspect})" end @options[:swarm_heartbeat_timeout] = seconds end |
#telemetry=(enabled) ⇒ Object
Opt into Wurk::Telemetry — a producer span carrying W3C trace context on the job hash at enqueue, a linked consumer span on execute:
Wurk.configure_server { |config| config.telemetry = true }
Loading wurk/telemetry is deferred to this call, which is the only
reason require "wurk" can leave opentelemetry-api untouched in an app
that doesn't trace.
Warns when the host opted in but the gem is missing: tracing then stays off, and a silent no-op is indistinguishable from a broken exporter.
555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 |
# File 'lib/wurk/configuration.rb', line 555 def telemetry=(enabled) guard_frozen! @options[:telemetry] = enabled ? true : false # Turning tracing back off still has to unregister, but must not *load* # the module to do it — that require is the whole cost this flag gates, # and a config that never opted in has nothing registered to undo. Tracked # per instance rather than off `defined?(Wurk::Telemetry)`: whether some # *other* config loaded the module says nothing about this one's chain. return if !@options[:telemetry] && !@telemetry_installed require_relative 'telemetry' @telemetry_installed = true Wurk::Telemetry.install!(self) return if !@options[:telemetry] || Wurk::Telemetry.available? logger.warn { 'config.telemetry = true but opentelemetry-api is not installed; tracing stays off' } end |
#telemetry? ⇒ Boolean
Returns whether the host asked for tracing. Half the gate; Telemetry.enabled? is the whole one.
542 |
# File 'lib/wurk/configuration.rb', line 542 def telemetry? = @options[:telemetry] == true |
#to_json ⇒ Object
207 |
# File 'lib/wurk/configuration.rb', line 207 def to_json(*) = Wurk.dump_json(@options) |
#topology ⇒ Object
Worker topology for the swarm. When the host hasn't declared one (the
railtie path), default to a single flat fork running the default
capsule's queues + concurrency. Assign a custom Wurk::Topology (via
topology=) for specialized slots.
940 941 942 |
# File 'lib/wurk/configuration.rb', line 940 def topology @topology ||= default_topology end |
#topology=(value) ⇒ Object
944 945 946 947 948 |
# File 'lib/wurk/configuration.rb', line 944 def topology=(value) guard_frozen! @topology_assigned = true @topology = value end |
#topology_assigned? ⇒ Boolean
Whether the app declared a topology, as opposed to the default one #topology builds. Only a swarm reads it.
952 |
# File 'lib/wurk/configuration.rb', line 952 def topology_assigned? = @topology_assigned == true |
#total_concurrency ⇒ Object
229 230 231 |
# File 'lib/wurk/configuration.rb', line 229 def total_concurrency @capsules.each_value.sum(&:concurrency) end |
#web ⇒ Object
Web UI configuration: the authorization hook and read-only mode. Returns
the process-wide Wurk::Web.config singleton so config.web.read_only = true and the engine middleware share one source of truth. The web layer
is an autoload (Wurk::LazyLoader), so standalone boot stays lean until this
is called.
Spec: docs/target/sidekiq-ent.md §9.2.
747 748 749 |
# File 'lib/wurk/configuration.rb', line 747 def web Wurk::Web.config end |
#web_pool_size ⇒ Object
Connections in the dedicated web pool. config.web_pool_size = N overrides
the default; independent of config.redis[:size], which sizes the worker
capsules — the two pools are deliberately disjoint (#101).
320 321 322 |
# File 'lib/wurk/configuration.rb', line 320 def web_pool_size @options[:web_pool_size] || WEB_POOL_DEFAULT_SIZE end |
#web_pool_size=(size) ⇒ Object
324 325 326 327 |
# File 'lib/wurk/configuration.rb', line 324 def web_pool_size=(size) guard_frozen! @options[:web_pool_size] = Integer(size) end |
#web_redis_pool ⇒ Object
Dedicated Redis pool for the dashboard / JSON API / SSE, disjoint from
every worker capsule's pool. Dashboard load — an API burst, a long-lived
SSE stream — can no longer drain the connections a co-located (embedded)
worker needs to fetch and heartbeat, and vice versa: the #101 0/N
pool-exhaustion incident. Lazy, so a headless worker that never serves the
dashboard builds nothing; web entry points route Wurk.redis here through
Wurk::Web::PoolScope. The Configuration instance is never frozen (only its
@options/@capsules are), so this ||= is safe to fire post-boot.
337 338 339 |
# File 'lib/wurk/configuration.rb', line 337 def web_redis_pool @web_redis_pool ||= build_redis_pool(size: web_pool_size, name: 'web', pool_timeout: WEB_POOL_TIMEOUT) end |