Class: Wurk::Sentry::ErrorHandler

Inherits:
Object
  • Object
show all
Defined in:
lib/wurk/sentry/error_handler.rb

Overview

config.error_handlers entry. Reports fetch-loop errors (context: "Error fetching job"), shutdown-path errors ("!shutdown"), unparseable payloads ("Invalid JSON"), the retry machinery's own meta-errors, and job failures ("Job raised exception").

A job failure is Middleware's to report whenever the exception passed through it: the middleware holds the job's scope and has already applied the terminal-attempt policy, so reporting it here too would double it. What reaches this handler unseen raised outside the middleware — the class failed to load, the reloader or an earlier middleware raised — and gets the same policy and scope here.

Handler signature is Sidekiq's: call(exception, context_hash, config).

Constant Summary collapse

JOB_FAILURE = 
'Job raised exception'
SEEN_BY_MIDDLEWARE =

Thread-local handshake with Middleware: the exception it last saw on this thread. Error handlers run synchronously on the processor thread that raised, so the slot is never read across threads.

:wurk_sentry_seen_by_middleware
REPORTED_ON_THREAD =

The exception this thread last captured, for DeathHandler's dedupe.

:wurk_sentry_reported_on_thread
DEFAULT_FILTERED_ERROR_CLASSES =

Transport blips the pool already retried before re-raising. Wurk's default handler logs these at WARN precisely because they are self-healing, and the fetch loop runs them in a tight sleep(1) cycle: on one production Dragonfly backend a single Sentry issue accumulated ~136,000 events from fetch blips while the job pipeline was completely healthy. They stay in the logs; they just stop paging anyone.

Deliberately not RedisClient::Error: the server answering MISCONF, OOM, READONLY, NOPERM or WRONGPASS is not a blip. It heals only when someone acts, and every write fails until then, so those are reported (rate-limited below).

[RedisClient::ConnectionError, ConnectionPool::TimeoutError].freeze

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(filter_transport_errors: true, filtered_error_classes: nil, report_after_attempts: nil, coalescer: ErrorCoalescer.new) ⇒ ErrorHandler

Returns a new instance of ErrorHandler.



65
66
67
68
69
70
71
72
 
# File 'lib/wurk/sentry/error_handler.rb', line 65

def initialize(filter_transport_errors: true, filtered_error_classes: nil, report_after_attempts: nil,
               coalescer: ErrorCoalescer.new)
  @filter_transport_errors = filter_transport_errors
  @default_filter = filtered_error_classes.nil?
  @filtered_error_classes = (filtered_error_classes || DEFAULT_FILTERED_ERROR_CLASSES).to_a.freeze
  @report_after_attempts = report_after_attempts
  @coalescer = coalescer
end

Instance Attribute Details

#filtered_error_classes ⇒ Object (readonly)

Returns the value of attribute filtered_error_classes.



63
64
65
 
# File 'lib/wurk/sentry/error_handler.rb', line 63

def filtered_error_classes
  @filtered_error_classes
end

Class Method Details

.forget_thread! ⇒ Object 



45
46
47
48
 
# File 'lib/wurk/sentry/error_handler.rb', line 45

def self.forget_thread!
  Thread.current[SEEN_BY_MIDDLEWARE] = nil
  Thread.current[REPORTED_ON_THREAD] = nil
end

.reported!(exception) ⇒ Object 



37
38
39
 
# File 'lib/wurk/sentry/error_handler.rb', line 37

def self.reported!(exception)
  Thread.current[REPORTED_ON_THREAD] = exception
end

.reported?(exception) ⇒ Boolean

Returns:

  • (Boolean) 


41
42
43
 
# File 'lib/wurk/sentry/error_handler.rb', line 41

def self.reported?(exception)
  Thread.current[REPORTED_ON_THREAD].equal?(exception)
end

.seen_by_middleware!(exception) ⇒ Object 



30
31
32
 
# File 'lib/wurk/sentry/error_handler.rb', line 30

def self.seen_by_middleware!(exception)
  Thread.current[SEEN_BY_MIDDLEWARE] = exception
end

Instance Method Details

#call(exception, context = {}, config = nil) ⇒ Object 



74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
 
# File 'lib/wurk/sentry/error_handler.rb', line 74

def call(exception, context = {}, config = nil)
  return nil unless Wurk::Sentry.enabled?
  return nil if control_flow?(exception)
  # Ahead of the transport filter: that filter quiets the infrastructure
  # loop, and a job that died on a Redis error is still a dead job.
  return capture_job_failure(exception, context, config) if job_failure?(context)
  return nil if filtered?(exception)

  extra = extra_for(context)
  if exception.is_a?(RedisClient::HasCommand)
    suppressed = @coalescer.admit(command_error_key(exception))
    return nil unless suppressed

    extra[:suppressed] = suppressed if suppressed.positive?
  end
  ::Sentry.capture_exception(exception, extra: extra, tags: tags_for(context))
  nil
end

#filtered?(exception) ⇒ Boolean

Returns:

  • (Boolean) 


93
94
95
96
97
98
99
100
101
 
# File 'lib/wurk/sentry/error_handler.rb', line 93

def filtered?(exception)
  return false unless @filter_transport_errors

  # READONLY and MASTERDOWN subclass ConnectionError, but the server did answer,
  # so the default passes them. A list the host wrote is taken at its word.
  return false if @default_filter && exception.is_a?(RedisClient::HasCommand)

  @filtered_error_classes.any? { |klass| exception.is_a?(klass) }
end