Class: Wurk::Fetcher

Inherits:
Object
  • Object
show all
Defined in:
lib/wurk/fetcher.rb,
lib/wurk/fetcher/capped.rb,
lib/wurk/fetcher/reaper.rb,
lib/wurk/fetcher/reliable.rb,
lib/wurk/fetcher/oom_drain.rb,
lib/wurk/fetcher/many_queues.rb,
lib/wurk/fetcher/at_maxmemory.rb,
lib/wurk/fetcher/orphan_grace.rb,
lib/wurk/fetcher/unit_of_work.rb,
lib/wurk/fetcher/orphan_reclaim.rb,
lib/wurk/fetcher/self_reconcile.rb,
lib/wurk/fetcher/private_list_key.rb,
lib/wurk/fetcher/unparseable_keys.rb

Overview

Abstract fetcher. Wurk::Fetcher::Reliable is the only implementation we ship and the only one we recommend — BLMOVE-based reliable fetch. No "basic fetch" mode.

Defined Under Namespace

Modules: AtMaxmemory, Capped, ManyQueues, OomDrain, PrivateListKey Classes: OrphanGrace, OrphanReclaim, Reaper, Reliable, SelfReconcile, UnparseableKeys

Instance Method Summary collapse

Instance Method Details

#bulk_requeue(in_progress) ⇒ Object 



14
 
# File 'lib/wurk/fetcher.rb', line 14

def bulk_requeue(in_progress); end

#flush_pending_acks ⇒ Object

Send any ACK the fetcher is holding back rather than sending immediately. Processor calls this as it stops, so a job that finished but whose ACK never caught a fetch to ride on can't be reclaimed as unfinished. No-op in the abstract base; only Reliable defers.



25
 
# File 'lib/wurk/fetcher.rb', line 25

def flush_pending_acks; end

#retrieve_work ⇒ Object

Instant nil spins Processor#run, which immediately fetches again. Concrete implementations must block or back off before returning nil when empty.

Raises:

  • (NotImplementedError) 


10
11
12
 
# File 'lib/wurk/fetcher.rb', line 10

def retrieve_work
  raise NotImplementedError, "#{self.class} must override #retrieve_work and block or back off when empty"
end

#terminate ⇒ Object

Quiet hook: Manager#quiet calls this so retrieve_work can short-circuit and stop pulling new work the instant a process is quieted. No-op in the abstract base; Reliable flips its drain flag.



19
 
# File 'lib/wurk/fetcher.rb', line 19

def terminate; end