What I learnt today from reading gems’ code

Today I was working on chillout.io client and while I was debugging some parts, I had a look at some Ruby gems. This is always an interesting experience because you can learn how other developers design their API and how different it can be from your approach.

Sidekiq

So here are some interesting bits from sidekiq code.

Sidekiq::Client initializer

module Sidekiq
  class Client
    def initialize(redis_pool=nil)
      @redis_pool = redis_pool ||
      Thread.current[:sidekiq_via_pool] ||
      Sidekiq.redis_pool
    end
  end
end

Quoting the documentation:

Sidekiq::Client normally uses the default Redis pool but you may pass a custom ConnectionPool if you want to shard your Sidekiq jobs across several Redis instances…

I generally don’t like globals as a gem consumer but sometimes they are convenient and provide the convention over configuration magical feeling.

The nice thing about this global is that you don’t need to use it. It is easily overridable with such constructor. If you have specific requirements, your own connection pool, special redis connection, multiple clients and multiple connections etc, etc, you can still get the work done.

Sidekiq::Client.new(ConnectionPool.new { Redis.new })

Delegating class methods

Going further with global which you don’t need to use.

module Sidekiq
  class Client
    def push(item)
      # ...
    end

    def self.push(item)
      new.push(item)
    end
  end
end

With this code, instead of

Sidekiq::Client.new().push(
  'queue' => 'one',
  'class' => MyWorker,
  'args'  => ['do_it']
)

you can do

Sidekiq::Client.push(
  'queue' => 'one',
  'class' => MyWorker,
  'args'  => ['do_it']
)

Again. No one forces you to use the class method. If for any reason, the first approach works better than the second, if you need to have a new instance with specific constructor arguments, do it. Sidekiq can handle both.

Sidekiq.redis_pool

module Sidekiq
  def self.redis_pool
    @redis ||= Sidekiq::RedisConnection.create
  end

  def self.redis=(hash)
    @redis = if hash.is_a?(ConnectionPool)
      hash
    else
      Sidekiq::RedisConnection.create(hash)
    end
  end
end

This redis=(hash) setter can handle a Hash with redis configuration options or a Sidekiq::ConnectionPool instance.

yielding for configuration

module Sidekiq
  def self.server?
    defined?(Sidekiq::CLI)
  end

  def self.configure_server
    yield self if server?
  end

  def self.server_middleware
    @server_chain ||= default_server_middleware
    yield @server_chain if block_given?
    @server_chain
  end

  def self.default_server_middleware
    Middleware::Chain.new
  end
end

Quoting the documentation:

Sidekiq has a similar notion of middleware to Rack: these are small bits of code that can implement functionality. Sidekiq breaks middleware into client-side and server-side.

Server-side middleware runs ‘around’ job processing.

Client-side middleware runs before the pushing of the job to Redis and allows you to modify/stop the job before it gets pushed.

So the sidekiq client is the app (usually a Rails app) responsible for pushing jobs and scheduling them.

Sidekiq server is the worker process that execute on a different machine for processing jobs in the background.

Sidekiq needs to know which mode it is in, and it needs to have the ability to have different configurations for both of them. Especially considering that usually it is the same Rails application running either in client mode (http application server such as puma or unicorn) or server mode (worker process executed with sidekiq command).

The configuration can be set such as:

Sidekiq.configure_server do |config|
  config.redis = { namespace: 'myapp', size: 25 }
  config.server_middleware do |chain|
    chain.add MyServerHook
  end
end
Sidekiq.configure_client do |config|
  config.redis = { namespace: 'myapp', size: 1 }
end

So the configure_server method yields the block only when the if-statement evaluates we are in a server process. It uses block for lazy configuration. It is not evaluated when unnecessary (in the client).

server_middleware yields for nicer readability, I believe. Especially in the case of many middlewares.

BTW. chillout.io client uses a middleware to schedule sending metrics when a background job is done.

ActiveSupport

ActiveSupport::TaggedLogging

ActiveSupport::TaggedLogging wraps any standard Logger object to provide tagging capabilities.

logger = ActiveSupport::TaggedLogging.new(Logger.new(STDOUT))
logger.tagged('BCX') { logger.info 'Stuff' }           # Logs "[BCX] Stuff"
logger.tagged('BCX', "Jason") { logger.info 'Puff' }   # Logs "[BCX] [Jason] Puff"

There is one method which brought my attention:

module ActiveSupport
  module TaggedLogging
    def flush
      clear_tags!
      super if defined?(super)
    end
  end
end

I’ve never seen this super if defined?(super) but it turns out it is useful to dynamically figure out if the ancestor defined given method and you should call it or this is the first module/class in inheritance chain which defines it.

class Fool
  def foo
    puts "foo from Fool"
  end
end

module Baron
  def bar
    puts "bar from Baron"
  end
end

module Bazinga
  def baz
    puts "baz from Bazinga"
    super if defined?(super)
  end
end

module Freddy
  def fred
    puts "fred from Freddy"
    super if defined?(super)
  end
end

class Powerful < Fool
  include Baron
  prepend Freddy

  def foo
    puts "foo from Powerful"
    super if defined?(super)
  end

  def bar
    puts "bar from Powerful"
    super if defined?(super)
  end

  def baz
    puts "baz from Powerful"
  end

  def fred
    puts "fred from Powerful"
  end

  def qux
    puts "qux from Powerful"
    super if defined?(super)
  end

  def corge
    puts "corge from Powerful"
    super
  end
end

p = Powerful.new
p.extend(Bazinga)

# inheritance
p.foo
# foo from Powerful
# foo from Fool

# module included in class
p.bar
# bar from Powerful
# bar from Baron

# object extended with module
p.baz
# baz from Bazinga
# baz from Powerful

# module prepended in class
p.fred
# fred from Freddy
# fred from Powerful

# nothing
p.qux
# qux from Powerful

# without `if defined?(super)`
p.corge
# corge from Powerful
# NoMethodError: super: no superclass method `corge' for #<Powerful:0x000000015e8390>

self.new in a module

Also, check this out.

module ActiveSupport
  module TaggedLogging
    def self.new(logger)
      logger.formatter ||= ActiveSupport::Logger::SimpleFormatter.new
      logger.formatter.extend Formatter
      logger.extend(self)
    end
  end
end

new is not used to create a new instance of TaggedLogging (after all it is a module, not a class) that would delegate to the logger as one could expect based on the API. Instead it extends the logger object with itself.

What I learnt today from reading gems' code