Minimal decoupled subsystems in your rails app

There are multiple ways to implement communication between two separate microservices in your application. Messaging is often the most recommended way of doing this. However, you probably heard that “You must be this tall to use microservices”. In other words, your project must be certain size and your organization must be mature (in terms of DevOps, monitoring, etc.) to split your app into separately running and deployed processes. But that doesn’t mean you can’t benefit from decoupling individual subsystems in your application earlier.

Example

Those subsystems in your application are called Bounded Contexts.

Now, imagine that you have two bounded contexts in your application:

• Season Passes [SP] - which takes care of managing season passes for football clubs.
• Identity & Access [I&A] - which takes care of authentication, registrations and permissions.

And the use-case that we will be working with is described as:

When Season Pass Holder is imported, create an account for her/him and send a welcome e-mail with password setting instructions..

The usual way to achieve it would be to wrap everything in a transaction, create a user, create a season pass and commit it. But we already decided to split our application into two separate parts. And we don’t want to operate on both of them directly. They have their responsibilities, and we want to keep them decoupled.

Evented way

So here is what we could do instead:

• [SP] create a season pass (without a reference to user_id)
• [SP] publish a domain event that season pass was imported
• [I&A] react to that domain event in Identity & Access part of our application and create the user account
• [I&A] in case of success, publish a domain event that User was imported (UserImported)
  • or if the user is already present on our platform, it would publish UserAlreadyRegistered domain event.
• [SP] react to UserImported or UserAlreadyRegistered domain event and update the user_id of created SeasonPass to the ID of the user.

It certainly sounds (and is) more complicated compared to our default solution. So we should only apply this tactic where it benefits us more than it costs.

But I assume that if you decided to separate some sub-systems of your applications into indepedent, decoupled units, you already weighted the pros and cons. So now, we are talking only about the execution.

About the cost

You might be thinking that there is big infrastructural cost in communicating via domain events. That you need to set up some message bus and think about event serialization. But big chances are things are easier than you expect them to be.

You can start small and straightforward and later change to more complex solutions when the need appears. And chances are you already have all the components required for it, but you never thought of them in such a way.

Do you use Rails 5 Active Job, or resque or sidekiq or delayed job or any similar tooling, for scheduling background jobs? Good, you can use them as message bus for asynchronous communication between two parts of your application. With #retry_job you can even think of it as at least one delivery in case of failures.

So the parts of your application (sub-systems, bounded-contexts) don’t need at the beginning to be deployed as separate applications (microservices). They don’t need a separate message bus such as RabbitMQ or Apache Kafka. At the start, all you need is a code which assumes asynchronous communication (and also embraces eventual consistency) and uses the tools that you have at your disposal.

Also, you don’t need any fancy serializer at the beginning such as message pack or protobuf. YAML or JSON can be sufficient when you keep communicating asynchronously within the same codebase (just different part of it).

Show me the code

Storing and publishing a domain event

We are going to use rails_event_store, but you could achieve the same results using any other pub-sub (e.g. wisper + wisper-sidekiq extension). rails_event_store has the benefit that your domain events will be saved in a database.

Domain event definition. This will be published when Season Pass is imported:

class Season::PassImported < RubyEventStore::Event
  SCHEMA = {
    id: Integer,
    barcode: String,
    first_name: String,
    last_name: String,
    email: String,
  }.freeze

  def self.strict(data:)
    ClassyHash.validate_strict(data, SCHEMA)
    new(data: data)
  end
end

Domain event handlers/callbacks. This is how we put messages on our background queue, treating it as a simple message bus. We use Rails 5 API here.

Rails.application.config.event_store.tap do |es|
  es.subscribe(->(event) do
    IdentityAndAccess::RegisterSeasonPassHolder.perform_later(YAML.dump(event))
  end, [Season::PassImported])

  es.subscribe(->(event) do
    Season::AssignUserIdToHolder.perform_later(YAML.dump(event))
  end, [IdentityAndAccess::UserImported, IdentityAndAccess::UserAlreadyRegistered])
end

Imagine this part of code (somewhere) responsible for importing season passes. It saves the pass and publishes PassImported event.

ActiveRecord::Base.transaction do
  pass = Season::Pass.create!(...)
  event_store.publish_event(Season::PassImported.strict(data: {
    id: pass.id,
    barcode: pass.barcode,
    first_name: pass.holder.first_name,
    last_name: pass.holder.last_name,
    email: pass.holder.email,
  }), stream_name: "pass$#{pass.id}")
end

When event_store saves and publishes the Season::PassImported event, it will also be queued for processing by IdentityAndAccess::RegisterSeasonPassHolder background job (handler equivalent in DDD world).

Reacting to the PassImported event

These are the events that will be published by Identity and Access bounded context:

class IdentityAndAccess::UserImported < RubyEventStore::Event
  SCHEMA = {
    id: Integer,
    email: String,
  }.freeze

  def self.strict(data:)
    ClassyHash.validate_strict(data, SCHEMA)
    new(data: data)
  end
end

class IdentityAndAccess::UserAlreadyRegistered < RubyEventStore::Event
  SCHEMA = {
    id: Integer,
    email: String,
  }.freeze

  def self.strict(data:)
    ClassyHash.validate_strict(data, SCHEMA)
    new(data: data)
  end
end

This is how Identity And Access context reacts to the fact that Season Pass was imported.

module IdentityAndAccess
  class RegisterSeasonPassHolder < ApplicationJob
    queue_as :default

    def perform(serialized_event)
      event = YAML.load(serialized_event)
      ActiveRecord::Base.transaction do
        user = User.create!(email: event.data.email)
        event_store.publish_event(UserImported.strict(data: {
          id: user.id,
          email: user.email,
        }), stream_name: "user$#{user.id}")
      end
    rescue User::EmailTaken => exception
      event_store.publish_event(UserAlreadyRegistered.strict(data: {
        id: exception.user_id,
        email: exception.email,
      }), stream_name: "user$#{exception.user_id}")
    end
  end
end

(ps. if you think that we should not use exceptions for control-flow, or that exceptions are slow, keep it to yourself. This is not a debate about such topic. I am pretty sure you can imagine exception-less solution)

Reacting to the UserAlreadyRegistered/UserImported event

Reminder how when an event is published we schedule something to happen in the other part of the app.

Rails.application.config.event_store.tap do |es|
  es.subscribe(->(event) do
    IdentityAndAccess::RegisterSeasonPassHolder.perform_later(YAML.dump(event))
  end, [Season::PassImported])

  es.subscribe(->(event) do
    Season::AssignUserIdToHolder.perform_later(YAML.dump(event))
  end, [IdentityAndAccess::UserImported, IdentityAndAccess::UserAlreadyRegistered])
end

Season Pass Bounded Context reacts to either UserImported or UserAlreadyRegistered by saving the reference to user_id. It does not have direct access to User class. It just holds a reference.

module Season
  class AssignUserIdToHolder < ApplicationJob
    queue_as :default

    def perform(serialized_event)
      event = YAML.load(serialized_event)
      ActiveRecord::Base.transaction do
        Pass.all_with_holder_email!(event.data.email).each do |pass|
          pass.set_holder_user_id(event.data.id)
          event_store.publish_event(Season::PassHolderUserAssigned.strict(data: {
            pass_id: pass.id,
            user_id: pass.holder.user_id,
          }), stream_name: "pass$#{pass.id}")
        end
      end
    end

  end
end

When your needs grow

Now imagine that the needs of Identity And Access grow a lot. We would like to extract it as a small application (microservice) and scale separately. Maybe deploy much more instances than the rest of our app needs? Maybe ship it with JRuby instead of MRI. Maybe expose it to other applications that will now use it for authentication and managing users as well? Can it be done?

Yes. Switch to a serious message bus that can be used between separate apps, and use a proper serialization format (not YAML, because YAML is connected to class names and you won’t have identical class names between two separate apps).

Your code already assumes asynchronous communication between Season Passes and Identity&Access so you are safe to do so.

Did you like it?

Make sure to check our books and upcoming Smart Income For Developers Bundle.

Disclaimer

• This is an oversimplified example to show you the idea :)

Read more

• Evented Rails: Decoupling domains in Rails with Wisper pub/sub events
• Domain-Driven Design Distilled
• Implementing DDD book
• Reactive Messaging Patterns with the Actor Model
• Microservice Prerequisites
• Worried about rollbacks?