LUIDIBLU / tech blog
PT EN
PT EN

What's the truth about Sidekiq?

13 posts · 8,990 words · 46 min of reading · since 2016

0% read

What's the truth about Sidekiq?

Queues, workers, and asynchronous task processing with Sidekiq and Ruby on Rails.

TL;DR Too lazy to read? Click here.

Usually, when we talk about the standard architecture (MVC, RAILS + HEROKU), traffic has a long journey to make. Throughout that journey (from request to response), we block our UX from returning absolutely anything.

The standard Heroku process that handles these requests is also blocked. If the application takes longer than 30 seconds, it closes the connection and returns a TIMED OUT.

It (Heroku) does this because what you actually pay for on Heroku is the number of application processes (known as Dynos).

There are TWO TYPES of Dynos:

  • Web Dyno
  • Worker Dyno

At a high level, a Dyno is a process that runs in your Rails application.
A Web Dyno is a subtype specifically designed to handle web traffic.

Heroku places all Web Dynos behind something we call a “Routing Mash”.

Request -> HTTP Reverse Proxy -> HTTP cache -> Routing Mash -> Dyno Grid

In this case, instead of taking 30 seconds, we can add the request to a list and immediately return a message to the user along the lines of “We’ve added your request to our list.”

Basically, it’s a key-value store without having to model the data in a sequential database (yes, Redis).

So, instead of calling model.rb to generate the report, we put a message in the queue and it returns to the user immediately. Meanwhile, our queue also communicates with a worker.

A worker is a Ruby object in a separate process from the one handling the Web Request, within the same deployment.

Because the service is separate from the Web, it is not subject to the 30-second rules. In other words, when the Worker finishes the process, we can decide what to do with the data. For example, we could send the user an email or upload it to a Bucket.

We usually add the data to the cache so the user can view it on the Web without having to recalculate it.

We usually always use Redis with Sidekiq….

Redis => Database.
Sidekiq => Library for handling processes and Rails applications.

P.S.: Sidekiq comes with its own UI, which is built in Sinatra, so you need to import Sinatra.

Examples:

ruby#routes.rb

require ‘sidekiq/web’
mount Sidekiq::Web => “/sidekiq”

$ mkdir workers
$ cd workers && touch report_worker.rb

#report_worker.rb

class ReportWorker
  include Sidekiq::worker
  sidekiq_options retry:false

  def perform(start_date, end_date)
    puts("#{start_date} até #{end_date}")
  end
end

In the example above, we use retry: false because, by default, if the worker fails, it will try to run the process until it succeeds.
Every Sidekiq Worker has a method called “Perform”. It is called when something in the queue is processed.

You can put parameters in the queue ;).

ruby#controller.rb

def report
  ReportWorker.perform_async("01-01-2020", "01-01-2021")
  render text: "Request Comum"
end

And finally, to at least understand how Sidekiq runs, we use the following to run it in the terminal:

sidekiq -c 1

It runs all processes with one thread (by default), but you can increase that number.
By default, it knows the queue is in Redis because it reads port 6379 (the Redis default).

Okay, Diego, but how can I deploy this to the cloud?

Here’s my answer, my friend:

In your Rails application, do the following:

text#procfile

reportworker: bundle sidekiq -c 1

This way, it will run the same command we used in our local terminal.

:) I don’t know how to end an article, so….

$ systemctl stop medium-text

Visitor dossier

Read only in your browser. The edge answers and keeps no log. Nothing is stored or sent to third parties.

Shortcuts

/⌘K
search and commands
?
this table
p
visitor dossier
t
light / dark theme
.
12-column grid
jk
next / previous post
enter
open the focused post
g hg pg tg sg r
go to home, posts, tags, about, rss
esc
close dialogs