San Francisco, USA

5214F Diamond Heights Blvd #553
San Francisco, CA 94131

Pune, India

203, Jewel Towers, 2nd Floor
Lane Number 5, Koregaon Park
Pune 411001, India

301 - 275 - 3997
hello@BigBinary.com

Rails 6.1 tracks Active Storage variant in the database

This blog is part of our Rails 6.1 series.

Active Storage variants are the transformation of the original image. These variants can be used as thumbnails, avatars, etc.

Active Storage generates variants on demand by downloading the original image. The image is transformed into a variant and is stored to the third party services like S3.

When a request to fetch a variant for an Active Storage object is made, Rails checks if the variant is already been processed and is already available on S3 or not. But to do so Rails has to make a call to find out if the variant is available on S3. This extra call adds to the latency.

Active Storage has to wait until the image variant check call is completed because S3 might not return the image when a GET request is made due to eventual consistency. This way Rails avoid downloading a broken image from S3 and uploading broken image variant to S3 in case the variant is not present.

In Rails 6.1, Active Storage tracks the presence of the variant in the database. This change avoids unnecessary variant presence remote request made to the S3 and directly fetches or generates a image variant.

In Rails 6.1, the configuration to allow variant tracking in the database is by default set to true.

config.active_storage.track_variants: true

Check out the pull request for more details on this.


Ruby 2.7 adds Enumerable#filter_map

This blog is part of our Ruby 2.7 series. Ruby 2.7.0 was released on Dec 25, 2019.

Ruby 2.7 adds Enumerable#filter_map which is a combination of filter + map as the name indicates. The ‘filter_map’ method filters and map the enumerable elements within a single iteration.

Before Ruby 2.7, we could have achieved the same with 2 iterations using select & map combination or map & compact combination.

irb> numbers = [3, 6, 7, 9, 5, 4]

# we can use select & map to find square of odd numbers

irb> numbers.select { |x| x.odd? }.map { |x| x**2 }
=> [9, 49, 81, 25]

# or we can use map & compact to find square of odd numbers

irb> numbers.map { |x| x**2 if x.odd? }.compact
=> [9, 49, 81, 25]

Ruby 2.7

Ruby 2.7 adds Enumerable#filter_map which can be used to filter & map the elements in a single iteration and which is more faster when compared to other options described above.

irb> numbers = [3, 6, 7, 9, 5, 4]
irb> numbers.filter_map { |x| x**2 if x.odd? }
=> [9, 49, 81, 25]

The original discussion had started 8 years back. Here is the latest thread and github commit for reference.


Ruby 2.7 deprecates conversion of keyword arguments

A notable change has been announced for Ruby 3 for which deprecation warning has been added in Ruby 2.7. Ruby 2.7 deprecated automatic conversion of keyword arguments and positional arguments. This conversion will be completely removed in Ruby 3.

Ruby 2.7 NEWS has listed the spec of keyword arguments for Ruby 3.0. We will take the examples mentioned there and for each scenario we will look into how we can fix them in the existing codebase.

Scenario 1

When method definition accepts keyword arguments as the last argument.

def sum(a: 0, b: 0)
  a + b
end

Passing exact keyword arguments in a method call is acceptable, but in applications we usually pass a hash to a method call.

sum(a: 2, b: 4) # OK

sum({ a: 2, b: 4 }) # Warned

In this case, we can add a double splat operator to the hash to avoid deprecation warning.

sum(**{ a: 2, b: 4 }) # OK

Scenario 2

When method call passes keyword arguments but does not pass enough required positional arguments.

If the number of positional arguments doesn’t match with method definition, then keyword arguments passed in method call will be considered as the last positional argument to the method.

def sum(num, x: 0)
  num.values.sum + x
end
sum(a: 2, b: 4) # Warned

sum(a: 2, b: 4, x: 6) # Warned

To avoid deprecation warning and for code to be compatible with Ruby 3, we should pass hash instead of keyword arguments in method call.

sum({ a: 2, b: 4 }) # OK

sum({ a: 2, b: 4}, x: 6) # OK

Scenario 3

When a method accepts a hash and keyword arguments but method call passes only hash or keyword arguments.

If a method arguments are a mix of symbol keys and non-symbol keys, and the method definition accepts either one of them then Ruby splits the keyword arguments but also raises a warning.

def sum(num={}, x: 0)
  num.values.sum + x
end
sum("x" => 2, x: 4) # Warned

sum(x: 2, "x" => 4) # Warned

To fix this warning, we should pass hash separately as defined in the method definition.

sum({ "x" => 4 }, x: 2) # OK

Scenario 4

When an empty hash with double splat operator is passed to a method that doesn’t accept keyword arguments.

Passing keyword arguments using double splat operator to a method that doesn’t accept keyword argument will send empty hash similar to earlier version of Ruby but will raise a warning.

def sum(num)
  num.values.sum
end
numbers = {}
sum(**numbers) # Warned

To avoid this warning, we should change method call to pass hash instead of using double splat operator.

numbers = {}
sum(numbers) # OK

Added support for non-symbol keys

In Ruby 2.6.0, support for non-symbol keys in method call was removed. It is added back in Ruby 2.7. When method accepts arbitrary keyword arguments using double splat operator then non-symbol keys can also be passed.

def sum(**num)
  num.values.sum
end
ruby 2.6.5
sum("x" => 4, "y" => 3)
=> ArgumentError (wrong number of arguments (given 1, expected 0))

sum(x: 4, y: 3)
=> 7
ruby 2.7.0
sum("x" => 4, "y" => 3)
=> 7

sum(x: 4, y: 3)
=> 7

Added support for **nil

Ruby 2.7 added support for **nil to explicitly mention if a method doesn’t accept any keyword arguments in method call.

def sum(a, b, **nil)
  a + b
end

sum(2, 3, x: 4)
=> ArgumentError (no keywords accepted)

To suppress above deprecation warnings we can use -W:no-deprecated option.

In conclusion, Ruby 2.7 has worked big steps towards changing specification of keyword arguments which will be completely changed in Ruby 3.

For more information on discussion, code changes and official documentation, please head to Feature #14183 discussion, pull request and NEWS release.


Ruby 2.7 adds Enumerator::Lazy#eager

This blog is part of our Ruby 2.7 series. Ruby 2.7.0 was released on Dec 25, 2019.

Ruby 2.0 introduced Enumerator::Lazy, a special type of enumerator which helps us in processing chains of operations on a collection without actually executing it instantly.

By applying Enumerable#lazy method on any enumerable object, we can convert that object into Enumerator::Lazy object. The chains of actions on this lazy enumerator will be evaluated only when it is needed. It helps us in processing operations on large collections, files and infinite sequences seamlessly.

# This line of code will hang and you will have to quit the console by Ctrl+C.
irb> list = (1..Float::INFINITY).select { |i| i%3 == 0 }.reject(&:even?)

# Just adding `lazy`, the above line of code now executes properly
# and returns result without going to infinite loop. Here the chains of
# operations are performed as and when it is needed.
irb> lazy_list = (1..Float::INFINITY).lazy.select { |i| i%3 == 0 }.reject(&:even?)
=> #<Enumerator::Lazy: ...>

irb> lazy_list.first(5)
=> [3, 9, 15, 21, 27]

When we chain more operations on Enumerable#lazy object, it again returns lazy object without executing it. So, when we pass lazy objects to any method which expects a normal enumerable object as an argument, we have to force evaluation on lazy object by calling to_a method or it’s alias force.

# Define a lazy enumerator object.
irb> list = (1..30).lazy.select { |i| i%3 == 0 }.reject(&:even?)
=> #<Enumerator::Lazy: #<Enumerator::Lazy: ... 1..30>:select>:reject>

# The chains of operations will return again a lazy enumerator.
irb> result = list.select { |x| x if x <= 15 }
=> #<Enumerator::Lazy: #<Enumerator::Lazy: ... 1..30>:select>:reject>:map>

# It returns error when we call usual array methods on result.
irb> result.sample
irb> NoMethodError (undefined method `sample'
irb> for #<Enumerator::Lazy:0x00007faab182a5d8>)

irb> result.length
irb> NoMethodError (undefined method `length'
irb> for #<Enumerator::Lazy:0x00007faab182a5d8>)

# We can call the normal array methods on lazy object after forcing
# its actual execution with methods as mentioned above.
irb> result.force.sample
=> 9

irb> result.to_a.length
=> 3

The Enumerable#eager method returns a normal enumerator from a lazy enumerator, so that lazy enumerator object can be passed to any methods which expects a normal enumerable object as an argument. Also, we can call other usual array methods on the collection to get desired results.

# By adding eager on lazy object, the chains of operations would return
# actual result here. If lazy object is passed to any method, the
# processed result will be received as an argument.
irb> eager_list = (1..30).lazy.select { |i| i%3 == 0 }.reject(&:even?).eager
=> #<Enumerator: #<Enumerator::Lazy: ... 1..30>:select>:reject>:each>

irb> result = eager_list.select { |x| x if x <= 15 }
irb> result.sample
=> 9

irb> result.length
=> 3

The same way, we can use eager method when we pass lazy enumerator as an argument to any method which expects a normal enumerator.

irb> list = (1..10).lazy.select { |i| i%3 == 0 }.reject(&:even?)
irb> def display(enum)
irb>   enum.map { |x| p x }
irb> end

irb>  display(list)
=> #<Enumerator::Lazy: #<Enumerator::Lazy: ... 1..30>:select>:reject>:map>

irb> eager_list = (1..10).lazy.select { |i| i%3 == 0 }.reject(&:even?).eager
irb> display(eager_list)
=> 3
=> 9

Here’s the relevant commit and feature discussion for this change.


Ruby 2.7 introduces numbered parameters as default block parameters

This blog is part of our Ruby 2.7 series. Ruby 2.7.0 was released on Dec 25, 2019.

At some point, all of us have used names like a, n, i etc for block parameters. Below are few examples where numbered parameters can come in handy.

> (1..10).each { |n| p n * 3 }

> { a: [1, 2, 3], b: [2, 4, 6], c: [3, 6, 9] }.each { |_k, v| p v }

> [10, 100, 1000].each_with_index { |n, i| p n, i }

Ruby 2.7 introduces a new way to access block parameters. Ruby 2.7 onwards, if block parameters are obvious and we wish to not use absurd names like n or i etc, we can use numbered parameters which are available inside a block by default.

We can use _1 for first parameter, _2 for second parameter and so on.

Here’s how Ruby 2.7 provides numbered parameters inside a block. Below shown are the examples from above, only this time using numbered parameters.

> (1..10).each { p _1 * 3 }

> { a: [1, 2, 3], b: [2, 4, 6], c: [3, 6, 9] }.each { p _2 }

> [10, 100, 1000].each_with_index { p _1, _2 }

Like mentioned in News-2.7.0 docs, Ruby now raises a warning if we try to define local variable in the format _1. Local variable will have precedence over numbered parameter inside the block.

> _1 = 0
> => warning: `_1' is reserved for numbered parameter; consider another name

> [10].each { p _1 }
> => 0

Numbered parameters are not accessible inside the block if we define ordinary parameters. If we try to access _1 when ordinary parameters are defined, then ruby raises SyntaxError like shown below.

> ["a", "b", "c"].each_with_index { |alphabet, index| p _1, _2}

=> SyntaxError ((irb):1: ordinary parameter is defined)

This feature was suggested 9 years back and came back in discussion last year. After many suggestions community agreed to use _1 syntax.

Head to following links to read the discussion behind numbered parameters, Feature #4475 and Discussion #15723.

Here’s relevant commit for this feature.


Rails 6 fixes a bug where after_commit callbacks are called on failed update in a transaction block

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Rails 6 fixes a bug where after_commit callbacks are called on failed update in a transaction block.

Let’s checkout the bug in Rails 5.2 and the fix in Rails 6.

Rails 5.2

Let’s define an after_commit callback in User model and try updating an invalid user object in a transaction block.

>> class User < ApplicationRecord
>>   validates :name, :email, presence: true
>>
>>   after_commit :show_success_message
>>
>>   private
>>
>>     def show_success_message
>>       p 'User has been successfully saved into the database.'
>>     end
>> end

=> :show_success_message

>> user = User.create(name: 'Jon Snow', email: 'jon@bigbinary.com')
begin transaction
User Create (0.8ms)  INSERT INTO "users" ("name", "email", "created_at", "updated_at") VALUES (?, ?, ?, ?)  [["name", "Jon Snow"], ["email", "jon@bigbinary.com"], ["created_at", "2019-07-14 15:35:33.517694"], ["updated_at", "2019-07-14 15:35:33.517694"]]
commit transaction
"User has been successfully saved into the database."

=> #<User id: 1, name: "Jon Snow", email: "jon@bigbinary.com", created_at: "2019-07-14 15:35:33", updated_at: "2019-07-14 15:35:33">

>> User.transaction do
>>   user.email = nil
>>   p user.valid?
>>   user.save
>> end
begin transaction
false
commit transaction
"User has been successfully saved into the database."

=> false

As we can see here, that that the after_commit callback show_success_message was called even if object was never saved in the transaction.

Rails 6.0.0.rc1

Now, let’s try the same thing in Rails 6.

>> class User < ApplicationRecord
>>   validates :name, :email, presence: true
>>
>>   after_commit :show_success_message
>>
>>   private
>>
>>     def show_success_message
>>       p 'User has been successfully saved into the database.'
>>     end
>> end

=> :show_success_message

>> user = User.create(name: 'Jon Snow', email: 'jon@bigbinary.com')
SELECT sqlite_version(*)
begin transaction
User Create (1.0ms)  INSERT INTO "users" ("name", "email", "created_at", "updated_at") VALUES (?, ?, ?, ?)  [["name", "Jon Snow"], ["email", "jon@bigbinary.com"], ["created_at", "2019-07-14 15:40:54.022045"], ["updated_at", "2019-07-14 15:40:54.022045"]]
commit transaction
"User has been successfully saved into the database."

=> #<User id: 1, name: "Jon Snow", email: "jon@bigbinary.com", created_at: "2019-07-14 15:40:54", updated_at: "2019-07-14 15:40:54">

>> User.transaction do
>>   user.email = nil
>>   p user.valid?
>>   user.save
>>   end
false

=> false

Now, we can see that after_commit callback was never called if the object was not saved.

Here is the relevant issue and the pull request.


Ruby 2.7 adds Enumerable#tally

This blog is part of our Ruby 2.7 series. Ruby 2.7.0 was released on Dec 25, 2019.

Let’s say that we have to find the frequency of each element of an array.

Before Ruby 2.7, we could have achived it using group_by or inject.

irb> scores = [100, 35, 70, 100, 70, 30, 35, 100, 45, 30]

# we can use group_by to group the scores

irb> scores.group_by { |v| v }.map { |k, v| [k, v.size] }.to_h
=> {100=>3, 35=>2, 70=>2, 30=>2, 45=>1}

# or we can use inject to group the scores

irb> scores.inject(Hash.new(0)) {|hash, score| hash[score] += 1; hash }
=> {100=>3, 35=>2, 70=>2, 30=>2, 45=>1}

Ruby 2.7

Ruby 2.7 adds Enumerable#tally which can be used to find the frequency. Tally makes the code more readable and intutive. It returns a hash where keys are the unique elements and values are its corresponding frequency.

irb> scores = [100, 35, 70, 100, 70, 30, 35, 100, 45, 30]
irb> scores.tally
=> {100=>3, 35=>2, 70=>2, 30=>2, 45=>1}

Check out the github commit for more details on this.


Rails 6.1 introduces class_names helper

This blog is part of our Rails 6.1 series.

Rails 6.1 adds class_names view helper method to conditionally add CSS classes. class_names helper accepts String, Hash and Array as arguments and returns string of class names built from arguments.

Before Rails 6.1, conditional classes were added by using conditional statements. Let’s take an example of adding an active class to navigation link based on the current page.

Rails 6.0.0

<li class="<%= current_page?(dashboards_path) ? 'active' : '' %>">
  <%= link_to "Home", dashboards_path %>
</li>

Rails 6.1.0

>> class_names(active: current_page?(dashboards_path))
=> "active"

# Default classes can be added with conditional classes
>> class_names('navbar', { active: current_page?(dashboards_path) })
=> "navbar active"

# class_names helper rejects empty strings, nil, false arguments.
>> class_names(nil, '', false, 'navbar' {active: current_page?(dashboards_path)})
=> "navbar active"
<li class="<%= class_names(active: current_page?(dashboards_path)) %>">
  <%= link_to "Home", dashboards_path %>
</li>

Check out the pull request for more details on this.


Rails Multiple Polymorphic Joins

Having polymorphic associations in Rails can be a hard nut to crack. It enforces restriction on joins association which makes it difficult to write complex queries.

Consider following architecture where Defects can be of InspectedTruck or InspectedTrailer associated polymorphically.

  class InspectedTruck
    has_many :defects, as: :associated_object
  end

  class InspectedTrailer
    has_many :defects, as: :associated_object
  end

  class Defect
    belongs_to :associated_object, polymorphic: true
  end

Finding defects for inspected trucks using joins will raise error.

  => Defect.joins(:associated_object).load
  ActiveRecord::EagerLoadPolymorphicError: Cannot eagerly load the polymorphic association :associated_object

We need to write a raw sql INNER JOIN to fetch trucks with defects. Following query runs perfectly fine.

  sql = "INNER JOIN inspected_trucks ON inspected_trucks.id = defects.associated_object_id"
  Defect.joins(sql).load

We faced a scenario in one of our applications with multiple polymorphic joins. We needed to build a single query which lists vehicle inspection time, truck or trailer number and defect name (if available on the inspected item).

  class Truck
    # attributes :number
    has_many :inspected_trucks
  end

  class Trailer
    # attributes :number
    has_many :inspected_trailers
  end

  class VehicleInspectionReport
    # attribites :inspection_time
    has_one :inspected_truck, class_name: "InspectedTruck"
    has_many :inspected_trailers, class_name: "InspectedTrailer"
  end

  class InspectedTruck
    belongs_to :truck
    has_many :defects, as: :associated_object
  end

  class InspectedTrailer
    belongs_to :trailer
    has_many :defects, as: :associated_object
  end

  class Defect
    # attributes :name
    belongs_to :associated_object, polymorphic: true
  end

The task here was to query VehicleInspectionReport joining other five different tables and select required attributes to show. But the challenge here was posed by polymorphic association.

We had to come up with a way to query InspectedTruck and InspectedTrailer as a single dataset. We identified the dataset a kind of Single Table Inheritance (STI) dataset. And came up with following subquery.

  SELECT id AS associated_object_id, 'InspectedTruck' AS associated_object_type, vehicle_inspection_report_id, truck_id, NULL trailer_id
  FROM inspected_trucks
    UNION
  SELECT id AS associated_object_id, 'InspectedTrailer' AS associated_object_type, vehicle_inspection_report_id, NULL truck_id, trailer_id
  FROM inspected_trailers

This subquery gave us all inspected items in a single dataset and we could refer this dataset in a form of STI.

We were then able to build the final query using above subquery.

Add a scope in VehicleInspectionReport to join inspected items.

  class VehicleInspectionReport
    # attribites :inspection_time

    INSPECTED_ITEMS_RAW_SQL = "(
                              SELECT id, 'InspectedTruck' AS object_type, vehicle_inspection_report_id, truck_id, NULL trailer_id
                                FROM inspected_trucks
                              UNION
                              SELECT id, 'InspectedTrailer' AS object_type, vehicle_inspection_report_id, NULL truck_id, trailer_id
                                FROM inspected_trailers
                            ) AS inspected_items"

    has_one :inspected_truck, class_name: "InspectedTruck"
    has_many :inspected_trailers, class_name: "InspectedTrailer"

    scope :joins_with_inspected_items, -> { joins("INNER JOIN #{INSPECTED_ITEMS_RAW_SQL} ON vehicle_inspection_reports.id = inspected_items.vehicle_inspection_report_id") }
  end

joins_with_inspected_items scope on VehicleInspectionReport will work in a way of joining a STI table (inspected_items) on VehicleInspectionReport. We can now chain any query which require inspected items. Example:

  VehicleInspectionReport.select("defects.id AS defect_id,
                                  defects.name AS description,
                                  trucks.truck_number AS truck_number,
                                  trailers.number AS trailer_number,
                                  vehicle_inspection_reports.inspection_time AS inspection_time")
        .joins_with_inspected_items
        .joins("LEFT JOIN defects ON inspected_items.id = defects.associated_object_id
                  AND defects.associated_object_type = inspected_items.object_type")
        .joins("LEFT JOIN trucks ON inspected_items.truck_id = trucks.id")
        .joins("LEFT JOIN trailers ON inspected_items.trailer_id = trailers.id")
        .where("inspected_items.id IS NOT NULL")
        .order('truck_number, trailer_number, inspection_time DESC')

The underlying concept here is to structure STI dataset from polymorphic architecture. Notice the use of inspected_items dataset in a form of STI using inspected_items.associated_object_id AND inspected_items.associated_object_type.


Rails 6 adds rails db:prepare to migrate or setup a database

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Rails 6 adds rails db:prepare to migrate or setup a database if it doesn’t exist.

Before Rails 6, we had to run the following tasks to set up the database.

# create the database
rails db:create

# run the migrations
rails db:migrate

# prepopulate the database with initial/default data
rails db:seed  

Rails 6

Rails 6, adds rails db:prepare to get rid of running all the above tasks individually. rails db:prepare first calls the migrate to run the migrations, but if the database doesn’t exist, migrate throws an ActiveRecord::NoDatabaseError. Once it is catched, it performs the following operations:

  • Creates the database.
  • Loads the schema.
  • Seeds the database.

Thus, rails db:prepare saves a lot of time spent on running database tasks individually while setting up an application and finishes it with just one command.

Here is the relevant pull request.


Rails 6.1 adds *_previously_was attribute methods

This blog is part of our Rails 6.1 series.

Rails 6.1 adds *_previously_was attribute methods for dirty tracking the previous attribute value after the model is saved or reset. *_previously_was returns the previous attribute value that was changed before the model was saved

Before Rails 6.1, to retrieve the previous attribute value, we used *_previous_change or previous_changes.

Here is how it can be used.

Rails 6.0.0

>> user = User.new
=> #<User id: nil, name: nil, email: nil, created_at: nil, updated_at: nil>

>> user.name = "Sam"

# *_was returns the original value. In this example, the name was initially nil.
>> user.name_was
=> nil
>> user.save!

# After save, the original value is set to "Sam". To retrieve the
# previous value, we had to use `previous_changes`.
>> user.previous_changes[:name]
=> [nil, "Sam"]

Rails 6.1.0

>> user = User.find_by(name: "Sam")
=> #<User id: 1, name: "Sam", email: nil, created_at: "2019-10-14 17:53:06", updated_at: "2019-10-14 17:53:06">

>> user.name = "Nick"
>> user.name_was
=> "Sam"

>> user.save!

>> user.previous_changes[:name]
=> ["Sam", "Nick"]

# *_previously_was returns the previous value.
>> user.name_previously_was
=> "Sam"

# After reload, all the dirty tracking
# attributes is reset.
>> user.reload
>> user.name_previously_was
=> nil

Check out the pull request for more details on this.


Rails 6 adds guard against DNS Rebinding attacks

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

In a DNS Rebinding attack, a malicious webpage runs client-side script when it is loaded, to attack endpoints within a given network.

What is DNS Rebinding attack?

DNS Rebinding can be summarized as follows.

  • An unsuspecting victim is tricked into loading rebinding.network which is resolved by a DNS server controlled by a malicious entity.
  • Victims web browser sends a DNS query and gets the real IP address, say 24.56.78.99 of http://rebinding.network. This DNS server also sets a very short TTL value ( say 1 second ) on the response so that the client won’t cache this response for long.
  • The script on this webpage cannot attack services running in local network due to CORS restrictions imposed by victims web browser. Instead it starts sending a suspicious POST request to http://rebinding.network/setup/reboot with a JSON payload {params: factory-reset}.
  • First few requests are indeed sent to 24.56.78.99 (real IP address), with the DNS info from the cache, but then the browser sends out a DNS query for rebinding.network when it observes that the cache has gone stale.
  • When the malicious DNS server gets the request for a second time, instead of responding with 24.56.78.99 (which is the real IP address of rebinding.network), it responds with 192.168.1.90, an address at which, a poorly secured smart device runs.

Using this exploit, an attacker is able to factory-reset a device which relied on security provided by local network.

This attack is explained in much more detail in this blog post.

How does it affect Rails?

Rails’s web console was particularly vulnerable to a Remote Code Execution (RCE) via a DNS Rebinding.

In this blog post, Ben Murphy goes into technical details of exploiting this vulnerability to open Calculator app (only works in OS X).

How does Rails 6 mitigate DNS Rebinding?

Rails mitigates DNS Rebinding attack by maintaining a whitelist of domains from which it can receive requests. This is achieved with a new HostAuthorization middleware. This middleware leverages the fact that HOST request header is a forbidden header.

# taken from Rails documentation

# Allow requests from subdomains like `www.product.com` and
# `beta1.product.com`.
Rails.application.config.hosts << ".*\.product\.com/"

In the above example, Rails would render a blocked host template, if it receives requests from domains outside of above whitelist.

In development environment, default whitelist includes 0.0.0.0/0, ::0 (CIDR notations for IPv4 and IPv6 default routes) and localhost. For all other environments, config.hosts is empty and host header checks are not done.


Rails 6 adds ActiveStorage::Blob#open

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Rails 6 adds ActiveStorage::Blob#open which downloads a blob to a tempfile on disk and yields the tempfile.

>> blob = ActiveStorage::Blob.first
=> <ActiveStorage::Blob id: 1, key: "6qXeoibkvohP4VJiU4ytaEkH", filename: "Screenshot 2019-08-26 at 10.24.40 AM.png", ..., created_at: "2019-08-26 09:57:30">

>> blob.open do |tempfile|
>>   puts tempfile.path  #do some processing
>> end
# Output: /var/folders/67/3n96myxs1rn5q_c47z7dthj80000gn/T/ActiveStorage-1-20190826-73742-mve41j.png

Processing a blob

Let’s take an example of a face detection application where the user images are uploaded. Let’s assume that the images are uploaded on S3.

Before Rails 6, we will have to download the image in system’s memory, process it with an image processing program and then send the processed image back to the S3 bucket.

The overhead

If the processing operation is successful, the original file can be deleted from the system. We need to take care of a lot of uncertain events from the download phase till the phase when the processed image is created.

ActiveStorage::Blob#open to the rescue

ActiveStorage::Blob#open, abstracts away all this complications and gives us a tempfile which is closed and unlinked once the block is executed.

 1. open takes care of handling all the fanfare of getting a blob object to a tempfile.  2. open takes care of the tempfile cleanup after the block.

>> blob = ActiveStorage::Blob.first
>> blob.open do |tempfile|
>>   tempfile  #do some processing
>> end
   # once the given block is executed
   # the tempfile is closed and unlinked

=> #<Tempfile: (closed)> 

By default, tempfiles are created in Dir.tmpdir directory, but ActiveStorage::Blob#open also takes an optional argument tmpdir to set a custom directory for storing the tempfiles.

>> Dir.tmpdir
=> "/var/folders/67/3n96myxs1rn5q_c47z7dthj80000gn/T"

>> blob = ActiveStorage::Blob.first
>> blob.open(tmpdir: "/desired/path/to/save") do |tempfile|
>>   puts tempfile.path  #do some processing
>> end

Here is the relevant commit.


Rails 6 adds ActionMailer#email_address_with_name

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

When using ActionMailer::Base#mail, if we want to display name and email address of the user in email, we can pass a string in format "John Smith" <john@example.com> in to, from or reply_to options.

Before Rails 6, we had to join name and email address using string interpolation as mentioned in Rails 5.2 Guides and shown below.

  email_with_name = %("John Smith" <john@example.com>)
  mail(
    to: email_with_name,
    subject: 'Hey Rails 5.2!'
  )

Problem with string interpolation is it doesn’t escape unexpected special characters like quotes(“) in the name.

Here’s an example.

Rails 5.2

irb(main):001:0> %("John P Smith" <john@example.com>)
=> "\"John P Smith\" <john@example.com>"

irb(main):002:0> %('John "P" Smith' <john@example.com>)
=> "'John \"P\" Smith' <john@example.com>"

Rails 6 adds ActionMailer::Base#email_address_with_name to join name and email address in the format "John Smith" <john@example.com> and take care of escaping special characters.

Rails 6.1.0.alpha

irb(main):001:0> ActionMailer::Base.email_address_with_name("john@example.com", "John P Smith")
=> "John P Smith <john@example.com>"

irb(main):002:0> ActionMailer::Base.email_address_with_name("john@example.com", 'John "P" Smith')
=> "\"John \\\"P\\\" Smith\" <john@example.com>"
  mail(
    to: email_address_with_name("john@example.com", "John Smith"),
    subject: 'Hey Rails 6!'
  )

Here’s the relevant pull request for this change.


Rails 6 raises ArgumentError if param contains colon

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

The :param option in routes is used to override default resource identifier i.e. :id.

Let’s take for an example that we want product :name to be as the default resource identifier instead of :id while defining routes for products. In this case, :param option comes handy. We will see below how we can use this option.

Before Rails 6, if resource custom param contains a colon, Rails used to consider that as an extra param which should not be the case because it sneaks in an extra param.

An issue was raised in Aug, 2017 which was later fixed in February this year.

So, now Rails 6 raises ArgumentError if a resource custom param contains a colon(:).

Let’s checkout how it works.

Rails 5.2

Let’s create routes for products with custom param as name/:pzn.

>> Rails.application.routes.draw do
>>   resources :products, param: 'name/:pzn'
>> end
$ rake routes | grep products
products     GET    /products(.:format)                    products#index
             POST   /products(.:format)                    products#create
new_product  GET    /products/new(.:format)                products#new
edit_product GET    /products/:name/:pzn/edit(.:format)    products#edit
product      GET    /products/:name/:pzn(.:format)         products#show
             PATCH  /products/:name/:pzn(.:format)         products#update
             PUT    /products/:name/:pzn(.:format)         products#update
             DELETE /products/:name/:pzn(.:format)         products#destroy

As we can see, Rails also considers :pzn as a parameter.

Now let’s see how it works in Rails 6.

Rails 6.0.0.rc1

>> Rails.application.routes.draw do
>>   resources :products, param: 'name/:pzn'
>> end
$ rake routes | grep products

rake aborted!
ArgumentError: :param option can't contain colons
/Users/amit/.rvm/gems/ruby-2.6.3/gems/actionpack-6.0.0.rc1/lib/action_dispatch/routing/mapper.rb:1149:in `initialize'
/Users/amit/.rvm/gems/ruby-2.6.3/gems/actionpack-6.0.0.rc1/lib/action_dispatch/routing/mapper.rb:1472:in `new'
/Users/amit/.rvm/gems/ruby-2.6.3/gems/actionpack-6.0.0.rc1/lib/action_dispatch/routing/mapper.rb:1472:in `block in resources'
...
...
...

Here is the relevant issue and the pull request.


Rails 6 introduces new code loader called Zeitwerk

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Zeitwerk is the new code loader that comes with Rails 6 by default. In addition to providing autoloading, eager loading, and reloading capabilities, it also improves the classical code loader by being efficient and thread safe. According to the author of Zeitwerk, Xavier Noria, one of the main motivations for writing Zeitwerk was to keep code DRY and to remove the brittle require calls.

Zeitwerk is available as a gem with no additional dependencies. It means any regular Ruby project can use Zeitwerk.

How to use Zeitwerk

Zeitwerk is baked in a Rails 6 project, thanks to the Zeitwerk-Rails integration. For a non-Rails project, adding the following into the project’s entry point sets up Zeitwerk.

loader = Zeitwerk::Loader.new
loader.push_dir(...)
loader.setup

For gem maintainers, Zeitwerk provides the handy .for_gem utility method

The following example from Zeitwerk documentation illustrates the usage of Zeitwerk.for_gem method.

#lib/my_gem.rb (main file)

require "zeitwerk"
loader = Zeitwerk::Loader.for_gem
loader.setup

module MyGem
  # Since the setup has been performed, at this point we are already
  # able to reference project constants, in this case MyGem::MyLogger.
  include MyLogger
end

How does Zeitwerk work?

Before we look into Zeitwerk’s internals, the following section provides a quick refresher on constant-resolution in Ruby and how classical code loader of Rails works.

Ruby’s constant resolution looks for a constant in the following places.

  • In each entry of Module.nesting
  • In each entry of Module.ancestors

It triggers ‘constant_missing’ callback when it can’t find the constant.

Ruby used to look for constants in Object.ancestors as well, but that seems not the case anymore. An in-depth explanation of constant resolution can be found at Conrad Irwin’s blog.

Classical Code Loader in Rails

Classical code loader (code loader in Rails version < 6.0) achieves autoloading by overriding Module#const_missing and loads the missing constant without the need for an explicit require call as long as the code follows certain conventions.

  • The file should be within a directory in ActiveSupport::Dependencies.autoload_paths
  • A file should be named after the class, i.e Admin::RoutesController => admin/routes_controller.rb
Zeitwerk Mode

Zeitwerk takes an entirely different approach in autoloading by registering constants to be autoloaded by Ruby.

Consider the following configuration in which Zeitwerk manages lib directory and lib has automobile.rb file.

loader.push_dir('./lib')

Zeitwerk then uses Module.autoload to tell Ruby that “Automobile” can be found in “lib/automobile.rb”.

autoload "Automobile", "lib/automobile.rb"

Unlike classical loader, Zeitwerk takes module nesting into account while loading constants by leveraging the new Tracepoint API to go look for constants defined in subdirectories when a new class or module is defined.

Let us look at an example to understand this better.

class Automobile
  # => Tracepoint hook triggers here.
  # include Engine
end

When the tracepoint hook triggers, Zeitwerk checks for an automobile directory in the same level as automobile.rb and sets up Module.autoload for that directory and all the files (in this case ./automobile/engine.rb) within that directory.

Conclusion

Previously in Rails, we had a code loader that was riddled with gotchas and struggled to be thread safe. Zeitwerk does a better job by leveraging the new Ruby standard API and matches Ruby’s semantics for constants.


Rails 6 adds ActiveSupport::ActionableError

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

When working in a team on a Rails application, we often bump into PendingMigrationError or other errors that need us to run a rails command, rake task etc.

Rails introduced a way to resolve such frequent errors in development from error page itself.

Rails 6 added ActiveSupport::ActionableError module to define actions we want perform on errors, right from the error page.

For example, this is how PendingMigrationError page looks like in Rails 6.

How Actionable error looks like in Rails 6

By default, a button is added on error screen that says Run pending migrations. Clicking on this button would dispatch rails db:migrate action. Page will reload once migrations run successfully.

We can also define custom actions to execute on errors.

How to define actions on error?

We need to include ActiveSupport::ActionableError module in our error class. We can monkey patch an existing error class or define custom error class.

#action api is provided to define actions on error. First argument in #action is name of the action. This string would be displayed on the button on error page. Second argument is a block where we can write commands or code to fix the error.

Let’s take an example of seeding posts data from controller, if posts not already present.

# app/controllers/posts_controller.rb

class PostsController < ApplicationController

  def index
    @posts = Post.all
    if @posts.empty?
      raise PostsMissingError
    end
  end

end
# app/errors/posts_missing_error.rb

class PostsMissingError < StandardError

  include ActiveSupport::ActionableError

  action "seed posts data" do
    Rails::Command.invoke 'posts:seed'
  end

end
# lib/tasks/posts.rake

namespace :posts do

  desc 'posts seed task'
  task :seed do
    Post.create(title: 'First Post')
  end

end
# app/views/posts/index.html.erb

<% @posts.each do |post| %>
  <%= post.title %>
<% end %>

Let’s check /posts (posts#index action) when no posts are present. We would get an error page with an action button on it as shown below.

Actionable error - seed posts data

Clicking on seed posts data action button will run our rake task and create posts. Rails will automatically reload /posts after running rake task.

Posts index page

ActionDispatch::ActionableExceptions middleware takes care of invoking actions from error page. ActionableExceptions middleware dispatches action to ActionableError and redirects back when action block has successfully run. Action buttons are added on error page from this middleware template.

Checkout the pull request for more information on actionable error.


This is how our workspace looks like

BigBinary has been remote and flexible since the start, and it’s one of the best things a company can offer. You don’t need to spend hours commuting, you can work when you feel productive. Working remotely also means that you have the flexibility of working from Starbucks, from a library or from your home. You can set up your own workspace at home and still have the office-like feeling.

We recently got a chance to see workspaces of our colleagues and everyone shared photos of environments they work in on Slack and it was fun seeing everyone’s desk and the setup they have. We thought it would be fun to share a peek at the home offices we have. Here we go.

Akhil Gautam

BigBinary Remote Workspace

Amit Choudhary

BigBinary Remote Workspace

Chimed Palden

BigBinary Remote Workspace

Chirag Shah

BigBinary Remote Workspace BigBinary Remote Workspace

Ershad Kunnakkadan

BigBinary Remote Workspace

Mohit Natoo

BigBinary Remote Workspace

BigBinary Remote Workspace BigBinary Remote Workspace

Neeraj Singh

BigBinary Remote Workspace BigBinary Remote Workspace

Nitin Kalasannavar

BigBinary Remote Workspace

Paras Bansal

BigBinary Remote Workspace

Pranav Raj

BigBinary Remote Workspace

Prathamesh Sonpatki

BigBinary Remote Workspace

Rahul Mahale

BigBinary Remote Workspace

Rishi Mohan

BigBinary Remote Workspace BigBinary Remote Workspace BigBinary Remote Workspace BigBinary Remote Workspace BigBinary Remote Workspace

Shibin Madassery

BigBinary Remote Workspace

Sony Mathew

BigBinary Remote Workspace

Sunil Kumar

BigBinary Remote Workspace

Tyler and Naiara

BigBinary Remote Workspace

Unnikrishnan KP

BigBinary Remote Workspace

Vishal Telangre

BigBinary Remote Workspace


Rails 6 add_foreign_key & remove_foreign_key SQLite3

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Rails provides add_foreign_key to add foreign key constraint for a column on a table.

It also provides remove_foreign_key to remove the foreign key constraint.

Before Rails 6, add_foreign_key and remove_foreign_key were not supported for SQLite3.

Rails 6 now adds this support. Now, we can create and remove foreign key constraints using add_foreign_key and remove_foreign_key in SQLite3.

Let’s checkout how it works.

Rails 5.2

We have two tables named as orders and users. Now, let’s add foreign key constraint of users in orders table using add_foreign_key and then try removing it using remove_foreign_key.

>> class AddUserReferenceToOrders < ActiveRecord::Migration[6.0]
>>   def change
>>     add_column :orders, :user_id, :integer
>>     add_foreign_key :orders, :users
>>   end
>> end

=> :change

>> AddUserReferenceToOrders.new.change
-- add_column(:orders, :user_id, :integer)
   (1.2ms)  ALTER TABLE "orders" ADD "user_id" integer
   -> 0.0058s
-- add_foreign_key(:orders, :users)
   -> 0.0000s

=> nil

>> class RemoveUserForeignKeyFromOrders < ActiveRecord::Migration[6.0]
>>   def change
>>     remove_foreign_key :orders, :users
>>   end
>> end

=> :change

>> RemoveUserForeignKeyFromOrders.new.change
-- remove_foreign_key(:orders, :users)
   -> 0.0001s

=> nil

We can see that add_foreign_key and remove_foreign_key are ignored by Rails 5.2 with SQLite3.

Rails 6.0.0.rc1

We have two tables named as orders and users. Now, let’s add foreign key constraint of users in orders table using add_foreign_key.

>> class AddUserReferenceToOrders < ActiveRecord::Migration[6.0]
>>   def change
>>     add_column :orders, :user_id, :integer
>>     add_foreign_key :orders, :users
>>   end
>> end

=> :change

>> AddUserReferenceToOrders.new.change
-- add_column(:orders, :user_id, :integer)
   (1.0ms)  SELECT sqlite_version(*)
   (2.9ms)  ALTER TABLE "orders" ADD "user_id" integer
   -> 0.0091s
-- add_foreign_key(:orders, :users)
   (0.0ms)  begin transaction
   (0.1ms)  PRAGMA foreign_keys
   (0.1ms)  PRAGMA defer_foreign_keys
   (0.0ms)  PRAGMA defer_foreign_keys = ON
   (0.1ms)  PRAGMA foreign_keys = OFF
   (0.2ms)  CREATE TEMPORARY TABLE "aorders" ("id" integer NOT NULL PRIMARY KEY, "number" varchar DEFAULT NULL, "total" decimal DEFAULT NULL, "completed_at" datetime DEFAULT NULL, "created_at" datetime(6) NOT NULL, "updated_at" datetime(6) NOT NULL, "user_id" integer DEFAULT NULL)
   (0.1ms)  INSERT INTO "aorders" ("id","number","total","completed_at","created_at","updated_at","user_id")
                     SELECT "id","number","total","completed_at","created_at","updated_at","user_id" FROM "orders"
   (0.3ms)  DROP TABLE "orders"
   (0.1ms)  CREATE TABLE "orders" ("id" integer NOT NULL PRIMARY KEY, "number" varchar DEFAULT NULL, "total" decimal DEFAULT NULL, "completed_at" datetime DEFAULT NULL, "created_at" datetime(6) NOT NULL, "updated_at" datetime(6) NOT NULL, "user_id" integer DEFAULT NULL, CONSTRAINT "fk_rails_f868b47f6a"
FOREIGN KEY ("user_id")
  REFERENCES "users" ("id")
)
   (0.1ms)  INSERT INTO "orders" ("id","number","total","completed_at","created_at","updated_at","user_id")
                     SELECT "id","number","total","completed_at","created_at","updated_at","user_id" FROM "aorders"
   (0.1ms)  DROP TABLE "aorders"
   (0.0ms)  PRAGMA defer_foreign_keys = 0
   (0.0ms)  PRAGMA foreign_keys = 1
   (0.6ms)  commit transaction
   -> 0.0083s

=> []

>> class RemoveUserForeignKeyFromOrders < ActiveRecord::Migration[6.0]
>>   def change
>>     remove_foreign_key :orders, :users
>>   end
>> end

=> :change

>> RemoveUserForeignKeyFromOrders.new.change
-- remove_foreign_key(:orders, :users)
   (1.4ms)  SELECT sqlite_version(*)
   (0.0ms)  begin transaction
   (0.0ms)  PRAGMA foreign_keys
   (0.0ms)  PRAGMA defer_foreign_keys
   (0.0ms)  PRAGMA defer_foreign_keys = ON
   (0.0ms)  PRAGMA foreign_keys = OFF
   (0.2ms)  CREATE TEMPORARY TABLE "aorders" ("id" integer NOT NULL PRIMARY KEY, "number" varchar DEFAULT NULL, "total" decimal DEFAULT NULL, "completed_at" datetime DEFAULT NULL, "created_at" datetime(6) NOT NULL, "updated_at" datetime(6) NOT NULL, "user_id" integer DEFAULT NULL)
   (0.3ms)  INSERT INTO "aorders" ("id","number","total","completed_at","created_at","updated_at","user_id")
                     SELECT "id","number","total","completed_at","created_at","updated_at","user_id" FROM "orders"
   (0.4ms)  DROP TABLE "orders"
   (0.1ms)  CREATE TABLE "orders" ("id" integer NOT NULL PRIMARY KEY, "number" varchar DEFAULT NULL, "total" decimal DEFAULT NULL, "completed_at" datetime DEFAULT NULL, "created_at" datetime(6) NOT NULL, "updated_at" datetime(6) NOT NULL, "user_id" integer DEFAULT NULL)
   (0.1ms)  INSERT INTO "orders" ("id","number","total","completed_at","created_at","updated_at","user_id")
                     SELECT "id","number","total","completed_at","created_at","updated_at","user_id" FROM "aorders"
   (0.1ms)  DROP TABLE "aorders"
   (0.0ms)  PRAGMA defer_foreign_keys = 0
   (0.0ms)  PRAGMA foreign_keys = 1
   (0.7ms)  commit transaction
   -> 0.0179s

=> []

Now, let’s remove foreign key constraint of users from orders table using remove_foreign_key.

>> class RemoveUserForeignKeyFromOrders < ActiveRecord::Migration[6.0]
>>   def change
>>     remove_foreign_key :orders, :users
>>   end
>> end

=> :change

>> RemoveUserForeignKeyFromOrders.new.change
-- remove_foreign_key(:orders, :users)
   (1.4ms)  SELECT sqlite_version(*)
   (0.0ms)  begin transaction
   (0.0ms)  PRAGMA foreign_keys
   (0.0ms)  PRAGMA defer_foreign_keys
   (0.0ms)  PRAGMA defer_foreign_keys = ON
   (0.0ms)  PRAGMA foreign_keys = OFF
   (0.2ms)  CREATE TEMPORARY TABLE "aorders" ("id" integer NOT NULL PRIMARY KEY, "number" varchar DEFAULT NULL, "total" decimal DEFAULT NULL, "completed_at" datetime DEFAULT NULL, "created_at" datetime(6) NOT NULL, "updated_at" datetime(6) NOT NULL, "user_id" integer DEFAULT NULL)
   (0.3ms)  INSERT INTO "aorders" ("id","number","total","completed_at","created_at","updated_at","user_id")
                     SELECT "id","number","total","completed_at","created_at","updated_at","user_id" FROM "orders"
   (0.4ms)  DROP TABLE "orders"
   (0.1ms)  CREATE TABLE "orders" ("id" integer NOT NULL PRIMARY KEY, "number" varchar DEFAULT NULL, "total" decimal DEFAULT NULL, "completed_at" datetime DEFAULT NULL, "created_at" datetime(6) NOT NULL, "updated_at" datetime(6) NOT NULL, "user_id" integer DEFAULT NULL)
   (0.1ms)  INSERT INTO "orders" ("id","number","total","completed_at","created_at","updated_at","user_id")
                     SELECT "id","number","total","completed_at","created_at","updated_at","user_id" FROM "aorders"
   (0.1ms)  DROP TABLE "aorders"
   (0.0ms)  PRAGMA defer_foreign_keys = 0
   (0.0ms)  PRAGMA foreign_keys = 1
   (0.7ms)  commit transaction
   -> 0.0179s

=> []

We can see here that with Rails 6, add_foreign_key and remove_foreign_key work and were able to add and remove foreign key constraint respectively.

Here is the relevant pull request.


Rails 6 adds ActionDispatch::Request::Session#dig

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Rails 6 added ActionDispatch::Request::Session#dig.

This works the same way as Hash#dig.

It extracts the nested value specified by the sequence of keys.

Hash#dig was introduced in Ruby 2.3.

Before Rails 6, we can achieve the same thing by first converting session to a hash and then calling Hash#dig on it.

Let’s checkout how it works.

Rails 5.2

Let’s add some user information in session and use dig after converting it to a hash.

>> session[:user] = { email: 'jon@bigbinary.com', name: { first: 'Jon', last: 'Snow' }  }

=> {:email=>"jon@bigbinary.com", :name=>{:first=>"Jon", :last=>"Snow"}}

>> session.to_hash

=> {"session_id"=>"5fe8cc73c822361e53e2b161dcd20e47", "_csrf_token"=>"gyFd5nEEkFvWTnl6XeVbJ7qehgL923hJt8PyHVCH/DA=", "return_to"=>"http://localhost:3000", "user"=>{:email=>"jon@bigbinary.com", :name=>{:first=>"Jon", :last=>"Snow"}}}


>> session.to_hash.dig("user", :name, :first)

=> "Jon"

Rails 6.0.0.rc1

Let’s add the same information to session and now use dig on session object without converting it to a hash.

>> session[:user] = { email: 'jon@bigbinary.com', name: { first: 'Jon', last: 'Snow' }  }

=> {:email=>"jon@bigbinary.com", :name=>{:first=>"Jon", :last=>"Snow"}}

>> session.dig(:user, :name, :first)

=> "Jon"

Here is the relevant pull request.