San Francisco, USA

5214F Diamond Heights Blvd #553
San Francisco, CA 94131

Pune, India

203, Jewel Towers, 2nd Floor
Lane Number 5, Koregaon Park
Pune 411001, India

301 - 275 - 3997
hello@BigBinary.com

Rails 6 adds ActionDispatch::Request::Session#dig

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Rails 6 added ActionDispatch::Request::Session#dig.

This works the same way as Hash#dig.

It extracts the nested value specified by the sequence of keys.

Hash#dig was introduced in Ruby 2.3.

Before Rails 6, we can achieve the same thing by first converting session to a hash and then calling Hash#dig on it.

Let’s checkout how it works.

Rails 5.2

Let’s add some user information in session and use dig after converting it to a hash.

>> session[:user] = { email: 'jon@bigbinary.com', name: { first: 'Jon', last: 'Snow' }  }

=> {:email=>"jon@bigbinary.com", :name=>{:first=>"Jon", :last=>"Snow"}}

>> session.to_hash

=> {"session_id"=>"5fe8cc73c822361e53e2b161dcd20e47", "_csrf_token"=>"gyFd5nEEkFvWTnl6XeVbJ7qehgL923hJt8PyHVCH/DA=", "return_to"=>"http://localhost:3000", "user"=>{:email=>"jon@bigbinary.com", :name=>{:first=>"Jon", :last=>"Snow"}}}


>> session.to_hash.dig("user", :name, :first)

=> "Jon"

Rails 6.0.0.rc1

Let’s add the same information to session and now use dig on session object without converting it to a hash.

>> session[:user] = { email: 'jon@bigbinary.com', name: { first: 'Jon', last: 'Snow' }  }

=> {:email=>"jon@bigbinary.com", :name=>{:first=>"Jon", :last=>"Snow"}}

>> session.dig(:user, :name, :first)

=> "Jon"

Here is the relevant pull request.


Rails 6 marks arrays of translations as trusted safe by using the '_html' suffix

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Before Rails 6

Before Rails 6, keys with the _html suffix in the language locale files are automatically marked as HTML safe. These HTML safe keys do not get escaped when used in the views.

# config/locales/en.yml

en:
  home:
    index:
      title_html: <h2>We build web & mobile applications</h2>
      description_html: We are a dynamic team of <em>developers</em> and <em>designers</em>.
      sections:
        blogs:
          title_html: <h3>Blogs & publications</h3>
          description_html: We regularly write our blog. Our blogs are covered by <strong>Ruby Inside</strong> and <strong>Ruby Weekly Newsletter</strong>.
<!-- app/views/home/index.html.erb -->

<%= t('.title_html') %>
<%= t('.description_html') %>

<%= t('.sections.blogs.title_html') %>
<%= t('.sections.blogs.description_html') %>

Once rendered, this page looks like this.

rails-6-supports-marking-arrays-of-translations-as-html-safe/before-rails-6-i18n-_html-suffix-without-array-key.png

This way of marking translations as HTML safe by adding _html suffix to the keys does not work as expected when the value is an array.

# config/locales/en.yml

en:
  home:
    index:
      title_html: <h2>We build web & mobile applications</h2>
      description_html: We are a dynamic team of <em>developers</em> and <em>designers</em>.
      sections:
        blogs:
          title_html: <h3>Blogs & publications</h3>
          description_html: We regularly write our blog. Our blogs are covered by <strong>Ruby Inside</strong> and <strong>Ruby Weekly Newsletter</strong>.
        services:
          title_html: <h3>Services we offer</h3>
          list_html:
            - <strong>Ruby on Rails</strong>
            - React.js &#9883;
            - React Native &#9883; &#128241;
<!-- app/views/home/index.html.erb -->

<%= t('.title_html') %>
<%= t('.description_html') %>

<%= t('.sections.blogs.title_html') %>
<%= t('.sections.blogs.description_html') %>

<%= t('.sections.services.title_html') %>
<ul>
  <% t('.sections.services.list_html').each do |service| %>
    <li><%= service %></li>
  <% end %>
<ul>

The rendered page escapes the unsafe HTML while rendering the array of translations for the key .sections.services.list_html even though that key has the _html suffix.

rails-6-supports-marking-arrays-of-translations-as-html-safe/before-rails-6-i18n-_html-suffix-with-array-key.png

A workaround is to manually mark all the translations in that array as HTML safe using the methods such as #raw or #html_safe.

<!-- app/views/home/index.html.erb -->

<%= t('.title_html') %>
<%= t('.description_html') %>

<%= t('.sections.blogs.title_html') %>
<%= t('.sections.blogs.description_html') %>

<%= t('.sections.services.title_html') %>
<ul>
  <% t('.sections.services.list_html').each do |service| %>
    <li><%= service.html_safe %></li>
  <% end %>
<ul>

rails-6-supports-marking-arrays-of-translations-as-html-safe/rails-6-i18n-array-key-with-_html-suffix.png

Arrays of translations are trusted as HTML safe by using the ‘_html’ suffix in Rails 6

In Rails 6, the unexpected behavior of not marking an array of translations as HTML safe even though the key of that array has the _html suffix is fixed.

# config/locales/en.yml

en:
  home:
    index:
      title_html: <h2>We build web & mobile applications</h2>
      description_html: We are a dynamic team of <em>developers</em> and <em>designers</em>.
      sections:
        blogs:
          title_html: <h3>Blogs & publications</h3>
          description_html: We regularly write our blog. Our blogs are covered by <strong>Ruby Inside</strong> and <strong>Ruby Weekly Newsletter</strong>.
        services:
          title_html: <h3>Services we offer</h3>
          list_html:
            - <strong>Ruby on Rails</strong>
            - React.js &#9883;
            - React Native &#9883; &#128241;
<!-- app/views/home/index.html.erb -->

<%= t('.title_html') %>
<%= t('.description_html') %>

<%= t('.sections.blogs.title_html') %>
<%= t('.sections.blogs.description_html') %>

<%= t('.sections.services.title_html') %>
<ul>
  <% t('.sections.services.list_html').each do |service| %>
    <li><%= service %></li>
  <% end %>
<ul>

rails-6-supports-marking-arrays-of-translations-as-html-safe/rails-6-i18n-array-key-with-_html-suffix.png

We can see above that we no longer need to manually mark the translations as HTML safe for the key .sections.services.title_html using the methods such as #raw or #html_safe since that key has the _html suffix.


To learn more about this feature, please checkout rails/rails#32361.


Rails 6 adds filter_attributes on ActiveRecord::Base

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

A lot of times, we ask user for sensitive data such as password, credit card number etc. We should not be able to see this information in logs. So, there must be a way in Rails to filter out these parameters from logs.

Rails provides a way of doing this. We can add parameters to Rails.application.config.filter_parameters.

There is one more way of doing this in Rails. We can also use https://api.rubyonrails.org/classes/ActionDispatch/Http/FilterParameters.html.

However there is still a security issue when we call inspect on an ActiveRecord object for logging purposes. In this case, Rails does not consider Rails.application.config.filter_parameters and displays the sensitive information.

Rails 6 fixes this. It considers Rails.application.config.filter_parameters while inspecting an object.

Rails 6 also provides an alternative way to filter columns on ActiveRecord level by adding filter_attributes on ActiveRecord::Base.

In Rails 6, filter_attributes on ActiveRecord::Base takes priority over Rails.application.config.filter_parameters.

Let’s checkout how it works.

Rails 6.0.0.rc1

Let’s create a user record and call inspect on it.

>> class User < ApplicationRecord
>>  validates :email, :password, presence: true
>> end

=> {:presence=>true}

>> User.create(email: 'john@bigbinary.com', password: 'john_wick_bigbinary')
BEGIN
  User Create (0.6ms)  INSERT INTO "users" ("email", "password", "created_at", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id"  [["email", "john@bigbinary.com"], ["password", "john_wick_bigbinary"], ["created_at", "2019-05-17 21:34:34.504394"], ["updated_at", "2019-05-17 21:34:34.504394"]]
COMMIT

=> #<User id: 2, email: "john@bigbinary.com", password: [FILTERED], created_at: "2019-05-17 21:34:34", updated_at: "2019-05-17 21:34:34">

We can see that password is filtered as it is added to Rails.application.config.filter_parameters by default in config/initializers/filter_parameter_logging.rb.

Now let’s add just :email to User.filter_attributes

>> User.filter_attributes = [:email]

=> [:email]

>> User.first.inspect
SELECT "users".* FROM "users" ORDER BY "users"."id" ASC LIMIT $1  [["LIMIT", 1]]

=> "#<User id: 2, email: [FILTERED], password: \"john_wick_bigbinary\", created_at: \"2019-05-17 21:34:34\", updated_at: \"2019-05-17 21:34:34\">"

We can see here that User.filter_attributes took priority over Rails.application.config.filter_parameters and removed filtering from password and filtered just email.

Now, let’s add both :email and :password to User.filter_attributes.

>> User.filter_attributes = [:email, :password]

=> [:email, :password]

>> User.first.inspect

=> "#<User id: 2, email: [FILTERED], password: [FILTERED], created_at: \"2019-05-17 21:34:34\", updated_at: \"2019-05-17 21:34:34\">"

We can see that now both email and password are filtered out.

Here is the relevant pull request.


Rails 6 raises ArgumentError for invalid :limit and :precision

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Rails 6 raises ArgumentError when :limit and :precision are used with invalid datatypes.

Before Rails 6, it used to return ActiveRecord::ActiveRecordError.

Let’s checkout how it works.

Rails 5.2

Let’s create an orders table and try using :limit with a column named as quantity with data type integer.

>> class CreateOrders < ActiveRecord::Migration[5.2]
>>   def change
>>     create_table :orders do |t|
>>       t.string :item
>>       t.integer :quantity, limit: 10
>>
>>       t.timestamps
>>     end
>>   end
>> end

=> :change

>> CreateOrders.new.change
-- create_table(:orders)

=> Traceback (most recent call last):
        2: from (irb):11
        1: from (irb):3:in 'change'
ActiveRecord::ActiveRecordError (No integer type has byte size 10. Use a numeric with scale 0 instead.)

We can see that use of :limit with integer column raises ActiveRecord::ActiveRecordError in Rails 5.2.

Now let’s try using :precision of 10 with a datetime column.

>> class CreateOrders < ActiveRecord::Migration[5.2]
>>   def change
>>     create_table :orders do |t|
>>       t.string :item
>>       t.integer :quantity
>>       t.datetime :completed_at, precision: 10
>>
>>       t.timestamps
>>     end
>>   end
>> end

=> :change

>> CreateOrders.new.change
-- create_table(:orders)

=> Traceback (most recent call last):
        2: from (irb):12
        1: from (irb):3:in 'change'
ActiveRecord::ActiveRecordError (No timestamp type has precision of 10. The allowed range of precision is from 0 to 6)

We can see that invalid value of :precision with datetime column also raises ActiveRecord::ActiveRecordError in Rails 5.2.

Rails 6.0.0.rc1

Let’s create an orders table and try using :limit with a column named as quantity with data type integer in Rails 6.

>> class CreateOrders < ActiveRecord::Migration[6.0]
>>   def change
>>     create_table :orders do |t|
>>       t.string :item
>>       t.integer :quantity, limit: 10
>>
>>       t.timestamps
>>     end
>>   end
>> end

=> :change

>> CreateOrders.new.change
-- create_table(:orders)

=> Traceback (most recent call last):
        2: from (irb):11
        1: from (irb):3:in 'change'
ArgumentError (No integer type has byte size 10. Use a numeric with scale 0 instead.)

We can see that use of :limit with integer column raises ArgumentError in Rails 6.

Now let’s try using :precision of 10 with a datetime column.

>> class CreateOrders < ActiveRecord::Migration[6.0]
>>   def change
>>     create_table :orders do |t|
>>       t.string :item
>>       t.integer :quantity
>>       t.datetime :completed_at, precision: 10
>>
>>       t.timestamps
>>     end
>>   end
>> end

=> :change

>> CreateOrders.new.change
-- create_table(:orders)

=> Traceback (most recent call last):
        2: from (irb):12
        1: from (irb):3:in 'change'
ArgumentError (No timestamp type has precision of 10. The allowed range of precision is from 0 to 6)

We can see that invalid value of :precision with datetime column also raises ArgumentError in Rails 6.

Here is the relevant pull request.


Rails 6 allows passing custom configuration to ActionCable::Server::Base

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Before Rails 6, Action Cable server used default configuration on boot up, unless custom configuration is provided explicitly.

Custom configuration can be mentioned in either config/cable.yml or config/application.rb as shown below.

# config/cable.yml

production:
  url: redis://redis.example.com:6379
  adapter: redis
  channel_prefix: custom_

Or

# config/application.rb

config.action_cable.cable = { adapter: "redis", channel_prefix: "custom_" }

In some cases, we need another Action Cable server running separately from application with a different set of configuration.

Problem is that both approaches mentioned earlier set Action Cable server configuration on application boot up. This configuration can not be changed for the second server.

Rails 6 has added a provision to pass custom configuration. Rails 6 allows us to pass ActionCable::Server::Configuration object as an option when initializing a new Action Cable server.

config = ActionCable::Server::Configuration.new
config.cable = { adapter: "redis", channel_prefix: "custom_" }

ActionCable::Server::Base.new(config: config)

For more details on Action Cable configurations, head to Action Cable docs.

Here’s the relevant pull request for this change.


Rails 6 adds support of symbol keys with ActiveSupport::HashWithIndifferentAccess#assoc

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Rails 6 added support of symbol keys with ActiveSupport::HashWithIndifferentAccess#assoc.

Please note that documentation of ActiveSupport::HashWithIndifferentAccess#assoc in Rails 5.2 shows that ActiveSupport::HashWithIndifferentAccess#assoc works with symbol keys but it doesn’t.

In Rails 6, ActiveSupport::HashWithIndifferentAccess implements a hash where string and symbol keys are considered to be the same.

Before Rails 6, HashWithIndifferentAccess#assoc used to work with just string keys.

Let’s checkout how it works.

Rails 5.2

Let’s create an object of ActiveSupport::HashWithIndifferentAccess and call assoc on that object.

>> info = { name: 'Mark', email: 'mark@bigbinary.com' }.with_indifferent_access

=> {"name"=>"Mark", "email"=>"mark@bigbinary.com"}

>> info.assoc(:name)

=> nil

>> info.assoc('name')

=> ["name", "Mark"]

We can see that assoc does not work with symbol keys with ActiveSupport::HashWithIndifferentAccess in Rails 5.2.

Rails 6.0.0.beta2

Now, let’s call assoc on the same hash in Rails 6 with both string and symbol keys.

>> info = { name: 'Mark', email: 'mark@bigbinary.com' }.with_indifferent_access

=> {"name"=>"Mark", "email"=>"mark@bigbinary.com"}

>> info.assoc(:name)

=> ["name", "Mark"]

>> info.assoc('name')

=> ["name", "Mark"]

As we can see, assoc works perfectly fine with both string and symbol keys with ActiveSupport::HashWithIndifferentAccess in Rails 6.

Here is the relevant pull request.


Rails 6 preserves status of #html_safe? on sliced and multiplied HTML safe strings

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Before Rails 6

Before Rails 6, calling #html_safe? on a slice of an HTML safe string returns nil.

>> html_content = "<div>Hello, world!</div>".html_safe
# => "<div>Hello, world!</div>"
>> html_content.html_safe?
# => true
>> html_content[0..-1].html_safe?
# => nil

Also, before Rails 6, the ActiveSupport::SafeBuffer#* method does not preserve the HTML safe status as well.

>> line_break = "<br />".html_safe
# => "<br />"
>> line_break.html_safe?
# => true
>> two_line_breaks = (line_break * 2)
# => "<br /><br />"
>> two_line_breaks.html_safe?
# => nil

Rails 6 returns expected status of #html_safe?

In Rails 6, both of the above cases have been fixed properly.

Therefore, we will now get the status of #html_safe? as expected.

>> html_content = "<div>Hello, world!</div>".html_safe
# => "<div>Hello, world!</div>"
>> html_content.html_safe?
# => true
>> html_content[0..-1].html_safe?
# => true

>> line_break = "<br />".html_safe
# => "<br />"
>> line_break.html_safe?
# => true
>> two_line_breaks = (line_break * 2)
# => "<br /><br />"
>> two_line_breaks.html_safe?
# => true

Please check rails/rails#33808 and rails/rails#36012 for the relevant changes.


Recyclable cache keys in Rails

Recyclable cache keys or cache versioning was introduced in Rails 5.2. Large applications frequently need to invalidate their cache because cache store has limited memory. We can optimize cache storage and minimize cache miss using recyclable cache keys.

Recyclable cache keys is supported by all cache stores that ship with Rails.

Before Rails 5.2, cache_key’s format was {model_name}/{id}-{update_at}. Here model_name and id are always constant for an object and updated_at changes on every update.

Rails 5.1

>> post = Post.last

>> post.cache_key
=> "posts/1-20190522104553296111"

# Update post
>> post.touch

>> post.cache_key
=> "posts/1-20190525102103422069" # cache_key changed

In Rails 5.2, #cache_key returns {model_name}/{id} and new method #cache_version returns {updated_at}.

Rails 5.2

>> ActiveRecord::Base.cache_versioning = true

>> post = Post.last

>> post.cache_key
=> "posts/1"

>> post.cache_version
=> "20190522070715422750"

>> post.cache_key_with_version
=> "posts/1-20190522070715422750"

Let’s update post instance and check cache_key and cache_version’s behaviour.

>> post.touch

>> post.cache_key
=> "posts/1" # cache_key remains same

>> post.cache_version
=> "20190527062249879829" # cache_version changed

To use cache versioning feature, we have to enable ActiveRecord::Base.cache_versioning configuration. By default cache_versioning config is set to false for backward compatibility.

We can enable cache versioning configuration globally as shown below.

ActiveRecord::Base.cache_versioning = true
# or
config.active_record.cache_versioning = true

Cache versioning config can be applied at model level.

class Post < ActiveRecord::Base
  self.cache_versioning = true
end

# Or, when setting `#cache_versioning` outside the model -

Post.cache_versioning = true

Let’s understand the problem step by step with cache keys before Rails 5.2.

Rails 5.1 (without cache versioning)

1. Write post instance to cache using fetch api.

>> before_update_cache_key = post.cache_key
=> "posts/1-20190527062249879829"

>> Rails.cache.fetch(before_update_cache_key) { post }
=> #<Post id: 1, title: "First Post", created_at: "2019-05-22 17:23:22", updated_at: "2019-05-27 06:22:49">

2. Update post instance using touch.

>> post.touch
   (0.1ms)  begin transaction
  Post Update (1.6ms)  UPDATE "posts" SET "updated_at" = ? WHERE "posts"."id" = ?  [["updated_at", "2019-05-27 08:01:52.975653"], ["id", 1]]
   (1.2ms)  commit transaction
=> true

3. Verify stale cache_key in cache store.

>> Rails.cache.fetch(before_update_cache_key)
=> #<Post id: 1, title: "First Post", created_at: "2019-05-22 17:23:22", updated_at: "2019-05-27 06:22:49">

4. Write updated post instance to cache using new cache_key.

>> after_update_cache_key = post.cache_key
=> "posts/1-20190527080152975653"

>> Rails.cache.fetch(after_update_cache_key) { post }
=> #<Post id: 1, title: "First Post", created_at: "2019-05-22 17:23:22", updated_at: "2019-05-27 08:01:52">

5. Cache store now has two copies of post instance.

>> Rails.cache.fetch(before_update_cache_key)
=> #<Post id: 1, title: "First Post", created_at: "2019-05-22 17:23:22", updated_at: "2019-05-27 06:22:49">

>> Rails.cache.fetch(after_update_cache_key)
=> #<Post id: 1, title: "First Post", created_at: "2019-05-22 17:23:22", updated_at: "2019-05-27 08:01:52">

cache_key and its associated instance becomes irrelevant as soon as an instance is updated. But it stays in cache store until it is manually invalidated.

This sometimes result in overflowing cache store with stale keys and data. In applications that extensively use cache store, a huge chunk of cache store gets filled with stale data frequently.

Now let’s take a look at the same example. This time with cache versioning to understand how recyclable cache keys help optimize cache storage.

Rails 5.2 (cache versioning)

1. Write post instance to cache store with version option.

>> ActiveRecord::Base.cache_versioning = true

>> post = Post.last

>> cache_key = post.cache_key
=> "posts/1"

>> before_update_cache_version = post.cache_version
=> "20190527080152975653"

>> Rails.cache.fetch(cache_key, version: before_update_cache_version) { post }
=> #<Post id: 1, title: "First Post", created_at: "2019-05-22 17:23:22", updated_at: "2019-05-27 08:01:52">

2. Update post instance.

>> post.touch
   (0.1ms)  begin transaction
  Post Update (0.4ms)  UPDATE "posts" SET "updated_at" = ? WHERE "posts"."id" = ?  [["updated_at", "2019-05-27 09:09:15.651029"], ["id", 1]]
   (0.7ms)  commit transaction
=> true

3. Verify stale cache_version in cache store.

>> Rails.cache.fetch(cache_key, version: before_update_cache_version)
=> #<Post id: 1, title: "First Post", created_at: "2019-05-22 17:23:22", updated_at: "2019-05-27 08:01:52">

4. Write updated post instance to cache.

>> after_update_cache_version = post.cache_version
=> "20190527090915651029"

>> Rails.cache.fetch(cache_key, version: after_update_cache_version) { post }
=> #<Post id: 1, title: "First Post", created_at: "2019-05-22 17:23:22", updated_at: "2019-05-27 09:09:15">

5. Cache store has replaced old copy of post with new version automatically.

>> Rails.cache.fetch(cache_key, version: before_update_cache_version)
=> nil

>> Rails.cache.fetch(cache_key, version: after_update_cache_version)
=> #<Post id: 1, title: "First Post", created_at: "2019-05-22 17:23:22", updated_at: "2019-05-27 09:09:15">

Above example shows how recyclable cache keys maintains single, latest copy of an instance. Stale versions are removed automatically when new version is added to cache store.

Rails 6 added #cache_versioning for ActiveRecord::Relation.

ActiveRecord::Base.collection_cache_versioning configuration should be enabled to use cache versioning feature on collections. It is set to false by default.

We can enable this configuration as shown below.

ActiveRecord::Base.collection_cache_versioning = true
# or
config.active_record.collection_cache_versioning = true

Before Rails 6, ActiveRecord::Relation had cache_key in format {table_name}/query-{query-hash}-{count}-{max(updated_at)}.

In Rails 6, cache_key is split in stable part cache_key - {table_name}/query-{query-hash} and volatile part cache_version - {count}-{max(updated_at)}.

For more information, check out blog on ActiveRecord::Relation#cache_key in Rails 5.

Rails 5.2

>> posts = Post.all

>> posts.cache_key
=> "posts/query-00644b6a00f2ed4b925407d06501c8fb-3-20190522172326885804"

Rails 6

>> ActiveRecord::Base.collection_cache_versioning = true

>> posts = Post.all

>> posts.cache_key
=> "posts/query-00644b6a00f2ed4b925407d06501c8fb"

>> posts.cache_version
=> "3-20190522172326885804"

Cache versioning works similarly for ActiveRecord::Relation as ActiveRecord::Base.

In case of ActiveRecord::Relation, if number of records change and/or record(s) are updated, then same cache_key is written to cache store with new cache_version and updated records.

Conclusion

Previously, cache invalidation had to be done manually either by deleting cache or setting cache expire duration. Cache versioning invalidates stale data automatically and keeps latest copy of data, saving on storage and performance drastically.

Check out the pull request and commit for more details.


Rails 6 deprecates where.not working as NOR and will change to NAND in Rails 6.1

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

A notable deprecation warning has been added in Rails 6 when using where.not with multiple attributes.

Before Rails 6, if we use where.not with multiple attributes, it applies logical NOR (NOT(A) AND NOT(B)) in WHERE clause of the query. This does not always work as expected.

Let’s look at an example to understand this better.

We have Post model with a polymorphic association.

Rails 5.2
>> Post.all
=> #<ActiveRecord::Relation [
#<Post id: 1, title: "First Post", source_type: "Feed", source_id: 100>,
#<Post id: 2, title: "Second Post", source_type: "Feed", source_id: 101>]>

>> Post.where(source_type: "Feed", source_id: 100)
=> #<ActiveRecord::Relation [#<Post id: 1, title: "First Post", source_type: "Feed", source_id: 100>]>

>> Post.where.not(source_type: "Feed", source_id: 100)
=> #<ActiveRecord::Relation []>

In the last query, we expect ActiveRecord to fetch one record.

Let’s check SQL generated for the above case.

>> Post.where.not(source_type: "Feed", source_id: 100).to_sql

=> SELECT "posts".* FROM "posts" WHERE "posts"."source_type" != 'Feed' AND "posts"."source_id" != 100

where.not applies AND to the negation of source_type and source_id, and fails to fetch expected records.

In such cases, correct implementation of where.not would be logical NAND (NOT(A) OR NOT(B)).

Let us query posts table using NAND this time.

>> Post.where("source_type != 'Feed' OR source_id != 100")

   SELECT "posts".* FROM "posts" WHERE (source_type != 'Feed' OR source_id != 100)

=> #<ActiveRecord::Relation [#<Post id: 2, title: "Second Post", source_type: "Feed", source_id: 101>]>

Above query works as expected and returns one record. Rails 6.1 will change where.not working to NAND similar to the above query.

Rails 6.0.0.rc1
>> Post.where.not(source_type: "Feed", source_id: 100)

DEPRECATION WARNING: NOT conditions will no longer behave as NOR in Rails 6.1. To continue using NOR conditions, NOT each conditions manually (`.where.not(:source_type => ...).where.not(:source_id => ...)`). (called from irb_binding at (irb):1)

=> #<ActiveRecord::Relation []>

It is well mentioned in deprecation warning that if we wish to use NOR condition with multiple attributes, we can chain multiple where.not using a single predicate.

>> Post.where.not(source_type: "Feed").where.not(source_id: 100)

Here’s the relevant discussion and pull request for this change.


Rails 6 adds support for Optimizer Hints

Rails 6 has added support to provide optimizer hints.

What is Optimizer Hints?

Many relational database management systems (RDBMS) have a query optimizer. The job of the query optimizer is to determine the most efficient and fast plan to execute a given SQL query. Query optimizer has to consider all possible query execution plans before it can determine which plan is the optimal plan for executing the given SQL query and then compile and execute that query.

An optimal plan is chosen by the query optimizer by calculating the cost of each possible plans. Typically, when the number of tables referenced in a join query increases, then the time spent in query optimization grows exponentially which often affects the system’s performance. The fewer the execution plans the query optimizer needs to evaluate, the lesser time is spent in compiling and executing the query.

As an application designer, we might have more context about the data stored in our database. With the contextual knowledge about our database, we might be able to choose a more efficient execution plan than the query optimizer.

This is where the optimizer hints or optimizer guidelines come into picture.

Optimizer hints allow us to control the query optimizer to choose a certain query execution plan based on the specific criteria. In other words, we can hint the optimizer to use or ignore certain optimization plans using optimizer hints.

Usually, optimizer hints should be provided only when executing a complex query involving multiple table joins.

Note that the optimizer hints only affect an individual SQL statement. To alter the optimization strategies at the global level, there are different mechanisms supported by different databases. Optimizer hints provide finer control over other mechanisms which allow altering optimization plans by other means.

Optimizer hints are supported by many databases such as MySQL, PostgreSQL with the help of pg_hint_plan extension, Oracle, MS SQL, IBM DB2, etc. with varying syntax and options.

Optimizer Hints in Rails 6

Before Rails 6, we have to execute a raw SQL query to use the optimizer hints.

query = "SELECT
            /*+ JOIN_ORDER(articles, users) MAX_EXECUTION_TIME(60000) */
            articles.*
         FROM articles
         INNER JOIN users
         ON users.id = articles.user_id
         WHERE (published_at > '2019-02-17 13:15:44')
        ".squish

ActiveRecord::Base.connection.execute(query)

In the above query, we provided two optimizer hints to MySQL .

/*+ HINT_HERE ANOTHER_HINT_HERE ... */

Another approach to use optimizer hints prior to Rails 6 is to use a monkey patch like this.

In Rails 6, using optimizer hints is easier.

The same example looks like this in Rails 6.

Article
  .joins(:user)
  .where("published_at > ?", 2.months.ago)
  .optimizer_hints(
    "JOIN_ORDER(articles, users)",
    "MAX_EXECUTION_TIME(60000)"
  )

This produces the same SQL query as above but the result is of type ActiveRecord::Relation.

In PostgreSQL (using the pg_hint_plan extension), the optimizer hints have a different syntax.

Article
  .joins(:user)
  .where("published_at > ?", 2.months.ago)
  .optimizer_hints("Leading(articles users)", "SeqScan(articles)")

Please checkout the documentation of each database separately to learn the support and syntax of optimizer hints.

To learn more, please checkout this PR which introduced the #optimization_hints method to Rails 6.

Bonus example: Using optimizer hints to speedup a slow SQL statement in MySQL

Consider that we have articles table with some indexes.

class CreateArticles < ActiveRecord::Migration[6.0]
  def change
    create_table :articles do |t|
      t.string :title, null: false
      t.string :slug, null: false
      t.references :user
      t.datetime :published_at
      t.text :description

      t.timestamps

      t.index :slug, unique: true
      t.index [:published_at]
      t.index [:slug, :user_id]
      t.index [:published_at, :user_id]
      t.index [:title, :slug]
    end
  end
end

Let’s try to fetch all the articles which have been published in the last 2 months.

>> Article.joins(:user).where("published_at > ?", 2.months.ago)
# Article Load (10.5ms)  SELECT `articles`.* FROM `articles` INNER JOIN `users` ON `users`.`id` = `articles`.`user_id` WHERE (published_at > '2019-02-17 11:38:18.647296')
=> #<ActiveRecord::Relation [#<Article id: 20, title: "Article 20", slug: "article-20", user_id: 1, ...]>

Let’s use EXPLAIN to investigate why it is taking 10.5ms to execute this query.

>> Article.joins(:user).where("published_at > ?", 2.months.ago).explain
# Article Load (13.9ms)  SELECT `articles`.* FROM `articles` INNER JOIN `users` ON `users`.`id` = `articles`.`user_id` WHERE (published_at > '2019-02-17 11:39:05.380577')
=> # EXPLAIN for: SELECT `articles`.* FROM `articles` INNER JOIN `users` ON `users`.`id` = `articles`.`user_id` WHERE (published_at > '2019-02-17 11:39:05.380577')
# +--------+----------+----------------+-----------+------+----------+-------+
# | select |   table  | possible_keys  | key       | rows | filtered | Extra |
# | _type  |          |                |           |      |          |       |
# +--------+----------+----------------+-----------+------+----------+-------+
# | SIMPLE |   users  | PRIMARY        | PRIMARY   | 2    | 100.0    | Using |
# |        |          |                |           |      |          | index |
# +--------+----------+----------------+-----------+------+----------+-------+
# | SIMPLE | articles | index          | index     | 9866 | 10.0     | Using |
# |        |          | _articles      | _articles |      |          | where |
# |        |          | _on_user_id,   | _on       |      |          |       |
# |        |          | index          | _user_id  |      |          |       |
# |        |          | _articles      |           |      |          |       |
# |        |          | _on            |           |      |          |       |
# |        |          | _published_at, |           |      |          |       |
# |        |          | index          |           |      |          |       |
# |        |          | _articles      |           |      |          |       |
# |        |          | _on            |           |      |          |       |
# |        |          | _published_at  |           |      |          |       |
# |        |          | _and_user_id   |           |      |          |       |
# +--------+----------+----------------+-----------+------+----------+-------+

According to the above table, it appears that the query optimizer is considering users table first and then the articles table.

The rows column indicates the estimated number of rows the query optimizer must examine to execute the query.

The filtered column indicates an estimated percentage of table rows that will be filtered by the table condition.

The formula rows x filtered gives the number of rows that will be joined with the following table.

Also,

  • For users table, the number of rows to be joined with the following table is 2 x 100% = 2,
  • For articles table, the number of rows to be joined with the following table is 500 * 7.79 = 38.95.

Since the articles tables contain more records which references very few records from the users table, it would be better to consider the articles table first and then the users table.

We can hint MySQL to consider the articles table first as follows.

>> Article.joins(:user).where("published_at > ?", 2.months.ago).optimizer_hints("JOIN_ORDER(articles, users)")
# Article Load (2.2ms)  SELECT `articles`.* FROM `articles` INNER JOIN `users` ON `users`.`id` = `articles`.`user_id` WHERE (published_at > '2019-02-17 11:54:06.230651')
=> #<ActiveRecord::Relation [#<Article id: 20, title: "Article 20", slug: "article-20", user_id: 1, ...]>

Note that it took 2.2ms now to fetch the same records by providing JOIN_ORDER(articles, users) optimization hint.

Let’s try to EXPLAIN what changed by using this JOIN_ORDER(articles, users) optimization hint.

>> Article.joins(:user).where("published_at > ?", 2.months.ago).optimizer_hints("JOIN_ORDER(articles, users)").explain
# Article Load (4.1ms)  SELECT /*+ JOIN_ORDER(articles, users) */ `articles`.* FROM `articles` INNER JOIN `users` ON `users`.`id` = `articles`.`user_id` WHERE (published_at > '2019-02-17 11:55:24.335152')
=> # EXPLAIN for: SELECT /*+ JOIN_ORDER(articles, users) */ `articles`.* FROM `articles` INNER JOIN `users` ON `users`.`id` = `articles`.`user_id` WHERE (published_at > '2019-02-17 11:55:24.335152')
# +--------+----------+----------------+-----------+------+----------+--------+
# | select |   table  | possible_keys  | key       | rows | filtered | Extra  |
# | _type  |          |                |           |      |          |        |
# +--------+----------+----------------+-----------+------+----------+--------+
# | SIMPLE | articles | index          | index     | 769  | 100.0    | Using  |
# |        |          | _articles      | _articles |      |          | index  |
# |        |          | _on_user_id,   | _on       |      |          | condi  |
# |        |          | index          | _publish  |      |          | tion;  |
# |        |          | _articles      | ed_at,    |      |          | Using  |
# |        |          | _on            |           |      |          | where  |
# |        |          | _published_at, |           |      |          |        |
# |        |          | index          |           |      |          |        |
# |        |          | _articles      |           |      |          |        |
# |        |          | _on            |           |      |          |        |
# |        |          | _published_at  |           |      |          |        |
# |        |          | _and_user_id   |           |      |          |        |
# +--------+----------+----------------+-----------+------+----------+--------+
# | SIMPLE | users    | PRIMARY        | PRIMARY   | 2    | 100.0    | Using  |
# |        |          |                |           |      |          | index  |
# +--------+----------+----------------+-----------+------+----------+--------+

The result of the EXPLAIN query shows that the articles table was considered first and then the users table as expected. We can also see that the index_articles_on_published_at index key was considered from the possible keys to execute the given query. The filtered column for both tables shows that the number of filtered rows was 100% which means no filtering of rows occurred.

We hope this example helps in understanding how to use #explain and #optimization_hints methods in order to investigate and debug the performance issues and then fixing it.


Rails 6 reports object allocations made while rendering view templates

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Recently, Rails 6 added allocations feature to ActiveSupport::Notifications::Event. Using this feature, an event subscriber can see how many number of objects were allocated during the event’s start time and end time. We have written in detail about this feature here.

By taking the benefit of this feature, Rails 6 now reports the allocations made while rendering a view template, a partial and a collection.

Started GET "/articles" for ::1 at 2019-04-15 17:24:09 +0530
Processing by ArticlesController#index as HTML
  Rendering articles/index.html.erb within layouts/application
  Rendered shared/_ad_banner.html.erb (Duration: 0.1ms | Allocations: 6)
  Article Load (1.3ms)  SELECT "articles".* FROM "articles"
  ↳ app/views/articles/index.html.erb:5
  Rendered collection of articles/_article.html.erb [100 times] (Duration: 6.1ms | Allocations: 805)
  Rendered articles/index.html.erb within layouts/application (Duration: 17.6ms | Allocations: 3901)
Completed 200 OK in 86ms (Views: 83.6ms | ActiveRecord: 1.3ms | Allocations: 29347)

Notice the Allocations: information in the above logs.

We can see that

  • 6 objects were allocated while rendering shared/_ad_banner.html.erb view partial,
  • 805 objects were allocated while rendering a collection of 100 articles/_article.html.erb view partials,
  • and 3901 objects were allocated while rendering articles/index.html.erb view template.

We can use this information to understand how much time was spent while rendering a view template and how many objects were allocated in the process’ memory between the time when that view template had started rendering and the time when that view template had finished rendering.

To learn more about this feature, please check rails/rails#34136.

Note that we can also collect this information by subscribing to Action View hooks.

ActiveSupport::Notifications.subscribe /^render_.+.action_view$/ do |event|
  views_path = Rails.root.join("app/views/").to_s
  template_identifier = event.payload[:identifier]
  template_name = template_identifier.sub(views_path, "")
  message = "[#{event.name}] #{template_name} (Allocations: #{event.allocations})"

  ViewAllocationsLogger.log(message)
end

This should log something like this.

[render_partial.action_view] shared/_ad_banner.html.erb (Allocations: 43)

[render_collection.action_view] articles/_article.html.erb (Allocations: 842)

[render_template.action_view] articles/index.html.erb (Allocations: 4108)

Rails 6 adds ActiveRecord::Relation#annotate

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Rails 6 has added ActiveRecord::Relation#annotate to allow adding comments to the SQL queries generated by the ActiveRecord::Relation instance.

Here is how it can be used.

>> User.annotate("User whose name starts with 'A'").where("name LIKE ?", "A%")

SELECT "users".* FROM "users"
WHERE (name LIKE 'A%')
/* User whose name starts with 'A' */
LIMIT ?  [["LIMIT", 11]]

ActiveRecord::Relation#annotate allows to add multiple annotations on a query

>> bigbinary = Organization.find_by!(name: "BigBinary")
>> User.annotate("User whose name starts with 'A'")
       .annotate("AND belongs to BigBinary organization")
       .where("name LIKE ?", "A%")
       .where(organization: bigbinary)

SELECT "users".* FROM "users"
WHERE (name LIKE 'A%') AND "users"."organization_id" = ?
/* Users whose name starts with 'A' */
/* AND belongs to BigBinary organization */
LIMIT ?  [["organization_id", 1], ["LIMIT", 11]]

Also, ActiveRecord::Relation#annotate allows annotating scopes and model associations.

class User < ActiveRecord::Base
  scope :active, -> { where(status: 'active').annotate("Active users") }
end

>> User.active
SELECT "users".* FROM "users"
/* Active users */
LIMIT ?  [["LIMIT", 11]]

Check out the pull request for more details on this.


Rails 6 adds hooks to Active Job around retries and discards

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Before Rails 6

Before Rails 6, we have to provide a custom block to perform custom logging and monitoring around retries and discards of the jobs defined using Active Job framework.

class Container::DeleteJob < ActiveJob::Base
  retry_on Timeout::Error, wait: 2.seconds, attempts: 3 do |job, error|
    message = "Stopped retrying #{job.class} (JID #{job.job_id})
               with #{job.arguments.join(', ')} due to
               '#{error.class} - #{error.message}'.
               This job was retried for #{job.executions} times.".squish

    BackgroundJob::ErrorLogger.log(message)
  end

  discard_on Container::NotFoundError do |job, error|
    message = "Discarded #{job.class} (JID #{job.job_id})
               with #{job.arguments.join(', ')} due to
               '#{error.class} - #{error.message}' error.".squish

    BackgroundJob::ErrorLogger.log(message)
  end

  def perform(container_id)
    Container::DeleteService(container_id).process

    # Will raise Container::NotFoundError
    # if no container is found with 'container_id'.

    # Might raise Timeout::Error when the remote API is not responding.
  end
end

Notice the custom blocks provided to retry_on and discard_on methods to an individual job in the above example.

Extracting such custom logic to a base class or to a 3rd-party gem is possible but it will be non-standard and will be a bit difficult task.

An alternative approach is to subscribe to the hooks instrumented using Active Support Instrumentation API which is a standard and recommended way. Prior versions of Rails 6 already instruments some hooks such as enqueue_at.active_job, enqueue.active_job, perform_start.active_job, and perform.active_job. Unfortunately no hook is instrumented around retries and discards of an Active Job prior to Rails 6.

Rails 6

Rails 6 has introduced hooks to Active Job around retries and discards to which one can easily subscribe using Active Support Instrumentation API to perform custom logging and monitoring or to collect any custom information.

The newly introduced hooks are enqueue_retry.active_job, retry_stopped.active_job and discard.active_job.

Let’s discuss each of these hooks in detail.

Note that whenever we say a job, it means a job of type ActiveJob.

enqueue_retry.active_job

The enqueue_retry.active_job hook is instrumented when a job is enqueued to retry again due to occurrence of an exception which is configured using the retry_on method in the job’s definition. This hook is triggered only when above condition is satisfied and the number of executions of the job is less than the number of attempts defined using the retry_on method. The number of attempts is by default set to 5 if not defined explicitly.

This is how we would subscribe to this hook and perform custom logging in our Rails application.

ActiveSupport::Notifications.subscribe "enqueue_retry.active_job" do |*args|
  event = ActiveSupport::Notifications::Event.new *args
  payload = event.payload
  job = payload[:job]
  error = payload[:error]
  message = "#{job.class} (JID #{job.job_id})
             with arguments #{job.arguments.join(', ')}
             will be retried again in #{payload[:wait]}s
             due to error '#{error.class} - #{error.message}'.
             It is executed #{job.executions} times so far.".squish

  BackgroundJob::Logger.log(message)
end

Note that the BackgroundJob::Logger above is our custom logger. If we want, we can add any other logic instead.

We will change the definition of Container::DeleteJob job as below.

class Container::DeleteJob < ActiveJob::Base
  retry_on Timeout::Error, wait: 2.seconds, attempts: 3

  def perform(container_id)
    Container::DeleteService(container_id).process

    # Will raise Timeout::Error when the remote API is not responding.
  end
end

Let’s enqueue this job.

Container::DeleteJob.perform_now("container-1234")

Assume that this job keeps throwing Timeout::Error exception due to a network issue. The job will be retried twice since it is configured to retry when a Timeout::Error exception occurs up to maximum 3 attempts. While retrying this job, Active Job will instrument enqueue_retry.active_job hook along with the necessary job payload.

Since we have already subscribed to this hook, our subscriber would log something like this with the help of BackgroundJob::Logger.log.

Container::DeleteJob (JID 770f4f2a-daa7-4c1e-be51-24adc04b4cb2) with arguments container-1234 will be retried again in 2s due to error 'Timeout::Error - Couldn't establish connection to cluster within 10s'. It is executed 1 times so far.

Container::DeleteJob (JID 770f4f2a-daa7-4c1e-be51-24adc04b4cb2) with arguments container-1234 will be retried again in 2s due to error 'Timeout::Error - Couldn't establish connection to cluster within 10s'. It is executed 2 times so far.
retry_stopped.active_job

The retry_stopped.active_job hook is triggered when a job is retried up to the available number of attempts.

Let’s see how this hook is triggered.

Along with the subscription for the enqueue_retry.active_job hook, let’s subscribe to the retry_stopped.active_job hook, too.

ActiveSupport::Notifications.subscribe "retry_stopped.active_job" do |*args|
  event = ActiveSupport::Notifications::Event.new *args
  payload = event.payload
  job = payload[:job]
  error = payload[:error]
  message = "Stopped processing #{job.class} (JID #{job.job_id})
             further with arguments #{job.arguments.join(', ')}
             since it failed due to '#{error.class} - #{error.message}' error
             which reoccurred #{job.executions} times.".squish

  BackgroundJob::Logger.log(message)
end

Let’s keep the Container::DeleteJob job’s definition unchanged and enqueue it again.

Container::DeleteJob.perform_now("container-1234")

We will assume that the job will keep throwing Timeout::Error exception due to a network issue.

In the logs recorded using BackgroundJob::Logger.log, we should see something like this.

Container::DeleteJob (JID 770f4f2a-daa7-4c1e-be51-24adc04b4cb2) with arguments container-1234 will be retried again in 2s due to error 'Timeout::Error - Couldn't establish connection to cluster within 10s'. It is executed 1 times so far.

Container::DeleteJob (JID 770f4f2a-daa7-4c1e-be51-24adc04b4cb2) with arguments container-1234 will be retried again in 2s due to error 'Timeout::Error - Couldn't establish connection to cluster within 10s'. It is executed 2 times so far.

Stopped processing Container::DeleteJob (JID 770f4f2a-daa7-4c1e-be51-24adc04b4cb2) further with arguments container-1234 since it failed due to 'Timeout::Error - Couldn't establish connection to cluster within 10s' error which reoccurred 3 times.

Notice the last entry in the logs above and its order.

discard.active_job

The discard.active_job hook is triggered when a job’s further execution is discarded due to occurrence of an exception which is configured using discard_on method.

To see how this hook is triggered, we will subscribe to the discard.active_job hook.

ActiveSupport::Notifications.subscribe "discard.active_job" do |*args|
  event = ActiveSupport::Notifications::Event.new *args
  payload = event.payload
  job = payload[:job]
  error = payload[:error]
  message = "Discarded #{job.class} (JID #{job.job_id})
             with arguments #{job.arguments.join(', ')}
             due to '#{error.class} - #{error.message}' error.".squish

  BackgroundJob::Logger.log(message)
end

We will configure our Container::DeleteJob job to discard when Container::NotFoundError exception occurs while executing the job.

class Container::DeleteJob < ActiveJob::Base
  discard_on Container::NotFoundError
  retry_on Timeout::Error, wait: 2.seconds, attempts: 3

  def perform(container_id)
    Container::DeleteService(container_id).process

    # Will raise Container::NotFoundError
    # if no container is found with 'container_id'.

    # Will raise Timeout::Error when the remote API is not responding.
  end
end

Let’s enqueue this job and assume that it would throw Container::NotFoundError exception.

Container::DeleteJob.perform_now("unknown-container-9876")

We should see following in the logs recorded by BackgroundJob::Logger.log.

Discarded Container::DeleteJob (JID e9b1cb5c-6d2d-49ae-b1d7-fef44f09ab8d) with arguments unknown-container-9876 due to 'Container::NotFoundError - Container 'unknown-container-9876' was not found' error.

Notes

  1. These new hooks are also instrumented for the jobs which are enqueued using perform_later method since both perform_now and perform_later calls perform method under the hood.

  2. Active Job already subscribes to the these hooks and writes them using the Rails’ default logger.

  3. If a block is provided to retry_on or discard_on methods then an applicable hook is instrumented first and then the given block is yielded.


Rails 6 adds support for Multi Environment credentials

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

In Rails 5.2, encrypted credentails are stored in the file config/credentials.yml.enc. This is a single flat file which is encrypted by the key located in config/master.key.

Rails 5.2 does not support storing credentials of different environments with different encryption keys. If we want environment specific encrypted credentials, we’ll have to follow this workaround.

Rails 6 has added support for Multi Environment credentials. With this change, credentials that belong to different environments can be stored in separate files with their own encryption key.

Let’s see how this works in Rails 6.0.0.beta3

Rails 6.0.0.beta3

If we want to add credentials to be used in staging environment, we can run

rails edit:credentials --environment staging

This will create the credentials file config/credentials/staging.yml.enc and a staging specific encryption key config/credentials/staging.key and open the credentials file in your text editor.

Let’s add our AWS access key id here.

aws:
  access_key_id: "STAGING_KEY"

We can then access the access_key_id in staging environment.

>> RAILS_ENV=staging rails c

pry(main)> Rails.application.credentials.aws[:access_key_id]

=> "STAGING_KEY"

Which takes precedence: Global or Environment Specific credentials?

Credentials added to global file config/credentials.yml.enc will not be loaded in environments which have their own environment specific credentials file (config/credentials/$environment.yml.enc).

So if we decide to add the following to the global credentials file, these credentials will not be available in staging. Since we already have a environment specific credentials file for staging.

aws:
  access_key_id: "DEFAULT_KEY"
stripe:
  secret_key: "DEFAULT_SECRET_KEY"
>> RAILS_ENV=staging rails c

pry(main)> Rails.application.credentials.aws[:access_key_id]

=> "STAGING_KEY"

pry(main)> Rails.application.credentials.stripe[:secret_key]

Traceback (most recent call last):
        1: from (irb):6
NoMethodError (undefined method `[]' for nil:NilClass)

Here is the relevant pull request


Rails 6 adds before? and after? to Date and Time

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Rails 6 adds before? and after? to Date , DateTime , Time and ActiveSupport::TimeWithZone classes.

before? and after? are aliases to < (less than) and > (greater than) methods respectively.

Let’s checkout how it works.

Rails 5.2

Let’s try calling before? on a date object in Rails 5.2.

>> Date.new(2019, 3, 31).before?(Date.new(2019, 4, 1))

=> NoMethodError: undefined method 'before?' for Sun, 31 Mar 2019:Date
	from (irb):1

>> Date.new(2019, 3, 31) < Date.new(2019, 4, 1)

=> true

Rails 6.0.0.beta2

Now, let’s compare Date , DateTime , Time and ActiveSupport::TimeWithZone objects using before? and after? in Rails 6.

>> Date.new(2019, 3, 31).before?(Date.new(2019, 4, 1))

=> true

>> Date.new(2019, 3, 31).after?(Date.new(2019, 4, 1))

=> false

>> DateTime.parse('2019-03-31').before?(DateTime.parse('2019-04-01'))

=> true

>> DateTime.parse('2019-03-31').after?(DateTime.parse('2019-04-01'))

=> false

>> Time.parse('2019-03-31').before?(Time.parse('2019-04-01'))

=> true

>> Time.parse('2019-03-31').after?(Time.parse('2019-04-01'))

=> false

>> ActiveSupport::TimeWithZone.new(Time.utc(2019, 3, 31, 12, 0, 0), ActiveSupport::TimeZone["Eastern Time (US & Canada)"]).before?(ActiveSupport::TimeWithZone.new(Time.utc(2019, 4, 1, 12, 0, 0), ActiveSupport::TimeZone["Eastern Time (US & Canada)"]))

=> true

>> ActiveSupport::TimeWithZone.new(Time.utc(2019, 3, 31, 12, 0, 0), ActiveSupport::TimeZone["Eastern Time (US & Canada)"]).after?(ActiveSupport::TimeWithZone.new(Time.utc(2019, 4, 1, 12, 0, 0), ActiveSupport::TimeZone["Eastern Time (US & Canada)"]))

=> false

Here is the relevant pull request for adding before? and after? methods and the pull request for moving before? and after? to DateAndTime::Calculations.


Rails 6 adds Array#extract!

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Rails 6 added extract! on Array class. extract! removes and returns the elements for which the given block returns true.

extract! is different from reject! in the way that reject! returns the array after removing the elements whereas extract! returns removed elements from the array.

Let’s checkout how it works.

Rails 6.0.0.beta2

Let’s pluck all the user emails and then extract emails which include gmail.com.

>> emails = User.pluck(:email)
SELECT "users"."email" FROM "users"

=> ["amit.choudhary@bigbinary.com", "amit@gmail.com", "mark@gmail.com", "sam@gmail.com"]

>> emails.extract! { |email| email.include?('gmail.com') }

=> ["amit@gmail.com", "mark@gmail.com", "sam@gmail.com"]

>> emails

=> ["amit.choudhary@bigbinary.com"]

>> emails = User.pluck(:email)
SELECT "users"."email" FROM "users"

=> ["amit.choudhary@bigbinary.com", "amit@gmail.com", "mark@gmail.com", "sam@gmail.com"]

>> emails.reject! { |email| email.include?('gmail.com') }

=> ["amit.choudhary@bigbinary.com"]

>> emails

=> ["amit.choudhary@bigbinary.com"]

Here is the relevant pull request.


Rails 6 adds Enumerable#index_with

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Rails 6 added index_with on Enumerable module. This will help in creating a hash from an enumerator with default or fetched values.

Before Rails 6, we can achieve this by calling map along with to_h.

index_with takes both value or a block as a parameter.

Let’s checkout how it works.

Rails 5.2

Let’s create a hash from an array in Rails 5.2 using map and to_h.

>> address = Address.first
SELECT "addresses".* FROM "addresses"
ORDER BY "addresses"."id" ASC LIMIT $1  [["LIMIT", 1]]

=> #<Address id: 1, first_name: "Amit", last_name: "Choudhary", state: "California", created_at: "2019-03-21 10:03:57", updated_at: "2019-03-21 10:03:57">

>> NAME_ATTRIBUTES = [:first_name, :last_name]

=> [:first_name, :last_name]

>> NAME_ATTRIBUTES.map { |attr| [attr, address.public_send(attr)] }.to_h

=> {:first_name=>"Amit", :last_name=>"Choudhary"}

Rails 6.0.0.beta2

Now let’s create the same hash from the array using index_with in Rails 6.

>> address = Address.first
SELECT "addresses".* FROM "addresses"
ORDER BY "addresses"."id" ASC LIMIT $1  [["LIMIT", 1]]

=> #<Address id: 1, first_name: "Amit", last_name: "Choudhary", state: "California", created_at: "2019-03-21 10:02:47", updated_at: "2019-03-21 10:02:47">

>> NAME_ATTRIBUTES = [:first_name, :last_name]

=> [:first_name, :last_name]

>> NAME_ATTRIBUTES.index_with { |attr| address.public_send(attr) }

=> {:first_name=>"Amit", :last_name=>"Choudhary"}

>> NAME_ATTRIBUTES.index_with('Default')

=> {:first_name=>"Default", :last_name=>"Default"}

Here is the relevant pull request.


Rails 6 adds private option to delegate method

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Rails 6 adds :private option to delegate method. After this addition, we can delegate methods in the private scope.

Let’s checkout how it works.

Rails 6.0.0.beta2

Let’s create two models named as Address and Order. Let’s also delegate validate_state method in Order to Address.

class Address < ApplicationRecord
  validates :first_name, :last_name, :state, presence: true

  DELIVERABLE_STATES = ['New York']

  def validate_state
    unless DELIVERABLE_STATES.include?(state)
      errors.add(:state, :invalid)
    end
  end
end

class Order < ApplicationRecord
  belongs_to :address

  delegate :validate_state, to: :address
end

>> Order.first
SELECT "orders".* FROM "orders"
ORDER BY "orders"."id" ASC LIMIT $1  [["LIMIT", 1]]

=> #<Order id: 1, amount: 0.1e2, address_id: 1, created_at: "2019-03-21 10:02:58", updated_at: "2019-03-21 10:17:44">

>> Address.first
SELECT "addresses".* FROM "addresses"
ORDER BY "addresses"."id" ASC LIMIT $1  [["LIMIT", 1]]

=> #<Address id: 1, first_name: "Amit", last_name: "Choudhary", state: "California", created_at: "2019-03-21 10:02:47", updated_at: "2019-03-21 10:02:47">

>> Order.first.validate_state
SELECT "orders".* FROM "orders"
ORDER BY "orders"."id" ASC LIMIT $1  [["LIMIT", 1]]

SELECT "addresses".* FROM "addresses"
WHERE "addresses"."id" = $1 LIMIT $2  [["id", 1], ["LIMIT", 1]]

=> ["is invalid"]

Now, let’s add private: true to the delegation.

class Order < ApplicationRecord
  belongs_to :address

  delegate :validate_state, to: :address, private: true
end

>> Order.first.validate_state
SELECT "orders".* FROM "orders"
ORDER BY "orders"."id" ASC LIMIT $1  [["LIMIT", 1]]

=> Traceback (most recent call last):
        1: from (irb):7
NoMethodError (private method 'validate_state' called for #<Order:0x00007fb9d72fc1f8>
Did you mean?  validate)

As we can see, Rails now raises an exception of private method called if private option is set with delegate method.

Here is the relevant pull request.


Rails 6 allows spaces in postgres table names

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Rails 6 allows spaces in tables names in PostgreSQL. Before Rails 6, if we try to create a table named as user reviews, Rails tries to create a table named as reviews in schema named as user.

Let’s checkout how it works.

Rails 5.2

Let’s create a table user reviews in Rails 5.2.

>> class CreateUserReviews < ActiveRecord::Migration[5.2]
>>   def change
>>     create_table 'user reviews' do |t|
>>       t.string :value
>>
>>       t.timestamps
>>     end
>>   end
>> end

=> :change

>> CreateUserReviews.new.change
-- create_table("user reviews")
CREATE TABLE "user"."reviews" ("id" bigserial primary key, "value" character varying, "created_at" timestamp NOT NULL, "updated_at" timestamp NOT NULL)

=> Traceback (most recent call last):
        2: from (irb):10
        1: from (irb):3:in 'change'
ActiveRecord::StatementInvalid (PG::InvalidSchemaName: ERROR:  schema "user" does not exist)
LINE 1: CREATE TABLE "user"."reviews" ("id" bigserial primary key, "...
                     ^
: CREATE TABLE "user"."reviews" ("id" bigserial primary key, "value" character varying, "created_at" timestamp NOT NULL, "updated_at" timestamp NOT NULL)

We can see that Rails 5.2 raised an exception and tried to create table named as reviews in user schema.

Rails 6.0.0.beta2

Now, let’s create a table user reviews in Rails 6.

>> class CreateUserReviews < ActiveRecord::Migration[6.0]
>>   def change
>>     create_table 'user reviews' do |t|
>>       t.string :value
>>
>>       t.timestamps
>>     end
>>   end
>> end

=> :change

>> CreateUserReviews.new.change
-- create_table("user reviews")
CREATE TABLE "user reviews" ("id" bigserial primary key, "value" character varying, "created_at" timestamp(6) NOT NULL, "updated_at" timestamp(6) NOT NULL)

=> #<PG::Result:0x00007f9d633c5458 status=PGRES_COMMAND_OK ntuples=0 nfields=0 cmd_tuples=0>

Now, we can see that the SQL generated is correct and Rails successfully created a table named as user reviews.

Here is the relevant pull request.


Rails 6 adds if_not_exists option to create_table

This blog is part of our Rails 6 series. Rails 6.0 was recently released.

Rails 6 added if_not_exists to create_table option to create a table if it doesn’t exist.

Before Rails 6, we could use ActiveRecord::Base.connection.table_exists?.

Default value of if_not_exists option is false.

Rails 5.2

Let’s create users table in Rails 5.2.

>> class CreateUsers < ActiveRecord::Migration[6.0]
>>   def change
>>     create_table :users do |t|
>>       t.string :name, index: { unique: true }
>>
>>       t.timestamps
>>     end
>>   end
>> end

>> CreateUsers.new.change
-- create_table(:users)
CREATE TABLE "users" ("id" bigserial primary key, "name" character varying, "created_at" timestamp NOT NULL, "updated_at" timestamp NOT NULL)

=> #<PG::Result:0x00007fd73e711cf0 status=PGRES_COMMAND_OK ntuples=0 nfields=0 cmd_tuples=0>

Now let’s try creating users table again with if_not_exists option.

>> class CreateUsers < ActiveRecord::Migration[6.0]
>>   def change
>>     create_table :users, if_not_exists: true do |t|
>>       t.string :name, index: { unique: true }
>>
>>       t.timestamps
>>     end
>>   end
>> end

>> CreateUsers.new.change
-- create_table(:users, {:if_not_exists=>true})
CREATE TABLE "users" ("id" bigserial primary key, "name" character varying, "created_at" timestamp NOT NULL, "updated_at" timestamp NOT NULL)

=> Traceback (most recent call last):
        2: from (irb):121
        1: from (irb):114:in 'change'
ActiveRecord::StatementInvalid (PG::DuplicateTable: ERROR:  relation "users" already exists)
: CREATE TABLE "users" ("id" bigserial primary key, "name" character varying, "created_at" timestamp NOT NULL, "updated_at" timestamp NOT NULL)

We can see that Rails 5.2 ignored if_not_exists option and tried creating the table again.

Now let’s try ActiveRecord::Base.connection.table_exists? with Rails 5.2.

>> class CreateUsers < ActiveRecord::Migration[5.2]
>>   def change
>>     unless ActiveRecord::Base.connection.table_exists?('users')
>>       create_table :users do |t|
>>         t.string :name
>>
>>         t.timestamps
>>       end
>>     end
>>   end
>> end

>> CreateUsers.new.change

=> nil

We can see that create_table :users never executed because ActiveRecord::Base.connection.table_exists?('users') returned true.

Rails 6.0.0.beta2

Let’s create users table in Rails 6 with if_not_exists option set as true.

>> class CreateUsers < ActiveRecord::Migration[6.0]
>>   def change
>>     create_table :users, if_not_exists: true do |t|
>>       t.string :name, index: { unique: true }
>>
>>       t.timestamps
>>     end
>>   end
>> end

>> CreateUsers.new.change
-- create_table(:users, {:if_not_exists=>true})
CREATE TABLE IF NOT EXISTS "users" ("id" bigserial primary key, "name" character varying, "created_at" timestamp(6) NOT NULL, "updated_at" timestamp(6) NOT NULL)

=> #<PG::Result:0x00007fc4614fef48 status=PGRES_COMMAND_OK ntuples=0 nfields=0 cmd_tuples=0>

>> CreateUsers.new.change
-- create_table(:users, {:if_not_exists=>true})
CREATE TABLE IF NOT EXISTS "users" ("id" bigserial primary key, "name" character varying, "created_at" timestamp(6) NOT NULL, "updated_at" timestamp(6) NOT NULL)

=> #<PG::Result:0x00007fc46513fde0 status=PGRES_COMMAND_OK ntuples=0 nfields=0 cmd_tuples=0>

We can see that no exception was raised when we tried creating users table the second time.

Now let’s see what happens if we set if_not_exists to false.

>> class CreateUsers < ActiveRecord::Migration[6.0]
>>   def change
>>     create_table :users, if_not_exists: false do |t|
>>       t.string :name, index: { unique: true }
>>
>>       t.timestamps
>>     end
>>   end
>> end

>> CreateUsers.new.change
-- create_table(:users, {:if_not_exists=>false})
CREATE TABLE "users" ("id" bigserial primary key, "name" character varying, "created_at" timestamp(6) NOT NULL, "updated_at" timestamp(6) NOT NULL)

=> Traceback (most recent call last):
        2: from (irb):23
        1: from (irb):15:in `change'
ActiveRecord::StatementInvalid (PG::DuplicateTable: ERROR:  relation "users" already exists
)

As we can see, Rails raised an exception here because if_not_exists was set to false.

Here is the relevant pull request.