Validate multiple contexts together in Rails 5

This blog is part of our Rails 5 series.

Active Record validation is a well-known and widely used functionality of Rails. Slightly lesser popular is Rails’s ability to validate on custom context.

If used properly, contextual validations can result in much cleaner code. To understand validation context, we will take example of a form which is submitted in multiple steps:

class MultiStepForm < ActiveRecord::Base
  validate :personal_info
  validate :education, on: :create
  validate :work_experience, on: :update
  validate :final_step, on: :submission

  def personal_info
    # validation logic goes here..
  end

  # Smiliary all the validation methods go here.
end

Let’s go through all the four validations one-by-one.

1. personal_info validation has no context defined (notice the absence of on:). Validations with no context are executed every time a model save is triggered. Please go through all the triggers here.

2. education validation has context of :create. It is executed only when a new object is created.

3. work_experience validation is in :update context and gets triggered for updates only. :create and :update are the only two pre-defined contexts.

4. final_step is validated using a custom context named :submission. Unlike above scenarios, it needs to be explicitly triggered like this:

form = MultiStepForm.new

# Either
form.valid?(:submission)

# Or
form.save(context: :submission)

valid? runs the validation in given context and populates errors. save would first call valid? in the given context and persist the changes if validations pass. Otherwise populates errors.

One thing to note here is that when we validate using an explicit context, Rails bypasses all other contexts including :create and :update.

Now that we understand validation context, we can switch our focus to validate multiple context together enhancement in Rails 5.

Let’s change our contexts from above example to

class MultiStepForm < ActiveRecord::Base
  validate :personal_info, on: :personal_submission
  validate :education, on: :education_submission
  validate :work_experience, on: :work_ex_submission
  validate :final_step, on: :final_submission

  def personal_info
    # code goes here..
  end

  # Smiliary all the validation methods go here.
end

For each step, we would want to validate the model with all previous steps and avoid all future steps. Prior to Rails 5, this can be achieved like this:

class MultiStepForm < ActiveRecord::Base
  #...

  def save_personal_info
    self.save if self.valid?(:personal_submission)
  end

  def save_education
    self.save if self.valid?(:personal_submission)
              && self.valid?(:education_submission)
  end

  def save_work_experience
    self.save if self.valid?(:personal_submission)
              && self.valid?(:education_submission)
              && self.valid?(:work_ex_submission)
  end

  # And so on...
end

Notice that valid? takes only one context at a time. So we have to repeatedly call valid? for each context.

This gets simplified in Rails 5 by enhancing valid? and invalid? to accept an array. Our code changes to:

class MultiStepForm < ActiveRecord::Base
  #...

  def save_personal_info
    self.save if self.valid?(:personal_submission)
  end

  def save_education
    self.save if self.valid?([:personal_submission,
                              :education_submission])
  end

  def save_work_experience
    self.save if self.valid?([:personal_submission,
                              :education_submission,
                              :work_ex_submission])
  end
end

A tad bit cleaner I would say.

Rails 5 changes protect_from_forgery execution order

This blog is part of our Rails 5 series.

What makes Rails a great framework to work with is its sane conventions over configuration. Rails community is always striving to keep these conventions relevant over time. In this blog, we will see why and what changed in execution order of protect_from_forgery.

protect_from_forgery protects applications against CSRF. Follow that link to read up more about CSRF.

What

If we generate a brand new Rails application in Rails 4.x then application_controller will look like this.

class ApplicationController < ActionController::Base
  protect_from_forgery with: :exception
end

Looking it at the code it does not look like protect_from_forgery is a before_action call but in reality that’s what it is. Since protect_from_forgery is a before_action call it should follow the order of how other before_action are executed. But this one is special in the sense that protect_from_forgery is executed first in the series of before_action no matter where protect_from_forgery is mentioned. Let’s see an example.

class ApplicationController < ActionController::Base
  before_action :load_user
  protect_from_forgery with: :exception
end

In the above case even though protect_from_forgery call is made after load_user, the protection execution happens first. And we can’t do anything about it. We can’t pass any option to stop Rails from doing this.

Rails 5 changes this behavior by introducing a boolean option called prepend. Default value of this option is false. What it means is, now protect_from_forgery gets executed in order of call. Of course, this can be overridden by passing prepend: true as shown below and now protection call will happen first just like Rails 4.x.

class ApplicationController < ActionController::Base
  before_action :load_user
  protect_from_forgery with: :exception, prepend: true
end

Why

There isn’t any real advantage in forcing protect_from_forgery to be the first filter in the chain of filters to be executed. On the flip side, there are cases where output of other before_action should decide the execution of protect_from_forgery. Let’s see an example.

class ApplicationController < ActionController::Base
  before_action :authenticate
  protect_from_forgery unless: -> { @authenticated_by.oauth? }

  private
    def authenticate
      if oauth_request?
        # authenticate with oauth
        @authenticated_by = 'oauth'.inquiry
      else
        # authenticate with cookies
        @authenticated_by = 'cookie'.inquiry
      end
    end
end

Above code would fail in Rails 4.x, as protect_from_forgery, though called after :authenticate, actually gets executed before it. Due to which we would not have @authenticated_by set properly.

Whereas in Rails 5, protect_from_forgery gets executed after :authenticate and gets skipped if authentication is oauth.

Upgrading to Rails 5

Let’s take an example to understand how this change might affect the upgrade of applications from Rails 4 to Rails 5.

class ApplicationController < ActionController::Base
  before_action :set_access_time
  protect_from_forgery

  private
    def set_access_time
      current_user.access_time = Time.now
      current_user.save
    end
end

In Rails 4.x, set_access_time is not executed for bad requests. But it gets executed in Rails 5 because protect_from_forgery is called after set_access_time.

Saving data (current_user.save) in before_action is anyways a big enough violation of the best practices, but now those persistences would leave us vulnerable to CSRF if they are called before protect_from_forgery is called.

Rails 5 provides application config to use UUID as primary key

This blog is part of our Rails 5 series.

UUIDs are a popular alternative to auto-incremental integer primary keys.

create_table :users, id: :uuid do |t|
  t.string :name
end

Notice that id: :uuid is passed to create_table. This is all we need to do to have UUID as primary key for users.

Now, if an application is designed to use UUID instead of Integer, then chances are that new tables too would use UUID as primary key. And it can easily get repetitive to add id: :uuid in create_table , every time a new model is generated.

Rails 5 comes up with a solution. We need to set primary key as UUID in config/application.rb.

config.active_record.primary_key = :uuid

This automatically adds id: :uuid to create_table in all future migrations.

One small recurrence off our plate.

Rails 5 changed Active Job default adapter from Inline to Async

This blog is part of our Rails 5 series.

Active Job has built-in adapters for multiple queuing backends among which two are intended for development and testing. They are Active Job Inline Adapter and Active Job Async Adapter.

These adapters can be configured as follows.

# for Active Job Inline
Rails.application.config.active_job.queue_adapter = :inline

# for Active Job Async
Rails.application.config.active_job.queue_adapter = :async

In Rails 4.x the default queue adapter is :inline. In Rails 5 it has been changed to :async by DHH.

Asynchronous Execution

In case of inline, as the name suggests, execution of the job happens in the same process that invokes the job. In case of Async adapter, the job is executed asynchronously using in-process thread pool.

AsyncJob makes use of a concurrent-ruby thread pool and the data is retained in memory. Since the data is stored in memory, if the application restarts, this data is lost. Hence, AsyncJob should not be used in production.

Running in future

AsyncJob supports running the job at some time in future through perform_later. Inline executes the job immediately and does not support running the job in future.

Both Active Job Async and Active Job Inline do not support configuring priorities among queues, timeout in execution and retry intervals/counts.

Advantage of having Async as default adapter

In Rails 4.x where Inline is the default adapter, the test cases were mistakenly dependent on job’s behavior that happens synchronously in development/testing environment. Using Async adapter ,by default, will help users have tests not rely on such synchronous behavior.

It’s a step closer to simulating your production environment where jobs are executed asynchronously with more persistent backends.

Consider an example, where in an e-commerce site upon every order placed an email is sent.

test "order is created successfully" do
  # Code to test record in orders table is created
end

test "order email is sent" do
  # Code to test order email is sent
end

The process of sending email can be part of a job which is invoked from an after_create callback in Order model.

class Order < ActiveRecord::Base

  after_create :send_order_email

  def send_order_email
    # Invoke the job of sending an email asynchronously.
  end

end

When Inline adapter is used, any wrongly configured email settings will cause both the above tests to fail. This is because the process of sending the email happens within the process of order creation and any error in sending the email would kill the process if unhandled.

Support for left outer join in Rails 5

This blog is part of our Rails 5 series.

Suppose in a blog application there are authors and posts. A post belongs to an author, while author has many posts.

The app needs to show a list of all the authors along with a number of posts that they have written.

For this, we need to join author and posts table with “left outer join”. More about “left outer join” here, here and here .

In Rails 4.x, we need to write the SQL for left outer join manually as Active Record does not have support for outer joins.

authors = Author.join('LEFT OUTER JOIN "posts" ON "posts"."author_id" = "authors"."id"')
                .uniq
                .select("authors.*, COUNT(posts.*) as posts_count")
                .group("authors.id")

Rails 5 has added left_outer_joins method.

authors = Author.left_outer_joins(:posts)
                .uniq
                .select("authors.*, COUNT(posts.*) as posts_count")
                .group("authors.id")

It also allows to perform the left join on multiple tables at the same time.

>> Author.left_joins :posts, :comments
  Author Load (0.1ms)  SELECT "authors".* FROM "authors" LEFT OUTER JOIN "posts" ON "posts"."author_id" = "authors"."id" LEFT OUTER JOIN "comments" ON "comments"."author_id" = "authors"."id"

If you feel left_outer_joins is too long to type, then Rails 5 also has an alias method left_joins.

has_secure_token to generate unique random token in Rails 5

This blog is part of our Rails 5 series.

We sometimes need unique and random tokens in our web apps. Here is how we typically build it.

class User < ActiveRecord::Base

  before_create :set_access_token

  private

  def set_access_token
    self.access_token = generate_token
  end

  def generate_token
    loop do
      token = SecureRandom.hex(10)
      break token unless User.where(access_token: token).exists?
    end
  end
end

has_secure_token in Rails 5

Rails 5 has added has_secure_token method to generate a random alphanumeric token for a given column.

class User < ApplicationRecord
  has_secure_token
end

By default, Rails assumes that the attribute name is token. We can provide a different name as a parameter to has_secure_token if the attribute name is not token.

class User < ApplicationRecord
  has_secure_token :password_reset_token
end

The above code assumes that we already have password_reset_token attribute in our model.

>> user = User.new
>> user.save
=> true

>> user.password_reset_token
=> 'qjCbex522DfVEVd5ysUWppWQ'

The generated tokens are URL safe and are of fixed length strings.

Migration helper for generating token

We can also generate migration for token similar to other data types.

$ rails g migration add_auth_token_to_user auth_token:token
class AddAuthTokenToUser < ActiveRecord::Migration[5.0]
  def change
    add_column :users, :auth_token, :string
    add_index :users, :auth_token, unique: true
  end
end

Notice that migration automatically adds index on the generated column with unique constraint.

We can also generate a model with the token attribute.

$ rails g model Product access_token:token
class CreateProducts < ActiveRecord::Migration[5.0]
  def change
    create_table :products do |t|
      t.string :access_token

      t.timestamps
    end
    add_index :products, :access_token, unique: true
  end
end

Model generator also adds has_secure_token method to the model.

class Product < ApplicationRecord
  has_secure_token :access_token
end

Regenerating tokens

Sometimes we need to regenerate the tokens based on some expiration criteria.

In order to do that, we can simply call regenerate_#{token_attribute_name} which would regenerate the token and save it to its respective attribute.

>> user = User.first
=> <User id: 11, name: 'John', email: 'john@example.com',
         token: "jRMcN645BQyDr67yHR3qjsJF",
         password_reset_token: "qjCbex522DfVEVd5ysUWppWQ">

>> user.password_reset_token
=> "qjCbex522DfVEVd5ysUWppWQ"

>> user.regenerate_password_reset_token
=> true

>> user.password_reset_token
=> "tYYVjnCEd1LAXvmLCyyQFzbm"

Beware of race condition

It is possible to generate a race condition in the database while generating the tokens. So it is advisable to add a unique index in the database to deal with this unlikely scenario.

Suppress save events in Rails 5

This blog is part of our Rails 5 series.

Rails 5 added suppress method which is used to prevent the receiver from being saved during the given block.

Use case for suppress method

Let’s say, we have an E-commerce application, which has many products. Whenever new product is launched then subscribed customers are notified about it.

class Product < ApplicationRecord
  has_many :notifications
  belongs_to :seller

  after_save :send_notification

  def launch!
    update_attributes!(launched: true)
  end

  private

  def send_notification
    notifications.create(message: 'New product Launched', seller: seller)
  end
end

class Notification < ApplicationRecord
  belongs_to :product
  belongs_to :seller

  after_create :send_notifications

  private

  def send_notifications
    # Sends notification about product to customers.
  end
end

class Seller < ApplicationRecord
  has_many :products
end

This creates a notification record every time we launch a product.

>> Notification.count
=> 0

>> seller = Seller.last
=> <Seller id: 6, name: "John">

>> product = seller.products.create(name: 'baseball hat')
=> <Product id: 4, name: "baseball hat", seller_id: 6>

>> product.launch!

>> Notification.count
=> 1

Now, we have a situation where we need to launch a product but we don’t want to send notifications about it.

Before Rails 5, this was possible only by adding more conditions.

ActiveRecord::Base.Suppress in Rails 5

In Rails 5, we can use ActiveRecord::Base.suppress method to suppress creating of notifications as shown below.

class Product < ApplicationRecord
  def launch_without_notifications
    Notification.suppress do
      launch!
    end
  end
end

>> Notification.count
=> 0

>> product = Product.create!(name: 'tennis hat')
=> <Event id: 1, name: "tennis hat">

>> product.launch_without_notifications

>> Notification.count
=> 0

As we can see, no new notifications were created when product is launched inside Notification.suppress block.

Checkout the pull request to gain better understanding of how suppress works.

Rails 5 makes rendering partial from cache substantially faster

This blog is part of our Rails 5 series.

Let’s have a look at Rails view code that renders partial using a collection.

# index.html.erb
<%= render partial: 'todo', collection: @todos %>

# _todo.html.erb
<% cache todo do %>
  <%= todo.name %>
<% end %>

In the above case Rails will do one fetch from the cache for each todo.

Fetch is usually pretty fast with any caching solution, however, one fetch per todo can make the app slow.

Gem multi_fetch_fragments fixed this issue by using read_multi api provided by Rails.

In a single call to cache, this gem fetches all the cache fragments for a collection. The author of the gem saw 78% speed improvement by using this gem.

The features of this gem have been folded into Rails 5.

To get benefits of collection caching, just add cached: true as shown below.

# index.html.erb
<%= render partial: 'todo', collection: @todos, cached: true %>

# _todo.html.erb
<% cache todo do %>
  <%= todo.name %>
<% end %>

With cached: true present, Rails will use read_multi to the cache store instead of reading from it every partial.

Rails will also log cache hits in the logs as below.

  Rendered collection of todos/_todo.html.erb [100 / 100 cache hits] (339.5ms)

Checkout the pull request to gain better understanding about how collection caching works.

Rails 5 switches from strong etags to weak etags

This blog is part of our Rails 5 series.

ETag, short for entity tag, is a part of HTTP header and is used for web cache validation. ETag is a digest of the resource that uniquely identifies specific version of the resource. This helps browser and web servers determine if resource in the browser’s cache is exactly same as the resource on the server.

Strong v/s Weak ETags

ETag supports strong and weak validation of the resource.

Strong ETag indicates that resource content is same for response body and the response headers.

Weak ETag indicates that the two representations are semantically equivalent. It compares only the response body.

Weak ETags are prefixed with W\ and thus one can easily distinguish between Weak ETags and Strong ETags.

"543b39c23d8d34c232b457297d38ad99"    – Strong ETag
W/"543b39c23d8d34c232b457297d38ad99"  – Weak ETag

W3 has an example page to illustrate how ETag matching works.

When server receives a request, it returns an ETag header as part of HTTP response. This ETag represents state of the resource. For the subsequent HTTP requests, client sends this ETag via If-None-Match header to identify if the resource is changed or not. The server will compare the current ETag and the one sent by the client. If ETag matches, server responds with 304 Not modified. This means resource content in the client’s cache is up-to-date. If resource is changed, server will send updated resource along with the new ETag.

Let’s see it in action.

ETags in Rails 4.x

Rails 4.x generates strong ETags by default i.e without W/ prefix.

class ItemsController < ApplicationController
  def show
    @item = Item.find(params[:id])
    fresh_when @item
  end
end

We are making first request to the server.

$ curl -i http://localhost:3000/items/1

HTTP/1.1 200 OK
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Etag: "618bbc92e2d35ea1945008b42799b0e7"
Last-Modified: Sat, 30 Jan 2016 08:02:12 GMT
Content-Type: text/html; charset=utf-8
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: 98359119-14ae-4e4e-8174-708abbc3fd4b
X-Runtime: 0.412232
Server: WEBrick/1.3.1 (Ruby/2.2.2/2015-04-13)
Date: Fri, 04 Mar 2016 10:50:38 GMT
Content-Length: 1014
Connection: Keep-Alive

For the next request, we will send ETag that was sent by the sever. And notice that server returns 304 Not Modified.

$ curl -i -H 'If-None-Match: "618bbc92e2d35ea1945008b42799b0e7"' http://localhost:3000/items/1

HTTP/1.1 304 Not Modified
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Etag: "618bbc92e2d35ea1945008b42799b0e7"
Last-Modified: Sat, 30 Jan 2016 08:02:12 GMT
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: e4447f82-b96c-4482-a5ff-4f5003910c18
X-Runtime: 0.012878
Server: WEBrick/1.3.1 (Ruby/2.2.2/2015-04-13)
Date: Fri, 04 Mar 2016 10:51:22 GMT
Connection: Keep-Alive

Rails 5 sets Weak ETags by default

In Rails 5, all ETags generated by Rails will be weak by default.

$ curl -i http://localhost:3000/items/1

HTTP/1.1 200 OK
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Etag: W/"b749c4dd1b20885128f9d9a1a8ba70b6"
Last-Modified: Sat, 05 Mar 2016 00:00:00 GMT
Content-Type: text/html; charset=utf-8
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: a24b986c-74f0-4e23-9b1d-0b52cb3ef906
X-Runtime: 0.038372
Server: WEBrick/1.3.1 (Ruby/2.2.3/2015-08-18)
Date: Fri, 04 Mar 2016 10:48:35 GMT
Content-Length: 1906
Connection: Keep-Alive

Now for the second request, server will return 304 Not Modified response as before, but the ETag is weak ETag.

$ curl -i -H 'If-None-Match: W/"b749c4dd1b20885128f9d9a1a8ba70b6"' http://localhost:3000/items/1

HTTP/1.1 304 Not Modified
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Etag: W/"b749c4dd1b20885128f9d9a1a8ba70b6"
Last-Modified: Sat, 05 Mar 2016 00:00:00 GMT
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: 7fc8a8b9-c7ff-4600-bf9b-c847201973cc
X-Runtime: 0.005469
Server: WEBrick/1.3.1 (Ruby/2.2.3/2015-08-18)
Date: Fri, 04 Mar 2016 10:49:27 GMT
Connection: Keep-Alive

Why this change?

Rails does not perform strong validation of ETags as implied by strong ETags spec. Rails just checks whether the incoming ETag from the request headers matches with the ETag of the generated response. It does not do byte by byte comparison of the response.

This was true even before Rails 5. So this change is more of a course correction. Rack also generates weak ETags by default because of similar reasons.

Mark Notthingham is chair of HTTP Working Group and he has written about etags which has some useful links to other ETag resources.

How to use strong ETags in Rails 5

If we want to bypass default Rails 5 behavior to use strong ETags then we can do by following way.

class ItemsController < ApplicationController
  def show
    @item = Item.find(params[:id])
    fresh_when strong_etag: @item
  end
end

This will generate strong Etag i.e without W/ prefix.

Parameter filtering enhancement in Rails 5

This blog is part of our Rails 5 series.

For security reasons, we do not want sensitive data like passwords, credit card information, auth keys etc to appear in log files.

Rails makes it very easy to filter such data. Just add following line in application.rb to filter sensitive information.

config.filter_parameters += [:password]

Now the log file will show [FILTERED] instead of real password value.

This replacement of password with [FILTERED] is done recursively.

{user_name: "john", password: "123"}
{user: {name: "john", password: "123"}}
{user: {auth: {id: "john", password: "123"}}}

In all the above cases, “123” would be replaced by “[FILTERED]”.

Now think of a situation where we do not want to filter all the occurrence of a key. Here is an example.

{credit_card: {number: "123456789", code: "999"}}
{user_preference: {color: {name: "Grey", code: "999999"}}}

We definitely want to filter [:credit_card][:code] but we want [:color][:code] to show up in the log file.

This can be achieved in Rails 5.

The application.rb changes from

config.filter_parameters += ["code"]

to

config.filter_parameters += ["credit_card.code"]

In this case so long as parent of code is credit_card Rails will filter the data.

Rails 5 adds http_cache_forever to allow caching of response forever

This blog is part of our Rails 5 series.

Rails 5 allows to cache HTTP responses forever by introducing http_cache_forever method.

Sometimes, we have static pages that never/rarely change.

# app/controllers/home_controller.rb
class HomeController < ApplicationController
  def index
    render
  end
end

# app/views/home/index.html.erb
<h1>Welcome</h1>

Let’s see log for the above action.

Processing by HomeController#index as HTML
  Rendered home/index.html.erb within layouts/application (1.3ms)
Completed 200 OK in 224ms (Views: 212.4ms | ActiveRecord: 0.0ms)

 And so on for every request for this action.

There is no change in the response and still we are rendering same thing again and again and again.

Rails 5 introduces http_cache_forever

When response does not change then we want browsers and proxies to cache it for a long time.

Method http_cache_forever allows us to set response headers to tell browsers and proxies that response has not modified.

# app/controllers/home_controller.rb
class HomeController < ApplicationController
  def index
    http_cache_forever(public: true) {}
  end
end

# OR
class HomeController < ApplicationController
  def index
    http_cache_forever(public: true) do
      render
    end
  end
end


# app/views/home/index.html.erb
<h1>Welcome</h1>

Now let’s look at the log for the modified code.

# When request is made for the first time.

Processing by HomeController#index as HTML
  Rendered home/index.html.erb within layouts/application (1.3ms)
Completed 200 OK in 224ms (Views: 212.4ms | ActiveRecord: 0.0ms)

# For consecutive requests for the same page

Processing by HomeController#index as HTML
Completed 304 Not Modified in 2ms (ActiveRecord: 0.0ms)

On first hit, we serve the request normally but, then on each subsequent request cache is revalidated and a “304 Not Modified” response is sent to the browser.

Options with http_cache_forever

By default, HTTP responses are cached only on the user’s web browser. To allow proxies to cache the response, we can set public to true to indicate that they can serve the cached response.

Use http_cache_forever with caution

By using this method, Cache-Control: max-age=3155760000 is set as response header and browser/proxy won’t revalidate the resource back to the server unless force reload is done.

In case force reload is done, Cache-Control: max-age=0 is set as request header.

In this case, browser will receive the changed resource whether ETag is changed or not.

http_cache_forever is literally going to set the headers to cache it for 100 years and developers would have to take extra steps to revalidate it. So, this should be used with extra care.

Better exception responses in Rails 5 API apps

This blog is part of our Rails 5 series.

Rails 4.x returns error information in HTML page whenever there is any exception, in the development environment.

This is fine for normal HTML requests. But traditionally, Rails always returned with HTML response for exceptions for all requests, including JSON on XML requests in development.

We can now generate API only apps in Rails 5. In case of such apps, it’s better to have the error message in the format in which request was made. Having an HTML response for a JSON endpoint like http://localhost:3000/posts.json is not going to help in debugging why the exception happened.

New config option debug_exception_response_format

Rails 5 has introduced new configuration to respond with proper format for exceptions.

# config/environments/development.rb
config.debug_exception_response_format = :api

Let’s see an example of the response received with this configuration.

$ curl localhost:3000/posts.json
{"status":404,"error":"Not Found","exception":"#\u003cActionController::RoutingError: No route matches [GET] \"/posts.json\"\u003e","traces":{"Application Trace":[...],"Framework Trace":[...]}}

The status key will represent HTTP status code and error key will represent the corresponding Rack HTTP status.

exception will print the output of actual exception in inspect format.

traces will contain application and framework traces similar to how they are displayed in HTML error page.

By default, config.debug_exception_response_format is set to :api so as to render responses in the same format as requests.

If you want the original behavior of rendering HTML pages, you can configure this option as follows.

# config/environments/development.rb
config.debug_exception_response_format = :default

Use file_fixture to access test files in Rails 5

This blog is part of our Rails 5 series.

While writing tests sometimes we need to read files to compare the output. For example a test might want to compare the API response with pre determined data stored in a file.

Here is an example.

# In test helper
def file_data(name)
  File.read(Rails.root.to_s + "/tests/support/files/#{name}")
end

# In test
class PostsControllerTest < ActionDispatch::IntegrationTest
  setup do
    @post = posts(:one)
  end

  test "should get index" do
    get posts_url, format: :json

    assert_equal file_data('posts.json'), response.body
  end
end

File Fixtures in Rails 5

In Rails 5, we can now organize such test files as fixtures.

Newly generated Rails 5 applications, will have directory test/fixtures/files to store such test files.

These test files can be accessed using file_fixture helper method in tests.

require 'test_helper'

class PostsControllerTest < ActionDispatch::IntegrationTest
  setup do
    @post = posts(:one)
  end

  test "should get index" do
    get posts_url, format: :json

    assert_equal response.body, file_fixture('posts.json').read
  end
end

The file_fixture method returns Pathname object, so it’s easy to extract file specific information.

file_fixture('posts.json').read
file_fixture('song.mp3').size

Migrations are versioned in Rails 5

This blog is part of our Rails 5 series.

We will see how migrations in Rails 5 differ by looking at different cases.

Case I

In Rails 4.x command

rails g model User name:string

will generate migration as shown below.

class CreateUsers < ActiveRecord::Migration
  def change
    create_table :users do |t|
      t.string :name
      t.timestamps null: false
    end
  end
end

In Rails 5 the same command will generate following migration.

class CreateUsers < ActiveRecord::Migration[5.0]
  def change
    create_table :users do |t|
      t.string :name
      t.timestamps
    end
  end
end

Let’s see the generated schema after running migration generated in Rails 5.

sqlite> .schema users
CREATE TABLE "users" ("id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, "name" varchar, "created_at" datetime NOT NULL, "updated_at" datetime NOT NULL);
sqlite>

Rails 5 added the NOT NULL constraints on the timestamps columns even though not null constraint was not specified in the migration.

Case II

Let’s look at another example.

In Rails 4.x command

rails g model Task user:references

would generate following migration.

class CreateTasks < ActiveRecord::Migration
  def change
    create_table :tasks do |t|
      t.references :user, index: true, foreign_key: true
      t.timestamps null: false
    end
  end
end

In Rails 5.0, same command will generate following migration.

class CreateTasks < ActiveRecord::Migration[5.0]
  def change
    create_table :tasks do |t|
      t.references :user, foreign_key: true

      t.timestamps
    end
  end
end

There is no mention of index: true in the above migration. Let’s see the generated schema after running Rails 5 migration.

sqlite> .schema tasks
CREATE TABLE "tasks" ("id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, "user_id" integer, "created_at" datetime NOT NULL, "updated_at" datetime NOT NULL);

CREATE INDEX "index_tasks_on_user_id" ON "tasks" ("user_id");

As you can see, an index on user_id column is added even though it’s not present in the migration.

Migration API has changed in Rails 5

Rails 5 has changed migration API because of which even though null: false options is not passed to timestamps when migrations are run then not null is automatically added for timestamps.

Similarly, we want indexes for referenced columns in almost all cases. So Rails 5 does not need references to have index: true. When migrations are run then index is automatically created.

Now let’s assume that an app was created in Rails 4.x. It has a bunch of migrations. Later the app was upgraded to Rails 5. Now when older migrations are run then those migrations will behave differently and will create a different schema file. This is a problem.

Solution is versioned migrations.

Versioned migrations in Rails 5

Let’s look at the migration generated in Rails 5 closely.

class CreateTasks < ActiveRecord::Migration[5.0]
  def change
    create_table :tasks do |t|
      t.references :user, index: true, foreign_key: true
      t.timestamps null: false
    end
  end
end

In this case CreateUsers class is now inheriting from ActiveRecord::Migration[5.0] instead of ActiveRecord::Migration.

Here [5.0] is Rails version that generated this migration.

Solving the issue with older migrations

Whenever Rails 5 runs migrations, it checks the class of the current migration file being run. If it’s 5.0, it uses the new migration API which has changes like automatically adding null: false to timestamps.

But whenever the class of migration file is other than ActiveRecord::Migration[5.0], Rails will use a compatibility layer of migrations API. Currently this compatibility layer is present for Rails 4.2. What it means is that all migration generated prior to usage of Rails 5 will be treated as if they were generate in Rails 4.2.

You will also see a deprecation warning asking user to add the version of the migration to the class name for older migrations.

So if you are migrating a Rails 4.2 app, all of your migrations will have class ActiveRecord::Migration. If you run those migrations in Rails 5, you will see a warning asking to add version name to the class name so that class name looks like ActiveRecord::Migration[4.2].

Rails 5 improves redirect_to :back with new redirect_back method

This blog is part of our Rails 5 series.

In Rails 4.x, for going back to previous page we use redirect_to :back.

However sometimes we get ActionController::RedirectBackError exception when HTTP_REFERER is not present.

class PostsController < ApplicationController
  def publish
    post = Post.find params[:id]
    post.publish!

    redirect_to :back
  end
end

This works well when HTTP_REFERER is present and it redirects to previous page.

Issue comes up when HTTP_REFERER is not present and which in turn throws exception.

To avoid this exception we can use rescue and redirect to root url.

class PostsController < ApplicationController
  rescue_from ActionController::RedirectBackError, with: :redirect_to_default

  def publish
    post = Post.find params[:id]
    post.publish!
    redirect_to :back
  end

  private

  def redirect_to_default
    redirect_to root_path
  end
end

Improvement in Rails 5

In Rails 5, redirect_to :back has been deprecated and instead a new method has been added called redirect_back.

To deal with the situation when HTTP_REFERER is not present, it takes required option fallback_location.

class PostsController < ApplicationController

  def publish
    post = Post.find params[:id]
    post.publish!

    redirect_back(fallback_location: root_path)
  end
end

This redirects to HTTP_REFERER when it is present and when HTTP_REFERER is not present then redirects to whatever is passed as fallback_location.

Rails 5 allows configuring queue name for mailers

This blog is part of our Rails 5 series.

In Rails 4.2, Active Job was integrated with Action Mailer to send emails asynchronously.

Rails provides deliver_later method to enqueue mailer jobs.

class UserMailer < ApplicationMailer
  def send_notification(user)
    @user = user
    mail(to: user.email)
  end
end

> UserMailer.send_notification(user).deliver_later
=> <ActionMailer::DeliveryJob:0x007ff602dd3128 @arguments=["UserMailer", "send_notification", "deliver_now", <User id: 1, name: "John", email: "john@bigbinary.com">], @job_id="d0171da0-86d3-49f4-ba03-37b37d4e8e2b", @queue_name="mailers", @priority=nil>

Note that the task of delivering email was put in queue called mailers.

In Rails 4.x, all background jobs are given queue named “default” except for mailers. All outgoing mails are given the queue named “mailers” and we do not have the option of changing this queue name from “mailers” to anything else.

Since Rails 4.x comes with minimum of two queues it makes difficult to use queuing services like que which relies on applications having only one queue.

Customizing queue name in Rails 5

In Rails 5, we can now change queue name for mailer jobs using following configuration.

config.action_mailer.deliver_later_queue_name = 'default'

class UserMailer < ApplicationMailer
  def send_notification(user)
    @user = user
    mail(to: user.email)
  end
end

2.2.2 :003 > user = User.last
=> <User id: 6, name: "John", email: "john@bigbinary.com">

2.2.2 :004 > UserMailer.send_notification(user).deliver_later
=> <ActionMailer::DeliveryJob:0x007fea2182b2d0 @arguments=["UserMailer", "send_notification", "deliver_now", <User id: 1, name: "John", email: "john@bigbinary.com">], @job_id="316b00b2-64c8-4a2d-8153-4ce7abafb28d", @queue_name="default", @priority=nil>

Rails 5 handles DateTime with better precision

This blog is part of our Rails 5 series.

MySQL 5.6.4 and up has added fractional seconds support for TIME, DATETIME, and TIMESTAMP values, with up to microseconds (6 digits) precision.

Adding precision to migration

To add precision on datetime column we need to add limit option to it. By default it is set to 0.

def change
  add_column :users, :last_seen_at, :datetime, limit: 6
end

This adds precision(6) to last_seen_at column in users table.

Rails 4.x behavior

Let’s look at the some of the examples with different precision values.

The task here is to set end_of_day value to updated_at column.

With precision set to 6

user = User.first
user.updated_at
=> Mon, 18 Jan 2016 10:13:10 UTC +00:00

user.updated_at = user.updated_at.end_of_day
=> Mon, 18 Jan 2016 23:59:59 UTC +00:00

user.save
'UPDATE `users` SET `updated_at` = '2016-01-18 23:59:59.999999' WHERE `users`.`id` = 1'

user.updated_at
=> Mon, 18 Jan 2016 23:59:59 UTC +00:00

user.reload
user.updated_at
=> Mon, 18 Jan 2016 23:59:59 UTC +00:00

Everything looks good here.

But let’s look at what happens when precision is set to 0.

With precision set to 0

user = User.first
user.updated_at
=> Mon, 18 Jan 2016 10:13:10 UTC +00:00

user.updated_at = user.updated_at.end_of_day
=> Mon, 18 Jan 2016 23:59:59 UTC +00:00

user.save
'UPDATE `users` SET `updated_at` = '2016-01-18 23:59:59.999999' WHERE `users`.`id` = 1'

user.updated_at
=> Mon, 18 Jan 2016 23:59:59 UTC +00:00

So far everything looks good here too. Now let’s see what happens when we reload this object.

user.reload
user.updated_at
=> Tue, 19 Jan 2016 00:00:00 UTC +00:00

As we can clearly see after the reload updated_at value has been rounded off from 2016-01-18 23:59:59.999999 to 2016-01-19 00:00:00. It might seem like a small issue but notice that date has changed from 01/18 to 01/19 because of this rounding.

Improvement in Rails 5

Rails team fixed this issue by removing fractional part if mysql adapter does not support precision.

Here are the two relevant commits to this change.

With precision set to 0

user.updated_at
=> Tue, 19 Jan 2016 00:00:00 UTC +00:00

user.updated_at = user.updated_at.tomorrow.beginning_of_day - 1
=> Tue, 19 Jan 2016 23:59:59 UTC +00:00

user.save
'UPDATE `users` SET `updated_at` = '2016-01-19 23:59:59' WHERE `users`.`id` = 1'

user.reload

user.updated_at
=> Tue, 19 Jan 2016 23:59:59 UTC +00:00

If precision is not set then fractional part gets stripped and date is not changed.

Active Support Improvements in Rails 5

This blog is part of our Rails 5 series.

Rails 5 has added some nice enhancements to Active Support. This blog will go over some of those changes.

Improvements in Date, Time and Datetime

prev_day and next_day

As the name of the methods suggests, next_day returns next calendar date.

Similarly, prev_day returns previous calendar date.

Time.current
=> Fri, 12 Feb 2016 08:53:31 UTC +00:00

Time.current.next_day
=> Sat, 13 Feb 2016 08:53:31 UTC +00:00

Time.current.prev_day
=> Thu, 11 Feb 2016 08:53:31 UTC +00:00

Support for same_time option to next_week and prev_week

In Rails 4.x next_week returns beginning of next week and prev_week returns beginning of previous week.

In Rails 4.x these two methods also accept week day as a parameter.

Time.current
=> Fri, 12 Feb 2016 08:53:31 UTC +00:00

Time.current.next_week
=> Mon, 15 Feb 2016 00:00:00 UTC +00:00

Time.current.next_week(:tuesday)
=> Tue, 16 Feb 2016 00:00:00 UTC +00:00

Time.current.prev_week(:tuesday)
=> Tue, 02 Feb 2016 00:00:00 UTC +00:00

By using week day as parameter we can get the date one week from now but the returned date is still the beginning of that date. How do we get one week from the current time.

Rails 5 add an additional option same_time: true to solve this problem.

Using this option, we can now get next week date from the current time.

Time.current
=> Fri, 12 Feb 2016 09:15:10 UTC +00:00

Time.current.next_week
=> Mon, 15 Feb 2016 00:00:00 UTC +00:00

Time.current.next_week(same_time: true)
=> Mon, 15 Feb 2016 09:15:20 UTC +00:00

Time.current.prev_week
=> Mon, 01 Feb 2016 00:00:00 UTC +00:00

Time.current.prev_week(same_time: true)
=> Mon, 01 Feb 2016 09:16:50 UTC +00:00

on_weekend?

This method returns true if the receiving date/time is a Saturday or Sunday.

Time.current
=> Fri, 12 Feb 2016 09:47:40 UTC +00:00

Time.current.on_weekend?
=> false

Time.current.tomorrow
=> Sat, 13 Feb 2016 09:48:47 UTC +00:00

Time.current.tomorrow.on_weekend?
=> true

on_weekday?

This method returns true if the receiving date/time is not a Saturday or Sunday.

Time.current
=> Fri, 12 Feb 2016 09:47:40 UTC +00:00

Time.current.on_weekday?
=> true

Time.current.tomorrow
=> Sat, 13 Feb 2016 09:48:47 UTC +00:00

Time.current.tomorrow.on_weekday?
=> false

next_weekday and prev_weekday

next_weekday returns next day that is not a weekend.

Similarly, prev_weekday returns last day that is not a weekend.

Time.current
=> Fri, 12 Feb 2016 09:47:40 UTC +00:00

Time.current.next_weekday
=> Mon, 15 Feb 2016 09:55:14 UTC +00:00

Time.current.prev_weekday
=> Thu, 11 Feb 2016 09:55:33 UTC +00:00

Time.days_in_year

# Gives number of days in current year, if year is not passed.
Time.days_in_year
=> 366

# Gives number of days in specified year, if year is passed.
Time.days_in_year(2015)
=> 365

Improvements in Enumerable

pluck

pluck method is now added to Enumerable objects.

users = [{id: 1, name: 'Max'}, {id: 2, name: 'Mark'}, {id: 3, name: 'George'}]

users.pluck(:name)
=> ["Max", "Mark", "George"]

# Takes multiple arguments as well
users.pluck(:id, :name)
=> [[1, "Max"], [2, "Mark"], [3, "George"]]

one great improvement in ActiveRecord due to this method addition is that when relation is already loaded then instead of firing query with pluck, it uses Enumerable#pluck to get data.

# In Rails 4.x
users = User.all
SELECT `users`.* FROM `users`

users.pluck(:id, :name)
SELECT "users"."id", "users"."name" FROM "users"

=> [[2, "Max"], [3, "Mark"], [4, "George"]]


# In Rails 5
users = User.all
SELECT "users".* FROM "users"

# does not fire any query
users.pluck(:id, :name)
=> [[1, "Max"], [2, "Mark"], [3, "George"]]

without

This method returns a copy of enumerable without the elements passed to the method.

vehicles = ['Car', 'Bike', 'Truck', 'Bus']

vehicles.without("Car", "Bike")
=> ["Truck", "Bus"]

vehicles = {car: 'Hyundai', bike: 'Honda', bus: 'Mercedes', truck: 'Tata'}

vehicles.without(:bike, :bus)
=> {:car=>"Hyundai", :truck=>"Tata"}

Array#second_to_last and Array#third_to_last

['a', 'b', 'c', 'd', 'e'].second_to_last
=> "d"

['a', 'b', 'c', 'd', 'e'].third_to_last
=> "c"

PR for these methods can be found here.

Integer#positive? and Integer#negative?

positive? returns true if integer is positive.

negative? returns true if integer is negative.

4.positive?
=> true

4.negative?
=> false

-4.0.positive?
=> false

-4.0.negative?
=> true

Commit for these methods can be found here.

These changes have now been added to Ruby 2.3 also.

Array#inquiry

Rails team has added ArrayInquirer to ActiveSupport which gives a friendlier way to check its contents.

Array#inquiry is a shortcut for wrapping the receiving array in an ArrayInquirer

users = [:mark, :max, :david]

array_inquirer1 = ActiveSupport::ArrayInquirer.new(users)

# creates ArrayInquirer object which is same as array_inquirer1 above
array_inquirer2 = users.inquiry

array_inquirer2.class
=> ActiveSupport::ArrayInquirer

# provides methods like:

array_inquirer2.mark?
=> true

array_inquirer2.john?
=> false

array_inquirer2.any?(:john, :mark)
=> true

array_inquirer2.any?(:mark, :david)
=> true

array_inquirer2.any?(:john, :louis)
=> false

Rails 5 improves searching for routes with advanced options

This blog is part of our Rails 5 series.

rails routes shows all the routes in the application.

$ rake routes

Prefix       Verb   URI Pattern                   Controller#Action
wishlist_user GET    /users/:id/wishlist(.:format) users#wishlist
        users GET    /users(.:format)              users#index
              POST   /users(.:format)              users#create
     new_user GET    /users/new(.:format)          users#new
    edit_user GET    /users/:id/edit(.:format)     users#edit
         user GET    /users/:id(.:format)          users#show
              PATCH  /users/:id(.:format)          users#update
              PUT    /users/:id(.:format)          users#update
              DELETE /users/:id(.:format)          users#destroy
     products GET    /products(.:format)           products#index
              POST   /products(.:format)           products#create
and so on ......

This list can be lengthy and it could be difficult to locate exactly what user is looking for.

Ways to search specific routes prior to Rails 5

To see only specific routes we can use commands like grep.

$ rake routes | grep products
Prefix       Verb   URI Pattern                   Controller#Action
products      GET    /products(.:format)           products#index
              POST   /products(.:format)           products#create

Options with Rails 5

Rails 5 has added options in rails routes to perform pattern matching on routes.

Use option -c to search for routes related to controller. Also remember that Rails does case insensitive search. So rails routes -c users is same as rails routes -c Users.

# Search for Controller name
$ rails routes -c users
       Prefix Verb   URI Pattern                   Controller#Action
wishlist_user GET    /users/:id/wishlist(.:format) users#wishlist
        users GET    /users(.:format)              users#index
              POST   /users(.:format)              users#create

# Search for namespaced Controller name.
$ rails routes -c admin/users
         Prefix Verb   URI Pattern                     Controller#Action
    admin_users GET    /admin/users(.:format)          admin/users#index
                POST   /admin/users(.:format)          admin/users#create

# Search for namespaced Controller name.
$ rails routes -c Admin::UsersController
         Prefix Verb   URI Pattern                     Controller#Action
    admin_users GET    /admin/users(.:format)          admin/users#index
                POST   /admin/users(.:format)          admin/users#create

Use -g option to do general purpose pattern matching. This results in any routes that partially matches Prefix, Controller#Action or the URI pattern.

# Search with pattern
$ rails routes -g wishlist
       Prefix Verb URI Pattern                   Controller#Action
wishlist_user GET  /users/:id/wishlist(.:format) users#wishlist

# Search with HTTP Verb
$ rails routes -g POST
    Prefix Verb URI Pattern            Controller#Action
           POST /users(.:format)       users#create
           POST /admin/users(.:format) admin/users#create
           POST /products(.:format)    products#create

# Search with URI pattern
$ rails routes -g admin
       Prefix Verb   URI Pattern                     Controller#Action
  admin_users GET    /admin/users(.:format)          admin/users#index
              POST   /admin/users(.:format)          admin/users#create

Note that using CONTROLLER=some_controller has now been deprecated. This had the same effect as searching for a controller specific route.

Rails 5 makes belongs_to association required by default

This blog is part of our Rails 5 series.

In Rails 5, whenever we define a belongs_to association, it is required to have the associated record present by default after this change.

It triggers validation error if associated record is not present.

class User < ApplicationRecord
end

class Post < ApplicationRecord
  belongs_to :user
end

post = Post.create(title: 'Hi')
=> <Post id: nil, title: "Hi", user_id: nil, created_at: nil, updated_at: nil>

post.errors.full_messages.to_sentence
=> "User must exist"

As we can see, we can’t create any post record without having an associated user record.

How to achieve this behavior before Rails 5

In Rails 4.x world To add validation on belongs_to association, we need to add option required: true .

class User < ApplicationRecord
end

class Post < ApplicationRecord
  belongs_to :user, required: true
end

post = Post.create(title: 'Hi')
=> <Post id: nil, title: "Hi", user_id: nil, created_at: nil, updated_at: nil>

post.errors.full_messages.to_sentence
=> "User must exist"

By default, required option is set to false.

Opting out of this default behavior in Rails 5

We can pass optional: true to the belongs_to association which would remove this validation check.

class Post < ApplicationRecord
  belongs_to :user, optional: true
end

post = Post.create(title: 'Hi')
=> <Post id: 2, title: "Hi", user_id: nil>

But, what if we do not need this behavior anywhere in our entire application and not just a single model?

Opting out of this default behavior for the entire application

New Rails 5 application comes with an initializer named new_framework_defaults.rb.

When upgrading from older version of Rails to Rails 5, we can add this initializer by running bin/rails app:update task.

This initializer has config named Rails.application.config.active_record.belongs_to_required_by_default = true

For new Rails 5 application the value is set to true but for old applications, this is set to false by default.

We can turn off this behavior by keeping the value to false.

Rails.application.config.active_record.belongs_to_required_by_default = false

class Post < ApplicationRecord
  belongs_to :user
end

post = Post.create(title: 'Hi')
=> <Post id: 3, title: "Hi", user_id: nil, created_at: "2016-02-11 12:36:05", updated_at: "2016-02-11 12:36:05">