This blog is part of our Rails 5 series.

While writing tests sometimes we need to read files to compare the output. For example a test might want to compare the API response with pre determined data stored in a file.

Here is an example.

# In test helper
def file_data(name)
  File.read(Rails.root.to_s + "/tests/support/files/#{name}")
end

# In test
class PostsControllerTest < ActionDispatch::IntegrationTest
  setup do
    @post = posts(:one)
  end

  test "should get index" do
    get posts_url, format: :json

    assert_equal file_data('posts.json'), response.body
  end
end

File Fixtures in Rails 5

In Rails 5, we can now organize such test files as fixtures.

Newly generated Rails 5 applications, will have directory test/fixtures/files to store such test files.

These test files can be accessed using file_fixture helper method in tests.

require 'test_helper'

class PostsControllerTest < ActionDispatch::IntegrationTest
  setup do
    @post = posts(:one)
  end

  test "should get index" do
    get posts_url, format: :json

    assert_equal response.body, file_fixture('posts.json').read
  end
end

The file_fixture method returns Pathname object, so it’s easy to extract file specific information.

file_fixture('posts.json').read
file_fixture('song.mp3').size

This blog is part of our Rails 5 series.

We will see how migrations in Rails 5 differ by looking at different cases.

Case I

In Rails 4.x command

rails g model User name:string

will generate migration as shown below.

class CreateUsers < ActiveRecord::Migration
  def change
    create_table :users do |t|
      t.string :name
      t.timestamps null: false
    end
  end
end

In Rails 5 the same command will generate following migration.

class CreateUsers < ActiveRecord::Migration[5.0]
  def change
    create_table :users do |t|
      t.string :name
      t.timestamps
    end
  end
end

Let’s see the generated schema after running migration generated in Rails 5.

sqlite> .schema users
CREATE TABLE "users" ("id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, "name" varchar, "created_at" datetime NOT NULL, "updated_at" datetime NOT NULL);
sqlite>

Rails 5 added the NOT NULL constraints on the timestamps columns even though not null constraint was not specified in the migration.

Case II

Let’s look at another example.

In Rails 4.x command

rails g model Task user:references

would generate following migration.

class CreateTasks < ActiveRecord::Migration
  def change
    create_table :tasks do |t|
      t.references :user, index: true, foreign_key: true
      t.timestamps null: false
    end
  end
end

In Rails 5.0, same command will generate following migration.

class CreateTasks < ActiveRecord::Migration[5.0]
  def change
    create_table :tasks do |t|
      t.references :user, foreign_key: true

      t.timestamps
    end
  end
end

There is no mention of index: true in the above migration. Let’s see the generated schema after running Rails 5 migration.

sqlite> .schema tasks
CREATE TABLE "tasks" ("id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, "user_id" integer, "created_at" datetime NOT NULL, "updated_at" datetime NOT NULL);

CREATE INDEX "index_tasks_on_user_id" ON "tasks" ("user_id");

As you can see, an index on user_id column is added even though it’s not present in the migration.

Migration API has changed in Rails 5

Rails 5 has changed migration API because of which even though null: false options is not passed to timestamps when migrations are run then not null is automatically added for timestamps.

Similarly, we want indexes for referenced columns in almost all cases. So Rails 5 does not need references to have index: true. When migrations are run then index is automatically created.

Now let’s assume that an app was created in Rails 4.x. It has a bunch of migrations. Later the app was upgraded to Rails 5. Now when older migrations are run then those migrations will behave differently and will create a different schema file. This is a problem.

Solution is versioned migrations.

Versioned migrations in Rails 5

Let’s look at the migration generated in Rails 5 closely.

class CreateTasks < ActiveRecord::Migration[5.0]
  def change
    create_table :tasks do |t|
      t.references :user, index: true, foreign_key: true
      t.timestamps null: false
    end
  end
end

In this case CreateUsers class is now inheriting from ActiveRecord::Migration[5.0] instead of ActiveRecord::Migration.

Here [5.0] is Rails version that generated this migration.

Solving the issue with older migrations

Whenever Rails 5 runs migrations, it checks the class of the current migration file being run. If it’s 5.0, it uses the new migration API which has changes like automatically adding null: false to timestamps.

But whenever the class of migration file is other than ActiveRecord::Migration[5.0], Rails will use a compatibility layer of migrations API. Currently this compatibility layer is present for Rails 4.2. What it means is that all migration generated prior to usage of Rails 5 will be treated as if they were generate in Rails 4.2.

You will also see a deprecation warning asking user to add the version of the migration to the class name for older migrations.

So if you are migrating a Rails 4.2 app, all of your migrations will have class ActiveRecord::Migration. If you run those migrations in Rails 5, you will see a warning asking to add version name to the class name so that class name looks like ActiveRecord::Migration[4.2].

This blog is part of our Rails 5 series.

In Rails 4.x, for going back to previous page we use redirect_to :back.

However sometimes we get ActionController::RedirectBackError exception when HTTP_REFERER is not present.

class PostsController < ApplicationController
  def publish
    post = Post.find params[:id]
    post.publish!

    redirect_to :back
  end
end

This works well when HTTP_REFERER is present and it redirects to previous page.

Issue comes up when HTTP_REFERER is not present and which in turn throws exception.

To avoid this exception we can use rescue and redirect to root url.

class PostsController < ApplicationController
  rescue_from ActionController::RedirectBackError, with: :redirect_to_default

  def publish
    post = Post.find params[:id]
    post.publish!
    redirect_to :back
  end

  private

  def redirect_to_default
    redirect_to root_path
  end
end

Improvement in Rails 5

In Rails 5, redirect_to :back has been deprecated and instead a new method has been added called redirect_back.

To deal with the situation when HTTP_REFERER is not present, it takes required option fallback_location.

class PostsController < ApplicationController

  def publish
    post = Post.find params[:id]
    post.publish!

    redirect_back(fallback_location: root_path)
  end
end

This redirects to HTTP_REFERER when it is present and when HTTP_REFERER is not present then redirects to whatever is passed as fallback_location.

This blog is part of our Rails 5 series.

In Rails 4.2, Active Job was integrated with Action Mailer to send emails asynchronously.

Rails provides deliver_later method to enqueue mailer jobs.

class UserMailer < ApplicationMailer
  def send_notification(user)
    @user = user
    mail(to: user.email)
  end
end

> UserMailer.send_notification(user).deliver_later
=> <ActionMailer::DeliveryJob:0x007ff602dd3128 @arguments=["UserMailer", "send_notification", "deliver_now", <User id: 1, name: "John", email: "john@bigbinary.com">], @job_id="d0171da0-86d3-49f4-ba03-37b37d4e8e2b", @queue_name="mailers", @priority=nil>

Note that the task of delivering email was put in queue called mailers.

In Rails 4.x, all background jobs are given queue named “default” except for mailers. All outgoing mails are given the queue named “mailers” and we do not have the option of changing this queue name from “mailers” to anything else.

Since Rails 4.x comes with minimum of two queues it makes difficult to use queuing services like que which relies on applications having only one queue.

Customizing queue name in Rails 5

In Rails 5, we can now change queue name for mailer jobs using following configuration.

config.action_mailer.deliver_later_queue_name = 'default'

class UserMailer < ApplicationMailer
  def send_notification(user)
    @user = user
    mail(to: user.email)
  end
end

2.2.2 :003 > user = User.last
=> <User id: 6, name: "John", email: "john@bigbinary.com">

2.2.2 :004 > UserMailer.send_notification(user).deliver_later
=> <ActionMailer::DeliveryJob:0x007fea2182b2d0 @arguments=["UserMailer", "send_notification", "deliver_now", <User id: 1, name: "John", email: "john@bigbinary.com">], @job_id="316b00b2-64c8-4a2d-8153-4ce7abafb28d", @queue_name="default", @priority=nil>

This blog is part of our Rails 5 series.

MySQL 5.6.4 and up has added fractional seconds support for TIME, DATETIME, and TIMESTAMP values, with up to microseconds (6 digits) precision.

Adding precision to migration

To add precision on datetime column we need to add limit option to it. By default it is set to 0.

def change
  add_column :users, :last_seen_at, :datetime, limit: 6
end

This adds precision(6) to last_seen_at column in users table.

Rails 4.x behavior

Let’s look at the some of the examples with different precision values.

The task here is to set end_of_day value to updated_at column.

With precision set to 6

user = User.first
user.updated_at
=> Mon, 18 Jan 2016 10:13:10 UTC +00:00

user.updated_at = user.updated_at.end_of_day
=> Mon, 18 Jan 2016 23:59:59 UTC +00:00

user.save
'UPDATE `users` SET `updated_at` = '2016-01-18 23:59:59.999999' WHERE `users`.`id` = 1'

user.updated_at
=> Mon, 18 Jan 2016 23:59:59 UTC +00:00

user.reload
user.updated_at
=> Mon, 18 Jan 2016 23:59:59 UTC +00:00

Everything looks good here.

But let’s look at what happens when precision is set to 0.

With precision set to 0

user = User.first
user.updated_at
=> Mon, 18 Jan 2016 10:13:10 UTC +00:00

user.updated_at = user.updated_at.end_of_day
=> Mon, 18 Jan 2016 23:59:59 UTC +00:00

user.save
'UPDATE `users` SET `updated_at` = '2016-01-18 23:59:59.999999' WHERE `users`.`id` = 1'

user.updated_at
=> Mon, 18 Jan 2016 23:59:59 UTC +00:00

So far everything looks good here too. Now let’s see what happens when we reload this object.

user.reload
user.updated_at
=> Tue, 19 Jan 2016 00:00:00 UTC +00:00

As we can clearly see after the reload updated_at value has been rounded off from 2016-01-18 23:59:59.999999 to 2016-01-19 00:00:00. It might seem like a small issue but notice that date has changed from 01/18 to 01/19 because of this rounding.

Improvement in Rails 5

Rails team fixed this issue by removing fractional part if mysql adapter does not support precision.

Here are the two relevant commits to this change.

• Commit for support of precision with mysql

• Commit for checking precision on columns

With precision set to 0

user.updated_at
=> Tue, 19 Jan 2016 00:00:00 UTC +00:00

user.updated_at = user.updated_at.tomorrow.beginning_of_day - 1
=> Tue, 19 Jan 2016 23:59:59 UTC +00:00

user.save
'UPDATE `users` SET `updated_at` = '2016-01-19 23:59:59' WHERE `users`.`id` = 1'

user.reload

user.updated_at
=> Tue, 19 Jan 2016 23:59:59 UTC +00:00

If precision is not set then fractional part gets stripped and date is not changed.

This blog is part of our Rails 5 series.

Rails 5 has added some nice enhancements to Active Support. This blog will go over some of those changes.

Improvements in Date, Time and Datetime

prev_day and next_day

As the name of the methods suggests, next_day returns next calendar date.

Similarly, prev_day returns previous calendar date.

Time.current
=> Fri, 12 Feb 2016 08:53:31 UTC +00:00

Time.current.next_day
=> Sat, 13 Feb 2016 08:53:31 UTC +00:00

Time.current.prev_day
=> Thu, 11 Feb 2016 08:53:31 UTC +00:00

Support for same_time option to next_week and prev_week

In Rails 4.x next_week returns beginning of next week and prev_week returns beginning of previous week.

In Rails 4.x these two methods also accept week day as a parameter.

Time.current
=> Fri, 12 Feb 2016 08:53:31 UTC +00:00

Time.current.next_week
=> Mon, 15 Feb 2016 00:00:00 UTC +00:00

Time.current.next_week(:tuesday)
=> Tue, 16 Feb 2016 00:00:00 UTC +00:00

Time.current.prev_week(:tuesday)
=> Tue, 02 Feb 2016 00:00:00 UTC +00:00

By using week day as parameter we can get the date one week from now but the returned date is still the beginning of that date. How do we get one week from the current time.

Rails 5 add an additional option same_time: true to solve this problem.

Using this option, we can now get next week date from the current time.

Time.current
=> Fri, 12 Feb 2016 09:15:10 UTC +00:00

Time.current.next_week
=> Mon, 15 Feb 2016 00:00:00 UTC +00:00

Time.current.next_week(same_time: true)
=> Mon, 15 Feb 2016 09:15:20 UTC +00:00

Time.current.prev_week
=> Mon, 01 Feb 2016 00:00:00 UTC +00:00

Time.current.prev_week(same_time: true)
=> Mon, 01 Feb 2016 09:16:50 UTC +00:00

on_weekend?

This method returns true if the receiving date/time is a Saturday or Sunday.

Time.current
=> Fri, 12 Feb 2016 09:47:40 UTC +00:00

Time.current.on_weekend?
=> false

Time.current.tomorrow
=> Sat, 13 Feb 2016 09:48:47 UTC +00:00

Time.current.tomorrow.on_weekend?
=> true

on_weekday?

This method returns true if the receiving date/time is not a Saturday or Sunday.

Time.current
=> Fri, 12 Feb 2016 09:47:40 UTC +00:00

Time.current.on_weekday?
=> true

Time.current.tomorrow
=> Sat, 13 Feb 2016 09:48:47 UTC +00:00

Time.current.tomorrow.on_weekday?
=> false

next_weekday and prev_weekday

next_weekday returns next day that is not a weekend.

Similarly, prev_weekday returns last day that is not a weekend.

Time.current
=> Fri, 12 Feb 2016 09:47:40 UTC +00:00

Time.current.next_weekday
=> Mon, 15 Feb 2016 09:55:14 UTC +00:00

Time.current.prev_weekday
=> Thu, 11 Feb 2016 09:55:33 UTC +00:00

Time.days_in_year

# Gives number of days in current year, if year is not passed.
Time.days_in_year
=> 366

# Gives number of days in specified year, if year is passed.
Time.days_in_year(2015)
=> 365

Improvements in Enumerable

pluck

pluck method is now added to Enumerable objects.

users = [{id: 1, name: 'Max'}, {id: 2, name: 'Mark'}, {id: 3, name: 'George'}]

users.pluck(:name)
=> ["Max", "Mark", "George"]

# Takes multiple arguments as well
users.pluck(:id, :name)
=> [[1, "Max"], [2, "Mark"], [3, "George"]]

one great improvement in ActiveRecord due to this method addition is that when relation is already loaded then instead of firing query with pluck, it uses Enumerable#pluck to get data.

# In Rails 4.x
users = User.all
SELECT `users`.* FROM `users`

users.pluck(:id, :name)
SELECT "users"."id", "users"."name" FROM "users"

=> [[2, "Max"], [3, "Mark"], [4, "George"]]


# In Rails 5
users = User.all
SELECT "users".* FROM "users"

# does not fire any query
users.pluck(:id, :name)
=> [[1, "Max"], [2, "Mark"], [3, "George"]]

without

This method returns a copy of enumerable without the elements passed to the method.

vehicles = ['Car', 'Bike', 'Truck', 'Bus']

vehicles.without("Car", "Bike")
=> ["Truck", "Bus"]

vehicles = {car: 'Hyundai', bike: 'Honda', bus: 'Mercedes', truck: 'Tata'}

vehicles.without(:bike, :bus)
=> {:car=>"Hyundai", :truck=>"Tata"}

Array#second_to_last and Array#third_to_last

['a', 'b', 'c', 'd', 'e'].second_to_last
=> "d"

['a', 'b', 'c', 'd', 'e'].third_to_last
=> "c"

PR for these methods can be found here.

Integer#positive? and Integer#negative?

positive? returns true if integer is positive.

negative? returns true if integer is negative.

4.positive?
=> true

4.negative?
=> false

-4.0.positive?
=> false

-4.0.negative?
=> true

Commit for these methods can be found here.

These changes have now been added to Ruby 2.3 also.

Array#inquiry

Rails team has added ArrayInquirer to ActiveSupport which gives a friendlier way to check its contents.

Array#inquiry is a shortcut for wrapping the receiving array in an ArrayInquirer

users = [:mark, :max, :david]

array_inquirer1 = ActiveSupport::ArrayInquirer.new(users)

# creates ArrayInquirer object which is same as array_inquirer1 above
array_inquirer2 = users.inquiry

array_inquirer2.class
=> ActiveSupport::ArrayInquirer

# provides methods like:

array_inquirer2.mark?
=> true

array_inquirer2.john?
=> false

array_inquirer2.any?(:john, :mark)
=> true

array_inquirer2.any?(:mark, :david)
=> true

array_inquirer2.any?(:john, :louis)
=> false

This blog is part of our Rails 5 series.

rails routes shows all the routes in the application.

$ rake routes

Prefix       Verb   URI Pattern                   Controller#Action
wishlist_user GET    /users/:id/wishlist(.:format) users#wishlist
        users GET    /users(.:format)              users#index
              POST   /users(.:format)              users#create
     new_user GET    /users/new(.:format)          users#new
    edit_user GET    /users/:id/edit(.:format)     users#edit
         user GET    /users/:id(.:format)          users#show
              PATCH  /users/:id(.:format)          users#update
              PUT    /users/:id(.:format)          users#update
              DELETE /users/:id(.:format)          users#destroy
     products GET    /products(.:format)           products#index
              POST   /products(.:format)           products#create
and so on ......

This list can be lengthy and it could be difficult to locate exactly what user is looking for.

Ways to search specific routes prior to Rails 5

To see only specific routes we can use commands like grep.

$ rake routes | grep products
Prefix       Verb   URI Pattern                   Controller#Action
products      GET    /products(.:format)           products#index
              POST   /products(.:format)           products#create

Options with Rails 5

Rails 5 has added options in rails routes to perform pattern matching on routes.

Controller specific search

Use option -c to search for routes related to controller. Also remember that Rails does case insensitive search. So rails routes -c users is same as rails routes -c Users.

# Search for Controller name
$ rails routes -c users
       Prefix Verb   URI Pattern                   Controller#Action
wishlist_user GET    /users/:id/wishlist(.:format) users#wishlist
        users GET    /users(.:format)              users#index
              POST   /users(.:format)              users#create

# Search for namespaced Controller name.
$ rails routes -c admin/users
         Prefix Verb   URI Pattern                     Controller#Action
    admin_users GET    /admin/users(.:format)          admin/users#index
                POST   /admin/users(.:format)          admin/users#create

# Search for namespaced Controller name.
$ rails routes -c Admin::UsersController
         Prefix Verb   URI Pattern                     Controller#Action
    admin_users GET    /admin/users(.:format)          admin/users#index
                POST   /admin/users(.:format)          admin/users#create

Pattern specific search

Use -g option to do general purpose pattern matching. This results in any routes that partially matches Prefix, Controller#Action or the URI pattern.

# Search with pattern
$ rails routes -g wishlist
       Prefix Verb URI Pattern                   Controller#Action
wishlist_user GET  /users/:id/wishlist(.:format) users#wishlist

# Search with HTTP Verb
$ rails routes -g POST
    Prefix Verb URI Pattern            Controller#Action
           POST /users(.:format)       users#create
           POST /admin/users(.:format) admin/users#create
           POST /products(.:format)    products#create

# Search with URI pattern
$ rails routes -g admin
       Prefix Verb   URI Pattern                     Controller#Action
  admin_users GET    /admin/users(.:format)          admin/users#index
              POST   /admin/users(.:format)          admin/users#create

Note that using CONTROLLER=some_controller has now been deprecated. This had the same effect as searching for a controller specific route.

This blog is part of our Rails 5 series.

In Rails 5, whenever we define a belongs_to association, it is required to have the associated record present by default after this change.

It triggers validation error if associated record is not present.

class User < ApplicationRecord
end

class Post < ApplicationRecord
  belongs_to :user
end

post = Post.create(title: 'Hi')
=> <Post id: nil, title: "Hi", user_id: nil, created_at: nil, updated_at: nil>

post.errors.full_messages.to_sentence
=> "User must exist"

As we can see, we can’t create any post record without having an associated user record.

How to achieve this behavior before Rails 5

In Rails 4.x world To add validation on belongs_to association, we need to add option required: true .

class User < ApplicationRecord
end

class Post < ApplicationRecord
  belongs_to :user, required: true
end

post = Post.create(title: 'Hi')
=> <Post id: nil, title: "Hi", user_id: nil, created_at: nil, updated_at: nil>

post.errors.full_messages.to_sentence
=> "User must exist"

By default, required option is set to false.

Opting out of this default behavior in Rails 5

We can pass optional: true to the belongs_to association which would remove this validation check.

class Post < ApplicationRecord
  belongs_to :user, optional: true
end

post = Post.create(title: 'Hi')
=> <Post id: 2, title: "Hi", user_id: nil>

But, what if we do not need this behavior anywhere in our entire application and not just a single model?

Opting out of this default behavior for the entire application

New Rails 5 application comes with an initializer named new_framework_defaults.rb.

When upgrading from older version of Rails to Rails 5, we can add this initializer by running bin/rails app:update task.

This initializer has config named Rails.application.config.active_record.belongs_to_required_by_default = true

For new Rails 5 application the value is set to true but for old applications, this is set to false by default.

We can turn off this behavior by keeping the value to false.

Rails.application.config.active_record.belongs_to_required_by_default = false

class Post < ApplicationRecord
  belongs_to :user
end

post = Post.create(title: 'Hi')
=> <Post id: 3, title: "Hi", user_id: nil, created_at: "2016-02-11 12:36:05", updated_at: "2016-02-11 12:36:05">

This blog is part of our Rails 5 series.

Before Rails 5, returning false from any before_ callback in ActiveModel or ActiveModel::Validations, ActiveRecord and ActiveSupport resulted in halting of callback chain.

class Order < ActiveRecord::Base

  before_save :set_eligibility_for_rebate
  before_save :ensure_credit_card_is_on_file

  def set_eligibility_for_rebate
    self.eligibility_for_rebate ||= false
  end

  def ensure_credit_card_is_on_file
    puts "check if credit card is on file"
  end
end

Order.create!
=> ActiveRecord::RecordNotSaved: ActiveRecord::RecordNotSaved

In this case the code is attempting to set the value of eligibility_for_rebate to false. However the side effect of the way Rails callbacks work is that the callback chain will be halted simply because one of the callbacks returned false.

Right now, to fix this we need to return true from before_ callbacks, so that callbacks are not halted.

Improvements in Rails 5

Rails 5 fixed this issue by adding throw(:abort) to explicitly halt callbacks.

Now, if any before_ callback returns false then callback chain is not halted.

class Order < ActiveRecord::Base

  before_save :set_eligibility_for_rebate
  before_save :ensure_credit_card_is_on_file

  def set_eligibility_for_rebate
    self.eligibility_for_rebate ||= false
  end

  def ensure_credit_card_is_on_file
    puts "check if credit card is on file"
  end

end

Order.create!
=> check if credit card is on file
=> <Order id: 4, eligibility_for_rebate: false>

To explicitly halt the callback chain, we need to use throw(:abort).

class Order < ActiveRecord::Base

  before_save :set_eligibility_for_rebate
  before_save :ensure_credit_card_is_on_file

  def set_eligibility_for_rebate
    self.eligibility_for_rebate ||= false
    throw(:abort)
  end

  def ensure_credit_card_is_on_file
    puts "check if credit card is on file"
  end

end

Order.create!
=> ActiveRecord::RecordNotSaved: Failed to save the record

Opting out of this behavior

The new Rails 5 application comes up with initializer named callback_terminator.rb.

ActiveSupport.halt_callback_chains_on_return_false = false

By default the value is to set to false.

We can turn off this default behavior by changing this configuration to true. However then Rails shows deprecation warning when false is returned from callback.

ActiveSupport.halt_callback_chains_on_return_false = true

class Order < ApplicationRecord
  before_save :set_eligibility_for_rebate
  before_save :ensure_credit_card_is_on_file

  def set_eligibility_for_rebate
    self.eligibility_for_rebate ||= false
  end

  def ensure_credit_card_is_on_file
    puts "check if credit card is on file"
  end
end

=> DEPRECATION WARNING: Returning `false` in Active Record and Active Model callbacks will not implicitly halt a callback chain in the next release of Rails. To explicitly halt the callback chain, please use `throw :abort` instead.
ActiveRecord::RecordNotSaved: Failed to save the record

How older applications will work with this change?

The initializer configuration will be present only in newly generated Rails 5 apps.

If you are upgrading from an older version of Rails, you can add this initializer yourself to enable this change for entire application.

This is a welcome change in Rails 5 which will help prevent accidental halting of the callbacks.

Bundler helps in managing gem dependencies of ruby projects. You can specify which gems and versions you need, bundler will install them and load them at runtime. Bundler ensures that gems you need are present in the environment you need.

Bundle configurations

Bundler gets its configurations from local application (app/.bundle/config), environment variables and user’s home directory (~/.bundle/config) in the order of priority.

To list all bundler configurations for the current bundle, run bundle config without any parameters. You will also get the location where the value is set.

$ bundle config
Settings are listed in order of priority. The top value will be used.

You might see different result based on configuration of bundler on your machine.

To get value for the specific configuration setting, run bundle config with name.

$ bundle config disable_multisource
Settings for `disable_multisource` in order of priority. The top value will be used
You have not configured a value for `disable_multistore`

Setting Configuration

To set the value of the configuration setting, use bundle config with name and value. Configuration will be stored in ~/.bundle/config.

$ bundle config build.pg --with-pg-config=/opt/local/lib/postgresql91/bin/pg_config

$ bundle config
Settings are listed in order of priority. The top value will be used.
build.pg
Set for the current user (/Users/username/.bundle/config): "--with-pg-config=/opt/local/lib/postgresql91/bin/pg_config"

If any config already has a value, it will be overwritten directly and user will be warned.

$ bundle config build.pg --with-pg-config=/usr/pgsql-9.1/bin/pg_config
You are replacing the current global value of build.pg, which is currently "--with-pg-config=/opt/local/lib/postgresql91/bin/pg_config"

Application level configuration

By default setting configuration value will set it for all projects on the machine. You can set configurations specific to local application with --local option. This will store value in app/.bundle/config.

$ bundle config --local auto_install false
You are replacing the current local value of auto_install, which is currently nil

$ bundle config auto_install
Settings for `auto_install` in order of priority. The top value will be used
Set for your local app (/Users/username/Documents/Workspace/app-name/.bundle/config): "false"
Set for the current user (/Users/username/.bundle/config): "true"

You can run bundle config with --global. This will set values at global level i.e. across all applications on the machine. It will be similar to running bundle config without any options.

Deleting configuration

You can delete configuration with --delete option.

$ bundle config --delete auto_install
$ bundle config
Settings are listed in order of priority. The top value will be used.
disable_multisource
Set for the current user (/Users/username/.bundle/config): "true"

build.pg
Set for the current user (/Users/username/.bundle/config): "--with-pg-config=/usr/pgsql-9.1/bin/pg_config"

This is not compatible with --local and --global. This will delete configuration from local and global resources.

Build Options

You can pass the flags required for installing particular gem to bundler with bundle config.

Many El Capitan users face an issue while installing eventmachine gem. The issue can be resolved by providing path to OpenSSL include directory while installing eventmachine.

$ gem install eventmachine -v '1.0.8' -- --with-cppflags=-I/usr/local/opt/openssl/include

As location of configuration will vary from machine to machine, we can set this with bundle config.

$ bundle config build.eventmachine --with-cppflags=-I/usr/local/opt/openssl/include

Now bundler will pick this configuration while installing eventmachine gem.

Configuration keys

Various configuration keys are available with bundler. These keys are available in two forms. You can specify them in canonical form with bundle config or set them in environment variable form. Following are canonical forms and their usage. Corresponding environment variables are specified in bracket.

auto_install

Setting auto_install config will enable automatic installing of gems instead of raising an error. This applies to show, binstubs, outdated, exec, open, console, license, clean commands.

Example, When you try to run bundle show for the gem which is not yet installed you will get an error.

$ bundle show pg
Could not find gem 'pg'.

You can set auto_install to remove this error and install gem.

$ bundle config auto_install true

$ bundle show pg
# Gem will get installed
/Users/username/.rvm/gems/ruby-2.2.2@gemset-auto/gems/pg-0.17.1

path (BUNDLE_PATH)

You can specify location to install your gems. Default path is $GEM_HOME for development and vendor/bundle when –deployment is used.

$ bundle config path NEW_PATH
$ bundle install
# Gems will get installed
#
#
Bundle complete! 4 Gemfile dependencies, 35 gems now installed.
Bundled gems are installed into NEW_PATH.

frozen (BUNDLE_FROZEN)

You can freeze changes to your Gemfile.

$ bundle config frozen true

If frozen is set and you try to run bundle install with changed Gemfile, you will get following warning.

You are trying to install in deployment mode after changing
your Gemfile. Run `bundle install` elsewhere and add the
updated Gemfile.lock to version control.

If this is a development machine, remove the /Users/username/Documents/Workspace/app-name/Gemfile freeze
by running `bundle install --no-deployment`.

You have added to the Gemfile:
* minitest-reporters

without (BUNDLE_WITHOUT)

You can skip installing groups of gems with bundle install. Specify : separated group names whose gems bundler should not install.

$ bundle config --local without development:test

$ bundle install
# This will install gems skipping development and test group gems.

bin (BUNDLE_BIN)

You can set the directory to install executables from gems in the bundle.

$ bundle config bin NEW_PATH

$ bundle install
# This will install executables in NEW_PATH

gemfile (BUNDLE_GEMFILE)

You can set the file which bundler should use as theGemfile. By default, bundler will use Gemfile. The location of this file also sets the root of the project, which is used to resolve relative paths in the Gemfile.

$ bundle config gemfile Gemfile-rails4

$ bundle install
# This will install gems from Gemfile-rails4 file.

ssl_ca_cert (BUNDLE_SSL_CA_CERT)

This specifies path to a designated CA certificate file or folder containing multiple certificates for trusted CAs in PEM format. You can specify your own https sources in Gemfile with corresponding certificates specified via bundle config.

$ bundle config ssl_ca_cert NEW_CERTIFICATE_PATH

ssl_client_cert (BUNDLE_SSL_CLIENT_CERT)

This specifies path to a designated file containing a X.509 client certificate and key in PEM format.

$ bundle config ssl_client_cert NEW_CERTIFICATE_PATH

cache_path (BUNDLE_CACHE_PATH)

You can set the path to place cached gems while running bundle package.

$ bundle config cache_path vendor/new-cache-path

$ bundle package
Using colorize 0.7.7
Using pg 0.17.1
Using bundler 1.11.2
Updating files in vendor/new-cache-path
  * colorize-0.7.7.gem
  * pg-0.17.1.gem
Bundle complete! 2 Gemfile dependencies, 3 gems now installed.
Use `bundle show [gemname]` to see where a bundled gem is installed.
Updating files in vendor/new-cache-path

disable_multisource (BUNDLE_DISABLE_MULTISOURCE)

When set, Gemfiles containing multiple sources will produce an error instead of a warning.

With Gemfile,

source 'https://rubygems.org'
source 'http://gems.github.com'

ruby '2.2.2'

When you try to run bundle install, you will get warning.

$ bundle install
Warning: this Gemfile contains multiple primary sources. Using `source` more than once without a block is a security risk, and may result in installing unexpected gems. To resolve this warning, use a block to indicate which gems should come from the secondary source. To upgrade this warning to an error, run `bundle config disable_multisource true`.

$ bundle config --local disable_multisource true

$ bundle install
[!] There was an error parsing `Gemfile`: Warning: this Gemfile contains multiple primary sources. Each source after the first must include a block to indicate which gems should come from that source. To downgrade this error to a warning, run `bundle config --delete disable_multisource`. Bundler cannot continue.

 #  from /Users/username/Documents/Workspace//Gemfile:2
 #  -------------------------------------------
 #  source 'https://rubygems.org'
 >  source 'http://gems.github.com'
 #
 #  -------------------------------------------

To ignore all bundle config on the machine and run bundle install, set BUNDLE_IGNORE_CONFIG environment variable.

In this blog, we will see how to plot a simple line chart using ReactJS and D3JS.

If you are not familiar with ReactJS then please take a look at official ReactJS webpage. You can also look at our Learn ReactJS in steps video series.

What is D3.js

D3.js is a Javascript library used to create interactive, dynamic visualizations.

Let’s take a step by step look at how we can integrate ReactJS with D3JS to plot some interactive visualizations.

Step 1 - Get ReactJS example working

We will be using JSFiddle example from ReactJS Docs to begin with. Fork the JSFiddle example and you should be good to go.

Step 2 - Add D3.js as an external resource

We will be using D3.js from Cloudflare CDN. Add D3.js as an external resource as shown in the image given below and type the following URL as an external resource.

https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.12/d3.js

Step 3 - Build ReactJS components to create visualizations with D3.js

Now let’s try to draw a Line Chart using D3.js.

Let’s create a Line component that renders line path for the data points provided.

const Line = React.createClass({

  propTypes: {
    path:         React.PropTypes.string.isRequired,
    stroke:       React.PropTypes.string,
    fill:         React.PropTypes.string,
    strokeWidth:  React.PropTypes.number
  },

  getDefaultProps() {
    return {
      stroke:       'blue',
      fill:         'none',
      strokeWidth:  3
    };
  },

  render() {
    let { path, stroke, fill, strokeWidth } = this.props;
    return (
      <path
        d={path}
        fill={fill}
        stroke={stroke}
        strokeWidth={strokeWidth}
        />
    );
  }

});

Here in above code, Line component renders an SVG path. Path data d is generated using D3 path functions.

Let’s create another component DataSeries that will render Line component for each series of data provided. This generates path based on xScale and yScale generated for plotting a line chart.

const DataSeries = React.createClass({

  propTypes: {
    colors:             React.PropTypes.func,
    data:               React.PropTypes.object,
    interpolationType:  React.PropTypes.string,
    xScale:             React.PropTypes.func,
    yScale:             React.PropTypes.func
  },

  getDefaultProps() {
    return {
      data:               [],
      interpolationType:  'cardinal',
      colors:             d3.scale.category10()
    };
  },

  render() {
    let { data, colors, xScale, yScale, interpolationType } = this.props;

    let line = d3.svg.line()
      .interpolate(interpolationType)
      .x((d) => { return xScale(d.x); })
      .y((d) => { return yScale(d.y); });

    let lines = data.points.map((series, id) => {
      return (
        <Line
          path={line(series)}
          stroke={colors(id)}
          key={id}
          />
      );
    });

    return (
      <g>
        <g>{lines}</g>
      </g>
    );
  }

});

Here in above code d3.svg.line creates a new line generator which expects input as a two-element array of numbers.

Now we will create LineChart component that will calculate xScale, yScale based on data and will render DataSeries by passing xScale, yScale, data (input x,y values), width, height for the chart.

const LineChart = React.createClass({

  propTypes: {
    width:  React.PropTypes.number,
    height: React.PropTypes.number,
    data:   React.PropTypes.object.isRequired
  },

  getDefaultProps(){
    return {
      width:  600,
      height: 300
    }
  },

  render() {
    let { width, height, data } = this.props;

    let xScale = d3.scale.ordinal()
                   .domain(data.xValues)
                   .rangePoints([0, width]);

    let yScale = d3.scale.linear()
                   .range([height, 10])
                   .domain([data.yMin, data.yMax]);

    return (
      <svg width={width} height={height}>
          <DataSeries
            xScale={xScale}
            yScale={yScale}
            data={data}
            width={width}
            height={height}
            />
      </svg>
    );
  }

});

Here d3.scale.ordinal constructs an ordinal scale that can have discrete domain while d3.scale.linear constructs a linear quantitative scale.

You can learn more about D3 Quantitative scales here.

Now we need to call LineDataSeries component with the data.

let data = {
  points: [
    [ { x: 0, y: 20 }, { x: 1, y: 30 }, { x: 2, y: 10 }, { x: 3, y: 5 },
      { x: 4, y: 8 }, { x: 5, y: 15 }, { x: 6, y: 10 } ]
    ,
    [ { x: 0, y: 8 }, { x: 1, y: 5 }, { x: 2, y: 20 }, { x: 3, y: 12 },
      { x: 4, y: 4 }, { x: 5, y: 6 }, { x: 6, y: 2 } ]
    ,
    [ { x: 0, y: 0 }, { x: 1, y: 5 }, { x: 2, y: 8 }, { x: 3, y: 2 },
      { x: 4, y: 6 }, { x: 5, y: 4 }, { x: 6, y: 2 } ]
    ],
  xValues: [0,1,2,3,4,5,6],
  yMin: 0,
  yMax: 30
};

ReactDOM.render(
  <LineChart
    data={data}
    width={600}
    height={300}
    />,
  document.getElementById('container')
);

An element with id container is replaced with content rendered by LineChart.

If we take a look at the output now, we see how the Line Chart gets plotted.

To build complex visualizations in a modularized fashion, we can use one of the open source libraries mentioned below based on their advantages and disadvantages.

ReactJS + D3.js Open Source Projects

Here are two popular open source ReactJS + D3.JS projects.

react-d3

Pros

• Supports Bar chart, Line chart, Area chart, Pie chart, Candlestick chart, Scattered chart and Treemap.
• Legend support.
• Tooltips support.

Cons

• No support for Animations. You can implement animations using D3 Transitions.
• Only stacked Bar chart support.

react-d3-components

Pros

• Custom scales support.
• Supports Bar chart (Stacked, Grouped), Line chart, Area chart, Pie chart, Scattered chart.
• Tooltips support.

Cons

• No Legend support.
• No support for Animations.

Summary

Below is final working example of JSFiddle built in the post.

This blog is part of our Rails 5 series.

Often while developing a Rails application you may look to have one of these caching techniques to boost the performance. Along with these, Rails 5 now provides a way of caching a collection of records, thanks to the introduction of the following method:

ActiveRecord::Relation#cache_key

What is collection caching?

Consider the following example where we are fetching a collection of all users belonging to city of Miami.

@users = User.where(city: 'miami')

Here @users is a collection of records and is an object of class ActiveRecord::Relation.

Whether the result of the above query would be same depends on following conditions.

• The query statement doesn’t change. If we change city name from “Miami” to “Boston” then result might change.
• No record is deleted. The count of records in the collection should be same.
• No record is added. The count of records in the collection should be same.

Rails community implemented caching for a collection of records . Method cache_key was added to ActiveRecord::Relation which takes into account many factors including query statement, updated_at column value and the count of the records in collection.

Understanding ActiveRecord::Relation#cache_key

We have object @users of class ActiveRecord::Relation. Now let’s execute cache_key method on it.

 @users.cache_key
 => "users/query-67ed32b36805c4b1ec1948b4eef8d58f-3-20160116111659084027"

Let’s try to understand each piece of the output.

users represents what kind of records we are holding. In this example we have collection of records of class User. Hence users is to illustrate that we are holding users records.

query- is hardcoded value and it will be same in all cases.

67ed32b36805c4b1ec1948b4eef8d58f is a digest of the query statement that will be executed. In our example it is MD5( "SELECT "users".* FROM "users" WHERE "users"."city" = 'Miami'")

3 is the size of collection.

20160116111659084027 is timestamp of the most recently updated record in the collection. By default, the timestamp column considered is updated_at and hence the value will be the most recent updated_at value in the collection.

Using ActiveRecord::Relation#cache_key

Let’s see how to use cache_key to actually cache data.

In our Rails application, if we want to cache records of users belonging to “Miami” then we can take following approach.

# app/controllers/users_controller.rb

class UsersController < ApplicationController

  def index
    @users = User.where(city: 'Miami')
  end
end

# users/index.html.erb

<% cache(@users) do %>
  <% @users.each do |user| %>
    <p> <%= user.city %> </p>
  <% end %>
<% end %>

# 1st Hit
Processing by UsersController#index as HTML
  Rendering users/index.html.erb within layouts/application
   (0.2ms)  SELECT COUNT(*) AS "size", MAX("users"."updated_at") AS timestamp FROM "users" WHERE "users"."city" = ?  [["city", "Miami"]]
Read fragment views/users/query-37a3d8c65b3f0f9ece7f66edcdcb10ab-4-20160704131424063322/30033e62b28c83f26351dc4ccd6c8451 (0.0ms)
  User Load (0.1ms)  SELECT "users".* FROM "users" WHERE "users"."city" = ?  [["city", "Miami"]]
Write fragment views/users/query-37a3d8c65b3f0f9ece7f66edcdcb10ab-4-20160704131424063322/30033e62b28c83f26351dc4ccd6c8451 (0.0ms)
Rendered users/index.html.erb within layouts/application (3.7ms)

# 2nd Hit
Processing by UsersController#index as HTML
  Rendering users/index.html.erb within layouts/application
   (0.2ms)  SELECT COUNT(*) AS "size", MAX("users"."updated_at") AS timestamp FROM "users" WHERE "users"."city" = ?  [["city", "Miami"]]
Read fragment views/users/query-37a3d8c65b3f0f9ece7f66edcdcb10ab-4-20160704131424063322/30033e62b28c83f26351dc4ccd6c8451 (0.0ms)
  Rendered users/index.html.erb within layouts/application (3.0ms)

From above, we can see that for the first hit, a count query is fired to get the the latest updated_at and size from the users collection.

Rails will write a new cache entry with a cache_key generated from above count query.

Now on second hit, it again fires count query and checks if cache_key for this query exists or not.

If cache_key is found, it loads data without firing SQL query.

What if your table doesn’t have updated_at column?

Previously we mentioned that cache_key method uses updated_at column. cache_key also provides an option of passing custom column as a parameter and then the highest value of that column among the records in the collection will be considered.

For example if your business logic considers a column named last_bought_at in products table as a factor to decide caching, then you can use the following code.

 products = Product.where(category: 'cars')
 products.cache_key(:last_bought_at)
 => "products/query-211ae6b96ec456b8d7a24ad5fa2f8ad4-4-20160118080134697603"

Edge cases to watch out for

Before you start using cache_key there are some edge cases to watch out for.

Consider you have an application where there are 5 entries in users table with city Miami.

Using limit puts incorrect size in cache key if collection is not loaded.

If you want to fetch three users belonging to city “Miami” then you would execute following query.

 users = User.where(city: 'Miami').limit(3)
 users.cache_key
 => "users/query-67ed32b36805c4b1ec1948b4eef8d58f-3-20160116144936949365"

Here users contains only three records and hence the cache_key has 3 for size of collection.

Now let’s try to execute same query without fetching the records first.

 User.where(name: 'Sam').limit(3).cache_key
 => "users/query-8dc512b1408302d7a51cf1177e478463-5-20160116144936949365"

You can see that the count in the cache is 5 this time even though we have set a limit to 3. This is because the implementation of ActiveRecord::Base#collection_cache_key executes query without limit to fetch the size of the collection.

Cache key doesn’t change when an existing record from a collection is replaced

I want 3 users in the descending order of ids.

 users1 = User.where(city: 'Miami').order('id desc').limit(3)
 users1.cache_key
 => "users/query-57ee9977bb0b04c84711702600aaa24b-3-20160116144936949365"

Above statement will give us users with ids [5, 4, 3].

Now let’s remove the user with id = 3.

 User.find(3).destroy

 users2 = User.where(first_name: 'Sam').order('id desc').limit(3)
 users2.cache_key
 => "users/query-57ee9977bb0b04c84711702600aaa24b-3-20160116144936949365"

Note that cache_key both users1 and users2 is exactly same. This is because none of the parameters that affect the cache key is changed i.e., neither the number of records, nor the query statement, nor the timestamp of the latest record.

There is a discussion undergoing about adding ids of the collection records as part of the cache key. This might help solve the problems discussed above.

Using group query gives incorrect size in the cache key

Just like limit case discussed above cache_key behaves differently when data is loaded and when data is not loaded in memory.

Let’s say that we have two users with first_name “Sam”.

First let’s see a case where collection is not loaded in memory.

 User.select(:first_name).group(:first_name).cache_key
 => "users/query-92270644d1ec90f5962523ed8dd7a795-1-20160118080134697603"

In the above case, the size is 1 in cache_key. For the system mentioned above, the sizes that you will get shall either be 1 or 5. That is, it is size of an arbitrary group.

Now let’s see when collection is first loaded.

 users = User.select(:first_name).group(:first_name)
 users.cache_key
 => "users/query-92270644d1ec90f5962523ed8dd7a795-2-20160118080134697603"

In the above case, the size is 2 in cache_key. You can see that the count in the cache key here is different compared to that where the collection was unloaded even though the query output in both the cases will be exactly same.

In case where the collection is loaded, the size that you get is equal to the total number of groups. So irrespective of what the records in each group are, we may have possibility of having the same cache key value.

This blog is part of our Rails 5 series.

In Rails 4 if I’m doing work related to caching then first I need to turn caching “on” by opening file config/environments/development.rb and changing following line.

config.action_controller.perform_caching = false

After changing the value from false to true, I need to restart the server.

This means that if I am testing caching behavior locally then every time I turn caching “on” or “off” I need to restart the server.

New command to create development cache in Rails 5

Rails 5 has introduced a new command to create development cache and help us test how caching behaves in development mode. Here is the issue and here is the pull request.

$ rails dev:cache
Development mode is now being cached.

Execution of the above command creates file caching-dev.txt in tmp directory.

How does it work?

In Rails 5 when a brand new Rails app is created then config/environments/development.rb file will have the following snippet of code.

if Rails.root.join('tmp/caching-dev.txt').exist?
  config.action_controller.perform_caching = true
  config.static_cache_control = "public, max-age=172800"
  config.cache_store = :mem_cache_store
else
  config.action_controller.perform_caching = false
  config.cache_store = :null_store
end

In the above code we are checking if the file tmp/caching-dev.txt is present and then use :mem_cache_store to enable caching only if the file is found.

Also, here is a snippet from the dev cache source code.

def dev_cache
  if File.exist? 'tmp/caching-dev.txt'
    File.delete 'tmp/caching-dev.txt'
    puts 'Development mode is no longer being cached.'
  else
    FileUtils.touch 'tmp/caching-dev.txt'
    puts 'Development mode is now being cached.'
  end

  FileUtils.touch 'tmp/restart.txt'
end

What is the advantage

The advantage is that we do not need to restart the server manually if we want to turn caching “on” or “off”. It is internally taken care by the dev_cache method that is executed when rails dev:cache is executed. You can see in the source code that tmp/restart.txt is being touched.

Please note that this feature is not supported by unicorn, thin and webrick. My guess is that DHH wants this feature because his team uses pow and pow restarts when tmp/restart.txt is touched. He also created an issue for spring to watch tmp/restart.txt long time back.

Disabling development cache

Execute the same command that was used to enable caching. If caching was previously enabled then it will be turned “off” now.

$ rails dev:cache
Development mode is no longer being cached.

By default PostgreSQL is configured to be bound to “localhost”.

$ netstat -nlt
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:11211         0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:5432          0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:3737          0.0.0.0:*               LISTEN
tcp6       0      0 :::22                   :::*                    LISTEN

As we can see above port 5432 is bound to 127.0.0.1. It means any attempt to connect to the postgresql server from outside the machine will be refused. We can try hitting the port 5432 by using telnet.

$ telnet 107.170.11.79 5432
Trying 107.170.11.79...
telnet: connect to address 107.170.11.79: Connection refused
telnet: Unable to connect to remote host

Configuring postgresql.conf

In order to fix this issue we need to find postgresql.conf. In different systems it is located at different place. I usually search for it.

$ find \ -name "postgresql.conf"
/var/lib/pgsql/9.4/data/postgresql.conf

Open postgresql.conf file and replace line

listen_addresses = 'localhost'

with

listen_addresses = '*'

Now restart postgresql server.

$ netstat -nlt
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 127.0.0.1:11211         0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:5432            0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:2812          0.0.0.0:*               LISTEN
tcp6       0      0 ::1:11211               :::*                    LISTEN
tcp6       0      0 :::22                   :::*                    LISTEN
tcp6       0      0 :::5432                 :::*                    LISTEN
tcp6       0      0 ::1:25                  :::*                    LISTEN

Here we can see that “Local Address” for port 5432 has changed to 0.0.0.0.

Configuring pg_hba.conf

Let’s try to connect to remote postgresql server using “psql”.

$ psql -h 107.170.158.89 -U postgres
psql: could not connect to server: Connection refused
	Is the server running on host "107.170.158.89" and accepting
	TCP/IP connections on port 5432?

In order to fix it, open pg_hba.conf and add following entry at the very end.

host    all             all              0.0.0.0/0                       md5
host    all             all              ::/0                            md5

The second entry is for IPv6 network.

Do not get confused by “md5” option mentioned above. All it means is that a password needs to be provided. If you want client to allow collection without providing any password then change “md5” to “trust” and that will allow connection unconditionally.

Restart postgresql server.

$ psql -h 107.170.158.89 -U postgres
Password for user postgres:
psql (9.4.1, server 9.4.5)
Type "help" for help.

postgres=# \l

You should be able to see list of databases.

Now we are able to connect to postgresql server remotely.

Please note that in the real world you should be using extra layer of security by using “iptables”.

This blog is part of our Rails 5 series.

In Rails 4 some commands start with rails and some commands start with rake. This could be quite confusing for people new to Rails. Let’s see an example.

Our task is to write a database migration and then to run that migration.

rails g migration create_users

Above command creates a migration. Now we need to run that migration.

rake db:migrate

As you can see first command starts with rails and second command starts with rake.

In order to consolidate them we can either use rails for everything or we can use rake for everything.

Choosing rails command over rake command

Some favor using rake over rails. But an important feature missing in Rake is the ability to pass arguments.

rails console development

In order to execute the above command using rake we will have to pass console and development as arguments. We can pass these values using environment variables. That would mean adding additional code in Rake task to fetch right values and then only we will be able to invoke command rails console development.

Rails 5 enables executing rake commands with rails

Rails core team decided to have consistency by enabling rails command to support everything that rake does.

For example in Rails 5 commands like db:migrate, setup, test etc which are part of rake command in Rails 4 are now being supported by rails command. However you can still choose to use rake to run those commands similar to how they were run in Rails 4. This is because Rails community has introduced Rake Proxy instead of completely moving the command options from rake to rails.

What happens internally is that when rails db:migrate command is executed, Rails checks if db:migrate is something that rails natively supports or not. In this case db:migrate is not natively supported by rails, so Rails delegates the execution to Rake via Rake Proxy.

If you want to see all the commands that is supported by rails in Rails 5 then you can get a long list of options by executing rails --help.

Use app namespace for framework tasks in Rails 5

As rails command is now preferred over rake command, few rails namespaced framework tasks started looking little odd.

$ rails rails:update

$ rails rails:template

$ rails rails:templates:copy

$ rails rails:update:configs

$ rails rails:update:bin

So, Rails team decided to change the namespace for these tasks from rails to app.

$ rails app:update

$ rails app:template

$ rails app:templates:copy

$ rails app:update:configs

$ rails app:update:bin

Using rails rails:update will now give deprecation warning like: DEPRECATION WARNING: Running update with the rails: namespace is deprecated in favor of app: namespace. Run bin/rails app:update instead.

More improvements in pipeline

In Rails 4, the routes are usually searched like this.

$ rake routes | grep pattern

There is an effort underway to have a Rails command which might work as shown below.

$ rails routes -g pattern

There is also an effort to enable lookup for controller like this.

$ rails routes -c some_controller

This blog is part of our Rails 5 series.

We have written an extensive blog on what CSRF is and what steps Rails 4 takes to prevent CSRF. We encourage you to read that blog to fully understand rest of this article.

Nested form can get around CSRF protection offered by Rails 4

A typical form generated in Rails 4 might look like this.

<form method= "post" action="/money_transfer">
  <input type="hidden" name="authenticity_token" value="token_value">
</form>

Using code-injection, a Hacker can add another form tag above the form tag generated by Rails using JavaScript. Now the markup looks like this.

<form method="post" action="http://www.fraud.com/fraud">
  <form method= "post" action="/money_transfer">
    <input type="hidden" name="authenticity_token" value="token_value">
  </form>
</form>

HTML specification does not allow nested forms.

Since nested forms are not allowed browser will accept the top most level form. In this case that happens to be the form created by the hacker. When this form is submitted then “authenticity_token” is also submitted and Rails will do its check and will say everything is looking good and thus hacker will be able to hack the site.

Rails 5 fixes the issue by generating a custom token for a form

In Rails 5, CSRF token can be added for each form. Each CSRF token will be valid only for the method/action of the form it was included in.

You can add following line to your controller to add authenticity token specific to method and action in each form tag of the controller.

class UsersController < ApplicationController
  self.per_form_csrf_tokens = true
end

Adding that code to each controller feels burdensome. In that case you can enable this behavior for all controllers in the application by adding following line to an initializer.

# config/application.rb
Rails.configuration.action_controller.per_form_csrf_tokens = true

This will add authenticity token specific to method and action in each form tag of the application. After adding that token the generated form might look like as shown below.

<form method= "post" action="/money_transfer">
  <input type="hidden" name="authenticity_token" value="money_transfer_post_action_token">
</form>

Authenticity token included here will be specific to action money_transfer and method post. Attacker can still grab authenticity_token here, but attack will be limited to money_transfer post action.

This blog is part of our Rails 5 series.

Rails request-response cycle is very easy to understand. A request hits the app, a route is matched to a controller action from routes.rb, and finally controller action processes the request and renders HTML or JSON based on the type of the request.

But sometimes we want to render our HTML or JSON response outside of this request-response cycle.

For example let’s say user is allowed to download PDF version of a report on web. This can be done using request-response cycle. We also need to send a weekly report to managers and the email should have the report as an attachment. Now we need to generate the same PDF but since emails are sent using background job the request-response cycle is missing.

Rails 5 has this feature baked in.

Let’s say we have a OrdersController and we want to render individual order outside of controller.

Fire up rails console and execute following command.

OrdersController.render :show, assigns: { order: Order.last }

This will render app/views/orders/show.html.erb with @order set to Order.last. Instance variables can be set using assigns in the same way we use them in controller actions. Those instance variables will be passed to the view that is going to be rendered.

Rendering partials is also possible.

OrdersController.render :_form, locals: { order: Order.last }

This will render app/views/orders/_form.html.erb and will pass order as local variable.

Say I want to render all orders, but in JSON format.

OrdersController.render json: Order.all
# => "[{"id":1, "name":"The Well-Grounded Rubyist", "author":"David A. Black"},
       {"id":2, "name":"Remote: Office not required", "author":"David & Jason"}]

Even rendering simple text is possible.

>> BooksController.render plain: 'this is awesome!'
  Rendered text template (0.0ms)
# => "this is awesome!"

Similar to text, we can also use render file and render template.

Request environment

A typical web request carries it’s own environment with it. We usually handle this environment using request.env in controllers. Certain gems like devise depends on env hash for information such as warden token.

So when we are rendering outside of controller, we need to make sure that the rendering happens with correct environment.

Rails provides a default rack environment for this purpose. The default options used can be accessed through renderer.defaults.

>> OrdersController.renderer.defaults
=> {:http_host=>"example.org", :https=>false, :method=>"get", :script_name=>"", :input=>""}

Internally, Rails will build a new Rack environment based on these options.

Customizing the environment

We can customize environment using method renderer. Let’s say that we need “method as post” and we want “https to be true” for our background job processing.

renderer = ApplicationController.renderer.new(method: 'post', https: true)
# => #<ActionController::Renderer:0x007fdf34453f10 @controller=ApplicationController, @defaults={:http_host=>"example.org", :https=>false, :method=>"get", :script_name=>"", :input=>""}, @env={"HTTP_HOST"=>"example.org", "HTTPS"=>"on", "REQUEST_METHOD"=>"POST", "SCRIPT_NAME"=>"", "rack.input"=>""}>

Now that we have our custom renderer we can use it to generate view.

renderer.render template: 'show', locals: { order: Order.last }

Overall this is a nice feature which enables reuse of existing code.

Recently we came across following error when mobile app was trying to upload images using backend API.

==> nginx-error.log <==

2015/12/16 07:34:13 [error] 1440#0: *1021901
sendfile() failed (32: Broken pipe) while sending request to upstream,
client: xxx.xx.xxx.xx, server: app-name.local,
request: "POST /api_endpoint HTTP/1.1",
upstream: "http://unix:/data/apps/app-name/shared/tmp/sockets/unicorn.sock:/api_endpoint",
host: "appname.com"

==> nginx-error.log <==

xxx.xx.xxx.xx - - [16/Dec/2015:07:34:13 - 0500]
"POST /api_endpoint HTTP/1.1" 502 1051 "-"
"Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Mobile/13B143 (366691552)"

The backend API was developed using Ruby on Rails and was powered using NGINX HTTP server along with unicorn application server.

Notice the the http response received was 502 which is documented in RFC as shown below.

10.5.3 502 Bad Gateway The server, while acting as a gateway or proxy, received an invalid response from the upstream server it accessed in attempting to fulfill the request.

Debugging the problem

From the first look it seemed that the client_max_body_size parameter for nginx was too less, which might cause the request to fail if the uploaded filesize is greater than the allowed limit.

nginx.conf
http {
  ...
  client_max_body_size 250M;
  ...
}

But that was not the case with our NGINX configuration. The NGINX was configured to accept file size upto 250 megabytes, while the file being uploaded was around 6 megabytes.

In our Rails code we are forcing all traffic to use HTTPS by having production.rb to have following setting.

config.force_ssl = true

What this setting does is, it forces HTTPS by redirecting HTTP requests to their HTTPS counterparts. So any request accessing http://domain.com/path will be redirected to https://domain.com/path

Forcing all traffic to HTTPS still does not explain the vague error sendfile() failed (32: Broken pipe) while sending request to upstream and why the HTTP 502 error which means Bad Gateway

Here is how our nginx.conf is configured.

server {
  ...
  listen 80;
  listen 443 ssl;
  ...
}

NGINX is configured to accept both HTTP and HTTPS requests. This is a pretty standard setting so that application can support both HTTP and HTTPS. What it means is that nginx will accept both HTTP and HTTPS request. It would not force HTTP request to be HTTPS. It would forward HTTP request to rails backend api.

Here is what happens when you upload a file

When we upload a file then NGINX takes that request and passes that request to Rails. NGINX expects a response from Rails only when NGINX is done sending the whole request.

Let’s see what happens when a user visits login page using HTTP. NGINX takes the request and passes the request to Rails. When NGINX is done handing over the request to Rails then NGINX expects a response from Rails. However in this case rather than sending 200 Rails sends a redirect over HTTPS. So now NGINX takes up the request again over HTTPS and then hands over request to Rails. Rails processes the request and returns 200. Everything works out.

Now let’s see what happens when a file is uploaded over HTTP.

User uploads a file over HTTP. NGINX takes up the request and starts sending the file to Rails. More details about the file upload process can be see at RFC. The data is sent to server in chunks. When first chunk is sent to server then server notices that data is being sent over HTTP and it immediately sends a response that data should be sent over HTTPS.

In the meantime on the client side , client is still pushing rest of the data to the server. Client expects a response from the server only when it is done pushing all the data. However in this case server sends a response when client is not expecting it.

NGINX at this time is all confused and thinks something has gone wrong with the gateway and returns HTTP/1.1 502 Bad Gateway and aborts the upload operation. And that how we get the 502 error.

So next time you see an error like this make sure that you are using https to upload the file if server is enforcing https.

Year 2015 was an exciting year for BigBinary !

We added more clients. We increased our team size. We wrote more blogs, spoke at more conferences, and made even more videos !

Here is the breakdown.

We love conferences

We presented at 10 conferences across 7 countries, on topics from Rails to ReactJS.

• Garden City RubyConf, India
• RubyConf, Philippines
• RubyConf, India
• Reddot RubyConf, Singapore
• JQuery Conference, India
• Deccan RubyConf, India
• Full Stack Fest, Spain
• RubyConf, Taiwan
• Rocky Mountain Ruby, Colorado
• RubyConf, Colombia

We are all in ReactJS

We at BigBinary adopted ReactJS pretty early. Early in the year we published series of videos titled Learn ReactJS in steps which takes a “Hello World” app into a full TODO application using ReactJS in incremental steps.

Vipul and Prathamesh are currently authoring a book on ReactJS. Check it out at ReactJS by Example- Building Modern Web Applications with React Book

We also started an ios app using React Native. It’s coming along pretty good and soon we should see it in app store.

We authored many blogs

We love sharing our experiences via blogs on various topics like ReactJS, Rails, React Native and Robot framework etc. Here are some of our blogs from 2015.

• How to obtain current time from a different timezone in Selenium IDE using javascript, Prabhakar, Jan 2015
• Author information in jekyll blog, Neeraj, Jan 2015
• Phone verification using SMS via Twilio, Santosh, Jan 2015
• Blue border around JWPLAYER video, Prathamesh, Feb 2015
• Gotcha with after_commit callback in Rails, Prathamesh, March 2015
• Voice based phone verification using twilio, Santosh, March 2015
• Verifying PubSub Services from Rails using Redis, Vipul, May 2015
• Using ReactJS with Rails Action Cable, Vipul, July 2015
• How to test React Native App on a real iPhone, Chirag, Aug 2015
• Optimize JavaScript code using BabelJS, Prathamesh, Aug 2015
• Configuring Pycharm IDE to run a Robot Framework test suite or a single test script, Prabhakar, Oct 2015
• Migrating rails app from postgresql to sql server, Rohit, Oct 2015
• Getting around Apple iTunesConnect account activation issue, Prathamesh, Oct 2015
• Rails 5 allows setting custom HTTP Headers for assets, Vipul, Oct 2015
• Using Stripe API in React Native with fetch, Chirag, Nov 2015
• How constant lookup and resolution works in Ruby on Rails, Mohit, Nov 2015
• Explicitly ssh into vagrant machine, Neeraj, Dec 2015
• ApplicationRecord in Rails 5, Prathamesh, Dec 2015

Video Summary

• Learn Ruby on Rails
  • Use uuid, X-Request-Id and tagged logging to debug rails application.
  • Rails development using vagrant.
  • Using ES6 in Rails application.
• Learn ReactJS in step
• Keep up with ReactJS
• Learn JavaScript
  • Refactor JavaScript code using module pattern
  • A review of tools to test ES6
• Learn Selenium

Open Source

Apart from our team members contributing to various OpenSource projects, we also support some projects from our team. This year, we added and helped build following projects-

• Wheel : Wheel is our Rails template for new Ruby on Rails projects, with sane defaults and setups for different environments, and common functionalities like image uploaders, debugging, etc.
• Mail Interceptor : Interception, Forwarding emails in Ruby on Rails application
• Handy : A collection handy tools and Rails tasks for your Project.
• Learn ReactJS in Steps : Collection of examples from Learn ReactJS in step video series.
• DoceboRuby : Ruby wrapper for Decobo LMS API
• Doctsplit Chef : Check cookbox for docsplit ruby gem
• Fixtures Dumper : Dump your Rails data to fixtures easily to a database to populate data.

Community Engagement

Along with speaking at various conferences, we also helped organize, our Fun edition of Pune’s regional RubyConf, DeccanRubyConf, and supported other Indian Conferences, including RubyConfIndia, GardenCity RubyConf.

We also help Pune’s local ruby meetup, which had a splendid engagement this year.

Overall we are super excited about what we accomplished in year 2015. We are looking forward to an exciting year of 2016!

This blog is part of our Rails 5 series.

If you run bin/rails -h in a Rails 5 app, you will see a new command for running tests.

$ bin/rails -h
Usage: rails COMMAND [ARGS]

The most common rails commands are:
 generate    Generate new code (short-cut alias: "g")
 console     Start the Rails console (short-cut alias: "c")
 server      Start the Rails server (short-cut alias: "s")
 test        Run tests (short-cut alias: "t")

Before Rails 5, we had to use bin/rake test to run tests. But in Rails 5, we can use bin/rails test. It is not just replacement of old rake task but it is backed by a test runner inspired from RSpec, minitest-reporters, maxitest and others.

Let’s see what bin/rails test can do.

Running a single test

Now it is possible to run a single test out of the box using the line number of the test.

$ bin/rails test test/models/user_test.rb:27

Rails will intelligently run the test from user_test.rb having line number 27. Note that the line number 27 does not need to be first line of the test. Below is an example.

22: def test_valid_user
23:   hash = { email: 'bob@exmaple.com',
24:            first_name: 'John',
25:            last_name: 'Smith' }
26:
27:   user = User.new hash
28:
29:   assert user.valid?
30: end

In the above case, test_valid_user can be run as long as the line number provided is between 22 and 30.

Running multiple tests

You can also pass multiple test paths and Rails will run all of those tests.

$ bin/rails test test/models/user_test.rb:27 test/models/post_test.rb:42

It is also possible to run all tests from a directory.

$ bin/rails test test/controllers test/integration

Improved failure messages

When a test fails, Rails displays a command which can be used to just run the failed test.

$ bin/rails t
Run options: --seed 51858

# Running:

.F

Failure:
PostsControllerTest#test_should_get_new:
Expected response to be a <success>, but was a <302> redirect to <http://test.host/posts>


bin/rails test test/controllers/posts_controller_test.rb:15

I can simply copy bin/rails test test/controllers/posts_controller_test.rb:15 and rerun the failing test.

Failing fast

By default when a test fails then rails reports about the test failures and then moves on to the next test. If you want to stop running the test when a test fails then use option -f.

$ bin/rails t -f
Run options: -f --seed 59599

# Running:

..F

Failure:
PostsControllerTest#test_should_get_new:
Expected response to be a <success>, but was a <302> redirect to <http://test.host/posts>


bin/rails test test/controllers/posts_controller_test.rb:15

Interrupted. Exiting...


Finished in 0.179125s, 16.7481 runs/s, 22.3308 assertions/s.

3 runs, 4 assertions, 1 failures, 0 errors, 0 skips

Defer test output until the end of the full test run

By default when a test fails then rails prints F and then details about the failure like what assertion failed and how to re run the test etc.

If you want to have a clean output of . and F and would like all the test failures report to come at the every end then use option -d.

$ bin/rails t -d
Run options: -d --seed 29906

# Running:

..F...F

Finished in 0.201320s, 34.7704 runs/s, 49.6721 assertions/s.

  1) Failure:
PostsControllerTest#test_should_create_post [/Users/prathamesh/Projects/fun/rails-5-test-runner-app/test/controllers/posts_controller_test.rb:19]:
"Post.count" didn't change by 1.
Expected: 3
  Actual: 2


  2) Failure:
PostsControllerTest#test_should_get_new [/Users/prathamesh/Projects/fun/rails-5-test-runner-app/test/controllers/posts_controller_test.rb:15]:
Expected response to be a <success>, but was a <302> redirect to <http://test.host/posts>

7 runs, 10 assertions, 2 failures, 0 errors, 0 skips

Failed tests:

bin/rails test test/controllers/posts_controller_test.rb:19
bin/rails test test/controllers/posts_controller_test.rb:15

Better backtrace output

By default when an error is encountered while running the test then the output does not contain full stacktrace. This makes debugging little bit difficult.

Error:
PostsControllerTest#test_should_create_post:
NameError: undefined local variable or method `boom' for #<PostsController:0x007f86bc62b728>
    app/controllers/posts_controller.rb:29:in `create'
    test/controllers/posts_controller_test.rb:20:in `block (2 levels) in <class:PostsControllerTest>'
    test/controllers/posts_controller_test.rb:19:in `block in <class:PostsControllerTest

Now we can use -b switch, which will display complete backtrace of error message.

$ bin/rails t -b

Error:
PostsControllerTest#test_should_create_post:
NameError: undefined local variable or method `boom' for #<PostsController:0x007fc53c4eb868>
    /rails-5-test-runner-app/app/controllers/posts_controller.rb:29:in `create'
    /sources/rails/actionpack/lib/action_controller/metal/basic_implicit_render.rb:4:in `send_action'
    /sources/rails/actionpack/lib/abstract_controller/base.rb:183:in `process_action'
    /sources/rails/actionpack/lib/action_controller/metal/rendering.rb:30:in `process_action'
    /sources/rails/actionpack/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
    /sources/rails/activesupport/lib/active_support/callbacks.rb:126:in `call'
.....
    /sources/rails/activesupport/lib/active_support/testing/assertions.rb:71:in `assert_difference'
    /rails-5-test-runner-app/test/controllers/posts_controller_test.rb:19:in `block in <class:PostsControllerTest>'

Leveraging power of Minitest

The test runner also leverages power of minitest by providing some handy options.

Switch -s to provide your own seed

Now we can also provide our own seed using -s switch.

$ bin/rails t --s 42000

Switch -n to run matching tests

Switch -n will run tests matching the given string or regular expression pattern.

$ bin/rails t -n "/create/"
Run options: -n /create/ --seed 24558

# Running:

E

Error:
PostsControllerTest#test_should_create_post:
NameError: undefined local variable or method `boom' for #<PostsController:0x007faa39c2df90>
    app/controllers/posts_controller.rb:29:in `create'
    test/controllers/posts_controller_test.rb:20:in `block (2 levels) in <class:PostsControllerTest>'
    test/controllers/posts_controller_test.rb:19:in `block in <class:PostsControllerTest>'


bin/rails test test/controllers/posts_controller_test.rb:18

Finished in 0.073857s, 13.5396 runs/s, 0.0000 assertions/s.

1 runs, 0 assertions, 0 failures, 1 errors, 0 skips

Verbose output

It is also possible to see the verbose output using -v switch. It shows time required to run each test. This would help in detecting slow running tests.

$ bin/rails t -v
Run options: -v --seed 30118

# Running:

PostsControllerTest#test_should_destroy_post = 0.07 s = .
PostsControllerTest#test_should_update_post = 0.01 s = .
PostsControllerTest#test_should_show_post = 0.10 s = .
PostsControllerTest#test_should_create_post = 0.00 s = F

Failure:
PostsControllerTest#test_should_create_post:
"Post.count" didn't change by 1.
Expected: 3
  Actual: 2

bin/rails test test/controllers/posts_controller_test.rb:19

PostsControllerTest#test_should_get_new = 0.02 s = .
PostsControllerTest#test_should_get_index = 0.01 s = .
PostsControllerTest#test_should_get_edit = 0.00 s = .

Finished in 0.210071s, 33.3220 runs/s, 47.6028 assertions/s.

7 runs, 10 assertions, 1 failures, 0 errors, 0 skips

Colored output

Now by default we will get colored output. No need to add additional gem to colored output.

With all these awesome features, testing Rails 5 apps has definitely become a better experience. Rails has shipped all these features within the framework itself so you don’t have to use multiple gems and libraries to achieve all of these things.