Between October 31,2015 and Sep 5, 2016 we wrote 80 blogs on changes in Rails 5.

Producing a blog every 4 days consistently over 310 days takes persistence and time - lots of it.

We needed to go through all the commits and then pick the ones which are worth writing about and then write about it. Going into this I knew it would be a hard task. Ruby on Rails is now a well crafted machine. In order to fully understand what’s going on in the code base we need to spend sufficient time on it.

However I was surprised by the thing that turned to be the hardest - telling story of the code change.

Every commit has a story. There is a reason for it. The commit itself might be minor but that code change in itself does not tell the full story.

For example take this commit. This commit is so simple that you might think it is not worth writing about. However in order to fully understand what it does we need to tell the full story which was captured in this blog.

Or take the case of Rails 5 officially supports MariaDB . The blog captures the full story and not just the code that changed.

Now you might say that I have cherry picked blog posts that favor my case. So let’s pick a blog which is simple.

You might wonder what could go wrong with a blog like this. As it turns out, plenty. That’s because writing a blog also requires defining the boundary of the blog. Deciding what to include and what to leave out is hard. One gets a feel for it only after writing it. And after having typed the words on screen, pruning is hard.

A good written article is simple writing. The problem with article which are simple to readers is that - well it is simple. So it feels to readers that writing it must be simple. Nothing can be further from the truth. It takes a lot of hard work to produce anything simple. It’s true in writing. And it’s true in producing software.

Coming back to the “Skipping Mailer” blog, it took quite a bit of back and forth to bring the blog to its essence. So yes the final output is quite short but that does not mean that it took short amount of time to produce it.

Tell a story even if you have 10 seconds

John Lasseter was working as an animator at Disney in 1984. He was just fired from Disney for promoting computer animations at Disney. Lasseter joins Lucasfilm. Lucasfilm renamed itself to Pixar Graphics Group and sold itself to Steve Jobs for $5 million.

Lasseter was tasked with producing a short film that would show the power of what computer animations could do so that Pixar Graphics Group can get some projects like producing TV commercials with cartoon characters and earn some money. Lasseter needed to produce a short film for the upcoming computer graphics animation conference.

His initial idea was to have a short movie having a plotless character. He presented this idea to a conference in Brussels. There Belgian animator Raoul Servais commented in slightly harsh tone that

No matter how short it is, it should have a beginning, a middle, and an end. Don’t forget the story.

Lasseter complained that it’s a pretty short movie and there might not be time to present a story.

Raoul Servais replied

You can tell a story in ten seconds.

Lasseter started developing a character. He came up with the idea of Luxo Jr.

Here is final production of Luxo Jr.

Luxo Jr. was a major hit at the conference. Crowd was on its feet in applause even before the two minutes film was over. Remember this is 1986 and Computer Animation was not much advanced at that time and this was the first movie ever made with the use of just computer graphics.

Lasseter later said that when audience was watching the movie they forgot that they were watching a computer animated film because the story took over them. He learned the lesson that technology should enable better story telling and technology in itself divorced from story telling would not advance the cause of Pixar.

Later John Lasseter went on to produce hits like Toy Story, A bug’s life, Toy Story 2, Cars, Cars 2, Monsters Inc, Finding Nemo and many more.

So you see even a great John Lasseter had to be reminded to tell a story.

Actual content over bullet points

Jeff Bezos is so focused on knowing the full story that he banned usage of power point in internal meetings and discussions. As per him it is easy to hide behind bullet points in a power point presentation.

He insisted on writing the full story in word document and distribute it to meeting attendees. The meetings starts with everyone head down reading the document.

He is also known for saying that if we are building a feature then we first need to know how it would be presented to the consumers when it is unveiled. We need to know the story we are going to tell them. Without the story we won’t have full picture of what we are going to build.

Learning to tell story is a journey

I’m glad that during the last 310 days 16 people contributed to the blog posts. The process of writing the posts at times was frustrating for a bunch of them. They had done the work of digging into the code and had posted their findings. Continuously getting feedback to edit the blog to build a nice coherent story where each paragraph is an extension of the previous paragraph is a downer. Some were dismayed at why we are spending so much energy on a technical blog.

However in the end we all are happy that we underwent this exercise. We could see the initial draft of the blog and the final version and we all could see the difference.

By no means we have mastered the art of storytelling. It’s a long journey. However we believe we are on the right path. Hopefully in coming months and years we at BigBinary would be able to bring to you more stories from changes in Rails and other places.

This blog is part of our Rails 5 series.

Rails already provides methods for creating class level and module level variables in the form of cattr_* and mattr_* suite of methods.

In Rails 5, we can go a step further and create thread specific class or module level variables.

Here is an example which demonstrates an example on how to use it.

module CurrentScope
  thread_mattr_accessor :user_permissions
end

class ApplicationController < ActionController::Base

  before_action :set_permissions

  def set_permissions
    user = User.find(params[:user_id])
    CurrentScope.user_permissions = user.permissions
  end

end

Now CurrentScope.user_permissions will be available till the lifetime of currently executing thread and all the code after this point can use this variable.

For example, we can access this variable in any of the models without explicitly passing current_user from the controller.

class BookingsController < ApplicationController
  def create
    Booking.create(booking_params)
  end
end

class Booking < ApplicationRecord
  validate :check_permissions

  private

  def check_permissions
    unless CurrentScope.user_permissions.include?(:create_booking)
      self.errors.add(:base, "Not permitted to allow creation of booking")
    end
  end
end

It internally uses Thread.current#[]= method, so all the variables are scoped to the thread currently executing. It will also take care of namespacing these variables per class or module so that CurrentScope.user_permissions and RequestScope.user_permissions will not conflict with each other.

If you have used PerThreadRegistry before for managing global variables, thread_mattr_* & thread_cattr_* methods can be used in place of it starting from Rails 5.

Globals are generally bad and should be avoided but this change provides nicer API if you want to fiddle with them anyway!

This blog is part of our Rails 5 series.

As a Rails developer, it was a familiar sign to see assets logs flooding the whole terminal in development mode.

Started GET "/assets/application.self-4a04ce68c5ebf2d39fba46316802f17d0a73fadc4d2da50a138d7a4bf2d26a84.css?body=1" for 127.0.0.1 at 2016-09-02 10:23:04 +0530
Started GET "/assets/bootstrap/transition.self-6ad2488465135ab731a045a8ebbe3ea2fc501aed286042496eda1664fdd07ba9.js?body=1" for 127.0.0.1 at 2016-09-02 10:23:04 +0530
Started GET "/assets/bootstrap/collapse.self-2eb697f62b587bb786ff940d82dd4be88cdeeaf13ca128e3da3850c5fcaec301.js?body=1" for 127.0.0.1 at 2016-09-02 10:23:04 +0530
Started GET "/assets/jquery_ujs.self-e87806d0cf4489aeb1bb7288016024e8de67fd18db693fe026fe3907581e53cd.js?body=1" for 127.0.0.1 at 2016-09-02 10:23:04 +0530
Started GET "/assets/jquery.self-660adc51e0224b731d29f575a6f1ec167ba08ad06ed5deca4f1e8654c135bf4c.js?body=1" for 127.0.0.1 at 2016-09-02 10:23:04 +0530

Fortunately, we could include quiet_assets gem in our application. It turns off the Rails asset pipeline log in development mode.

Started GET "/assets/application.js" for 127.0.0.1 at 2016-08-28 19:35:34

quiet_assets is part of Rails 5

Now quiet_assets gem is folded into Rails 5 itself.

A new configuration config.assets.quiet which when set to true, loads a rack middleware named Sprockets::Rails::QuietAssets. This middleware checks whether the current request matches assets prefix path and if it does, it silences that request.

This eliminates the need to add external gem for this.

By default, config.assets.quiet is set to true in development mode. So we don’t have to do anything. It just works out of the box.

Compatibility with older versions of Rails

This functionality has been backported to sprockets-rails 3.1.0 and is available in Rails 4.2.7 as well.

This blog is part of our Rails 5 series.

This blog requires understanding of what is autoloading. If you are not familiar with that then please refer to Autoloading and Reloading Constants article on Rails Guide.

Eagerload paths

Autoloading is not thread-safe and hence we need to make sure that all constants are loaded when application boots. The concept of loading all the constants even before they are actually needed is called “Eager loading”. In a way it is opposite of “Autoloading”. In the case of “Autoloading” the application does not load the constant until it is needed. Once a class is needed and it is missing then the application starts looking in “autoloading paths” to load the missing class.

eager_load_paths contains a list of directories. When application boots in production then the application loads all constants found in all directories listed in eager_load_paths.

We can add directories to eager_load_paths as shown below.

# config/application.rb

config.eager_load_paths << Rails.root.join('lib')

In Rails 5 autoloading is disabled for production environment by default

With this commit Rails will no longer do Autoloading in production after it has booted.

Rails will load all the constants from eager_load_paths but if a constant is missing then it will not look in autoload_paths and will not attempt to load the missing constant.

This is a breaking change for some applications. For vast majority of the applications this should not be an issue.

In the rare situation where our application still needs autoloading in the production environment, we can enable it by setting up enable_dependency_loading to true as follows:

# config/application.rb

config.enable_dependency_loading = true
config.autoload_paths << Rails.root.join('lib')

This blog is part of our Rails 5 series.

Adding HTTPS support is one of the first steps towards enhancing the security of a web application.

Even when a web app is available over https, some users may end up visiting the http version of the app, losing the security https provides.

It is important to redirect users to the https URLs whenever possible.

Forcing HTTPS in Rails

We can force users to use HTTPS by setting config.force_ssl = true.

If we look at Rails source code, we can see that when we set config.force_ssl = true, a middleware ActionDispatch::SSL, is inserted into our app’s middleware stack :

if config.force_ssl
  middleware.use ::ActionDispatch::SSL,config.ssl_options
end

This middleware, ActionDispatch::SSL is responsible for doing three things :

1. Redirect all http requests to their https equivalents.

2. Set secure flag on cookies to tell browsers that these cookies must not be sent for http requests.

3. Add HSTS headers to response.

Let us go through each of these.

Redirect all http requests to their https equivalents

In Rails 5, we can configure the behavior of redirection using the redirect key in the config.ssl_options configuration.

In previous versions of Rails, whenever an http request was redirected to https request, it was done with an HTTP 301 redirect.

Browsers cache 301 redirects. When forcing https redirects, if at any point we want to test the http version of the page, it would be hard to browse it, since the browser would redirect to the https version. Although this is the desired behavior, this is a pain during testing and deploying.

Rails 5 lets us specify the status code for redirection, which can be set to 302 or 307 for testing, and later to 301 when we are ready for deployment to production.

We can specify the options for redirection in Rails 5 as follows :

...
  config.force_ssl = true
  config.ssl_options = {  redirect: { status: 307, port: 81 } }
...

If a redirect status is not specified, requests are redirected with a 301 status code.

There is an upcoming change to make the status code used for redirecting any non-GET, non-HEAD http requests to 307 by default.

Other options accepted by ssl_options under redirect key are host and body .

Set secure flags on cookies

By setting the Secure flag on a cookie, the application can instruct the browser not to send the cookie in clear text. Browsers which support this flag will send such cookies only through HTTPS connections.

Setting secure flag on cookies is important to prevent cookie hijacking by man in the middle attacks.

In case of a “man in the middle” attack, the attacker places oneself between the user and the server. By doing this, attacker aims to collect cookies which are sent from user to server on every request. However, if we mark the cookies with sensitive information as Secure, those cookies won’t be sent on http requests. This ensures that the browser never sends cookies to an attacker who was impersonating the webserver at an http end point.

Upon enabling config.force_ssl = true, the ActionDispatch::SSL middleware sets the Secure flag on all cookies by default.

Set HSTS Headers on Responses

HSTS or “HTTP Strict Transport Security” is a security enhancement by which applications can specify themselves as HTTPS-only to complying browsers.

HSTS capabilities of a browser can be used by sending appropriate response headers from the server. When a domain is added to the HSTS list of a browser, the browser redirects to the https version of the URL without the help of the server.

Chrome maintains an HSTS Preload List with a list of domains which are hardcoded into chrome as HTTPS only. This list is also used by Firefox and Safari.

Rails 5 has a configuration flag to set the preload directive in the HSTS header and can be used as follows :

config.ssl_options = { hsts: { preload: true } }

We can also specify a max-age for the HSTS header.

Rails 5 by default sets the max-age of HSTS header to 180 days, which is considered as the lower bound by SSL Lab’s SSL Test . This period is also above the 18 week requirement for HSTS max-age mandated for inclusion in browser preload list.

We can specify a custom max-age by :

  config.ssl_options = { hsts: { expires: 10.days } }

In Rails 5, if we disable HSTS by setting :

config.ssl_options = { hsts: false }

Rails 5 will set the value of expires header to 0, so that browsers immediately stop treating the domain as HTTPS-only.

With custom redirect status and greater control over the HSTS header, Rails 5 lets us roll out HTTPS in a controlled manner, and makes rolling back of these changes easier.

This blog is part of our Rails 5 series.

Rails, by default, stores session data in cookies.

The cookies have a storage limit of 4K and cookie overflow exception is raised if we attempt to store more than 4K of data in it.

Cookie overflow issue with Rails 4.x

Flash messages are persisted across requests with the help of session storage.

Flash messages like flash.now are marked as discarded for next request. So, on next request, it gets deleted before reconstituting the values.

This unnecessary storage of discarded flash messages leads to more consumption of data in the cookie store. When the data exceeds 4K limit, Rails throws ActionDispatch::Cookies::CookieOverflow.

Let us see an example below to demonstrate this.

class TemplatesController < ApplicationController
  def search
    @templates = Template.search(params[:search])
    flash.now[:notice] = "Your search results for #{params[:search]}"
    flash[:alert] = "Alert message"
    p session[:flash]
  end
end

#logs

{"discard"=>["notice"],
"flashes"=>{"notice"=>"Your search results for #{Value of search params}",
"alert"=>"Alert message"}}

In the above example, it might be possible that params[:search] is large amount of data and it causes Rails to raise CookieOverflow as the session persists both flash.now[:notice] and flash[:alert] .

Rails 5 removes discarded flash messages

In Rails 5, discarded flash messages are removed before persisting into the session leading to less consumption of space and hence, fewer chances of CookieOverflow being raised.

class TemplatesController < ApplicationController
  def search
    @templates = Template.search(params[:search], params[:template])
    flash.now[:notice] = "Your search results for #{params[:search]} with template #{params[:template]}"
    flash[:alert] = "Alert message"
    p session[:flash]
  end
end

#logs

{"discard"=>[], "flashes"=>{"alert"=>"Alert message"}}

We can see from above example, that flash.now value is not added in session in Rails 5 leading to less chances of raising ActionDispatch::Cookies::CookieOverflow.

This blog is part of our Rails 5 series.

Rails 5 has deprecated usage of alias_method_chain in favor of Ruby’s built-in method Module#prepend.

What is alias_method_chain and when to use it

A lot of good articles have been written by some very smart people on the topic of “alias_method_chain”. So we will not be attempting to describe it here.

Ernier Miller wrote When to use alias_method_chain more than five years ago but it is still worth a read.

Using Module#prepend to solve the problem

Ruby 2.0 introduced Module#prepend which allows us to insert a module before the class in the class ancestor hierarchy.

Let’s try to solve the same problem using Module#prepend.

module Flanderizer
  def hello
    "#{super}-diddly"
  end
end

class Person
  def hello
    "Hello"
  end
end

# In ruby 2.0
Person.send(:prepend, Flanderizer)

# In ruby 2.1
Person.prepend(Flanderizer)

flanders = Person.new
puts flanders.hello #=> "Hello-diddly"

Now we are back to being nice to our neighbor which should make Ernie happy.

Let’s see what the ancestors chain looks like.

flanders.class.ancestors # => [Flanderizer, Person, Object, Kernel]

In Ruby 2.1 both Module#include and Module#prepend became a public method. In the above example we have shown both Ruby 2.0 and Ruby 2.1 versions.

When a new version of Rails comes out, one of the pain points is upgrading existing apps to the latest version.

A Rails upgrade can be boiled down to following essential steps :

1. Have a green build
2. Update the Rails version in Gemfile and bundle
3. Run the update task to update configuration files
4. Run tests and sanity checks to see if anything is broken by the upgrade and fix the issues
5. Repeat step 4!

Rails 5 comes with a lot of new features. Some of them, like not halting the callback chain when a callback returns false, are breaking changes for older apps.

To keep the upgrade process easier, Rails 5 has added feature flags for all of these breaking changes.

When we create a brand new Rails 5 app, all of the feature flags will be turned on. We can see these feature flags in config/initializers/new_framework_defaults.rb file.

But when we upgrade an app to Rails 5, just updating the Gemfile and bundling is not enough.

We need to run the bin/rails app:update task which will update few configurations and also add config/initializers/new_framework_defaults.rb file.

Rails will turn off all the feature flags in the config/initializers/new_framework_defaults.rb file while upgrading an older app. In this way our app won’t break due to the breaking features.

Let’s take a look at these configuration flags one by one.

Enable per-form CSRF tokens

Starting from Rails 5, each form will get its own CSRF token. This change will have following feature flag.

Rails.application.config.action_controller.per_form_csrf_tokens

For new apps, it will be set to true and for older apps upgraded to Rails 5, it will be set to false. Once we are ready to use this feature in our upgraded app, we just need to change it to true.

Enable HTTP Origin Header checking for CSRF mitigation

For additional defense against CSRF attacks, Rails 5 has a feature to check HTTP Origin header against the site’s origin. This will be disabled by default in upgraded apps using the following configuration option:

Rails.application.config.action_controller.forgery_protection_origin_check

We can set it to true to enable HTTP origin header check when we are ready to use this feature.

Make Ruby 2.4 preserve the timezone of the receiver

In Ruby 2.4 the to_time method for both DateTime and Time will preserve the timezone of the receiver when converting to an instance of Time. For upgraded apps, this feature is disabled by setting the following configuration option to false :

ActiveSupport.to_time_preserves_timezone

To use the Ruby 2.4+ default of to_time, set this to true .

Require belongs_to associations by default

In Rails 5, when we define a belongs_to association, the association record is required to be present.

In upgraded apps, this validation is not enabled. It is disabled using the following option:

Rails.application.config.active_record.belongs_to_required_by_default

We can update our code to use this feature and turn this on by changing the above option to true.

Do not halt callback chain when a callback returns false

In Rails 5, callback chain is not halted when a callback returns false. This change is turned off for backward compatibility with the following option set to true:

ActiveSupport.halt_callback_chains_on_return_false

We can use the new behavior of not halting the callback chain after making sure that our code does not break due to this change and changing the value of this config to false.

Configure SSL options to enable HSTS with subdomains

HTTP Strict Transport Security or HSTS, is a web security policy mechanism which helps to protect websites against protocol downgrade attacks and cookie hijacking. Using HSTS, we can ask browsers to make connections using only HTTPS. In upgraded apps, HSTS is not enabled on subdomains. In new apps HSTS is enabled using the following option :

Rails.application.config.ssl_options = { hsts: { subdomains: true } }

Having all these backward incompatible features which can be turned on one by one after the upgrade, in one file, eases the upgrade process. This initializer also has helpful comments explaining the features!

Happy Upgrading!

This blog is part of our Rails 5 series.

Cache Digests

After cache digests were introduced in Rails, all calls to #cache in views automatically append a digest of that template and all of its dependencies to the cache key.

So developers no longer need to manually discard cache for the specific templates they make changes to.

# app/views/users/show.html.erb
<% cache user do %>
  <h1>All Posts</h1>
  <%= render user.posts %>
<% end %>

# app/views/posts/_post.html.erb
<% cache post do %>
  <p> <%= post.content %></p>
  <p> <%= post.created_at.to_s %>
  <%= render 'posts/completed' %>
<% end %>

This creates a caching key something like this views/users/605416233-20129410191209/d9fb66b12bx8edf46707c67ab41d93cb2 which depends upon the template and its dependencies.

So, now if we change posts/_completed.html.erb, it will change cache key and thus it allows cache to expire automatically.

Explicit dependencies

As we saw in our earlier example, Rails was able to determine template dependencies implicitly. But, sometimes it is not possible to determine dependencies at all.

Let’s see an example below.

# app/views/users/show.html.erb
<% cache user do %>
  <h1>All Posts</h1>
  <%= render user.posts %>
<% end %>

# app/views/posts/_post.html.erb
<% cache post do %>
  <p> <%= post.content %></p>
  <p> <%= post.created_at.to_s %>
  <%= render_post_complete_or_not(post) %>
<% end %>

# app/helpers/posts_helper.rb

module PostsHelper
  def render_post_complete_or_not(post)
    if post.completed?
      render 'posts/complete'
    else
      render 'posts/incomplete'
    end
  end
end

To explicitly add dependency on this template, we need to add a comment in special format as follows.

  <%# Template Dependency: posts/complete %>
  <%# Template Dependency: posts/incomplete %>

If we have multiple dependencies, we need to add special comments for all the dependencies one by one.

# app/views/posts/_post.html.erb
<% cache post do %>
  <p> <%= post.content %></p>
  <p> <%= post.created_at.to_s %>

  <%# Template Dependency: posts/complete %>
  <%# Template Dependency: posts/incomplete %>
  <%= render_post_complete_or_not(post) %>
<% end %>

Using Wildcard in Rails 5

In Rails 5, we can now use a wildcard for adding dependencies on multiple files in a directory. So, instead of adding files one by one we can add dependency using wildcard.

# app/views/posts/_post.html.erb
<% cache post do %>
  <p> <%= post.content %></p>
  <p> <%= post.created_at.to_s %>

  <%# Template Dependency: posts/* %>
  <%= render_post_complete_or_not(post) %>
<% end %>

This blog is part of our Rails 5 series.

Rails has powerful tools to control caching of resources via HTTP such as fresh_when and stale?.

Previously we could only pass a single record to these methods but now Rails 5 adds support for accepting a collection of records as well. For example,

def index
  @posts = Post.all
  fresh_when(etag: @posts, last_modified: @posts.maximum(:updated_at))
end

or simply written as,

def index
  @posts = Post.all
  fresh_when(@posts)
end

This works with stale? method too, we can pass a collection of records to it. For example,

def index
  @posts = Post.all

  if stale?(@posts)
    render json: @posts
  end
end

To see this in action, let’s begin by making a request at /posts.

$ curl -I http://localhost:3000/posts

HTTP/1.1 200 OK
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
ETag: W/"a2b68b7a7f8c67f1b88848651a86f5f5"
Content-Type: text/html; charset=utf-8
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: 7c8457e7-9d26-4646-afdf-5eb44711fa7b
X-Runtime: 0.074238

In the second request, we would send the ETag in If-None-Match header to check if the data has changed.

$ curl -I -H 'If-None-Match: W/"a2b68b7a7f8c67f1b88848651a86f5f5"' http://localhost:3000/posts

HTTP/1.1 304 Not Modified
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
ETag: W/"a2b68b7a7f8c67f1b88848651a86f5f5"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: 6367b2a5-ecc9-4671-8a79-34222dc50e7f
X-Runtime: 0.003756

Since there’s no change, the server returned HTTP/1.1 304 Not Modified. If these requests were made from a browser, it would automatically use the version in its cache on the second request.

The second request was obviously faster as the server was able to save the time of fetching data and rendering it. This can be seen in Rails log,

Started GET "/posts" for ::1 at 2016-08-06 00:39:44 +0530
Processing by PostsController#index as HTML
   (0.2ms)  SELECT MAX("posts"."updated_at") FROM "posts"
   (0.1ms)  SELECT COUNT(*) AS "size", MAX("posts"."updated_at") AS timestamp FROM "posts"
  Rendering posts/index.html.erb within layouts/application
  Post Load (0.2ms)  SELECT "posts".* FROM "posts"
  Rendered posts/index.html.erb within layouts/application (2.0ms)
Completed 200 OK in 31ms (Views: 27.1ms | ActiveRecord: 0.5ms)


Started GET "/posts" for ::1 at 2016-08-06 00:39:46 +0530
Processing by PostsController#index as HTML
   (0.2ms)  SELECT MAX("posts"."updated_at") FROM "posts"
   (0.1ms)  SELECT COUNT(*) AS "size", MAX("posts"."updated_at") AS timestamp FROM "posts"
Completed 304 Not Modified in 2ms (ActiveRecord: 0.3ms)

Cache expires when collection of records is updated. For example, an addition of a new record to the collection or a change in any of the records (which changes updated_at) would change the ETag.

Now that Rails 5 supports collection of records in fresh_when and stale?, we have an improved system to cache resources and make our applications faster. This is more helpful when we have controller actions with time consuming data processing logic.

This blog is part of our Rails 5 series.

Fixtures

In Rails, for setting up test data we use fixtures. Fixtures are written as YAML files placed in test/fixtures directory in the Rails app.

The model name of a fixture is automatically picked up from the fixture file name.

Generally in Rails, the model name and table name follow a strict convention. The table for User model will be users. By this convention, the fixture file for User model is test/fixtures/users.yml.

But sometimes model names do not match directly with the table name. When we are building on top of a legacy application or we have namespacing of models, we might run into this scenario. In such cases detection of model name from fixture file name becomes difficult.

When automatic detection of model name from fixture file name fails, we can specify the table name using the set_fixture_class method. Take a look at our older blog for an example of how to do this.

One drawback of using this approach is that, the model name set using set_fixture_class is available only in the context of tests. When we run rake db:fixtures:load to load the fixtures, the tests are not run, and the fixture file is not associated with the model name we set using set_fixture_class. This will cause failure to load the fixtures correctly.

The Rails 5 way

In Rails 5 a new key is added to specify the model name for a fixture file.

Let us consider the example where our table was named morning_appts, and we used a more appropriately named model MorningAppointment to represent this table.

We can now set the table name in our fixture file test/fixtures/morning_appts.yml as follows :

_fixture:
  model_class: MorningAppointment
standup:
 name: Standup
 priority: 1

The special key _fixture in the fixture file is now used to store metadata about the fixture. model_class is the key we can use to specify the model name for the fixture.

We can now use this fixture to load test data using the rake task rake db:fixtures:load as well.

Happy Testing!

Generators in JavaScript are the functions that can be paused and resumed.

function* genFunc(){...}

Calling genFunc() doesn’t execute the body of the function.

const genObj = genFunc();

Instead, it returns generator object which can be used to control the execution of the generator.

genObj.next() will start executing the function and it will execute till it reaches yield keyword.

We can think of yield as return for now. It is similar to C# yield.

function* genFunc(){
  console.log('First');
  yield;
  console.log('Second');
}
const genObj = genFunc();
genObj.next();

/*
output

First
*/

Calling genObj.next() again will resume the further execution after yield.

function* genFunc(){
  console.log('First');
  yield;
  console.log('Second');
}
const genObj = genFunc();
genObj.next();
genObj.next();

/*
output

First
Second
*/

Here’s how the execution looks like:

yield is not allowed inside non-generator functions. That is, yielding in callbacks doesn’t work.

We can also pass the data to generator function via next.

function* genFunc(){
  console.log('First');
  const input = yield;
  console.log(input);
  console.log('Second');
}
const genObj = genFunc();
genObj.next();
genObj.next('Third');

/*
output

First
Third
Second
*/

We can retrieve the yielded values via the generator object genObj:

function* genFunc(){
  console.log('First');
  const input = yield;
  console.log(input);
  console.log('Second');
  yield 'Forth';
}
const genObj = genFunc();
genObj.next();
const result = genObj.next('Third');
console.log(result);

/*
output

First
Third
Second
{
  done: false,
  value: "Forth"
}
*/

We can have multiple yield in the function as shown in the above example.

Once, the execution of the generator function is completed, further calls to genObj.next() will have no effect.

Further reading

We highly recommend Exploring ES6 by Dr. Axel Rauschmayer to go deeper on this topic.

Recently we worked with a client where we had to run a part of their multi-threaded code in JRuby for performance reasons. They have been using CircleCI with MRI for running tests. In this post I will explain how we configured CircleCI to run the same tests using both JRuby and MRI.

CircleCI uses circle.yml file for configuration. Before configuring JRuby, this is how it looked like:

  machine:
    ruby:
      version:
        2.1.5

  dependencies:
    pre:
      - ./bundle_install_circle_ci.sh
    cache_directories:
      - "~/vendor/bundle"

  test:
    override:
      - bundle exec rspec --format progress --format documentation --format RspecJunitFormatter --out $CIRCLE_TEST_REPORTS/app.xml:
          parallel: true
          pwd: app
          files:
            - spec/**/*_spec.rb

Here are the steps to enable JRuby in CircleCI.

Specify the JDK version

We need to specify a JDK version before using JRuby.

  machine:
    java:
      version:
        openjdk7

Install proper dependencies

We needed to use JRuby 9.0.4.0 but the version of JRuby that came with Ubuntu 12.04 image of CircleCI was different. We added rvm install command as follows to install specific version that we wanted. Also we can configure any script (like bundle install) that needs to run before running tests.

  dependencies:
    pre:

      - rvm install jruby-9.0.4.0
      - ./bundle_install_jruby_circle_ci.sh

    cache_directories:
      - "~/vendor/bundle"

Configure JRuby

We used rvm-exec to set JRuby for running tests for this particular component in the test section. Otherwise by default it picks up MRI.

  test:
    override:

      - rvm-exec jruby-9.0.4.0 bash -c "bundle exec rspec --format progress --format documentation --format RspecJunitFormatter --out $CIRCLE_TEST_REPORTS/app_jruby.xml":
          parallel: true
          pwd: app
          files:
            - spec/**/*_spec.rb

Improving test runs on JRuby

Once we started running tests with JRuby, we observed it was taking comparatively slower to finish all tests. Most of the time was spent in starting the JVM. We made it faster by setting --dev parameter in JRUBY_OPTS environment variable. This parameter improves JRuby boot time and it shaved more than a minute time for us.

  machine:
    environment:
      JRUBY_OPTS: '--dev'

Here is the final circle.yml file:

# circle.yml

machine:
  ruby:
    version:
      2.1.5

  java:
    version:
      openjdk7

  environment:
    JRUBY_OPTS: '--dev'

dependencies:
  pre:
    - rvm install jruby-9.0.4.0
    - ./bundle_install_jruby_circle_ci.sh

  cache_directories:
    - "~/vendor/bundle"

test:
  override:
    - rvm-exec jruby-9.0.4.0 bash -c "bundle exec rspec --format progress --format documentation --format RspecJunitFormatter --out $CIRCLE_TEST_REPORTS/app_jruby.xml":
        parallel: true
        pwd: app
        files:
          - spec/**/*_spec.rb

    - bundle exec rspec --format progress --format documentation --format RspecJunitFormatter --out $CIRCLE_TEST_REPORTS/app_mri.xml:
        parallel: true
        pwd: app
        files:
          - spec/**/*_spec.rb

Using WebView in a React Native application allows us to reuse already built web pages.

With iOS 9 or higher if our application attempts to connect to any HTTP server that doesn’t support the latest SSL technology (TLSv1.2), WebView will fail and will not be able load the web pages.

Let’s see this in action. Here we are trying to access a HTTP site via WebView in React Native.

<WebView
  style={styles.container}
  source={ {uri: "http://del.icio.us"} } />

Running that on an iPhone simulator with iOS version 9.0 or greater would show following error.

Error Loading Page
Domain: NSURLErrorDomain
Error Code: -1022
Description: The resource could not be loaded because
the App Transport Security policy requires the use of a
secure connection

Ideally, the site we are trying to connect to should have HTTPS enabled. However there might be cases where we need to connect to sites where HTTPS is not enabled.

For example, while developing the app, we might want to connect to local server which is running just HTTP.

Fix - Using Xcode

To access HTTP sites inside our WebView, we need to open the project in Xcode and open the Info.plist file.

In the list of keys, we will find App Transport Security Settings.

When we expand the section, we would find localhost inside the Exception Domains and the key NSTemporaryExceptionAllowsInsecureHTTPLoads with value true.

Because of this setting when we are connecting to localhost then app runs even if server is running on HTTP and not on HTTPS.

So in order to make our non HTTPS site run, we need to add our website url to this whitelisting.

When we hover over the Exception Domains key, we would see a + sign at the right hand side.

Click on it and add our domain here. Set the type to dictionary.

Now click on the domain we just entered and add NSTemporaryExceptionAllowsInsecureHTTPLoads with type Boolean and value YES similar to the one present for localhost

Re-run the app using react-native run-ios from the terminal and now the site would be loaded.

If it doesn’t work, then prior to running the app, do a clean build from xcode.

Fix using any IDE

By making the changes from XCode, if we look at the changes in info.plist file, we would find a few lines of code added.

So if we don’t want to open Xcode for the fix, we can add the following lines directly in our info.plist.

Edit the node for the key NSAppTransportSecurity so that the whole node now looks like this:

<key>NSAppTransportSecurity</key>
<dict>
    <key>NSExceptionDomains</key>
    <dict>
        <key>del.icio.us</key>
        <dict>
            <key>NSTemporaryExceptionAllowsInsecureHTTPLoads</key>
            <true/>
        </dict>
        <key>localhost</key>
        <dict>
            <key>NSTemporaryExceptionAllowsInsecureHTTPLoads</key>
            <true/>
        </dict>
    </dict>
</dict>

Be sure to re-run the app using react-native run-ios. Now let’s see how to allow all the HTTP sites instead of whitelisting each and everyone

Using Xcode

Using Xcode : To allow all the non HTTPS sites, just delete the Exception Domains from Xcode inside info.plist and add a new key Allow Arbitrary Loads with the value true.

Using any IDE

Our NSAppTransportSecurity should just contain the following.

<key>NSAppTransportSecurity</key>
<dict>
    <key>NSAllowsArbitraryLoads</key>
    <true/>
</dict>

Using WebView in a React Native application allows us to reuse already built web pages.

HTTP Headers are name/value pairs that appear in both request and response messages. The purpose of headers is to supply the web server with additional information and control how content is returned.

In React Native, while opening web pages via WebView Component, we can pass headers to the HTTP request. Refer to our previous blog for more on this.

<WebView
  source={
    {
      uri: "http://localhost:3000",
      headers: {"custom-app-header": "react-native-ios-app"}
    }
  }
/>

But there is a bug. On subsequent requests from inside the WebView, the headers are not passed in the request.

First, let’s try to understand by recreating the bug. We have created a simple node server which will act as the backend for the application and log the request along with the custom header.

Here is our server.js

var http = require('http');
var port = 9000;

function logRequest(request) {
  console.log("Processing request for: ", request.url)
  console.log("Custom Header: ", request.headers["custom-app-header"])
  console.log("Request Processed\n")
}

http.createServer(function(request, response) {
  response.writeHead(200, {"Content-Type": "text/html"});
  switch(request.url) {
    case "/":
      response.write("<html><body>Welcome<a href='/bye'>Bye</a></body></html>");
      logRequest(request)
      break;
    case "/bye":
      response.write("<html><body>Bye<a href='/'>Welcome</a></body></html>");
      logRequest(request)
      break;
    default:
      break;
  }
  response.end();
}).listen(port);

As we can see, the welcome page has a link to bye and vice versa. Let’s start the node server by running node server.js.

When we run the app on the simulator, the welcome page opens up, and in the server log, we can verify that the request header is being passed.

Processing request for:  /
Custom Header:  react-native-ios-app
Request Processed

But when we click on the Bye link from the Welcome page, the server doesn’t receive the request header, which can be verified from the log.

Processing request for:  /bye
Custom Header:  undefined
Request Processed

And it can be verified again that for any subsequent clicks the request header does not get passed. We can click on Welcome and check the log again.

Processing request for:  /
Custom Header:  undefined
Request Processed

We recently encountered this bug and created an issue here. Untill the issue is fixed, we have found a workaround.

Workaround

WebView provides a prop onLoadStart which accepts a function that is invoked when the WebView starts loading.

We can use this prop to know when a link is clicked and then re-render the WebView component with the new url. Re-rendering the WebView component will load the page as if it’s the first page and then the request headers would be passed.

We know that in React, a component re-renders itself when any of its state changes. The only thing which changes here is the url, so let’s move the url to a state and initialize it to the Welcome page which is the root of the application. And then use the onLoadStart prop to change the url state to the clicked url.

Here’s the new code.

class testApp extends Component {
  state = {
    url: "http://localhost:3000",
  };

  render() {
    return (
      <WebView
        onLoadStart={(navState) => this.setState({url: navState.nativeEvent.url})}
        source={
          {
            uri: this.state.url,
            headers: {"custom-app-header": "react-native-ios-app"}
          }
        }
      />
    );
  }
}

Now when we run the app, we can verify in the backend that the request headers are being sent even when we click on Bye link.

Processing request for:  /bye
Custom Header:  undefined
Request Processed

Processing request for:  /bye
Custom Header:  react-native-ios-app
Request Processed

One thing to note here is that, when we click on the Bye link, the request is not intercepted from reaching the server. We are just resending the request by means of a component re-render with the new url.

Hence in the log, we see two requests. First request took place when user clicked on the link, and the second request occurred when the component got re-rendered with the required request headers.

This workaround might help us to pass the request headers which we intend to send to the backend server until the issue gets fixed.

This blog is part of our Rails 5 series.

We are all guilty of treating ActionController::Parameters as a plain hash at some point or the other. But with Rails 5, ActionController::Parameters will no longer inherit from HashWithIndifferentAccess.

Inheriting from HashWithIndifferentAccess allowed programmers to call enumerable methods over ActionController::Parameters, which caused ActionController::Parameters to lose its @permitted state there by rendering Strong Parameters as a barebone Hash. This change would discourage such operations.

However since this change would have meant a major impact on all of the upgrading applications as they would have crashed with a NoMethodErrorfor all of those undesired methods. Hence this feature would go through a deprecation cycle, showing deprecation warnings for all of those HashWithIndifferentAccess method usages.

class Parameters

...

def method_missing(method_sym, *args, &block)
  if @parameters.respond_to?(method_sym)
    message = <<-DEPRECATE.squish
      Method #{method_sym} is deprecated and will be removed in Rails 5.1,
      as `ActionController::Parameters` no longer inherits from
      hash. Using this deprecated behavior exposes potential security
      problems. If you continue to use this method you may be creating
      a security vulnerability in your app that can be exploited. Instead,
      consider using one of these documented methods which are not
      deprecated: http://api.rubyonrails.org/v#{ActionPack.version}/classes/ActionController/Parameters.html
    DEPRECATE
    ActiveSupport::Deprecation.warn(message)
    @parameters.public_send(method_sym, *args, &block)
  else
    super
  end
end

...

end

If you need to convert ActionController::Parameters in a true hash then it supports to_h method. Also ActionController::Parameters will continue to have methods like fetch, slice, slice!, except, except!, extract!, delete etc. You can take a detailed look at them here.

This blog is part of our Rails 5 series.

users(:id, :name)
posts(:id, :title, :user_id)
comments(:id, :description, :user_id, :post_id)

>> Post.joins(:comments).group(:user_id).count
Mysql2::Error: Column 'user_id' in field list is ambiguous: SELECT COUNT(*) AS count_all, user_id AS user_id FROM `posts` INNER JOIN `comments` ON `comments`.`post_id` = `posts`.`id` GROUP BY user_id

As we can see user_id has conflict in both projection and GROUP BY as they are not prepended with the table name posts in the generated SQL and thus, raising SQL error Column 'user_id' in field list is ambiguous.

Fix in Rails 5

This issue has been addressed in Rails 5 with this pull request.

With this fix, we can now group by columns having same name in both the tables.

users(:id, :name)
posts(:id, :title, :user_id)
comments(:id, :description, :user_id, :post_id)

>> Post.joins(:comments).group(:user_id).count
SELECT COUNT(*) AS count_all, "posts"."user_id" AS posts_user_id FROM "posts" INNER JOIN "comments" ON "comments"."post_id" = "posts"."id" GROUP BY "posts"."user_id"

=> { 1 => 1 }

This shows that now both projection and Group By are prepended with the posts table name and hence fixing the conflict.

This blog is part of our Rails 5 series.

Let’s assume that in our health care application we have a page which shows all Patients. This page also has a filter and it allows us to filter patients by their name.

We could implement the filter as shown here.

Patient.where("lower(first_name) = ?", first_name.downcase)

There might be many users with the same name. In such cases, to speed up the search process, we can add an index. But, adding a regular index will not trigger an index scan since we are using an expression in the where clause i.e lower(name). In such cases, we can leverage expression indexes given by PostgreSQL.

Before Rails 5 adding an expression index is not straightforward since the migrate api does not support it. In order to add one we would need to ditch schema.rb and start using structure.sql. We would also need to add following migration.

def up
  execute <<-SQL
    CREATE INDEX patient_lower_name_idx ON patients (lower(name));
  SQL
end

def down
  execute <<-SQL
    DROP INDEX patient_lower_name_idx;
  SQL
end

Rails 5 adds support for expression indexes

Rails 5 provides ability to add an expression index using add_index method as follows:

def change
  add_index :patients,
            'lower(last_name)',
            name: "index_patients_on_name_unique",
            unique: true
end

And we also get to keep schema.rb.

Time goes on. everyone is happy with the search functionality until one day a new requirement comes along which is, in short, to have partial matches on patient names.

We modify our search as follows:

User.where("lower(name) like ?", "%#{name.downcase}%")

Since the query is different from before, PostgreSQL query planner will not take the already existing btree index into account and will revert to a sequential scan.

Quoting directly from Postgresql documents,

'The operator classes text_pattern_ops, varchar_pattern_ops, and bpchar_pattern_ops support B-tree indexes on the types text, varchar, and char respectively. The difference from the default operator classes is that the values are compared strictly character by character rather than according to the locale-specific collation rules. This makes these operator classes suitable for use by queries involving pattern matching expressions (LIKE or POSIX regular expressions) when the database does not use the standard "C" locale.'

We need to add an operator class to the previous index for the query planner to utilize the index that we created earlier.

Rails 5 adds support for specifying operator classes on expression indexes

In order to add an index with an operator class we could write our migration as shown below.

def change
  remove_index :patients, name: :index_patients_on_name_unique
  add_index :patients,  'lower(last_name) varchar_pattern_ops',
                        name: "index_patients_on_name_unique",
                        unique: true
end

This blog is part of our Rails 5 series.

Rails 4.2 came with built-in support for executing jobs in the background using Active Job. Along with many enhancements to Active Job, Rails 5 provides the ability to attach arbitrary metadata to any job.

Consider the scenario where we would like to get notified when a job has failed for more than three times. With the new enhancement, we can make it work by overriding serialize and deserialize methods of the job class.

class DeliverWebhookJob < ActiveJob::Base
  def serialize
    super.merge('attempt_number' => (@attempt_number || 0) + 1)
  end

  def deserialize(job_data)
    super(job_data)
    @attempt_number = job_data['attempt_number']
  end

  rescue_from(TimeoutError) do |ex|
    notify_job_tried_x_times(@attempt_number) if @attempt_number == 3
    retry_job(wait: 10)
  end
end

Earlier, deserialization was performed by #deserialize class method and therefore was inaccessible from the job instance. With the new changes, deserialization is delegated to the deserialize method on job instance thereby allowing it to attach arbitrary metadata when it gets serialized and read it back when it gets performed.

This blog is part of our Rails 5 series.

Before Rails 5, partials name should start with underscore and should be followed by any combination of letters, numbers and underscores.

This rule was required because before commit, rendering a partial without giving :object or :collection used to generate a local variable with the partial name by default and a variable name in ruby can’t have dash and other things like that.

In the following case we have a file named _order-details.html.erb. Now let’s try to use this partial.

<!DOCTYPE html>
<html>
<body>
  <%= render :partial => 'order-details' %>
</body>
</html>

We will get following error, if we try to render above view in Rails 4.x.

ActionView::Template::Error (The partial name (order-details) is not a valid Ruby identifier;
make sure your partial name starts with underscore,
and is followed by any combination of letters, numbers and underscores.):
    2: <html>
    3: <body>
    4: Following code is rendered through partial named _order-details.erb
    5: <%= render :partial => 'order-details' %>
    6: </body>
    7: </html>

In the above the code failed because the partial name has a dash which is not a valid ruby variable name.

In Rails 5, we can give our partials any name which starts with underscore.