Optimize JavaScript code using BabelJS

Though Babel is a transpiler to convert ESNext code to ES5, it can be used to optimize the code as well.

Let’s say we want to convert following ES6 code using Babel.

let a = 10;
let b = 42;

if (a < b) {
  console.log("a is less than b");
} else {
  console.log("b is less than a");
}

It will get translated to:

"use strict";

var a = 10;
var b = 42;

if (a < b) {
  console.log("a is less than b");
} else {
  console.log("b is less than a");
}

All good so far. Let’s try Babel’s constant folding plugin. I am compiling the ES6 code from command line this time.

babel --plugins constant-folding index.js --out-file bundle.js -w

This gives us following output:

"use strict";

var a = 10;
var b = 42;

if (true) {
  console.log("a is less than b");
} else {
  console.log("b is less than a");
}

If condition changed from (a < b) to (true). The constant-folding plugin has smartly evaluated the conditional expression a < b and replaced the code with the result of that expression.

This plugin can also optimize other expressions as shown below.

// Unary operators
console.log(!true);

// Binary expression
console.log(20 + 22);

// Function calls
console.log(Math.min(91, 2 + 20));

Which gets optimized to:

// Unary operators
console.log(false);

// Binary expression
console.log(42);

// Function calls
console.log(22);

How does this actually work?

Though we are using a “constant folding plugin” to achieve this optimization, the real work happens in Babel itself.

For every expression in the code, the constant folding plugin calls evaluate function from Babel source code. This function checks whether it can confidently find end value for a given expression.

The evaluate function returns confidence level and end value for any given expression. Based on this “confidence level”, constant folding plugin replaces the expression with their end values altering our original code as follows.

How evaluate handles different cases

For code Math.min(10, 20), evaluate will return

{ confident: true, value: 10 }

For code a < b, evaluate will return

{ confident: true, value: true }.

But for user defined function like foo('bar') or browser defined console.log('hello'), evaluate will return

{ confident: false, value: undefined }.

In the above case “confident” value will be “false” even if function returns a constant value. For example for code foo(100), evaluate will return

{ confident: false, value: undefined }.

In the above case function foo will always return 100. Still Babel has low confidence level. Why? That’s because Babel sees that it is a function and it bails out. It does not even look inside to try to figure things out.

Here is evaluate code in Babel. You should check it out.

How much optimization is possible?

How much help we will get from Babel for optimizing our code? Will it optimize everything?

The answer is unfortunately no.

As of now, Babel optimizes logical, binary, conditional expressions. It can also evaluate function calls on literals like "babel".length confidently if the literal is string or number.

For function calls, it supports only certain callees like String, Number and Math. So call to a user defined function, even if it’s returning a fixed value, will not be optimized.

Experimental feature

This feature looks great. But it’s available as experimental feature. If you use the plugin you will get following warning. unless you enable experimental flag.

$ babel --plugins constant-folding index.js --out-file bundle.js -w

[BABEL] index.js: THE TRANSFORMER constant-folding HAS BEEN MARKED AS EXPERIMENTAL AND IS WIP. USE AT YOUR OWN RISK. THIS WILL HIGHLY LIKELY BREAK YOUR CODE SO USE WITH **EXTREME** CAUTION. ENABLE THE `experimental` OPTION TO IGNORE THIS WARNING.

In order to get rid of warning you need to pass --experimental flag like this.

$ babel --plugins constant-folding index.js --out-file bundle.js -w
--experimental

Eliminating dead code

In above code example, we know that the result of if (a < b) is true based on values of a and b. Since the result is not going to change no matter what there is no need to have the if and else clauses.

That’s dead code.

Can Babel help us eliminate dead code?

Yes with the help of minification.deadCodeElimination option.

babel --optional minification.deadCodeElimination index.js --out-file bundle.js -w

Which converts earlier code to:

"use strict";

console.log("a is less than b");

I will talk about how Babel can eliminate dead code in a later post.

How to test React Native App on a real iPhone

I have been developing a new iOS app using React Native. I have been testing it using simulator provided by Xcode. Now it’s time to test the app using a real device. One could ask why test on a real device if it works perfectly on a simulator. Here are some of the reasons.

  • There are a number of device specific features which are not testable on a simulator. Phone calls, camera, gps data, compass and vibration are a few of them.
  • Testing unexpected cases that can only be tested on a real device. Like how your application handles incoming calls, low memory situations, low disk space, limited data connectivity etc.
  • Hardware-wise, your device is different than your Mac. If your app is graphics intensive and requires a lot of CPU usage, it might work seamlessly on your simulator depending on your Mac’s specifications. But it might be laggy and glitchy on the real device.

Now let’s look into how you can start testing your React Native app on an real iPhone.

Step 1 - Plug in your device

Plug in your device to your Mac and open Xcode. You will be able to select your device to the right of the Play and Stop buttons.

select device

You might run into one of the following two errors if you have not enrolled into Apple’s Developer Program.

Error 1 : Failed to code sign

xcode error 1

xcode error 2

To fix above issues please enroll in Apple Developer program.

Error 2 : Disk Image could not be mounted

The Developer Disk Image could not be mounted :
You may be running a version of iOS that is not supported by this version of XCode

could not mount

This can happen if your iOS version is not supported by your current version of Xcode. To fix this, just update your Xcode to the latest version from the App Store.

Step 2 - Set right deployment target

In your Xcode project setup the Deployment Target should be set to less than or equal to the iOS version installed on the device.

If your iPhone is running iOS 7.0 and you have set “Deployment Target” as 8.0 then the app will not work. If your iPhone is running iOS 8.0 and you have set “Deployment Target” as 7.0 then the app will work.

deployment target

Step 3 - Fix “Could not connect to development server” error

So the app is installed and you can see the launch screen. That’s great. But soon you might get following error on your iPhone.

Could not connect to development server.
Ensure node server is running and available on the same
network – run ‘npm start’ from react-native root.

<img src=”/assets/testing_ios_app/connection-error-cf34a1f8eb8059baa422955e6a5c8361da648cf7564c657d469515eeb5e9b6f2.PNG” alt=”connection error” width=200=”true” height=350=”true” width=”640” height=”1136”>

To fix this open AppDelegate.m file in your project’s iOS folder.

Locate line mentioned below and replace localhost with your Mac IP address e.g. 192.168.x.x.

To find the ip address of your Mac this is what ReactNative suggests us to do.

you can get ip by typing ifconfig into the terminal and selecting the * inet value under en0:.

jsCodeLocation = [NSURL URLWithString:@"http://localhost:8081/index.ios.bundle"];

Save and click run again. This will fix the error.

Step 3 - Fix connecting to API hosted on local development server

Now the app is installed and you can navigate through the app screens. If the app attempts to make API call to the server running locally on the machine then depending on how the local server was started it could be a problem.

In my case, I have a Ruby on Rails application running on my local machine. I had started my rails server by executing command rails server.

To do this, the app should be able to access the rails server using the private ip of the server e.g. 192.168.x.x:3000. Turned out that I had started my rails server using command rails server.

Becase of change in Rails 4.2 rails server listens to localhost and not 0.0.0.0.

The rails guide further says following.

with this change you will no longer be able to access the Rails server from a different machine, for example if your development environment is in a virtual machine and you would like to access it from the host machine. In such cases, please start the server with rails server -b 0.0.0.0 to restore the old behavior.

In this case iPhone is trying to talk to Rails server so Rails server must be started using following command.

rails server -b 0.0.0.0
or
rails server --binding=0.0.0.0

By doing so, now you can connect to your Rails app from your local network, by browsing to http://192.168.x.x:3000.

Summary

These were some of the issues I came across while testing my iOS app on an actual device and how I fixed them. Hopefully, this blog post helps you fix these error cases get you started on testing quickly.

Using ReactJS with Rails Action Cable

Recent DHH, announced availability of alpha version of Action Cable.

Action Cable is still under heavy development but it comes with examples to demonstrate its usage.

Action Cable integrates websocket based real-time communication in Ruby on Rails applications. It allows building realtime applications like Chats, Status updates, etc.

Action Cable + React

Action Cable provides real time communication. ReactJS is a good tool to manage view complexity on the client side. Together they make it easy to develop snappy web applications which requires state management on the client side without too much work.

Anytime data changes the new data is instantly provided by Action Cable and the new data is shown on the view without user doing anything on the application by ReactJS.

Integrating React

The official Action Cable Example is a chat application. We will be building the same application using ReactJS.

First follow the instructions mentioned to get a working chat application using Action Cable.

Now that the chat application is working let’s get started with adding ReactJS to the application.

Please note that we have also posted a number of videos on learning ReactJS. Check them out if you are interested.

Step 1 - Add required gems to Gemfile

# react-rails isn't compatible yet with latest Sprockets.
# https://github.com/reactjs/react-rails/pull/322
gem 'react-rails', github: 'vipulnsward/react-rails', branch: 'sprockets-3-compat'

# Add support to use es6 based on top of babel, instead of using coffeescript
gem 'sprockets-es6'

Step 2 - Add required JavaScript files

Follow react-rails installation and run rails g react:install.

This will

  • create a components.js file.
  • create app/assets/javascripts/components/ directory.

Now put following lines in your application.js:

//= require react
//= require react_ujs
//= require components

Make sure your app/assets/javascripts/application.js looks like this

//= require jquery
//= require jquery_ujs
//= require turbolinks
//= require react
//= require react_ujs
//= require components
//= require cable

//= require channels
//= require_tree .

Step 3 - Setup Action Cable to start listening to events

We will be using es6, so lets replace the file app/assets/javascripts/channels/index.coffee, with app/assets/javascripts/channels/index.es6 and add following code.

var App = {};
App.cable = Cable.createConsumer('ws://localhost:28080');

Also remove file app/assets/javascripts/channels/comments.coffee, which is used to setup subscription. We will be doing this setup from our React Component.

Step 4 - Create CommentList React Component

Add following code to app/assets/javascripts/components/comments_list.js.jsx.

var CommentList = React.createClass({
  getInitialState(){
    let message = JSON.parse(this.props.message);
    return {message: message};
  },

  render() {
    let comments = this.state.message.comments.map((comment) => {
      return this.renderComment(comment);
    });

    return (
      <div>{comments}</div>
    );
  },

  renderComment(comment) {
    return (
      <article key={comment.id}>
        <h3>Comment by { comment.user.name }</h3>
        <p>{ comment.content }</p>
      </article>
    );
  }
});

Here we have defined a simple component to display a list of comments associated with a message. Message and associated comments are passed as props.

Step 5 - Create message JSON builder

Next we need to setup data needed to be passed to the component.

Add following code to app/views/messages/_message.json.jbuilder.

json.(message, :created_at, :updated_at, :title, :content, :id)
json.comments(message.comments) do |comment|
  json.extract! comment, :id, :content
  json.user do
    json.extract! comment.user, :id, :name
  end
end

This would push JSON data to our CommentList component.

Step 6 - Create Rails Views to display component

We now need to setup our views for Message and display of Comments.

We need form to create new Comments on messages. This already exists in app/views/comments/_new.html.erb and we will use it as is.

<%= form_for [ message, Comment.new ], remote: true do |form| %>
  <%= form.text_area :content, size: '100x20' %><br>
  <%= form.submit 'Post comment' %>
<% end %>

After creating comment we need to replace current form with new form, following view takes care of that.

From the file app/views/comments/create.js.erb delete the line containing following code. Please note that below line needs to be deleted.

$('#comments').append('<%=j render @comment %>');

We need to display the message details and render our component to display comments. Insert following code in app/views/messages/show.html.erb just before <%= render 'comments/comments', message: @message %>

<%= react_component 'CommentList', message: render(partial: 'messages/message.json', locals: {message: @message}) %>

After inserting the code would look like this.

<h1><%= @message.title %></h1>
<p><%= @message.content %></p>
<%= react_component 'CommentList', message: render(partial: 'messages/message.json', locals: {message: @message}) %>
<%= render 'comments/new', message: @message %>

Notice how we are rendering CommentList, based on Message json content from jbuilder view we created.

Step 7 - Setup Subscription to listen to Action Cable from React Component

To listen to new updates to comments, we need to setup subscription from Action Cable.

Add following code to CommentList component.

setupSubscription(){

  App.comments = App.cable.subscriptions.create("CommentsChannel", {
    message_id: this.state.message.id,

    connected: function () {
      setTimeout(() => this.perform('follow',
                                    { message_id: this.message_id}), 1000 );
    },

    received: function (data) {
      this.updateCommentList(data.comment);
    },

    updateCommentList: this.updateCommentList

    });
}

We need to also setup related AC Channel code on Rails end.

Make following code exists in app/channels/comments_channel.rb

class CommentsChannel < ApplicationCable::Channel
  def follow(data)
    stop_all_streams
    stream_from "messages:#{data['message_id'].to_i}:comments"
  end

  def unfollow
    stop_all_streams
  end
end

In our React Component, we use App.cable.subscriptions.create to create a new subscription for updates, and pass the channel we want to listen to. It accepts following methods for callback hooks.

  • connected: Subscription was connected successfully. Here we use perform method to call related action, and pass data to the method. perform('follow', {message_id: this.message_id}), 1000), calls CommentsChannel#follow(data).

  • received: We received new data notification from Rails. Here we take action to update our Component. We have passed updateCommentList: this.updateCommentList, which is a Component method that is called with data received from Rails.

Complete React Component

Here’s how our complete Component looks like.

var CommentList = React.createClass({
  getInitialState(){
    let message = JSON.parse(this.props.message);
    return {message: message};
  },

  render() {
    let comments = this.state.message.comments.map((comment) => {
      return this.renderComment(comment);
    });

    return (
        <div>{comments}</div>
    );
  },

  renderComment(comment) {
    return (
        <article key={comment.id}>
          <h3>Comment by { comment.user.name } </h3>
          <p>{ comment.content }</p>
        </article>
    );
  },

  componentDidMount() {
    this.setupSubscription();
  },

  updateCommentList(comment) {
    let message = JSON.parse(comment);
    this.setState({message: message});
  },

  setupSubscription() {

    App.comments = App.cable.subscriptions.create("CommentsChannel", {
      message_id: this.state.message.id,

      connected: function () {
        // Timeout here is needed to make sure Subscription
        // is setup properly, before we do any actions.
        setTimeout(() => this.perform('follow',
                                      {message_id: this.message_id}),
                                      1000);
      },

      received: function(data) {
        this.updateCommentList(data.comment);
      },

      updateCommentList: this.updateCommentList
    });
  }
});

Step 7 - Broadcast message when a new comment is created.

Our final piece is to broadcast new updates to message to the listeners, that have subscribed to the channel.

Add following code to app/jobs/message_relay_job.rb

class MessageRelayJob < ApplicationJob
  def perform(message)
    comment =  MessagesController.render(partial: 'messages/message',
                                         locals: {message: message})
    ActionCable.server.broadcast "messages:#{message.id}:comments",
                                 comment: comment
  end
end

which is then called from Comment model, like so-

Add this line to Comment model file app/model/comment.rb

after_commit { MessageRelayJob.perform_later(self.message) }

We are using message relay here, and will be getting rid of existing comment relay file - app/jobs/comment_relay_job.rb. We will also remove reference to CommentRelayJob from Comment model, since after_commit it now calls the MessageRelayJob.

Summary

Hopefully we have shown that Action Cable is going to be a good friend of ReactJS in future. Only time will tell.

Complete working example for Action Cable + ReactJS can be found here.

Verifying PubSub Services from Rails using Redis

Let’s say that we have a Service that reads and writes to Redis. We have BaseRedisService for managing connection, RedisWriterService to write to Redis and RedisReaderService to read from Redis.

require 'redis'

# Base class to manage connection
class BaseRedisService
 REDIS_HOST = '127.0.0.1'

 def connection
   if !(defined?(@@connection) && @@connection)
     @@connection =  Redis.new({ host: REDIS_HOST })
   end
   @@connection
 end

end
# Class to write to Redis
class RedisWriterService <  BaseRedisService
 attr_reader :key, :value

 def initialize key, value
   @key, @value = key, value
 end

 def process
   connection.set key, value
 end

end
# Class to read from Redis
class RedisReaderService < BaseRedisService
attr_reader :key

def initialize key
  @key = key
end

def process
  connection.get key
end

end

A test for the above mentioned case might look like this.

require 'test_helper'

class RedisPubSubServiceTest < ActiveSupport::TestCase

def test_writing_and_reading_value_using_redis
  value = 'Vipul A M'

  RedisWriterService.new('name', value).process
  assert_equal value, RedisReaderService.new('name').process
end

end

But now, we need to add PubSub to the service and verify that the service sends proper messages to Redis. For verifying from such a service, the service would need to listen to messages sent to Redis. Problem is that Redis listens in a loop. We would need to explicitly release control from listening and allow our tests to go ahead and verify some scenario.

Lets look at how these services would look.

class RedisPublisherService < BaseRedisService
attr_reader :options, :channel

def initialize channel, options
  @channel     = channel
  @options = options
end

def process
  connection.publish(options, channel)
end

end

and our Subscriber looks like this.

class RedisSubscriberService < BaseRedisService
attr_reader :channel

def initialize channel
  @channel = channel
end

def process
  connection.subscribe(channel) do |on|
    on.message do |channel, message|
      puts message
    end
  end
end

end

Notice that we don’t have control over returning value from the loop easily. Right now we just print on receiving a new message.

Now, lets start persisting our messages to some array in our Service. Also we will start exposing this from a thread variable so that it could be accessed from outside of execution of this listen loop.

class RedisSubscriberService < BaseRedisService
attr_reader :channel, :messages, :messages_count

def initialize channel, messages_count = 5
  @channel        = channel
  @messages       = []
  @messages_count = messages_count
end

def process
  connection.subscribe(channel) do |on|
    on.message do |channel, message|
      messages.unshift message
      Thread.current[:messages] = messages
    end
  end
end

end

We now have a way to access message state from the service to read any messages received by it. Lets say we define a new SubscriberService from a Thread, we could read messages like this.

subscriber = Thread.new { RedisSubscriberService.new('payment_alerts').process }
# Print first message from messages received
puts subscriber[:messages].first

Armed with this, we can now define a set of Rails helpers to use in our Rails tests.

module SubscriberHelpers
THREAD_PROCESS_TIMEOUT = 0.5

def setup_subscriber channel = 'test_channel'
  @subscriber = Thread.new { RedisSubscriberService.new(channel).process }
end

def teardown_subscriber
  @subscriber.kill
end

def with_subscriber
  yield
  @subscriber.join THREAD_PROCESS_TIMEOUT
end

def message_from_subscription
  @subscriber[:messages].first
end
end

Notice the with_subscriber method. It executes some code passed to it, then passes method execution to the subscriber process to read any messages sent and store onto messages store.

The count of the variable THREAD_PROCESS_TIMEOUT, can be experimented to set to a value that suites the system that’s being verified.

In our tests, we can now verify PubSub as-

require 'test_helper'

class RedisPubSubServiceTest < ActiveSupport::TestCase
include SubscriberHelpers

def setup
  setup_subscriber
end

def teardown
  teardown_subscriber
end

def test_writing_and_reading_back_values_from_pub_sub
  value = 'Vipul A M'

  with_subscriber do
    RedisPublisherService.new('test_channel', value).process
  end

  assert_equal value, message_from_subscription
end
end

Summary

We took a look at how PubSub based services can be verified by using threads and exposing messages from them, for verification. These can be tailored to support any similar PubSub service like Redis, and can be used to easily verify values being published from our services from Rails tests.

Voice based phone verification using twilio

In my previous blog post, I talked about how to do phone verification using SMS and now in this blog post I’m going to talk about how to do voice based phone verification using Twilio.

Requirement

Let’s change the requirement a bit this time.

  • After the user signs up, send an SMS verification. If SMS verification goes well then use that phone number for future use.
  • But if the user’s phone number doesn’t support the SMS feature, then call on user’s phone number for the verification.
  • In the call, ask the user to press 1 from keypad to complete the verification. If that user presses the 1 key, then mark the phone number as verified.

Step 1: Make a call for phone verification

We already handled SMS verification, so let’s add call related changes in PhoneVerificationService class.

class PhoneVerificationService
  attr_reader :user, :verification_through

  VERIFICATION_THROUGH_SMS  = :sms
  VERIFICATION_THROUGH_CALL = :call

  def initialize options
    @user                 = User.find(options[:user_id])
    @verification_through = options[:verification_through] || VERIFICATION_THROUGH_SMS
  end

  def process
    if verification_through == VERIFICATION_THROUGH_SMS
      perform_sms_verification
    else
      make_call
    end
  end

  private

  def from
    #phone number given by twilio
    Settings.twilio_number_for_app
  end

  def to
    "+1#{user.phone_number}"
  end

  def body
    "Please reply with this code '#{user.phone_verification_code}'" <<
    "to verify your phone number"
  end

  def send_sms
    Rails.logger.info "SMS: From: #{from} To: #{to} Body: \"#{body}\""

    twilio_client.account.messages.create ( from: from, to: to, body: body)
  end

  def make_call
    Rails.logger.info "Call: From: #{from} To: #{to}"

    twilio_client.account.calls.create( from: from, to: to, url: callback_url)
  end

  def perform_sms_verification
    begin
      send_sms
    rescue Twilio::REST::RequestError => e
      return make_call if e.message.include?('is not a mobile number')
      raise e.message
    end
  end

  def callback_url
    Rails.application.routes.url_helpers
      .phone_verifications_voice_url(host: Settings.app_host,
                                     verification_code: user.phone_verification_code)
  end

  def twilio_client
    @twilio ||= Twilio::REST::Client.new(Settings.twilio_account_sid,
                                         Settings.twilio_auth_token)
  end
end

In PhoneVerificationService class we have added some major changes:

  1. We have defined one more attribute reader verification_through and in initialize method we have set it.
  2. We have created two new constants VERIFICATION_THROUGH_SMS & VERIFICATION_THROUGH_CALL.
  3. We have changed process method and now we check which verification process should be taken place based on verification_through attribute.
  4. We also have added new methods perform_sms_verification, make_call and callback_url.

Let’s go through these newly added methods.

perform_sms_verification:

In this method, first we try to send an SMS verification. If SMS verification fails with error message like ‘is not a mobile number’ then in the rescue block, we make a phone verification call or we raise the error.

make_call:

In this method, we create an actual phone call and pass the all required info to twilio client object.

callback_url:

In this method, we set the callback_url which is required for Twilio for making a call. When we call the user for verification, our app needs to behave like a human and should ask to the user to press 1 to complete the verification (i.e. In the form of TwiML). This callback_url needs be to set in our app.

Step 2: Add voice action in phone verification controller

For callback_url, add route for voice action in config/routes.rb

post 'phone_verifications/voice' => 'phone_verifications#voice'

Add voice action and required code for it in PhoneVerificationsController.

class PhoneVerificationsController < ApplicationController
  skip_before_filter :verify_authenticity_token
  after_filter :set_header

  HUMAN_VOICE = 'alice'

  def voice
    verification_code = params[:verification_code]

    response = Twilio::TwiML::Response.new do |r|
      r.Gather numDigits: '1',
               action: "/phone_verifications/verify_from_voice?verification_code=#{verification_code}",
               method: 'post' do |g|

        g.Say 'Press 1 to verify your phone number.', voice: HUMAN_VOICE
      end
    end

    render_twiml response
  end

  def verify_from_message
    user = get_user_for_phone_verification
    user.mark_phone_as_verified! if user

    render nothing: true
  end

  private

  def get_user_for_phone_verification
    phone_verification_code = params['Body'].try(:strip)
    phone_number            = params['From'].gsub('+1', '')

    condition = { phone_verification_code: phone_verification_code,
                  phone_number: phone_number }

    User.unverified_phones.where(condition).first
  end

  def set_header
    response.headers["Content-Type"] = "text/xml"
  end

  def render_twiml(response)
    render text: response.text
  end
end

In this voice method, we have set up the Twilio response. When a call is made to the user, this response will get converted into robotic human voice. render_twiml method which sets twilio response in text form is required for Twilio APIs. Set up the response header in the set_header method which gets called in after_filter method.

Step 3: Set request URL for voice phone verification in Twilio

In voice action method, we have setup the request url in the action key of Twilio response object, that also needs to be set in your Twilio account. So when, user replies back to the call query, Twilio will make a request to our app and adds its own some values as parameters to the request. Using those parameters we handle the actual phone verification in the app.

Open twilio account and under NUMBERS section/tab, click on your Twilio number. Then in Voice section, add request URL with HTTP POST method. Add URL like this.

http://your.ngrok.com/phone_verifications/verify_from_voice/

We need Ngrok to expose local url to external world. Read more about it in my previous blog post.

Step 4: Add verify_from_voice action in phone verification controller

First add route for this action in config/routes.rb

post "phone_verifications/verify_from_voice" => "phone_verifications#verify_from_voice

Add this method, in your PhoneVerificationsController.

  def verify_from_voice
    response = Twilio::TwiML::Response.new do |r|
      if params['Digits'] == "1"
        user = get_user_for_phone_verification
        user.mark_phone_as_verified!

        r.Say 'Thank you. Your phone number has been verified successfully.',
               voice: HUMAN_VOICE
      else
        r.Say 'Sorry. Your phone number has not verified.',
               voice: HUMAN_VOICE
      end
    end

    render_twiml response
  end

Modify private method get_user_for_phone_verification to support voice verification in PhoneVerificationsController.

  def get_user_for_phone_verification
    if params['Called'].present?
      phone_verification_code = params['verification_code']
      phone_number            = params['To']
    else
      phone_verification_code = params['Body'].try(:strip)
      phone_number            = params['From']
    end
    condition = { phone_verification_code: phone_verification_code,
                  phone_number: phone_number.gsub('+1', '') }

    User.unverified_phones.where(condition).first
  end

In the verify_from_voice method, we get parameters Digits, To & verification_code from Twilio request. Using these parameters, we search for user in the database. If we get the proper user then we mark user’s phone number as verified phone number.

Gotcha with after_commit callback in Rails

after_commit callback in Rails is triggered after the end of transaction.

For eg. I have a Post model for which the number of lines of the content are calculated in the after_commit callback:

class Post < ActiveRecord::Base
  after_commit :calculate_total_lines, if: -> (post) { post.previous_changes.include?(:content) }

  def calculate_total_lines
    update! total_lines: content.split("\n").length
  end
end
post = Post.create! content: "Lets discuss Rails 5.\n", author: 'Prathamesh'
assert_equal 1, post.total_lines

Now lets wrap the creation of post inside a transaction block:

Post.transaction do
  post = Post.create! content: "Lets discuss Rails 5.\n", author: 'Prathamesh'
  assert_equal 1, post.total_lines
end

The test will fail now.

#   1) Failure:
# BugTest#test_within_transaction [after_commit_test.rb:45]:
# Expected: 1
#   Actual: nil

Why? Lets recall. after_commit callback will get executed after the end of transaction.

So until all the code inside transaction is completed, the callback is not going to get executed.

Here is a gist with complete test.

Next time you are using an after_commit callback and a transaction, make sure that code inside the transaction is not dependent on the result of the callback.

Blue border around JWPLAYER video

Latest versions of JWPlayer(6.9 onwards) adds blue border around the video when it is in focus.

JWPlayer blue border

This is because of the CSS class jwplayer-tab-focus.

The blue border around currently selected video allows to identify which instance of JWPlayer is in focus.

But with a single JWPlayer instance, it can be annoying.

To remove this blue border, we can override the default JWPlayer CSS as follows.

.jw-tab-focus:focus {
    outline: none;
}

To keep all the overridden CSS in once place, we can add this change in a separate file such as jwplayer_overrides.css.

Phone verification using SMS via Twilio

In this blog post, I’m going to show you how to do phone verification using SMS via Twilio. We will be using Twilio gem.

Requirement

  • When a user signs up , we want to send an SMS to that user with a random string to verify user’s phone number.
  • If that user replies back with that code, then verify user’s phone number in the app. Once the phone number is verified, then we can use that phone number for future use.

Step 1: Create required columns in users table

Let’s create phone_number, phone_verification_code and phone_verified columns in the users table.

 $ bin/rails generate migration AddPhoneAttributesToUsers phone_number:string phone_verification_code:string phone_verified:boolean

Then run the migration.

Step 2: Add phone number field in registration form

Add phone_number field in the registration form.

  <%= f.text_field :phone_number %>

Step 3: Send an SMS with verification code

When a user submits registration form, we need to send SMS if phone number is present. To handle this, add following code in User model.

class User < ActiveRecord::Base
  scope :unverified_phones,  -> { where(phone_verified: false) }

  before_save :set_phone_attributes, if: :phone_verification_needed?
  after_save :send_sms_for_phone_verification, if: :phone_verification_needed?

  def mark_phone_as_verified!
    update!(phone_verified: true, phone_verification_code: nil)
  end

  private

  def set_phone_attributes
    self.phone_verified = false
    self.phone_verification_code = generate_phone_verification_code

    # removes all white spaces, hyphens, and parenthesis
    self.phone_number.gsub!(/[\s\-\(\)]+/, '')
  end

  def send_sms_for_phone_verification
    PhoneVerificationService.new(user_id: id).process
  end

  def generate_phone_verification_code
    begin
     verification_code = SecureRandom.hex(3)
    end while self.class.exists?(phone_verification_code: verification_code)

    verification_code
  end

  def phone_verification_needed?
    phone_number.present? && phone_number_changed?
  end
end

We have added 2 major changes in the user model,

  • set_phone_attributes method is set in before_save callback.
  • send_sms_for_phone_verification method is set in after_save callback.

In set_phone_attributes method, we are setting up the phone attributes mainly sanitizing phone number and generating unique phone verification code. In send_sms_for_phone_verification method, we send SMS to the user. Creation of SMS for the phone verification message is handled in PhoneVerificationService class.

class PhoneVerificationService
  attr_reader :user

  def initialize(options)
    @user = User.find(options[:user_id])
  end

  def process
    send_sms
  end

  private

  def from
    # Add your twilio phone number (programmable phone number)
    Settings.twilio_number_for_app
  end

  def to
    # +1 is a country code for USA
    "+1#{user.phone_number}"
  end

  def body
    "Please reply with this code '#{user.phone_verification_code}' to
    verify your phone number"
  end

  def twilio_client
    # Pass your twilio account sid and auth token
    @twilio ||= Twilio::REST::Client.new(Settings.twilio_account_sid,
                                         Settings.twilio_auth_token)
  end

  def send_sms
    Rails.logger.info "SMS: From: #{from} To: #{to} Body: \"#{body}\""

    twilio_client.account.messages.create(
      from: from,
      to: to,
      body: body
    )
  end
end

In PhoneVerificationService class, we have defined user as an attribute reader and in initialize method, we have set the user object to it. Now if you see process method, it does lots of stuff for us.

Lets go through each method.

  • from - In this method, we set up the twilio phone number e.g. programmable number.
  • to - In this method, we set the phone number to which we want to send an SMS.
  • body - In this method, we build text message with verification code.
  • twilio_client - It creates twilio client based on twilio account sid and auth token.
  • send_sms - And last, it sends SMS to the user.

Step 4: Set request URL for phone verification in Twilio

As of now the system has the capability to send SMS to a user. Now we need to add capability to receive the SMS and to match the verification code.

First, we need to set up a request URL in Twilio account.

Open twilio account and under NUMBERS section/tab, click on your Twilio number. Then in Messaging section, add request URL with HTTP POST method. Add URL like this.

http://your.ngrok.com/phone_verifications/verify_from_message/

But to make it work, we need our Rails app on the public internet. There are two options for this:

  1. Deploy Rails app to your VPS or PaaS of choice.
  2. Use a tunneling service to take the server running on your development machine and make it available at an address on the public internet.

I’m going to use Ngrok tunneling service for the purpose of this blog. You can check this blog for more about its usage.

Step 5: Create phone verification controller

We need one more controller, which will handle phone verification when request comes from Twilio. When a user replies back with a verification code, it will trigger request URL through Twilio API. To handle that request, let’s add phone verifications controller.

  $ bin/rails generate controller phone_verifications

Add new route in config/routes.rb

post "phone_verifications/verify_from_message" => "phone_verifications#verify_from_message"

Add following code to the PhoneVerificationsController.

class PhoneVerificationsController < ApplicationController
  skip_before_action :verify_authenticity_token

  def verify_from_message
    user = get_user_for_phone_verification
    user.mark_phone_as_verified! if user

    render nothing: true
  end

  private

  def get_user_for_phone_verification
    phone_verification_code = params['Body'].try(:strip)
    phone_number            = params['From'].gsub('+1', '')

    condition = { phone_verification_code: phone_verification_code,
                  phone_number: phone_number }

    User.unverified_phones.where(condition).first
  end
end

In this controller, we added skip_before_action :verify_authenticity_token, this is because we need to allow twilio request in our Rails app which is actually an outside request (e.g. not secured request). This means we have disabled CSRF detection for this controller.

Now look at the verify_from_message method, in this method we take phone verification code and phone number from params hash. Using those data, we find the user from the database. Once we get the user, then we mark user’s phone number as a verified phone number.

Finally we are set to send business level text messages to the verified phone number.

This blog has more information about how to make secure twilio requests.

Author information in jekyll blog

BigBinary’s blog is powered by jekyll. In every blog we display author’s name, author’s twitter handle, author’s github id and author’s avatar. In this blog I’m going to discuss how we collect all that information in a simple manner.

We create a directory called _data in the root folder. This directory has a single file called authors.yml which in our case looks like this.

vipulnsward:
  name: Vipul
  avatar: http://bigbinary.com/assets/team/vipul.jpg
  github: vipulnsward
  twitter: vipulnsward
neerajdotname:
  name: Neeraj Singh
  avatar: http://bigbinary.com/assets/team/neeraj.jpg
  github: neerajdotname
  twitter: neerajdotname

We do not need to do anything to load authors.yml . It is automatically loaded by jekyll.

When we create a blog then the top of the blog looks like this.

---
layout: post
title: How to deploy jekyll site to heroku
categories: [Ruby]
author_github: neerajdotname
---

Notice the last line where we have put in the author’s github id. That’s the identifier we use to pull in author’s information.

In order to display author’s name we have following code in the layout.

<span class="author-name">
  {{ site.data.authors.[page.author_github].name }}
</span>

Similary to display author’s twitter handle and github id we have following code.

<a href="www.twitter.com/{{site.data.authors.[page.author_github].twitter}}">
  <i class="ico-twitter"></i>
</a>
<a href="www.github.com/{{site.data.authors.[page.author_github].github}}">
  <i class="ico-github"></i>
</a>

Now the blog will display the author information and all this information is nicely centralized in one single file.

How to obtain current time from a different timezone in Selenium IDE using javascript

I had encountered a problem in one of our projects where it was required to obtain the current time from a different timezone and then use it. I am in IST (Indian Standard Timezone - 5.50) and was required to get the time in EST (Eastern Standard Timezone + 5.00)

These are the set of commands I used to achieve that.

Command : storeEval
Target  : var d = new Date();
          d.setTime(new Date( (d.getTime() + (d.getTimezoneOffset() * 60000))            + (3600000 * (-5) ) ));
          d.toLocaleTimeString();
Value   : timeEst

Command : echo
Target  : ${timeEst}
Value   :

storeEval command is used to store the value in the variable timeEst after evaluating the javascript code.

var d = new Date(); a new date object is created.

d.getTime() method returns the number of milliseconds between midnight of January 1, 1970 and the specified date (as is present in the variable “d”).

d.getTimezoneOffset() method returns the time difference between UTC (Coordinated Universal Time) time and local time from the variable “d”, in minutes. For example, If our time zone is GMT-5.50, -330 will be returned.

Since getTime() is in milliseconds and getTimezoneOffset() is in minutes; getTimezoneOffset() is multiplied with 60000 to make it into milliseconds.

The expression 3600000 * (- 5) is required to make the time from UTC to EST. The difference between UTC and EST is -5 hours. So, to make the hours to milliseconds we need to multiply with 3600000 (60 min x 60 sec x 1000 millisec).

d.setTime() method is used to set the time of variable “d” after evaluating the expression.

With the above, the variable “d” is set with the date and time of the EST time.

d.toLocaleTimeString() method returns the time portion of a Date object as a string, using locale conventions.

The time string obtained above is stored in the variable timeEst.

echo ${timeEst}, the time string is displayed in the console.


Using est time

Selenkum javascript EST time


Using est date

Selenkum javascript EST date


2014 - Year in Community Engagement

At BigBinary our team loves to engage with the community as well as help out in different ways. We love contributing to OpenSource, speak and attend different conferences, and help out in community meetups, organizing conferences, and events like RailsGirls .

Short summary

In year 2014 we presented at following 8 conferences across 6 countries.

  • RubyConf Goa, India
  • RubyConf Philippines
  • RedDotRubyConf Singapore
  • DeccanRubyConf Pune, India
  • Madion+Ruby Wisconsin, USA
  • RubyConf Brazil
  • RubyKaigi Tokyo, Japan
  • Golden Gate Ruby Conference San Francisco, USA

Rails and Ruby Conferences

At the start of our travel, we visited RubyConfIndia 2014. The conference took place at an amazing beach resort in Goa. The two days of conference were full of fun and interactions with the best ruby people around India. During the conference our team announced launch of Ruby India to help spread ideas and experiments from the Ruby Indian Community, as well as highlight content from people.

Soon after, I visited Philippines, to conduct a workshop on “Contributing to Rails” at RubyConf Philippines. It was amazing to meet the growing Philippines community. I was happy to spend time with some amazing Rubyists, like Nick Sutterer, Zachary Scott, Konstantin Hasse, Koichi Sasada San, PJ Hagerty, Andre Arko and so on.

After that I and Prathamesh went to Singapore to speak at RedDotRubyConf. We spoke on Arel and ActiveRecord. “RedDorRubyConf” was our first joint talk together at a conference. Again we met a lot of awesome people like Piotr Solnica, Grzegorz Witek, Yinquan Teo, Sayanee Basu, Chinmay Pendharkar and Winston Teo Yong Wei. We also visited Marina Sand Bay and Sentosa Island.

Back in Pune, we hosted the first ever DeccanRubyConf. Our team was busy working on tasks right from building the website, inviting speakers, planning, and other arrangements. The conference had good talks and some really useful workshops. it was a fun one day conference, with attendance of over 170+ people.

The conference also saw our team announce the launch of RubyIndia Podcast, which does regular podcast interviews with notable people from the Ruby Community and Indian Community.

Prathamesh and I, then left on around a one and a half month travel, to attend and speak at multiple conferences.

We started with Madison+Ruby, in Madison, WI. After several missed flights, and a storm, we visited our first US conference after a travel of 48 hours. MadisonRuby was a conference like no other. Several topics touched the humane side of Ruby and the community. We spoke on ‘Building an own ORM using ARel’. Set in the cultural town of Madison, we immensely enjoyed the cheese-curds, farmers markets and game night arranged by the Conf team. A huge thanks to Jim and Jennifer Remsik, for hosting such an amazing event. Thanks also to Scott Ruttencutter for giving us space to work from his office and giving us a tour of the state capital.

We then visited Sao Paulo, Brazil for RubyConf Brazil and presented a talk on Building an ORM using ARel. It was a pleasure to meet Fabio Akita and the CodeMiner team. We made friends with Celso Fernandes and Paulo who were kind enough to help us around, since in Brazil primarily portuguese is spoken. We also met Rafael Franca and Carlos Antonio da Silva who have helped us a lot with Rails issue tracker.

Next Prathamesh headed to RubyKaigi, being held in Tokyo, Japan. He presented on Fixtures in Rails. He met Matz, creator of Ruby on his first day in Japan. Mostly all the core Ruby contributors attended RubyKaigi. He got to interact with Koichi Sasada San, Jonan Scheffler, Akira Matsuda, Godfrey Chan, Richard Schneeman and lot of awesome Rubyists. He also met with his JRuby Core Tom Enebo for the first time. Thanks to Yasuo Honda, Michael Reinsch for helping with Japanese food.

From Brazil, I first visited Miami, and was happy to visit The Lab Miami, WynCode and interact with Rubyists from Miami.

Before heading to San Francisco, for GoGaRuco, I was able to make a quick stop in Boston and visit AlterConf Boston. The theme of the conference was around diversity in tech and gaming industry.

My latest conference was in the amazing city on San Francisco. I presented about ‘Building an ORM’, at GoGaRuCo, which incidentally was the last ever GoGaRuCo. The conference taking place in San Francisco, saw an amazing turnout of crowd. I was able to interact with Sarah Allen, Yehuda Katz, Pat Allen, Sarah Mei, Ryan Davis. I spent most of the time along with Nathan Long, Randy Coulman, and Nathan’s friend Michael Gundlach, who is the creator of popular plugin Adblock. I also ran into Chris Eppstein, creator of compass. All around it was one of the most amazing interactions I had in a conference.

2014, was an amazing year for our team. Together we presented or were part of 8 conferences, launched RubyIndia Newsletter as well as the RubyIndia Podcast, started with 6 new video series on topics from ReactJS to Rubymotion to Selenium, published numerous blogs, and contributed to a number of OpenSource projects.

2015, starts with our team presenting at GardenCityRuby Conf. We hope to get more such chances to interact and help out the community. Onwards to a new year!

Migrating existing session cookies while upgrading to Rails 4.1 and above

Rails 4.1 introduced JSON serialization for cookies. Earlier all the cookies were serialized using Marshal library of Ruby. The marshalling of cookies can be unsafe because of the possibility of remote code execution vulnerability. So the change to :json is welcoming.

The new applications created with Rails 4.1 or 4.2 have :json as the default cookies serializer.

rake rails:update used for upgrading existing Rails apps to new versions rightly changes the serializer to :json.

Rails.application.config.action_dispatch.cookies_serializer = :json

Deserialization error

However that change can introduce an issue in the application.

Consider a scenario where the cookies are being used for session storage. Like many normal Rails apps, the current_user_id is being stored into the session.

session[:user_id] = current_user_id

Before Rails 4.1 the cookie will be handled by Marshal serializer.

cookie = Marshal.dump(current_user_id) # 42 => "\x04\bi/"
Marshal.load(cookie) # "\x04\bi/" => "42"

After the upgrade the application will try to unserialize cookies using JSON which were serialized using Marshal.

JSON.parse cookie # Earlier dumped using Marshal
# JSON::ParserError: 757: unexpected token at i/'

So the deserialization of the existing cookies will fail and users will start getting errors.

Hybrid comes to rescue

To prevent this Rails provides with a hybrid serializer. The hybrid serializer deserializes marshalled cookies and stores them in JSON format for the next use. All the new cookies will be serialized in the JSON format. This gives happy path for migrating existing marshaled cookies to new Rails versions like 4.1 and 4.2.

To use this hybrid serializer, set cookies_serializer config as :hybrid as follows:

Rails.application.config.action_dispatch.cookies_serializer = :hybrid

After this, all the existing marshalled cookies will be migrated to :json format properly and in the future upgrade of Rails, you can safely change the config from :hybrid to :json which is the default and safe value of this config.

Rails Girls Pune 2014

The second edition of RailsGirls Pune event was an amazing day spent with some equally amazing folks. The event took place on 13th of December and it saw a huge turnout of around 150+ women from various colleges and companies. It was a free event for beginners interested in learning about coding and building applications using Ruby on Rails.

BigBinary was happy to be one of the sponsors of the event.

The event was organized by Rajashree Malvade, Shifa Khan, Pooja Salpekar, Dominika Stempniewicz, and Magdalena Sitarek.

BigBinary team reached the venue, ThoughtWorks office, Pune, at about 8.30 AM. Rajashree did the introductions and Gautam Rege, and I did the kick off. Gautam introduced Ruby, along with how magical Ruby is and the importance of the event. I spent some time explaining how RailsGirls began, as well as Rails Bridge and other similar events.

RailsGirls pune kick-off

Next all instructors were grouped together. Grouping was done in such a way that advanced instructors were paired with intermediate and beginner instructors.

The talented folks from ThoughtWorks had created a fun movie explaining the three different tracks - beginner, intermediate and advanced into which the students were divided.

I, Prathamesh and Richa Trivedi took to one of the advanced track groups. We started off by pairing people to work with their partner and did a health check of everyones’ system. Many of the participants in our group had 1-2 years of professional experience in Java, .Net and so forth. This meant they were quite familiar with setting up various things on their machine and that was a great help. We started with basics of ruby- variables, loops, blocks, each, methods, classes, etc. This took about 2 hours and then we started with Rails and MVC.

Santosh paired with Dinesh and participated in intermediate track of a group of four students. They started with basics of Ruby and later started to build simple blog app using Rails and deployed apps to Heroku by the end of the day.

At about 11.30, Siddhant Chothe from TechVision, did an inspiring talk about Web Accessibility, Wiable gem, and his journey in Ruby & Rails world.

Siddhant's session

Then We did the Bentobox activity. Participants were handed a page listing various aspects of software development like infrastructure, frontend, application, storage in boxes. We read out technologies like XML, JSON, AJAX, MongoDB, etc, and asked everyone to write these on stickies and place them in appropriate boxes on the “Bentobox”. This was helpful for the participants to understand what different technologies are related to web-development and where they are used.

Then everyone broke out for lunch. Our enthusiastic lot stayed back to avoid the rush, and began with Rails development. We started by explaining basic MVC concepts and how Rails helps as a framework. We started with a simple App, and created “pages/home” static home page. This helped our group to understand rails generators, routes, controllers and views. With our first page up and running, we went for lunch.

After lunch, a session on Origami was conducted by Nima Mankar. It was a good stress buster after all of the first session’s bombardment over the participants.

Origami session

Our next objective was to build an app and deploy it to heroku. Our group started out to build “The Cat App”! We began with explaining controllers, CRUD operations, parts of URL, REST, etc. We created a Cat model, and everyone loved the beauty and simplicity of migrations and performing create, update, delete, find using ActiveRecord. We quickly moved on to building CatsController and CRUD operations on the same. We made sure we did not use scaffold, so as to explain the underlying magic, instead of scaffold hiding it away.

Richa with our group

Other part of our group

Soon everyone had a functional App, and it was fun to introduce GorbyPuff as the star of our App, whose images were displayed as cat records, which store name of the image and url to an image.

We then setup the Apps on heroku and were ready for the next part- Showcase. It was amazing to see so many groups complete the Apps and come up with fun, interesting and quirky ideas. One student created Boyfriend Expense (Kharcha) Management App.

App showcase

The day ended on a high note amid high enthusiasm from all the participants. We finished the workshop with a huge cake for everyone.

Overall, it was well-organized, fun, enthusiastic and a day well spent.

Thanks to Rajshree, Shifa, Pooja, Dominika and Magdalena for organizing such an awesome event. Both Pune edition events, have shown great interest, and it has left us all looking forward for the next one!

DRYing up Rails Views with View Carriers and Services

Anyone who has done any Rails development knows that views gets complicated very fast. Lately we’ve been experimenting with Carriers ( also called view services ) to clean up views.

There are many already solutions already existing to this problem like Draper Gem , or Cells from Trailblazer architecture.

We wanted to start with simplest solution by making use of simple ruby objects that takes care of business logic away from views.

A complex view

Consider this user model

class User
  def super_admin?
    self.role == 'super_admin'
  end

  def manager?
    self.role == 'manager'
  end
end

Here we have a view that displays appropriate profile link and changes css class based on the role of the user.

<% if @user.super_admin? %>
  <%= link_to 'All Profiles', profiles_path %>
<% elsif @user.manager? %>
  <%= link_to 'Manager Profile', manager_profile_path %>
<% end %>

<h3 class="<%= if @user.manager?
                  'hidden'
               elsif @user.super_admin?
                  'active'
               end %>">

   Hello, <%= @user.name %>
</h3>

Extracting logic to Rails helper

In the above case we can extract the logic from the view to a helper.

After the extraction the code might look like this

# app/helpers/users_helper.rb

module UsersHelper

  def class_for_user user
    if @user.manager?
      'hidden'
    elsif @user.super_admin?
      'active'
    end
  end

end

Now the view is much simpler.

<h3 class="<%= class_for_user(@user) %>">
   Hello, <%= @user.name %>
</h3>

Why not to use Rails helpers?

Above solution worked. However in a large Rails application it will start creating problems.

UsersHelper is a module and it is mixed into ApplicationHelper. So if the Rails project has large number of helpers then all of them are mixed into the ApplicationHelper and sometimes there is a name collision. For example let’s say that there is another helper called ShowingHelper and this helper also has method class_for_user. Now ApplicationHelper is mixing in both modules UsersHelper and ShowingHelper. One of those methods will be overridden and we would not even know about it.

Another issue is that all the helpers are modules not classes. Because they are not classes it becomes difficult to refactor helpers later. If a module has 5 methods and if we refactor two of the methods into two separate methods then we end up with seven methods. Now out of those seven methods in the helper only five of them should be public and the rest two should be private. However since all the helpers are modules it is very hard to see which of them are public and which of them are private.

And lastly writing tests for helpers is possible but testing a module directly feel weird since most of the time we test a class.

Carriers

Lets take a look at how we can extract the view logic using carriers.

class UserCarrier
  attr_reader :user

  def initialize user
    @user = user
  end

  def user_message_style_class
    if user.manager?
      'hidden'
    elsif user.super_admin?
      'active'
    end
  end
end

In our controller

  class UserController < ApplicationController
    def show
      @user = User.find(params[:id])
      @user_carrier = UserCarrier.new @user
    end
  end

Now the view looks like this

<% if @user.super_admin? %>
  <%= link_to 'All Profiles', profiles_path %>
<% elsif @user.manager?%>
  <%= link_to 'Manager Profile', manager_profile_path %>
<% end %>

<h3 class="<%= @user_carrier.user_message_style_class %>">
  Hello, <%= @user.name %>
</h3>

No html markup in the carriers

Even though carriers are used for presentation we stay away from having any html markup in our carriers. That is because once we open the door to having html markups in our carriers then carriers quickly get complicated and it becomes harder to test them.

No link_to in the carriers

Since carriers are plain ruby objects, there is no link_to and other helper methods usually. And we keep carriers that way. We do not do include ActionView::Helpers::UrlHelper because the job of the carrier is to present the data that can be used in link_to and complement the usage of link_to.

We believe that link_to belongs to the ERB file. However if we really need to have an abstraction over it then we can create a regular Rails helper method. We minimize usage of Rails helper, we do not avoid it altogether.

Overcoming Double Dots

Many times in our view we end up doing

  Email Preference for Tuesday: <%= @user.email_preferences.tuesday_preference %>

This is a violation of Law of Demeter . We call it “don’t use Double Dots”. Meaning don’t do @article.publisher.full_name.

Its just a matter of time before views code looks like this

  <%= @article.publisher.active.not_overdue.try(:full_name) %>

Since carriers encapsulate objects into classes, we can overcome this “double dots” issue by delegating behavior to appropriate object.

class UserCarrier
  attr_reader :user :email_preferences

  delegate :tuesday_preference, to: :email_preferences

  def initialize user
    @user = user
    @email_preferences = user.email_preferences
  end

end

After that refactoring we end up with cleaner views like.

  Email Preference for Tuesday: <%= @user_carrier.tuesday_preference %>

Note that “Double dot” is allowed at other parts of the code. We do not allow it in views.

Testing

Since the carriers are simple ruby objects it’s easy to test them.

require 'test_helper'

class UserCarrierTest < ActiveSupport::TestCase
  fixture :users

  def setup
    manager = users(:manager)
    @user_carrier = UserCarrier.new manager
  end

  def test_css_class_returned_for_manager
    assert_equal 'hidden', @user_carrier.user_message_style_class
  end

end

Summary

Carriers allow us to encapsulate complex business logic in simple ruby objects.

This helps us achieve clearer separation of concern, clean up our views and avoid skewed and complex views. Our views are free of “double dots” and we end up with simple tests which are easy to maintain.

We decided to call it a “carrier” and not “presenter” because the word “presenter” is overloaded and has many meanings.

We at BigBinary take a similar approach for extracting code from a fat controller or a fat model. You can find out more about it here.

Make Outbound Calls From the Browser to Phone using Twilio and Rails

In this blog post we will see how to make outbound phone calls from the browser to a phone using Twilio . We will make use of the Twilio-JS library and Twilio-Ruby gem.

The Rails App we will be creating, is based on the Twilio Client Quick-start tutorial. That Twilio tutorial makes use of Sinatra. We will see how we can achieve this in a Rails application.

Step 1 - Setup Twilio Credentials and TwiML App

We need to setup twilio credentials. We can find account ID and auth token from our account information.

When the call is made using browser then the phone that is receiving the call has to see a number from which the call is coming. So now we need to setup a Twilio verified number. This number will be used to place the outgoing calls from. How to setup a verified number can be found here.

When our app make a call from the browser using twilio-js client, Twilio first creates a new call connection from our Browser to Twilio. It then sends a request back to our server to get information about what to do next. We can respond by asking twilio to call a number, say something to the person after a call is connected, record a call etc.

Sending of this instructions is controlled by setting up a TwiML application. This application provides information about the end point on our server, where twilio should send the request to fetch instructions. TwiML is a set of instructions, that we can use to tell Twilio what to do in different cases like when an outbound phone call is made or when an inbound SMS message is received.

Given below is an example that will say a short message How are you today? in a call.

<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <Say voice="woman">How are you today?</Say>
</Response>

The TwiML app can be created here. Once the app is configured then we will get appsid.

We need to configure following information in our Rails Application:

twilio:
  verified_number: <%= ENV['TWILIO_VERIFIED_NUMBER']%>
  account_sid: <%= ENV['TWILIO_ACCOUNT_SID'] %>
  auth_token: <%= ENV['TWILIO_AUTH_TOKEN'] %>
  call_app_sid: <%= ENV['TWILIO_CALL_APP_SID'] %>

Step 2 - Generate capability token to be used by twilio-js

After we have the config setup, we will proceed to create the capability token. This token will be generated using the ruby gem, and passed to the javascript SDK. The token helps the twilio-js client determine, what permissions the application has like making calls, accepting calls, sending SMS, etc.

We define a TwilioTokenGeneratorService for this purpose.

class TwilioTokenGeneratorService

  def process
    capability = twilio_capability()
    capability.generate
  end

  private

  def twilio_capability
    capability ||= Twilio::Util::Capability.new Settings.twilio.account_sid, Settings.twilio.auth_token
    capability.allow_client_outgoing Settings.twilio.call_app_sid
    capability
  end

end

As you can see, we first define a new Twilio::Util::Capability instance and pass credentials to it. We then call allow_client_outgoing method and pass the client Sid to it. This is the identifier for the TwiML application we have previously created on Twilio. Calling allow_client_outgoing gives permission to the client to make outbound calls from Twilio. Finally we call the generate method to create a token from the capability object.

Step 3 - Define view elements and pass token to it

The generated token will now be passed to the Twilio JS client for connecting with Twilio. In our App we define CallsController, and index action in this controller. This action takes care of setting the capability token. Our index view consists of two buttons- to place and hangup a call, a number input field, call logs, and data field to pass capability token to the javascript bindings. We import the Twilio-JS library in the view. The css styling being used is from the Twilio example.

<div id="twilioToken" data-token="<%= @twilio_token %>"></div>

<button id="caller" class="call">Call</button>
<button id="hangup" class="hangup">Hangup</button>

<input type="text" id="phoneNumber" placeholder="Enter a phone number to call"/>

<div id="log">Loading pigeons...</div>

<script type="text/javascript" src="//static.twilio.com/libs/twiliojs/1.2/twilio.min.js"></script>

Step 4 - Define coffeescript bindings to handle TwilioDevice connection to Twilio

Next we setup coffeescript bindings to handle initialization of TwilioDevice and making use of entered number to place calls Twilio. We are taking care of various events like connect, disconnect, ready, etc. on TwilioDevice instance. More information about TwilioDevice usage can be found here.

class TwilioDevice
  constructor: ->
    @initTwilioDeviceBindings()
    @initFormBindings()

  initTwilioDeviceBindings: ->
    twilio_token = $('#twilioToken').data('token')
    twilio_device = Twilio.Device

    # Create the Client with a Capability Token
    twilio_device.setup(twilio_token, {debug: true});

    #/* Let us know when the client is ready. */
    twilio_device.ready ->
      $("#log").text("Ready")

    #/* Report any errors on the screen */
    twilio_device.error (error) ->
      $("#log").text("Error: " + error.message)

    #/* Log a message when a call connects. */
    twilio_device.connect (conn) ->
      $("#log").text("Successfully established call")

    #/* Log a message when a call disconnects. */
    twilio_device.disconnect (conn) ->
      $("#log").text("Call ended")

  initFormBindings: ->
    $('#caller').bind "click", (event) ->
      params = {"phone_number": $('#phoneNumber').val()}
      Twilio.Device.connect(params)

    $('#hangup').bind "click", (event) ->
      Twilio.Device.disconnectAll()

$ ->
  new TwilioDevice()

If we now load this page, we should be able to see our app saying its ready to take calls.

Twilio Rails App Ready

Step 5 - Define TwiML Response Generator Service

The final step before we place calls from our App is to handle callbacks from Twilio and return TwiML response. For this we are going to define TwilioCallTwiMLGeneratorService which takes care of generating this response. More information about how we need to define the response and individual fields can be found from Twilio’s docs.

What we need to define is a response as below:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Dial callerId="+15005550000">
    <Number>+15005550001</Number>
  </Dial>
</Response>

We are making use of two elements here - Dial, which makes Twilio place a call using defined callerId value, as the number from which the call is made, which is displayed on the callee’s phone. Note that this is the same verified number that we had specified before. Then we specify Number which is the number, to which we want to place the call to. This number is passed first by the javascript client to Twilio, and then back to our application by Twilio, which we use to generate the response as above.

We define our TwilioCallTwiMLGeneratorService to take in a phone number as parameter. It creates an instance of Twilio::TwiML::Response and tapping on this instance we provide Dial element with a :callerId value, and the Number to place the call to. We validate the number before passing it back, and return an error if the number is invalid.

class TwilioCallTwiMLGeneratorService
  attr_reader :phone_number
  VALID_PHONE_NUMBER_REGEX = /^[\d\+\-\(\) ]+$/ # Matches valid phone numbers acceptable to Twilio

  def initialize phone_number
    @phone_number = phone_number
  end

  def process
    Twilio::TwiML::Response.new do |r|
      if VALID_PHONE_NUMBER_REGEX.match(phone_number)
        r.Dial :callerId => Settings.twilio.verified_number do |d| # callerId is number from which call is made.
          d.Number(CGI::escapeHTML phone_number) # The number to call
        end
      else
        r.Error("Invalid number!")
      end
    end.text.strip
  end

end

Step 6 - Send TwiML response on Twilio callback

We are now set to define twilio’s callback handler. This will be handled by the create_call action in CallsController. Twilio will be sending this endpoint a POST request along with some information specified here. We make use of phone_number being passed to us by Twilio and pass it along to the TwilioCallTwiMLGeneratorService, which return us with valid TwiML response. Since TwiML is a flavor of XML, we make using render xml to return the response.

def create_call
  response_to_twilio_callback = TwilioCallTwiMLGeneratorService.new(call_params[:phone_number]).process
  render xml: response_to_twilio_callback
end

def call_params
  params.permit(:phone_number)
end

As create_call endpoint will be used by Twilio API, we need to skip authenticity token check for this action.

class CallsController < ApplicationController
  skip_before_action :verify_authenticity_token, only: [:create_call]
end

Finally we need to specify the callback url in our TwiML App on Twilio. For testing this locally, we can make use of a service like https://ngrok.com/, to expose this endpoint.

Twilio TwiML App setting

Our service is now ready to place calls. The complete Rails application code that we have created can be found here.

Happy calling everyone!

Tricks and Tips for using Fixtures effectively in Rails

Recently I gave a talk at RubyKaigi 2014 about Rails fixtures. In this blog post I will be discussing some of the tips and tricks for using fixtures effectively.

You can also the see the video of this talk here.

Don’t use ids.. unless required

In fixtures, we can specify id of a fixture.

john:
  id: 1
  email: john@example.com

I would recommend not to specify the id. Rails generates the id automatically if we don’t explicitly specify it. Moreover there are a few more advantages of not specifying the id.

Stable ids for every fixture.

Rails will generate the id based on key name. It will ensure that the id is unique for every fixture. It can also generate ids for uuid primary keys.

Labeled references for associations like belongs_to, has_many.

Lets say we have a users table. And a user has many cars.

Car ferrari belongs to john. So we have mentioned user_id as 1.

ferrari:
  name: ferrari
  make: 2014
  user_id: 1

When I’m looking at cars.yml I see user_id as 1. But now I to lookup to see which user has id as 1.

Here is another implementation.

john:
  name: john
  email: john@example.com

ferrari:
  name: ferrari
  make: 2014
  user: john

Notice that I no longer specify user_id for John. I have mentioned name. And now I can reference that name in cars.yml to mention that ferrari belongs to john.

How to set a value to nil from fixture

Let’s say that I have a boolean column which is false by default. But for an edge case, I want it to be nil. I can obviously mutate the data generated by fixture before testing. However I can achieve this in fixtures also.

Specify null to make the value nil

require 'yaml'
YAML.load "--- \n:private: null\n")
=> {:private=>nil}

As you can see above if the value is null then YAML will treat it as nil.

john:
  name: john
  email: john@example.com
  private: null

Leave the value blank to make the value nil

require 'yaml'
YAML.load "--- \n:private: \n")
=> {:private=>nil}

As you can see above if the value is blank then YAML will treat it as nil.

john:
  name: john
  email: john@example.com
  private:

When model name and table name does not match

Generally in Rails, the model name and table name follow a strict convention. Post model will have table name posts. Using this convention, the fixture file for Post models is obviously fixtures/posts.yml.

But sometimes models do not match directly with the table name. This could be because of legacy reason or because of namespacing of models. In such cases automatic detection of fixture files becomes difficult.

Rails provides set_fixture_class method for this purpose. This is a class method which accepts a hash where key should be name of the fixture or relative path to fixture file and value should be model class.

I can use this method inside test_helper.rb in any class inheriting from ActiveSupport::TestCase.

# test_helper.rb
class ActiveSupport::TestCase

  # table name is "morning_appts". It is being mapped to model "MorningAppointment".
  self.set_fixture_class morning_appts: MorningAppointment

  # in this case fixture is namespaced
  self.set_fixture_class '/legacy/users' => User

  # in this case the model is namespaced.
  self.set_fixture_class outdoor_games: Legacy::OutdoorGame
end

values interpolation using $LABEL

Rails provides many ways to keep our fixtures DRY. Label interpolation is one of them. It allows the use of key of fixture as a value in the fixture. For example:

john:
  name: john
  email: john@example.com

becomes:

john:
  name: $LABEL
  email: john@example.com

$LABEL is not a global variable here. Its just a placeholder. $LABEL is replaced by the key of the fixture. And as discussed earlier the key of the fixture in this case is john. So $LABLE has value john.

Before this PR, I could only use this feature if the value is exactly $LABEL. So if the email is john@example.com I could not use the $LABEL@example.com. But after this PR, I can $LABEL anywhere in the string, and Rails will replace it with the key.

So the earlier example becomes:

john:
  name: $LABEL
  email: $LABEL@example.com

YAML defaults

I use YAML defaults in database.yml for drying it up and keeping common configuration at one place.

defaults: &defaults
  adapter: postgresql
  encoding: utf8
  pool: 5
  host: localhost
  password:

development:
  <<: *defaults
  database: wheel_development

test:
  <<: *defaults
  database: wheel_test

production:
  <<: *defaults
  database: wheel_production

I can use it for drying up fixtures too for extracting common part in our fixtures.

DEFAULTS: &DEFAULTS
  company: BigBinary
  website: bigbinary.com
  blog: blog.bigbinary.com

john:
  <<: *DEFAULTS
  name: John Smith
  email: john@bigbinary.com

prathamesh:
  <<: *DEFAULTS
  name: Prathamesh Sonpatki
  email: prathamesh@bigbinary.com

Note the usage of key DEFAULTS for defining default fixture. Rails will automatically ignore any fixture with key DEFAULTS.

If we use any other key then a record with that key will also get inserted in the database.

Database specific tricks

Fixtures bypass the normal Active Record object creation process. After reading them from YAML file, they are inserted into database directly using insert query. So they skip callbacks and validations check. This also has an interesting side-effect which can be used for drying up fixtures.

Suppose we have fixture with timestamp:

john:
  name: John Smith
  email: john@example.com
  last_active_at: <%= Time.now %>

If I are using PostgreSQL, I can replace the last_active_at value with now:

john:
  name: John Smith
  email: john@example.com
  last_active_at: now

now is not a keyword here. It is just a string. The actual query looks like this:

INSERT INTO "users"
("name", "email", "last_active_at", "id")
VALUES
('John Smith', 'john@example.com', 'now',1144934)

So the value for last_active_at is still just now when the query is executed.

The magic starts as PostgreSQL starts reading the values. now is a shorthand for the current timestamp . As soon as Postgres reads it, it replaces now with the current timestamp and the column last_active_at gets populated with current timestamp.

I can also use the now() function instead of just now.

This function is available in PostgreSQL as well as MySQL. So the usage of now() works in both of these databases.

Selenium IDE - How I improved performance by reducing running time from 58 minutes to 15 minutes of selenium test

I wrote a bunch of selenium tests using Selenium IDE for a project. The selenium tests have proven to be very useful. However the tests take around 58 minutes to complete the full run.

Here are the specific steps I took which brought the running time to under 15 minutes.

Set to run at the maximum speed

Command Target Value
setSpeed 0

setSpeed command takes Target value in milliseconds. By setting the value to zero, I set the speed to maximum and the tests indeed ran fast. However, now I had lots of tests failing which were previously passing.

What happened.

In our tests real firefox browser is fired up and real elements are clicked. The application does make round trip to the rails server hosted on heroku.

By setting the selenium tests to the maximum speed the selenium tests started asserting for elements on the page even before the pages were fully loaded by the browser.

I needed sets of instructions using which I could tell selenium how long to wait for before asserting for elements.

Selenium provides a wonderful suite of commands which helped me fine tune the test run. Here I’m discussing some of those commands.

waitForVisible

This command is used to tell selenium to wait until the specified element is visible on the page.

In the below mentioned case, the Selenium IDE will wait until the element css=#text-a is visible on the page.

Command Target Value
waitForVisible css=#text-a

waitForText

This command is used to tell selenium to wait until a particular text is visible in the specified element.

In the case mentioned below, Selenium IDE will wait until the text violet is displayed in the element css=#text-a.

Command Target Value
waitForText css=#text-a violet


The difference between waitForVisible and waitForText is that waitForVisible waits until the specified element is visible on the page while waitForText waits until a particular text is visible in the specified element on the page.

waitForElementPresent

This command is used to tell Selenium to wait until the specified element is displayed on the page.

In the below mentioned case, the Selenium IDE will wait until the element css=a.button is displayed on the page.

Command Target Value
waitForElementPresent css=a.button

waitForVisible and waitForElementPresent seem very similar. It seems both of these commands do the same thing. There is a subtle difference though.

waitForVisible waits until the specified element is visible. Visibility of an element is manipulated by the settings of CSS properties. For example using display none; one can make an element not be visible at all.

In contrast the command waitForElementPresent waits until the specified element is present on the page in the form of html markup. This command does not give consideration to css settings.

refreshAndWait

This command is used to tell Selenium to wait until the page is refreshed and the targeted element is displayed on the web page.

In the example mentioned below, the Selenium IDE will wait until the page is refreshed and the targeted element css=span.button is displayed on the page.

Command Target Value
refreshAndWait css=span.button

clickAndWait

This command is used to tell selenium to wait until a particular button is clicked for submitting the form and the page starts reloading. The subsequent commands are paused until, the page is reloaded after the element is clicked on the page.

In the case mentioned below, Selenium IDE will wait until the page is reloaded after the specified element css=input#edit is clicked.

Command Target Value
clickAndWait css=input#edit

Selenium IDE commands used above and more are available at Selenium documentation .

DeccanRubyConf 2014. Hou De!

deccanrubyconflogo

I attended it, enjoyed it and took part as a volunteer in Pune’s first RubyConf - DeccanRubyConf 2014. As Hou De (let it be) name said, event went in the same way.

The day before conference, in the morning Vipul (one of the event organizers) and I picked up our guest speaker Koichi Sasada from the Pune Airport. Koichi is a Ruby core member and works for Heroku. We welcomed him and went to Hyatt Regency hotel where the event is taking place. Our guest checked into hotel and then we decided to go for a lunch at Malaka Spice restaurant.

We reached there and Koichi told us that he wanted non-spicy food (Safe food). We ordered non spicy food, but food was still too spicy for Koichi. However we enjoyed the food and had very good discussion about the Ruby internals, concurrency-parallelism, debugging in Ruby, Japanese culture and the Indian culture.

koichi & vipul

After lunch, we dropped Koichi off at the hotel and we left for our home.

Next morning was the event day and I woke up early and went to the event place. As part of volunteering team, I and other volunteers had tasks like giving pens, badges, stickers, T-shirts and coupon for night party to the attendees.

Attendees had started to come in slowly. Some attendees asked me about T-shirt size as I wore one of the conference T-shirts and from my T-shirt’s size they decided their T-shirt size. It was a great experience meeting with different kind of people from around the India.

Keynote by Koichi kicked the event off and he talked about Ruby 2.1 features like,

  • Required keyword parameter,
  • Rational number literal,
  • def returns symbol of method name
  • Runtime new features (String#scrub, Binding#local_time and etc.)

Then he talked about performance improvements, Ruby 2.2 and how to speed up Ruby Interpreter . Click here for more details about his talk.

koichi

attendees

In between the talks, some new attendees had come for the conference who had not registered for the conference. They told me that they thought it’s a regular Pune’s local Ruby Meetup. There was some miss-understandings but they seemed interested in attending the event. I contacted Gautam as he was one of the organizers and told him about the issue.

Attendees kept coming till the afternoon.

After Koichi’s talk, two sections had opened. One for talks and other for the workshops. TDD workshop was conducted by Sidu Ponnappa. I saw lots of attendees in this workshop and heard that it went very well.

tddworkshop

The next talk was on Requiem for a dream by Arnab Deka. He talked about various tips and tricks including using “Higher order functions and Concurrency” in Ruby and other programming languages like Clojure and Elixir.

After that, Rishi Jain talked about Game Development - The Ruby Way. He discussed how to build a game in Ruby using Gosu library . It was a very useful session for game developers. You can find out more about it here

Next talk was on Programming Ruby in Marathi by Ratnadeep Deshmane & his friend Aniket Awati. This was one of the best talks of the event. The way they used the similar words from Marathi for Ruby’s keywords and the examples made this talk remarkable. Their presentation style was nice too. Almost all attendees enjoyed this talk and they laughed a lot.

After this talk there was tea break for 15 minutes. Staffs from the Hyatt hotel were very helpful. There were serving tea and coffees to the attendees and overall did a good job of ensuring the the event cruised along smoothly. This is in sharp contrast to the service RubyConfIndia received from Lalit Resort.

After tea break, I didn’t get chance to attend other talks as attendees were still coming in and I was assisting them. But I heard almost all talks went very well.

In meantime, I was passing through main passage and I saw lighting talks board and decided to give lighting talk on my Ruby gem. Lighting talks is a short presentation that you can give about your achievement. You can also share your ideas and promote your library or any other projects.

Then we all had our lunch, lunch was good with lots of varieties with dessert.

After lunch I went for workshop on Deliver projects 30% faster, know your CSS by Aakash Dharmadhikari. Wanted to attend it fully, but some of the attendees had difficulties in internet connection, so I left the room to look into it.

Lightning talks were going to start so I took sometime to prepare for my presentation.

In the lighting talks, girls from Rails Girls Summer of Code, talked about their project and their progress on it. After that Prathamesh talked about RubyIndia.org and asked people to subscribe the newsletter. After that I gave talk on my Ruby gem RubySimpleSearch and you can find more on it here. The next speaker Rahul Mahale from Nashik asked people to help him in growing Ruby community in Nashik. All other lighting talks went very well.

santosh

After lighting talks, there was closing keynote On Solving Problems by Baishampayan Ghose. This talk made us think about how we write application in our daily routine. He talked about the architecture and he also explained the future is a function of past future = f(past). He also suggested that we should first understand the problem thoroughly and we should build the software. Talk was very informative and went very well.

After that Gautam came on to the stage and congratulated all the sponsors, organizers and volunteers. He also told that this event got large number of girls attendees than he had ever seen in any other conference.

After this event, there was party in Irish Village hotel. Me and my friends, we all went to the party. Party was superb and we all enjoyed it.

futsal

Thanks to all sponsors and organizers who made this event fun and enjoyable.

You can checkout more pictures of the conference from here.

Note: Photos are copyrighted by respective photo owners.

Flash access changes due to new serialization mechanism in Rails 4.1

Prior to upgrading to Rails 4.1 we had a helper to display flash messages and to add css class to the message based on flash type. Here is the code.

module FlashHelper
  ALERT_TYPES = [:success, :info, :warning, :danger]

  def bootstrap_flash
    flash_messages = []
    flash.each do |type, message|
      next if message.blank?

      type = :success if type == :notice
      type = :danger  if type == :alert
      type = :danger  if type == :error

      next unless ALERT_TYPES.include?(type)

      ....
      Array(message).each do |msg|
        text = content_tag(:div, msg.html_safe, :class => "alert fade in alert-#{type} ")
        flash_messages << text if msg
      end
    end
    flash_messages.join("\n").html_safe
  end
end

After upgrading to Rails 4.1, we started using the new Cookies serializer. Following code was added to an initializer.

  Rails.application.config.action_dispatch.cookies_serializer = :json

Soon after this our flash helper started misbehaving and all flash messages disappeared from the application.

JSON Cookies Serializer

Before we move ahead, a word on the new JSON Cookies Serializer. Applications created before Rails 4.1 uses Marshal to serialize cookie values into the signed and encrypted cookie jars.

Commits like this and this made it possible to have Cookies serializer and defaulted from Marshal Serializer to a secure Serializer using JSON.

The JSON Serializer works on JSON objects. Thus objects like Date and Time will be stored as strings. Hash keys will be stored as strings.

JSON serializer makes the application much safer since it is safer to pass around strings compare to passing around arbitrary values which is what was happens when values are marshalled and passed around.

Coming back to our problem, change Stringify the incoming hash in FlashHash coupled with above serialization changes meant that even if we put a symbol as a key in the flash we have to retrieve it as “string” since the keys are internally being converted into strings.

The difference is clearly illustrated below.

flash["string"] = "a string"
flash[:symbol] = "a symbol"

# Rails < 4.1
flash.keys # => ["string", :symbol]

# Rails >= 4.1
flash.keys # => ["string", "symbol"]

Solution

Now that we know the root cause of the problem the fix was simple. Instead of relying on symbols use “string” to access value from flash.

module BootstrapFlashHelper
  ALERT_TYPES = ['success', 'info', 'warning', 'danger']

  def bootstrap_flash
    flash_messages = []
    flash.each do |type, message|
      compare_type = type.to_s # Notice the stringifying of keys here, to make this work even with symbols.
      next if message.blank?

      compare_type = 'success' if compare_type == 'notice'
      compare_type = 'danger'  if compare_type == 'alert'
      compare_type = 'danger'  if compare_type == 'error'

      next unless ALERT_TYPES.include?(compare_type)

      Array(message).each do |msg|
        text = content_tag(:div, msg.html_safe, :class => "alert fade in alert-#{compare_type} ")
        flash_messages << text if msg
      end
    end
    flash_messages.join("\n").html_safe
  end
end

Issue with Delayed Job lifecyle and Postgres Errors

Recently in one of our projects, we experienced some strange errors from Delayed::Job. Delayed::Job workers started successfully, but when they were starting to lock the jobs, workers failed with PG::Error: no connection to server or PG::Error: FATAL: invalid frontend message type 60errors.

After some search, we found there had been such issues already experienced by others .

We started isolating the problem and digging through the recent changes we had made to the project. Since the last release the only significant modification had been made to internationalization. We had started using I18n-active_record .

# config/initializers/locale.rb

require 'i18n/backend/active_record'
Translation  = I18n::Backend::ActiveRecord::Translation

if (ActiveRecord::Base.connected? && Translation.table_exists?) ||
    in_delayed_job_process?

  I18n.backend = I18n::Backend::ActiveRecord.new
  I18n::Backend::ActiveRecord.send(:include, I18n::Backend::Memoize)
  I18n::Backend::ActiveRecord.send(:include, I18n::Backend::Flatten)
  I18n::Backend::Simple.send(:include, I18n::Backend::Memoize)
  I18n::Backend::Simple.send(:include, I18n::Backend::Pluralization)
  I18n.backend = I18n::Backend::Chain.new(I18n::Backend::Simple.new, I18n.backend)

end

for Delayed Job we had extra check as

def in_delayed_job_process?
  executable_name = File.basename $0
  arguments = $*
  rake_args_regex = /\Ajobs:/
  (executable_name == 'delayed_job') || (executable_name == 'rake' && arguments.find{ |v| v =~ rake_args_regex })
end

After some serious searching and digging through both Delayed::Job source code and how we were using to setup its config, we started noticing some issues.

The first thing we found was that the problem did not turn up when delayed job workers were started using rake jobs:work task.

After looking at DelayedJob internals we found that the main difference between a rake task and a binstub was in the fork method that was invoked in the binstub version. The binstub version was being executed seamlessly using Daemons#run_process method and had a slightly different lifecycle of execution.

DelayedJob lifecycle

Let’s take a look into DelayedJob internals before proceeding. DelayedJob has systems of the hooks that can be used by plugin-writers and in our applications. All this events functionality is hidden in Delayed::Lifecycle class. Each worker has its own instance of that class.

So, which events exactly do we have here?

Job-related events:

:enqueue
:perform
:error
:failure
:invoke_job

Worker-related events:

:execute
:loop
:perform
:error
:failure

You can setup callbacks to be run on before, after or around events simply using Delayed::Worker.lifecycle.before, Delayed::Worker.lifecycle.after and Delayed::Worker.lifecycle.around methods.

The Solution

Let’s move on to our problem. It turned out that delayed job active record gem was closing all database connections in before_fork hook and reestablishing them in after_fork hook. It was clear that I18n-active-record did not play well with this, causing the issue at hand.

We looked into DelayedJob lifecycle and chose before :execute hook, which was executed after all DelayedJob ActiveRecord backend connections manipulations.

Finally the locales initializer for delayed_job workers was changed to match as below:

require 'i18n/backend/active_record'
Translation  = I18n::Backend::ActiveRecord::Translation

Delayed::Worker.lifecycle.before :execute do
  if (ActiveRecord::Base.connected? && Translation.table_exists?) || in_delayed_job_process?
    I18n.backend = I18n::Backend::ActiveRecord.new

    I18n::Backend::ActiveRecord.send(:include, I18n::Backend::Memoize)
    I18n::Backend::ActiveRecord.send(:include, I18n::Backend::Flatten)
    I18n::Backend::Simple.send(:include, I18n::Backend::Memoize)
    I18n::Backend::Simple.send(:include, I18n::Backend::Pluralization)

    I18n.backend = I18n::Backend::Chain.new(I18n::Backend::Simple.new, I18n.backend)
  end
end

This helped us to mitigate the connection errors, and connections stopped dying abruptly.