Make Outbound Calls From the Browser to Phone using Twilio and Rails

In this blog post we will see how to make outbound phone calls from the browser to a phone using Twilio . We will make use of the Twilio-JS library and Twilio-Ruby gem.

The Rails App we will be creating, is based on the Twilio Client Quick-start tutorial. That Twilio tutorial makes use of Sinatra. We will see how we can achieve this in a Rails application.

Step 1 - Setup Twilio Credentials and TwiML App

We need to setup twilio credentials. We can find account ID and auth token from our account information.

When the call is made using browser then the phone that is receiving the call has to see a number from which the call is coming. So now we need to setup a Twilio verified number. This number will be used to place the outgoing calls from. How to setup a verified number can be found here.

When our app make a call from the browser using twilio-js client, Twilio first creates a new call connection from our Browser to Twilio. It then sends a request back to our server to get information about what to do next. We can respond by asking twilio to call a number, say something to the person after a call is connected, record a call etc.

Sending of this instructions is controlled by setting up a TwiML application. This application provides information about the end point on our server, where twilio should send the request to fetch instructions. TwiML is a set of instructions, that we can use to tell Twilio what to do in different cases like when an outbound phone call is made or when an inbound SMS message is received.

Given below is an example that will say a short message How are you today? in a call.

<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <Say voice="woman">How are you today?</Say>
</Response>

The TwiML app can be created here. Once the app is configured then we will get appsid.

We need to configure following information in our Rails Application:

twilio:
  verified_number: <%= ENV['TWILIO_VERIFIED_NUMBER']%>
  account_sid: <%= ENV['TWILIO_ACCOUNT_SID'] %>
  auth_token: <%= ENV['TWILIO_AUTH_TOKEN'] %>
  call_app_sid: <%= ENV['TWILIO_CALL_APP_SID'] %>

Step 2 - Generate capability token to be used by twilio-js

After we have the config setup, we will proceed to create the capability token. This token will be generated using the ruby gem, and passed to the javascript SDK. The token helps the twilio-js client determine, what permissions the application has like making calls, accepting calls, sending SMS, etc.

We define a TwilioTokenGeneratorService for this purpose.

class TwilioTokenGeneratorService

  def process
    capability = twilio_capability()
    capability.generate
  end

  private

  def twilio_capability
    capability ||= Twilio::Util::Capability.new Settings.twilio.account_sid, Settings.twilio.auth_token
    capability.allow_client_outgoing Settings.twilio.call_app_sid
    capability
  end

end

As you can see, we first define a new Twilio::Util::Capability instance and pass credentials to it. We then call allow_client_outgoing method and pass the client Sid to it. This is the identifier for the TwiML application we have previously created on Twilio. Calling allow_client_outgoing gives permission to the client to make outbound calls from Twilio. Finally we call the generate method to create a token from the capability object.

Step 3 - Define view elements and pass token to it

The generated token will now be passed to the Twilio JS client for connecting with Twilio. In our App we define CallsController, and index action in this controller. This action takes care of setting the capability token. Our index view consists of two buttons- to place and hangup a call, a number input field, call logs, and data field to pass capability token to the javascript bindings. We import the Twilio-JS library in the view. The css styling being used is from the Twilio example.

<div id="twilioToken" data-token="<%= @twilio_token %>"></div>

<button id="caller" class="call">Call</button>
<button id="hangup" class="hangup">Hangup</button>

<input type="text" id="phoneNumber" placeholder="Enter a phone number to call"/>

<div id="log">Loading pigeons...</div>

<script type="text/javascript" src="//static.twilio.com/libs/twiliojs/1.2/twilio.min.js"></script>

Step 4 - Define coffeescript bindings to handle TwilioDevice connection to Twilio

Next we setup coffeescript bindings to handle initialization of TwilioDevice and making use of entered number to place calls Twilio. We are taking care of various events like connect, disconnect, ready, etc. on TwilioDevice instance. More information about TwilioDevice usage can be found here.

class TwilioDevice
  constructor: ->
    @initTwilioDeviceBindings()
    @initFormBindings()

  initTwilioDeviceBindings: ->
    twilio_token = $('#twilioToken').data('token')
    twilio_device = Twilio.Device

    # Create the Client with a Capability Token
    twilio_device.setup(twilio_token, {debug: true});

    #/* Let us know when the client is ready. */
    twilio_device.ready ->
      $("#log").text("Ready")

    #/* Report any errors on the screen */
    twilio_device.error (error) ->
      $("#log").text("Error: " + error.message)

    #/* Log a message when a call connects. */
    twilio_device.connect (conn) ->
      $("#log").text("Successfully established call")

    #/* Log a message when a call disconnects. */
    twilio_device.disconnect (conn) ->
      $("#log").text("Call ended")

  initFormBindings: ->
    $('#caller').bind "click", (event) ->
      params = {"phone_number": $('#phoneNumber').val()}
      Twilio.Device.connect(params)

    $('#hangup').bind "click", (event) ->
      Twilio.Device.disconnectAll()

$ ->
  new TwilioDevice()

If we now load this page, we should be able to see our app saying its ready to take calls.

Twilio Rails App Ready

Step 5 - Define TwiML Response Generator Service

The final step before we place calls from our App is to handle callbacks from Twilio and return TwiML response. For this we are going to define TwilioCallTwiMLGeneratorService which takes care of generating this response. More information about how we need to define the response and individual fields can be found from Twilio’s docs.

What we need to define is a response as below:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Dial callerId="+15005550000">
    <Number>+15005550001</Number>
  </Dial>
</Response>

We are making use of two elements here - Dial, which makes Twilio place a call using defined callerId value, as the number from which the call is made, which is displayed on the callee’s phone. Note that this is the same verified number that we had specified before. Then we specify Number which is the number, to which we want to place the call to. This number is passed first by the javascript client to Twilio, and then back to our application by Twilio, which we use to generate the response as above.

We define our TwilioCallTwiMLGeneratorService to take in a phone number as parameter. It creates an instance of Twilio::TwiML::Response and tapping on this instance we provide Dial element with a :callerId value, and the Number to place the call to. We validate the number before passing it back, and return an error if the number is invalid.

class TwilioCallTwiMLGeneratorService
  attr_reader :phone_number
  VALID_PHONE_NUMBER_REGEX = /^[\d\+\-\(\) ]+$/ # Matches valid phone numbers acceptable to Twilio

  def initialize phone_number
    @phone_number = phone_number
  end

  def process
    Twilio::TwiML::Response.new do |r|
      if VALID_PHONE_NUMBER_REGEX.match(phone_number)
        r.Dial :callerId => Settings.twilio.verified_number do |d| # callerId is number from which call is made.
          d.Number(CGI::escapeHTML phone_number) # The number to call
        end
      else
        r.Error("Invalid number!")
      end
    end.text.strip
  end

end

Step 6 - Send TwiML response on Twilio callback

We are now set to define twilio’s callback handler. This will be handled by the create_call action in CallsController. Twilio will be sending this endpoint a POST request along with some information specified here. We make use of phone_number being passed to us by Twilio and pass it along to the TwilioCallTwiMLGeneratorService, which return us with valid TwiML response. Since TwiML is a flavor of XML, we make using render xml to return the response.

def create_call
  response_to_twilio_callback = TwilioCallTwiMLGeneratorService.new(call_params[:phone_number]).process
  render xml: response_to_twilio_callback
end

def call_params
  params.permit(:phone_number)
end

As create_call endpoint will be used by Twilio API, we need to skip authenticity token check for this action.

class CallsController < ApplicationController
  skip_before_action :verify_authenticity_token, only: [:create_call]
end

Finally we need to specify the callback url in our TwiML App on Twilio. For testing this locally, we can make use of a service like https://ngrok.com/, to expose this endpoint.

Twilio TwiML App setting

Our service is now ready to place calls. The complete Rails application code that we have created can be found here.

Happy calling everyone!

Tricks and Tips for using Fixtures effectively in Rails

Recently I gave a talk at RubyKaigi 2014 about Rails fixtures. In this blog post I will be discussing some of the tips and tricks for using fixtures effectively.

You can also the see the video of this talk here.

Don’t use ids.. unless required

In fixtures, we can specify id of a fixture.

john:
  id: 1
  email: john@example.com

I would recommend not to specify the id. Rails generates the id automatically if we don’t explicitly specify it. Moreover there are a few more advantages of not specifying the id.

Stable ids for every fixture.

Rails will generate the id based on key name. It will ensure that the id is unique for every fixture. It can also generate ids for uuid primary keys.

Labeled references for associations like belongs_to, has_many.

Lets say we have a users table. And a user has many cars.

Car ferrari belongs to john. So we have mentioned user_id as 1.

ferrari:
  name: ferrari
  make: 2014
  user_id: 1

When I’m looking at cars.yml I see user_id as 1. But now I to lookup to see which user has id as 1.

Here is another implementation.

john:
  name: john
  email: john@example.com

ferrari:
  name: ferrari
  make: 2014
  user: john

Notice that I no longer specify user_id for John. I have mentioned name. And now I can reference that name in cars.yml to mention that ferrari belongs to john.

How to set a value to nil from fixture

Let’s say that I have a boolean column which is false by default. But for an edge case, I want it to be nil. I can obviously mutate the data generated by fixture before testing. However I can achieve this in fixtures also.

Specify null to make the value nil

require 'yaml'
YAML.load "--- \n:private: null\n")
=> {:private=>nil}

As you can see above if the value is null then YAML will treat it as nil.

john:
  name: john
  email: john@example.com
  private: null

Leave the value blank to make the value nil

require 'yaml'
YAML.load "--- \n:private: \n")
=> {:private=>nil}

As you can see above if the value is blank then YAML will treat it as nil.

john:
  name: john
  email: john@example.com
  private:

When model name and table name does not match

Generally in Rails, the model name and table name follow a strict convention. Post model will have table name posts. Using this convention, the fixture file for Post models is obviously fixtures/posts.yml.

But sometimes models do not match directly with the table name. This could be because of legacy reason or because of namespacing of models. In such cases automatic detection of fixture files becomes difficult.

Rails provides set_fixture_class method for this purpose. This is a class method which accepts a hash where key should be name of the fixture or relative path to fixture file and value should be model class.

I can use this method inside test_helper.rb in any class inheriting from ActiveSupport::TestCase.

# test_helper.rb
class ActiveSupport::TestCase

  # table name is "morning_appts". It is being mapped to model "MorningAppointment".
  self.set_fixture_class morning_appts: MorningAppointment

  # in this case fixture is namespaced
  self.set_fixture_class '/legacy/users' => User

  # in this case the model is namespaced.
  self.set_fixture_class outdoor_games: Legacy::OutdoorGame
end

values interpolation using $LABEL

Rails provides many ways to keep our fixtures DRY. Label interpolation is one of them. It allows the use of key of fixture as a value in the fixture. For example:

john:
  name: john
  email: john@example.com

becomes:

john:
  name: $LABEL
  email: john@example.com

$LABEL is not a global variable here. Its just a placeholder. $LABEL is replaced by the key of the fixture. And as discussed earlier the key of the fixture in this case is john. So $LABLE has value john.

Before this PR, I could only use this feature if the value is exactly $LABEL. So if the email is john@example.com I could not use the $LABEL@example.com. But after this PR, I can $LABEL anywhere in the string, and Rails will replace it with the key.

So the earlier example becomes:

john:
  name: $LABEL
  email: $LABEL@example.com

YAML defaults

I use YAML defaults in database.yml for drying it up and keeping common configuration at one place.

defaults: &defaults
  adapter: postgresql
  encoding: utf8
  pool: 5
  host: localhost
  password:

development:
  <<: *defaults
  database: wheel_development

test:
  <<: *defaults
  database: wheel_test

production:
  <<: *defaults
  database: wheel_production

I can use it for drying up fixtures too for extracting common part in our fixtures.

DEFAULTS: &DEFAULTS
  company: BigBinary
  website: bigbinary.com
  blog: blog.bigbinary.com

john:
  <<: *DEFAULTS
  name: John Smith
  email: john@bigbinary.com

prathamesh:
  <<: *DEFAULTS
  name: Prathamesh Sonpatki
  email: prathamesh@bigbinary.com

Note the usage of key DEFAULTS for defining default fixture. Rails will automatically ignore any fixture with key DEFAULTS.

If we use any other key then a record with that key will also get inserted in the database.

Database specific tricks

Fixtures bypass the normal Active Record object creation process. After reading them from YAML file, they are inserted into database directly using insert query. So they skip callbacks and validations check. This also has an interesting side-effect which can be used for drying up fixtures.

Suppose we have fixture with timestamp:

john:
  name: John Smith
  email: john@example.com
  last_active_at: <%= Time.now %>

If I are using PostgreSQL, I can replace the last_active_at value with now:

john:
  name: John Smith
  email: john@example.com
  last_active_at: now

now is not a keyword here. It is just a string. The actual query looks like this:

INSERT INTO "users"
("name", "email", "last_active_at", "id")
VALUES
('John Smith', 'john@example.com', 'now',1144934)

So the value for last_active_at is still just now when the query is executed.

The magic starts as PostgreSQL starts reading the values. now is a shorthand for the current timestamp . As soon as Postgres reads it, it replaces now with the current timestamp and the column last_active_at gets populated with current timestamp.

I can also use the now() function instead of just now.

This function is available in PostgreSQL as well as MySQL. So the usage of now() works in both of these databases.

Selenium IDE - How I improved performance by reducing running time from 58 minutes to 15 minutes of selenium test

I wrote a bunch of selenium tests using Selenium IDE for a project. The selenium tests have proven to be very useful. However the tests take around 58 minutes to complete the full run.

Here are the specific steps I took which brought the running time to under 15 minutes.

Set to run at the maximum speed

Command Target Value
setSpeed 0

setSpeed command takes Target value in milliseconds. By setting the value to zero, I set the speed to maximum and the tests indeed ran fast. However, now I had lots of tests failing which were previously passing.

What happened.

In our tests real firefox browser is fired up and real elements are clicked. The application does make round trip to the rails server hosted on heroku.

By setting the selenium tests to the maximum speed the selenium tests started asserting for elements on the page even before the pages were fully loaded by the browser.

I needed sets of instructions using which I could tell selenium how long to wait for before asserting for elements.

Selenium provides a wonderful suite of commands which helped me fine tune the test run. Here I’m discussing some of those commands.

waitForVisible

This command is used to tell selenium to wait until the specified element is visible on the page.

In the below mentioned case, the Selenium IDE will wait until the element css=#text-a is visible on the page.

Command Target Value
waitForVisible css=#text-a

waitForText

This command is used to tell selenium to wait until a particular text is visible in the specified element.

In the case mentioned below, Selenium IDE will wait until the text violet is displayed in the element css=#text-a.

Command Target Value
waitForText css=#text-a violet


The difference between waitForVisible and waitForText is that waitForVisible waits until the specified element is visible on the page while waitForText waits until a particular text is visible in the specified element on the page.

waitForElementPresent

This command is used to tell Selenium to wait until the specified element is displayed on the page.

In the below mentioned case, the Selenium IDE will wait until the element css=a.button is displayed on the page.

Command Target Value
waitForElementPresent css=a.button

waitForVisible and waitForElementPresent seem very similar. It seems both of these commands do the same thing. There is a subtle difference though.

waitForVisible waits until the specified element is visible. Visibility of an element is manipulated by the settings of CSS properties. For example using display none; one can make an element not be visible at all.

In contrast the command waitForElementPresent waits until the specified element is present on the page in the form of html markup. This command does not give consideration to css settings.

refreshAndWait

This command is used to tell Selenium to wait until the page is refreshed and the targeted element is displayed on the web page.

In the example mentioned below, the Selenium IDE will wait until the page is refreshed and the targeted element css=span.button is displayed on the page.

Command Target Value
refreshAndWait css=span.button

clickAndWait

This command is used to tell selenium to wait until a particular button is clicked for submitting the form and the page starts reloading. The subsequent commands are paused until, the page is reloaded after the element is clicked on the page.

In the case mentioned below, Selenium IDE will wait until the page is reloaded after the specified element css=input#edit is clicked.

Command Target Value
clickAndWait css=input#edit

Selenium IDE commands used above and more are available at Selenium documentation .

DeccanRubyConf 2014. Hou De!

deccanrubyconflogo

I attended it, enjoyed it and took part as a volunteer in Pune’s first RubyConf - DeccanRubyConf 2014. As Hou De (let it be) name said, event went in the same way.

The day before conference, in the morning Vipul (one of the event organizers) and I picked up our guest speaker Koichi Sasada from the Pune Airport. Koichi is a Ruby core member and works for Heroku. We welcomed him and went to Hyatt Regency hotel where the event is taking place. Our guest checked into hotel and then we decided to go for a lunch at Malaka Spice restaurant.

We reached there and Koichi told us that he wanted non-spicy food (Safe food). We ordered non spicy food, but food was still too spicy for Koichi. However we enjoyed the food and had very good discussion about the Ruby internals, concurrency-parallelism, debugging in Ruby, Japanese culture and the Indian culture.

koichi & vipul

After lunch, we dropped Koichi off at the hotel and we left for our home.

Next morning was the event day and I woke up early and went to the event place. As part of volunteering team, I and other volunteers had tasks like giving pens, badges, stickers, T-shirts and coupon for night party to the attendees.

Attendees had started to come in slowly. Some attendees asked me about T-shirt size as I wore one of the conference T-shirts and from my T-shirt’s size they decided their T-shirt size. It was a great experience meeting with different kind of people from around the India.

Keynote by Koichi kicked the event off and he talked about Ruby 2.1 features like,

  • Required keyword parameter,
  • Rational number literal,
  • def returns symbol of method name
  • Runtime new features (String#scrub, Binding#local_time and etc.)

Then he talked about performance improvements, Ruby 2.2 and how to speed up Ruby Interpreter . Click here for more details about his talk.

koichi

attendees

In between the talks, some new attendees had come for the conference who had not registered for the conference. They told me that they thought it’s a regular Pune’s local Ruby Meetup. There was some miss-understandings but they seemed interested in attending the event. I contacted Gautam as he was one of the organizers and told him about the issue.

Attendees kept coming till the afternoon.

After Koichi’s talk, two sections had opened. One for talks and other for the workshops. TDD workshop was conducted by Sidu Ponnappa. I saw lots of attendees in this workshop and heard that it went very well.

tddworkshop

The next talk was on Requiem for a dream by Arnab Deka. He talked about various tips and tricks including using “Higher order functions and Concurrency” in Ruby and other programming languages like Clojure and Elixir.

After that, Rishi Jain talked about Game Development - The Ruby Way. He discussed how to build a game in Ruby using Gosu library . It was a very useful session for game developers. You can find out more about it here

Next talk was on Programming Ruby in Marathi by Ratnadeep Deshmane & his friend Aniket Awati. This was one of the best talks of the event. The way they used the similar words from Marathi for Ruby’s keywords and the examples made this talk remarkable. Their presentation style was nice too. Almost all attendees enjoyed this talk and they laughed a lot.

After this talk there was tea break for 15 minutes. Staffs from the Hyatt hotel were very helpful. There were serving tea and coffees to the attendees and overall did a good job of ensuring the the event cruised along smoothly. This is in sharp contrast to the service RubyConfIndia received from Lalit Resort.

After tea break, I didn’t get chance to attend other talks as attendees were still coming in and I was assisting them. But I heard almost all talks went very well.

In meantime, I was passing through main passage and I saw lighting talks board and decided to give lighting talk on my Ruby gem. Lighting talks is a short presentation that you can give about your achievement. You can also share your ideas and promote your library or any other projects.

Then we all had our lunch, lunch was good with lots of varieties with dessert.

After lunch I went for workshop on Deliver projects 30% faster, know your CSS by Aakash Dharmadhikari. Wanted to attend it fully, but some of the attendees had difficulties in internet connection, so I left the room to look into it.

Lightning talks were going to start so I took sometime to prepare for my presentation.

In the lighting talks, girls from Rails Girls Summer of Code, talked about their project and their progress on it. After that Prathamesh talked about RubyIndia.org and asked people to subscribe the newsletter. After that I gave talk on my Ruby gem RubySimpleSearch and you can find more on it here. The next speaker Rahul Mahale from Nashik asked people to help him in growing Ruby community in Nashik. All other lighting talks went very well.

santosh

After lighting talks, there was closing keynote On Solving Problems by Baishampayan Ghose. This talk made us think about how we write application in our daily routine. He talked about the architecture and he also explained the future is a function of past future = f(past). He also suggested that we should first understand the problem thoroughly and we should build the software. Talk was very informative and went very well.

After that Gautam came on to the stage and congratulated all the sponsors, organizers and volunteers. He also told that this event got large number of girls attendees than he had ever seen in any other conference.

After this event, there was party in Irish Village hotel. Me and my friends, we all went to the party. Party was superb and we all enjoyed it.

futsal

Thanks to all sponsors and organizers who made this event fun and enjoyable.

You can checkout more pictures of the conference from here.

Note: Photos are copyrighted by respective photo owners.

Flash access changes due to new serialization mechanism in Rails 4.1

Prior to upgrading to Rails 4.1 we had a helper to display flash messages and to add css class to the message based on flash type. Here is the code.

module FlashHelper
  ALERT_TYPES = [:success, :info, :warning, :danger]

  def bootstrap_flash
    flash_messages = []
    flash.each do |type, message|
      next if message.blank?

      type = :success if type == :notice
      type = :danger  if type == :alert
      type = :danger  if type == :error

      next unless ALERT_TYPES.include?(type)

      ....
      Array(message).each do |msg|
        text = content_tag(:div, msg.html_safe, :class => "alert fade in alert-#{type} ")
        flash_messages << text if msg
      end
    end
    flash_messages.join("\n").html_safe
  end
end

After upgrading to Rails 4.1, we started using the new Cookies serializer. Following code was added to an initializer.

  Rails.application.config.action_dispatch.cookies_serializer = :json

Soon after this our flash helper started misbehaving and all flash messages disappeared from the application.

JSON Cookies Serializer

Before we move ahead, a word on the new JSON Cookies Serializer. Applications created before Rails 4.1 uses Marshal to serialize cookie values into the signed and encrypted cookie jars.

Commits like this and this made it possible to have Cookies serializer and defaulted from Marshal Serializer to a secure Serializer using JSON.

The JSON Serializer works on JSON objects. Thus objects like Date and Time will be stored as strings. Hash keys will be stored as strings.

JSON serializer makes the application much safer since it is safer to pass around strings compare to passing around arbitrary values which is what was happens when values are marshalled and passed around.

Coming back to our problem, change Stringify the incoming hash in FlashHash coupled with above serialization changes meant that even if we put a symbol as a key in the flash we have to retrieve it as “string” since the keys are internally being converted into strings.

The difference is clearly illustrated below.

flash["string"] = "a string"
flash[:symbol] = "a symbol"

# Rails < 4.1
flash.keys # => ["string", :symbol]

# Rails >= 4.1
flash.keys # => ["string", "symbol"]

Solution

Now that we know the root cause of the problem the fix was simple. Instead of relying on symbols use “string” to access value from flash.

module BootstrapFlashHelper
  ALERT_TYPES = ['success', 'info', 'warning', 'danger']

  def bootstrap_flash
    flash_messages = []
    flash.each do |type, message|
      compare_type = type.to_s # Notice the stringifying of keys here, to make this work even with symbols.
      next if message.blank?

      compare_type = 'success' if compare_type == 'notice'
      compare_type = 'danger'  if compare_type == 'alert'
      compare_type = 'danger'  if compare_type == 'error'

      next unless ALERT_TYPES.include?(compare_type)

      Array(message).each do |msg|
        text = content_tag(:div, msg.html_safe, :class => "alert fade in alert-#{compare_type} ")
        flash_messages << text if msg
      end
    end
    flash_messages.join("\n").html_safe
  end
end

Issue with Delayed Job lifecyle and Postgres Errors

Recently in one of our projects, we experienced some strange errors from Delayed::Job. Delayed::Job workers started successfully, but when they were starting to lock the jobs, workers failed with PG::Error: no connection to server or PG::Error: FATAL: invalid frontend message type 60errors.

After some search, we found there had been such issues already experienced by others .

We started isolating the problem and digging through the recent changes we had made to the project. Since the last release the only significant modification had been made to internationalization. We had started using I18n-active_record .

# config/initializers/locale.rb

require 'i18n/backend/active_record'
Translation  = I18n::Backend::ActiveRecord::Translation

if (ActiveRecord::Base.connected? && Translation.table_exists?) ||
    in_delayed_job_process?

  I18n.backend = I18n::Backend::ActiveRecord.new
  I18n::Backend::ActiveRecord.send(:include, I18n::Backend::Memoize)
  I18n::Backend::ActiveRecord.send(:include, I18n::Backend::Flatten)
  I18n::Backend::Simple.send(:include, I18n::Backend::Memoize)
  I18n::Backend::Simple.send(:include, I18n::Backend::Pluralization)
  I18n.backend = I18n::Backend::Chain.new(I18n::Backend::Simple.new, I18n.backend)

end

for Delayed Job we had extra check as

def in_delayed_job_process?
  executable_name = File.basename $0
  arguments = $*
  rake_args_regex = /\Ajobs:/
  (executable_name == 'delayed_job') || (executable_name == 'rake' && arguments.find{ |v| v =~ rake_args_regex })
end

After some serious searching and digging through both Delayed::Job source code and how we were using to setup its config, we started noticing some issues.

The first thing we found was that the problem did not turn up when delayed job workers were started using rake jobs:work task.

After looking at DelayedJob internals we found that the main difference between a rake task and a binstub was in the fork method that was invoked in the binstub version. The binstub version was being executed seamlessly using Daemons#run_process method and had a slightly different lifecycle of execution.

DelayedJob lifecycle

Let’s take a look into DelayedJob internals before proceeding. DelayedJob has systems of the hooks that can be used by plugin-writers and in our applications. All this events functionality is hidden in Delayed::Lifecycle class. Each worker has its own instance of that class.

So, which events exactly do we have here?

Job-related events:

:enqueue
:perform
:error
:failure
:invoke_job

Worker-related events:

:execute
:loop
:perform
:error
:failure

You can setup callbacks to be run on before, after or around events simply using Delayed::Worker.lifecycle.before, Delayed::Worker.lifecycle.after and Delayed::Worker.lifecycle.around methods.

The Solution

Let’s move on to our problem. It turned out that delayed job active record gem was closing all database connections in before_fork hook and reestablishing them in after_fork hook. It was clear that I18n-active-record did not play well with this, causing the issue at hand.

We looked into DelayedJob lifecycle and chose before :execute hook, which was executed after all DelayedJob ActiveRecord backend connections manipulations.

Finally the locales initializer for delayed_job workers was changed to match as below:

require 'i18n/backend/active_record'
Translation  = I18n::Backend::ActiveRecord::Translation

Delayed::Worker.lifecycle.before :execute do
  if (ActiveRecord::Base.connected? && Translation.table_exists?) || in_delayed_job_process?
    I18n.backend = I18n::Backend::ActiveRecord.new

    I18n::Backend::ActiveRecord.send(:include, I18n::Backend::Memoize)
    I18n::Backend::ActiveRecord.send(:include, I18n::Backend::Flatten)
    I18n::Backend::Simple.send(:include, I18n::Backend::Memoize)
    I18n::Backend::Simple.send(:include, I18n::Backend::Pluralization)

    I18n.backend = I18n::Backend::Chain.new(I18n::Backend::Simple.new, I18n.backend)
  end
end

This helped us to mitigate the connection errors, and connections stopped dying abruptly.

RedDotRubyConf 2014

RedDotRubyConf2014logo

I and Vipul recently gave a talk at RedDotRubyConf on ActiveRecord can’t do it? Arel can!. It was our first trip to Singapore and we enjoyed the conference as well as Singapore a lot.

RDRC2014 was awesome.

Day 1

We reached the venue in time for Koichi’s keynote on Ruby.Inspect. He talked about various things related to development of Ruby including Ruby team at Heroku, recent releases of Ruby and new syntax introduced in Ruby 2.1. He also talked about performance improvements including Generational GC - RGenGC and upcoming features in Ruby 2.2.

After that in second part of the talk, he talked about inspection tools available in Ruby. It was a deep technical part for me and something to learn about. The message he gave from the talk was to become low level engineer.

Second talk of the conf was from T.J. Schuck about solving one of the hardest problems. Storing and retrieving passwords in a secure way. He talked about how increasing improvements in hardware pose a challenge as even if you use proper algorithm it can be cracked with high computing machines. It was interesting to know about internals of storing passwords. I had never cared too much about it :)

After the coffee break, Brandon Keepers from Github gave talk on Tending Your Open Source Garden. Github is still on Rails 2.3 and Brandon is working on bringing it up to new version. His talk was a great advice for those who want to contribute to open source and community. I think this talk resonated well with the audience as most of the crowd was new and interested in open source contributions.

Gautam Rege from Josh Software gave talk on Dark Side of Ruby. We had attended this talk at GCRC so we left the hall after some time and did our one last practice. But i heard the feedback was very well for this talk.

After the lunch, Keith Pitt talked about Guide to Continuous Deployment with Rails. He talked about keeping everything related to deployment from CI to migrations in sync. One of the interesting thing that i came to know from this talk was how to enable zero downtime deployments on Heroku using preboot feature.

Benjamin Tan gave talk on Ruby + Elixir: Polyglotting FTW! after that. He talked about Elixir. This talk was about looking beyond Ruby and adding another tool to our skills. Benjamin also gave some demos including the last one in which he used sidekiq with Elixir. The actual work was done by Elixir workers. I will definitely give a shot to Elixir in the coming days.

After that we gave our talk on Arel. I was a bit nervous as it was my first talk. But it went well. We finished a bit early than expected. But there was tea break after our talk :). We got some good feedback from the attendees and especially beginners who had not used Arel before. Our slides are here

After the tea break, lightning talks started. First talk was by Hiroshi Shibata about how anyone can contribute to Ruby to make it better. He talked about how to submit issues, feature requests using Redmine. After that William Notowidagdo gave talk on Building REST API using Grape. With Rails and Rails API gem it has become easy to generate an API. But we have grape also as a lightweight tool. Sayani Basu talked about how to make a podcast with Jekyll and other tools in 5 minutes.

We are planning to start a podcast here in India about Ruby community so it was good to know about it.

Update: Checkout our podcast .

After these awesome lightning talks, the last session of Day 1 started. There were talks on Fluentd and Domain driven design. Both were good to know as something outside of daily routine. Konstantin Haase’s last talk of the day was **Meta** talk. He talked on abstraction and how it happens in our mind. Our mind affects what we see, like we see magenta color. Similarly abstraction happens in mind. I had to concentrate a lot in this talk to understand it. But it was worth it.

Andddd that ended the first day of the conf. It was exciting and we were looking forward to second day.

Day 2

Day 2 started with Brian Helmkamp’s talk on Docker. We missed the initial part of the talk. He talked about basics of Docker, how to deploying in container environment. He also discussed about deploying a Rails app using docker and how it makes very easy to deploy different parts of the system using docker very easily.

Zachary Scott gave next talk introducing Ruby Core team and how it works, how it collaborates, developer meetings, how anyone can contribute to MRI. We also had a Friday hug in this talk :) This talk combined with Hiroshi’s lightning talk on the first day was great insight into CRuby development.

After the break, Pioter Solnica gave an excellent talk on Convenience vs Simplicity. He talked about convenience offered by ActiveRecord may not be simple to understand. The things such as input conversion, validation are convenient to use as a developer but not necessarily simple to understand. He also discussed presenters, immutable data structures, Adamantium for creating immutable objects in Ruby. In the second part of the talk, he talked about relations and how they can be used in composing queries. He explained this idea using Ruby Object Mapper. It uses Axiom as underlying relational algebra instead of Arel. Its an interesting project to checkout.

After that our very own Anil Wadghule talked on Solid Design Principles in Ruby. His emphasis was on following designs than patterns. He also showed code examples and refactored them after applying principles. His talk was good insight into understanding what are these Solid principles and how they can be applied in real life.

We skipped the session after lunch and roamed around talking with people. We had an interesting discussion about hiring Ruby on Rails developers, interview processes etc.

Then lightning talks started. Sheng-Loong-Su talked first on using Algorithms for Trading. He talked about collecting data using feeder, preparing trading signals using strategy and making decision based on trading signals. One of the best talks of day 2 was by Grzegorz Witek on how he is traveling the world without getting burned out and still happily programming. He talked about his experiences in different countries being a Nomadic Programmer. It was one of the best inspirational talks according to me. The last lightning talk was about Using Vagrant for setting up Dev environment by Shuwei and Arathi.

Nicholas Simmons talked on experience of building a Single page web app and back again to normal app. He gave real life metrics from Shopify and showed problems faced with single page apps, batman.js and how moving back to a normal app helped them.

Then chocolate man from Belgium, Christophe Philemotte, gave talk on Safety Nets: Learn to code with confidence. His talk was about how we can prevent code in long term using testing, static analysis using tools such as flog, flay, rubocop for removing duplication, reducing complexity, fixing warnings. He also talked about importance of code review. His code is present here. He also gave us excellent chocolates from Belgium.

And the last keynote by Aaron Patterson. As always, it was full of everything - tech stuff, jokes, puns.

He talked on how he is making performance improvements in Active Record, link generations. He showed some graphs with performance of various database adapters tested on Rails versions ranging from 2.3 to 4 to master. He urged everyone to report performance issues to the core team so that they are addressed quickly. This is the app used for doing performance testing by him.

And that ended talks at RDRC. We had an awesome after party where we discussed with lots of people about Ruby, Rails as well as other stuff. We would like to thank Winston for inviting us to RedDotRubyConf.

Checkout some of the pictures from the conference here by Nestor.

I am already looking at RDRC 2015.

Six Year Old Optional / Keyword Arguments bug

I recently conducted a workshop about Contributing to Open-Source at first-ever Rubyconf Philippines. In its introductory talk, I spoke about how Aaron Patterson, fixed a 6 year old bug about Optional Arguments, that existed in Rails.

Bug in ruby

Let’s try a small program.

class Lab

  def day
    puts 'invoked'
    'sunday'
  end

  def run
    day = day
  end

end

puts Lab.new.run

What do you think would be printed on your terminal when you run the above program.

If you are using ruby 2.1 or below then you will see nothing. Why is that ? That’s because of a bug in ruby.

This is bug number 9593 in ruby issue tracker.

In the statement day = day the left hand side variable assignment is stopping the call to method day. So the method day is never invoked.

Another variation of the same bug

class Lab

  def day
    puts 'invoked'
    'sunday'
  end

  def run( day: day)
  end

end

puts Lab.new.run

In the above case we are using the keyword argument feature added in Ruby 2.0 . If you are unfamiliar with keyword arguments feature of ruby then checkout this excellent video by Peter Cooper.

In this case again the same behavior is exhibited. The method day is never invoked.

How this bug affects Rails community

You might be thinking that I would never write code like that. Why would you have a variable name same as method name.

Well Rails had this bug because rails has code like this.

def has_cached_counter?(reflection = reflection)
end

In this case method reflection never got called and the variable reflection was always assigned nil.

Fixing the bug

Nobu fixed this bug in ruby 2.2.0. By the way Nobu is also known as “ruby patch-monster” because of amount of patches he applies to ruby.

So this bug is fixed in ruby 2.2.0. What about the people who are not using ruby 2.2.0.

The simple solution is not to omit the parameter. If we change the above code to

def has_cached_counter?(reflection = reflection())
end

then we are explicitly invoking the method reflection and the variable reflection will be assigned the output of method reflection.

And this is how Aaron Patterson fixed six years old bug.

How to deploy jekyll site to heroku

jekyll is an excellent tool for creating static pages and blogs. Our BigBinary blog is based on jekyll. Deploying our blog to heroku took longer than I had expected. I am outlining what I did to deploy BigBinary blog to heroku.

Add exclude vendor to _config.yml

Open _config.yml and add following line at the very bottom.

exclude: ['vendor']

Add Procfile

Create a new file called Procfile at the root of the project with following content.

web: bundle exec jekyll build && bundle exec thin start -p$PORT -V
console: echo console
rake: echo rake

Add Gemfile

Add Gemfile at the root of the project.

source 'https://rubygems.org'

gem 'jekyll', '2.4.0'
gem 'rake'
gem 'foreman'
gem 'thin'
gem 'rack-contrib'

Add config.ru

Add config.ru at the root of the project with following content.

require 'rack/contrib/try_static'

use Rack::TryStatic,
    :root => "_site",
    :urls => %w[/],
    :try => ['.html', 'index.html', '/index.html']

run lambda { |env|
  return [404, {'Content-Type' => 'text/html'}, ['Not Found']]
}

Test on local machine first

Test locally by executing bundle exec jekyll serve.

Push code to heroku

Now run bundle install and add the Gemfile.lock to the repository and push the repository to heroku.

How to add additional directories to test

In a project we needed to write different parsers for different services. Rather than putting all those parsers in app/models or in lib we created a new directory. We put all the parsers in app/parsers .

We put all the tests for these parsers in test/parsers directory.

We can run tests parsers individually by executing rake test test/parsers/email_parser_test.rb. However when we run rake then tests in test/parsers are not picked up.

We added following code to Rakefile to make rake pickup tests in test/parsers.

# Adding test/parsers directory to rake test.
namespace :test do
  desc "Test tests/parsers/* code"
  Rails::TestTask.new(parsers: 'test:prepare') do |t|
    t.pattern = 'test/parsers/**/*_test.rb'
  end
end

Rake::Task['test:run'].enhance ["test:parsers"]

Now when we run rake or rake test then tests under test/parsers are also picked up.

Above code adds a rake task rake test:parsers which would run all tests under test/parsers directory.

We can see this task by execute rake -T test.

$ rake -T test
rake test         # Runs test:units, test:functionals, test:integration together
rake test:all     # Run tests quickly by merging all types and not resetting db
rake test:all:db  # Run tests quickly, but also reset db
rake test:parsers # Test tests/parsers/* code
rake test:recent  # Run tests for {:recent=>["test:deprecated", "test:prepare"]} / Deprecated; Test recent changes

Configuring Log Formatting in Rails

Ideally we should be logging an exception in Rails like this.

begin
  raise "Amount must be more than zero"
rescue => exception
  Rails.logger.info exception
end

Above code would produce one line log message as shown below.

Amount must be more than zero

In order to get backtrace and other information about the exception we need to handle logging like this.

begin
  raise "Amount must be more than zero"
rescue => exception
  Rails.logger.info exception.class.to_s
  Rails.logger.info exception.to_s
  Rails.logger.info exception.backtrace.join("\n")
end

Above code would produce following log message.

RuntimeError
Amount must be more than zero
/Users/nsingh/code/bigbinary_llc/wheel/app/controllers/home_controller.rb:5:in `index'
/Users/nsingh/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/actionpack-4.0.2/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/Users/nsingh/.rbenv/versions/2.0.0-p247

Now let’s look at why Rails logger does not produce detailed logging and what can be done about it.

A closer look at Formatters

When we use Rails.logger.info(exception) then the output is formatted by ActiveSupport::Logger::SimpleFormatter. It is a custom formatter defined by Rails that looks like this.

# Simple formatter which only displays the message.
class SimpleFormatter < ::Logger::Formatter
  # This method is invoked when a log event occurs
  def call(severity, timestamp, progname, msg)
    "#{String === msg ? msg : msg.inspect}\n"
  end
end

As we can see it inherits from Logger::Formatter defined by Ruby Logger . It then overrides call method which is originally defined as

#Format = "%s, [%s#%d] %5s -- %s: %s\n"
def call(severity, time, progname, msg)
  Format % [severity[0..0], format_datetime(time), $$, severity, progname,
    msg2str(msg)]
end

......
......

def msg2str(msg)
  case msg
  when ::String
    msg
  when ::Exception
    "#{ msg.message } (#{ msg.class })\n" <<
      (msg.backtrace || []).join("\n")
  else
    msg.inspect
  end
end

When exception object is passed to SimpleFormatter then msg.inspect is called and that’s why we see the exception message without any backtrace.

The problem is that Rails’s SimpleFormatter’s call method is a bit dumb compared to Ruby logger’s call method.

Ruby logger’s method has a special check for exception messages. If the message it is going to print is of class Exception then it prints backtrace also.In comparison SimpleFormatter just prints msg.inspect for objects of Exception class.

Configuring logger

This problem can be solved by using config.logger.

From Rails Configuring Guides we have

config.logger accepts a logger conforming to the interface of Log4r or the default Ruby Logger class. Defaults to an instance of ActiveSupport::Logger, with auto flushing off in production mode.

So now we can configure Rails logger to not to be SimpleFomatter and go back to ruby’s logger.

Let’s set config.logger = ::Logger.new(STDOUT) in config/application.rb and then try following code.

begin
  raise "Amount must be more than zero"
rescue => exception
  Rails.logger.info exception
end

Now above code produces following log message.

I, [2013-12-17T01:05:41.944859 #13537]  INFO -- : Amount must be more than zero (RuntimeError)
test_app/app/controllers/page_controller.rb:3:in `index'
/Users/sward/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/actionpack-4.0.2/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/Users/sward/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/actionpack-4.0.2/lib/abstract_controller/base.rb:189:in `process_action'
/Users/sward/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/actionpack-4.0.2/lib/action_controller/metal/rendering.rb:10:in `process_action'
...<snip>...

Sending log to STDOUT is also a good practice

As per http://12factor.net/logs, an application should not concern itself much with the kind of logging framework being used. The application should write log to STDOUT and logging frameworks should operate on log streams.

Displaying non repeating random records

For one of our clients we need to display random records from the database. That’s easy enough. We can use random() function.

Batch.published_and_featured.order('random()')
                            .paginate(per_page: 20, page: params[:page])

Here we are using PostgreSQL database but ,I believe, above query will also work on MySQL.

The problem here is that if the user clicks on next page then we will try to get next set of 20 random records. And since these records are truly random, sometimes the user might see the records which has already been seen in the first page.

The fix is to make it random but not truly random. It needs to be random with a seed.

Fix in MySQL

In MySQL we can pass seed directly to random() function.

Batch.published_and_featured.order('random(0.3)')
                            .paginate(per_page: 20, page: params[:page])

Fix in PostgreSQL

In PostgreSQL it is a little more cumbersome. We first need to set seed and then the subsequent query’s usage of random() will make use of seed value.

Batch.connection.execute "SELECT setseed(0.2)"
Batch.published_and_featured.order('random()')
                            .paginate(per_page: 20, page: params[:page])

Set seed value in before_action

For different user we should use different seed value and this value should be random. So we set the seed value in before_action.

def set_seed
  cookies[:random_seed] ||= SecureRandom.random_number
end

Now change the query to use the seed value and we are all set.

Active Record is still magical

WickedGoodRubyConf

I gave a talk at Wicked Good Ruby Conference conference. The conference was very well organized and I had a lot of fun meeting new people.

Confreaks has put out the video. Slides are below too. I’m sorry about the bad audio.

Boston in November is just awesome. I had a lot of fun driving around and enjoying the fall color.


Getting arguments passed to command

In previous blog we discussed ruby code where we used ps -ocommand. In this blog let’s discuss how to get arguments passed to a command.

What is the issue

In the referred blog we are trying to find if --force or -f argument was passed to the git push command.

The kernel knows the arguments that was passed to the command. So the only way to find that answer would be to to ask kernel what was the full command. The tool to deal with such issues is ps.

In order to play with ps command let’s write a simple ruby program first.

# sl.rb
puts Process.pid
puts Process.ppid
sleep 99999999

In terminal execute ruby sl.rb. In another terminal execute ps.

$ ps
  PID TTY           TIME CMD
82246 ttys000    0:00.51 -bash
87070 ttys000    0:00.04 ruby loop.rb a, b, c
82455 ttys001    0:00.40 -bash

So here I have two bash shell open in two different tabs in my terminal. First terminal tab is running s1.rb. The second terminal tab is running ps. In the second terminal we can see the the arguments that were passed to program s1.

By default ps lists all the processes belonging to the user executing the command and the processes started from the current terminal.

Option -p

ps -p87070 would show result only for the given process id.

$ ps -p 87070
  PID TTY           TIME CMD
87070 ttys000    0:00.04 ruby loop.rb a, b, c

We can pass more than on process id.

$ ps -o pid,command -p87070,82246
  PID COMMAND
82246 -bash
87070 ruby loop.rb a, b, c

Option -o

ps -o can be used to select the attributes that we want to be shown. For example I want only pids to be shown.

$ ps -o pid
  PID
82246
87070
82455

Now I want pid and command.

$ ps -o pid,command
  PID COMMAND
82246 -bash
87070 ruby loop.rb a, b, c
82455 -bash

I want result only for a certain process id.

$ ps -o command -p87070
COMMAND
ruby loop.rb a, b, c

Now we have the arguments that were passed to the command. This is the code that article was talking about.

For the sake of completeness let’s see a few more options.

Option -e

ps -e would list all processes.

$ ps -e
  PID TTY           TIME CMD
    1 ??         2:56.20 /sbin/launchd
   11 ??         0:01.90 /usr/libexec/UserEventAgent (System)
   12 ??         0:02.11 /usr/libexec/kextd
   14 ??         0:09.00 /usr/sbin/notifyd
   15 ??         0:05.81 /usr/sbin/securityd -i
   ........................................
   ........................................

Option -f

ps -f would list a lot more attributes including ppid.

$ ps -f
  UID   PID  PPID   C STIME   TTY           TIME CMD
  501 82246 82245   0  2:06PM ttys000    0:00.51 -bash
  501 87070 82246   0  4:54PM ttys000    0:00.04 ruby loop.rb a, b, c
  501 82455 82452   0  2:07PM ttys001    0:00.42 -bash

What is ppid

In previous blog we discussed ruby code where we used two things: ppid and ps -ocommand. In this blog let’s discuss ppid. ps -ocommand is discussed in the next blog.

Parent process id is ppid

We know that every process has a process id. This is usually referred as pid. In *nix world every process has a parent process. And in ruby the way to get the “process id” of the parent process is through ppid.

Let’s see it in action. Time to fire up irb.

irb(main):002:0> Process.pid
=> 83132
irb(main):003:0> Process.ppid
=> 82455

Now keep the irb session open and go to anther terminal tab. In this new tab execute pstree -p 83132

$ pstree -p 83132
-+= 00001 root /sbin/launchd
 \-+= 00151 nsingh /sbin/launchd
   \-+= 00189 nsingh /Applications/Utilities/Terminal.app/Contents/MacOS/Terminal -psn_0_45067
     \-+= 82452 root login -pf nsingh
       \-+= 82455 nsingh -bash
         \--= 83132 nsingh irb

If pstree is not available then you can easily install it using brew install pstree.

As you can see from the output the process id 83132 is at the very bottom of the tree. The parent process id is 82455 which belongs to “bash shell”.

In irb session when we did Process.ppid then we got the same value 82455.

Do not allow force push to master

At BigBinary we create a branch for every issue. We deploy that branch and only when it is approved that branch is merged into master.

Time to time we rebase the branch. And after rebasing we need to do force push to send the changes to github. And once in a while someone force pushes into master by mistake. We recommend to set push.default to current to avoid such issues but still sometimes force push does happen in master.

In order to prevent such mistakes in future we are using pre-push hook. This is a small ruby program which runs before any git push command. If you are force pushing to master then it will reject the push like this.

*************************************************************************
Your attempt to FORCE PUSH to MASTER has been rejected.
If you still want to FORCE PUSH then you need to ignore the pre_push git hook by executing following command.
git push master --force --no-verify
*************************************************************************

Requirements

pre-push hook was added to git in version 1.8.2. So you need git 1.8.2 or higher. You can easily upgrade git by executing brew upgrade git .

$ git --version
git version 1.8.2.3

Seting up hooks

In order for these hooks to kick in they need to be setup.

First step is to clone the repo to your local machine. Now open ~/.gitconfig and add following line.

[init]
  templatedir= /Users/neeraj/code/tiny_scripts/git-hooks

Change the value /Users/neeraj/code/tiny_scripts/git-hooks to match with the directory of your machine.

Making existing repositories aware of this hook

Now pre-push hook is setup. Any new repository that you clone will have the feature of not being able to force push to master.

But existing repositories do not know about this git-hook. To make existing repositories aware of this hook execute following command on all repositories.

$ git init
Reinitialized existing Git repository in /Users/nsingh/dev/projects/streetcommerce/.git/

Now if you look into the .git/hooks directory of your project you should see a file called pre-push.

$ ls .git/hooks/pre-push
.git/hooks/pre-push

It means this project is all set with pre-push hook.

New repositories

When you clone a repository then git init is invoked automatically and you will get pre-push already copied for you. So you are all set for all future repositories too.

How to ignore pre-push hook

To ignore pre-push hook all you need to do is

# Use following command to ignore pre-push check and to force update master.
git push master --force --no-verify

The hook is here .

How to keep your fork up-to-date

Let’s say that I’m forking repo rails/rails. After the repo has been forked to my repository I will clone it on my local machine.

git clone git@github.com:neerajdotname/rails.git

Now cd rails and execute git remote -v . This is what I see.

origin git@github.com:neerajdotname/rails.git (fetch)
origin git@github.com:neerajdotname/rails.git (push)

Now I will add upstream remote by executing following command.

git remote add upstream git@github.com/rails/rails.git

After having done that when I execute git remote -v then I see

origin git@github.com:neerajdotname/rails.git (fetch)
origin git@github.com:neerajdotname/rails.git (push)
upstream git://github.com/rails/rails.git (fetch)
upstream git://github.com/rails/rails.git (push)

Now I want to make some changes to the code. After all this is why I forked the repo.

Let’s say that I want to add exception handling to the forked code I have locally. Then I create a branch called exception-handling and make all your changes in this branch. The key here is to not to make any changes to master branch. I try to keep master of my forked repository in sync with the master of the original repository where I forked it.

So now let’s create a branch and I will put in all my changes there.

git checkout -b exception-handling

In the Gemfile I will use this code like this

gem 'rails', github: 'neerajdotname/rails', branch: 'exception-handling'

A month has passed. In the meantime rails master has tons of changes. I want those changes in my exception-handling branch. In order to achieve that first I need to bring my local master up-to-date with rails master.

I need to switch to master branch and then I need to execute following commands.

git checkout master
git fetch upstream
git rebase upstream/master
git push

Now the master of forked repository is in-sync with the master of rails/rails. Now that master is up-to-date I need to pull in the changes in master in my exception-handling branch.

git checkout exception-handling
git rebase master
git push -f

Now my branch exception-handling has my fix on top of rails master.

How to setup Pinch to Zoom for an image in RubyMotion

In this post we will see how to build “pinch to zoom” functionality to zoom in an image in RubyMotion.

First let’s add a UIViewController that is initialized with an image.

class ImageViewController < UIViewController
  def initWithImage(image)
    @image = image
  end
end

UIScrollView and UIImageView

Now, we will add a UIScrollView with frame size set to full screen size and some other properties as listed below.

scrollView = UIScrollView.alloc.initWithFrame(UIScreen.mainScreen.bounds)
scrollView.scrollEnabled = false
scrollView.clipsToBounds = true
scrollView.contentSize = @image.size
scrollView.minimumZoomScale = 1.0
scrollView.maximumZoomScale = 4.0
scrollView.zoomScale = 0.3

Create a new UIImageView and add it to the scrollView created above.

imageView = UIImageView.alloc.initWithImage(@image)
imageView.contentMode = UIViewContentModeScaleAspectFit
imageView.userInteractionEnabled = true
imageView.frame = scrollView.bounds

We are setting the image view’s content mode to UIViewContentModeScaleAspectFit. Content mode can be set to either UIViewContentModeScaleToFill, UIViewContentModeAspectFill or UIViewContentModeScaleAspectFit depending on what suits your app. By default, contentMode property for most views is set to UIViewContentModeScaleToFill, which causes the view’s contents to be scaled to fit the new frame size. This Apple doc explains this behavior.

We need to add the above imageView as a subview to our scrollView.

scrollView.addSubview(imageView)
self.view.addSubview(@scrollView)

This is how our controller looks with all the above additions.

class ImageViewController < UIViewController

  def initWithImage(image)
    @image = image
    scrollView = UIScrollView.alloc.initWithFrame(UIScreen.mainScreen.bounds)
    scrollView.scrollEnabled = false
    scrollView.clipsToBounds = true
    scrollView.contentSize = @image.size
    scrollView.minimumZoomScale = 1.0
    scrollView.maximumZoomScale = 4.0
    scrollView.zoomScale = 0.3
    scrollView.delegate = self

    imageView = UIImageView.alloc.initWithImage(@image)
    imageView.contentMode = UIViewContentModeScaleToFill
    imageView.userInteractionEnabled = true
    imageView.frame = scrollView.bounds
    init
  end

end

ScrollView delegate

We must set a delegate for our scroll view to support zooming. The delegate object must conform to the UIScrollViewDelegate protocol. This is the reason we are setting scrollView.delegate = self above. The delegate class must implement viewForZoomingInScrollView and scrollViewDidZoom methods.

def viewForZoomingInScrollView(scrollView)
  scrollView.subviews.first
end

def scrollViewDidZoom(scrollView)
  if scrollView.zoomScale != 1.0
    scrollView.scrollEnabled = true
  else
    scrollView.scrollEnabled = false
  end
end

These two methods added above allow the scrollView to support pinch to zoom.

Supporting orientation changes

There is one more thing to do if we want to support orientations changes. We need to add the following methods:

def shouldAutorotateToInterfaceOrientation(*)
  true
end

def viewDidLayoutSubviews
  @scrollView.frame = self.view.bounds
end

We have to set the scrollView’s frame to view bounds in viewDidLayoutSubviews so that the scrollView frame is resized when the device orientation changes.

That’s it. With all those changes now our app supports orientation change and now we are able to pinch and zoom images.

Fix image orientation issue in RubyMotion

I’m building an app using RubyMotion. When I take picture then it all looks good. However when the picture is posted on web then the orientation of the picture is different.

UIImage and UIImageOrientation

UIImage in iOS has a property called UIImageOrientation. Image orientation affects the way the image data is displayed when drawn. The api docs mention that by default, images are displayed in the up orientation. However, if the image has associated metadata (such as EXIF information), then this property contains the orientation indicated by that metadata.

After using UIImagePickerController to take an image using the iPhone camera, I was using BubbleWrap to send the image to a webserver. When the image is taken in landscape/portrait mode, then the image appeared fine when it is viewed in the browser. But, when the image is sent back via api and is shown on the iphone, the image is rotated by 90 degrees if the image is taken in portrait mode. In exif metadata, iOS incorrectly sets the orientation to UIImageOrientationRight .

Here is how I fixed the image orientation issue:

if image.imageOrientation == UIImageOrientationUp
  return_image = image
else
  UIGraphicsBeginImageContextWithOptions(image.size, false, image.scale)
  image.drawInRect([[0,0], image.size])
  normalized_image = UIImage.UIGraphicsGetImageFromCurrentImageContext
  UIGraphicsEndImageContext()
  return_image = normalized_image
end

First, we are checking the image orientation of the image we have in hand. If the image orientation is UIImageOrientationUp, we don’t have to change anything. Otherwise we are redrawing the image and returning the normalized image.

Visitor pattern and double dispatch in ruby

Let’s say that we have an AST that holds integer nodes. We want to print double the value of all nodes. We can do something like this

class IntegerNode
  def initialize(value)
    @value = value
  end

  def double
    @value * 2
  end
end

class Ast
  def initialize
    @nodes = []
    @nodes << IntegerNode.new(2)
    @nodes << IntegerNode.new(3)
  end

  def print_double
    @nodes.each do |node|
      puts node.double
    end
  end
end

ast = Ast.new
ast.print_double # => 4 6

Above solution works. Now let’s try to print triple the value. In order to do that we need to change class IntegerNode. And IntegerNode has knowledge of how to print triple value. Tomorrow if we have another node called FloatNode then that node will have knowledge about how to double and triple the value.

Nodes are merely storing information. And the representation of data should be separate from the data itself. So IntegerNode and FloatNode should not know about how to double and triple.

To take the data representation code out of nodes we can make use of visitor pattern . Visitor pattern uses double dispatch .

Before we look at “double dispatch” let’s first look at “single dispatch”.

Single dispatch

When we invoke a method in ruby we are using single dispatch. In single dispatch, method invocation is done based on a single criteria: class of the object. Most of the object oriented programming languages use single dispatch system.

In the following case method double is invoked solely based on the class of node.

node.double

Double dispatch

As the name suggests in the case of Double dispatch dispatching depends on two things: class of the object and the class of the input object.

Ruby inherently does not support “Double dispatch”. We will see how to get around that issue shortly. First let’s see an example in Java which support Double dispatch. Java supports method overloading which allows two methods with same name to differ only in the type of argument it receives.

class Node
   def double(Integer value); value *2; end
   def double(String value); Integer.parseInt(value) * 2; end
end

node.double(2)
node.double("51")

In the above case the method that would be invoked is decided based on two things: class of the object ( node ) and the class of the value (Integer or String). That’s why this is called Double dispatch.

In ruby we can’t have two methods with same name and different signature because the second method would override the first method. In order to get around that limitation usually the method name has class name. Let’s try to write above java code in ruby.

class Node
  def accept value
   method_name = "visit_#{value.class}"
   send method_name
  end

  def visit_Integer value
   value * 2
  end

  def visit_String value
    value.to_i * 2
  end
end

If the above code is not very clear then don’t worry. We are going to look at visitor pattern in ruby and that will make the above code clearer.

Visitor pattern

Now let’s get back to the problem of traversing the AST. This time we are going to use “Double dispatch” so that node information is separate from representation information.

In visitor pattern nodes define a method called accept. That method accepts the visitor and then that method calls visit on visitor passing itself as self.

Below is a concrete example of visitor pattern. You can see that IntegerNode has method accepts which takes an instance of visitor as argument. And then visit method of visitor is invoked.

class Node
  def accept visitor
    raise NotImpelementedError.new
  end
end

module Visitable
  def accept visitor
    visitor.visit self
  end
end

class IntegerNode < Node
  include Visitable

  attr_reader :value
  def initialize value
    @value = value
  end
end

class Ast < Node
  def initialize
    @nodes = []
    @nodes << IntegerNode.new(2)
    @nodes << IntegerNode.new(3)
  end

  def accept visitor
    @nodes.each do |node|
      node.accept visitor
    end
  end
end

class DoublerVisitor
  def visit subject
    puts subject.value * 2
  end
end

class TriplerVisitor
  def visit subject
    puts subject.value * 3
  end
end

ast = Ast.new
puts "Doubler:"
ast.accept DoublerVisitor.new
puts "Tripler:"
ast.accept TriplerVisitor.new

# =>
Doubler:
4
6
Tripler:
6
9

Above code used only IntegerNode. In the next example I have added StringNode. Now notice how the visit method changed. Now based on the class of the argument the method to dispatch is being decided.

class Node
  def accept visitor
    raise NotImpelementedError.new
  end
end

module Visitable
  def accept visitor
    visitor.visit(self)
  end
end

class IntegerNode < Node
  include Visitable

  attr_reader :value
  def initialize value
    @value = value
  end
end

class StringNode < Node
  include Visitable

  attr_reader :value
  def initialize value
    @value = value
  end
end

class Ast < Node
  def initialize
    @nodes = []
    @nodes << IntegerNode.new(2)
    @nodes << StringNode.new("3")
  end

  def accept visitor
    @nodes.each do |node|
      node.accept visitor
    end
  end
end

class BaseVisitor
  def visit subject
    method_name = "visit_#{subject.class}".intern
    send(method_name, subject )
  end
end

class DoublerVisitor < BaseVisitor
  def visit_IntegerNode subject
    puts subject.value * 2
  end

  def visit_StringNode subject
    puts subject.value.to_i * 2
  end
end

class TriplerVisitor < BaseVisitor
  def visit_IntegerNode subject
    puts subject.value * 3
  end

  def visit_StringNode subject
    puts subject.value.to_i * 3
  end
end

ast = Ast.new
puts "Doubler:"
ast.accept DoublerVisitor.new
puts "Tripler:"
ast.accept TriplerVisitor.new

# =>
Doubler:
4
6
Tripler:
6
9

Real world usage

Arel uses visitor pattern to build query tailored to the specific database. You can see that it has a visitor class for sqlite3, mysql and Postgresql.

You can read more about “double dispatch” in this article by Aaron Patterson.