DeccanRubyConf 2014. Hou De!

deccanrubyconflogo

I attended it, enjoyed it and took part as a volunteer in Pune’s first RubyConf - DeccanRubyConf 2014. As Hou De (let it be) name said, event went in the same way.

The day before conference, in the morning Vipul (one of the event organizers) and I picked up our guest speaker Koichi Sasada from the Pune Airport. Koichi is a Ruby core member and works for Heroku. We welcomed him and went to Hyatt Regency hotel where the event is taking place. Our guest checked into hotel and then we decided to go for a lunch at Malaka Spice restaurant.

We reached there and Koichi told us that he wanted non-spicy food (Safe food). We ordered non spicy food, but food was still too spicy for Koichi. However we enjoyed the food and had very good discussion about the Ruby internals, concurrency-parallelism, debugging in Ruby, Japanese culture and the Indian culture.

koichi & vipul

After lunch, we dropped Koichi off at the hotel and we left for our home.

Next morning was the event day and I woke up early and went to the event place. As part of volunteering team, I and other volunteers had tasks like giving pens, badges, stickers, T-shirts and coupon for night party to the attendees.

Attendees had started to come in slowly. Some attendees asked me about T-shirt size as I wore one of the conference T-shirts and from my T-shirt’s size they decided their T-shirt size. It was a great experience meeting with different kind of people from around the India.

Keynote by Koichi kicked the event off and he talked about Ruby 2.1 features like,

  • Required keyword parameter,
  • Rational number literal,
  • def returns symbol of method name
  • Runtime new features (String#scrub, Binding#local_time and etc.)

Then he talked about performance improvements, Ruby 2.2 and how to speed up Ruby Interpreter . Click here for more details about his talk.

koichi

attendees

In between the talks, some new attendees had come for the conference who had not registered for the conference. They told me that they thought it’s a regular Pune’s local Ruby Meetup. There was some miss-understandings but they seemed interested in attending the event. I contacted Gautam as he was one of the organizers and told him about the issue.

Attendees kept coming till the afternoon.

After Koichi’s talk, two sections had opened. One for talks and other for the workshops. TDD workshop was conducted by Sidu Ponnappa. I saw lots of attendees in this workshop and heard that it went very well.

tddworkshop

The next talk was on Requiem for a dream by Arnab Deka. He talked about various tips and tricks including using “Higher order functions and Concurrency” in Ruby and other programming languages like Clojure and Elixir.

After that, Rishi Jain talked about Game Development - The Ruby Way. He discussed how to build a game in Ruby using Gosu library . It was a very useful session for game developers. You can find out more about it here

Next talk was on Programming Ruby in Marathi by Ratnadeep Deshmane & his friend Aniket Awati. This was one of the best talks of the event. The way they used the similar words from Marathi for Ruby’s keywords and the examples made this talk remarkable. Their presentation style was nice too. Almost all attendees enjoyed this talk and they laughed a lot.

After this talk there was tea break for 15 minutes. Staffs from the Hyatt hotel were very helpful. There were serving tea and coffees to the attendees and overall did a good job of ensuring the the event cruised along smoothly. This is in sharp contrast to the service RubyConfIndia received from Lalit Resort.

After tea break, I didn’t get chance to attend other talks as attendees were still coming in and I was assisting them. But I heard almost all talks went very well.

In meantime, I was passing through main passage and I saw lighting talks board and decided to give lighting talk on my Ruby gem. Lighting talks is a short presentation that you can give about your achievement. You can also share your ideas and promote your library or any other projects.

Then we all had our lunch, lunch was good with lots of varieties with dessert.

After lunch I went for workshop on Deliver projects 30% faster, know your CSS by Aakash Dharmadhikari. Wanted to attend it fully, but some of the attendees had difficulties in internet connection, so I left the room to look into it.

Lightning talks were going to start so I took sometime to prepare for my presentation.

In the lighting talks, girls from Rails Girls Summer of Code, talked about their project and their progress on it. After that Prathamesh talked about RubyIndia.org and asked people to subscribe the newsletter. After that I gave talk on my Ruby gem RubySimpleSearch and you can find more on it here. The next speaker Rahul Mahale from Nashik asked people to help him in growing Ruby community in Nashik. All other lighting talks went very well.

santosh

After lighting talks, there was closing keynote On Solving Problems by Baishampayan Ghose. This talk made us think about how we write application in our daily routine. He talked about the architecture and he also explained the future is a function of past future = f(past). He also suggested that we should first understand the problem thoroughly and we should build the software. Talk was very informative and went very well.

After that Gautam came on to the stage and congratulated all the sponsors, organizers and volunteers. He also told that this event got large number of girls attendees than he had ever seen in any other conference.

After this event, there was party in Irish Village hotel. Me and my friends, we all went to the party. Party was superb and we all enjoyed it.

futsal

Thanks to all sponsors and organizers who made this event fun and enjoyable.

You can checkout more pictures of the conference from here.

Note: Photos are copyrighted by respective photo owners.

Flash access changes due to new serialization mechanism in Rails 4.1

Prior to upgrading to Rails 4.1 we had a helper to display flash messages and to add css class to the message based on flash type. Here is the code.

module FlashHelper
  ALERT_TYPES = [:success, :info, :warning, :danger]

  def bootstrap_flash
    flash_messages = []
    flash.each do |type, message|
      next if message.blank?

      type = :success if type == :notice
      type = :danger  if type == :alert
      type = :danger  if type == :error

      next unless ALERT_TYPES.include?(type)

      ....
      Array(message).each do |msg|
        text = content_tag(:div, msg.html_safe, :class => "alert fade in alert-#{type} ")
        flash_messages << text if msg
      end
    end
    flash_messages.join("\n").html_safe
  end
end

After upgrading to Rails 4.1, we started using the new Cookies serializer. Following code was added to an initializer.

  Rails.application.config.action_dispatch.cookies_serializer = :json

Soon after this our flash helper started misbehaving and all flash messages disappeared from the application.

JSON Cookies Serializer

Before we move ahead, a word on the new JSON Cookies Serializer. Applications created before Rails 4.1 uses Marshal to serialize cookie values into the signed and encrypted cookie jars.

Commits like this and this made it possible to have Cookies serializer and defaulted from Marshal Serializer to a secure Serializer using JSON.

The JSON Serializer works on JSON objects. Thus objects like Date and Time will be stored as strings. Hash keys will be stored as strings.

JSON serializer makes the application much safer since it is safer to pass around strings compare to passing around arbitrary values which is what was happens when values are marshalled and passed around.

Coming back to our problem, change Stringify the incoming hash in FlashHash coupled with above serialization changes meant that even if we put a symbol as a key in the flash we have to retrieve it as “string” since the keys are internally being converted into strings.

The difference is clearly illustrated below.

flash["string"] = "a string"
flash[:symbol] = "a symbol"

# Rails < 4.1
flash.keys # => ["string", :symbol]

# Rails >= 4.1
flash.keys # => ["string", "symbol"]

Solution

Now that we know the root cause of the problem the fix was simple. Instead of relying on symbols use “string” to access value from flash.

module BootstrapFlashHelper
  ALERT_TYPES = ['success', 'info', 'warning', 'danger']

  def bootstrap_flash
    flash_messages = []
    flash.each do |type, message|
      compare_type = type.to_s # Notice the stringifying of keys here, to make this work even with symbols.
      next if message.blank?

      compare_type = 'success' if compare_type == 'notice'
      compare_type = 'danger'  if compare_type == 'alert'
      compare_type = 'danger'  if compare_type == 'error'

      next unless ALERT_TYPES.include?(compare_type)

      Array(message).each do |msg|
        text = content_tag(:div, msg.html_safe, :class => "alert fade in alert-#{compare_type} ")
        flash_messages << text if msg
      end
    end
    flash_messages.join("\n").html_safe
  end
end

Issue with Delayed Job lifecyle and Postgres Errors

Recently in one of our projects, we experienced some strange errors from Delayed::Job. Delayed::Job workers started successfully, but when they were starting to lock the jobs, workers failed with PG::Error: no connection to server or PG::Error: FATAL: invalid frontend message type 60errors.

After some search, we found there had been such issues already experienced by others .

We started isolating the problem and digging through the recent changes we had made to the project. Since the last release the only significant modification had been made to internationalization. We had started using I18n-active_record .

# config/initializers/locale.rb

require 'i18n/backend/active_record'
Translation  = I18n::Backend::ActiveRecord::Translation

if (ActiveRecord::Base.connected? && Translation.table_exists?) ||
    in_delayed_job_process?

  I18n.backend = I18n::Backend::ActiveRecord.new
  I18n::Backend::ActiveRecord.send(:include, I18n::Backend::Memoize)
  I18n::Backend::ActiveRecord.send(:include, I18n::Backend::Flatten)
  I18n::Backend::Simple.send(:include, I18n::Backend::Memoize)
  I18n::Backend::Simple.send(:include, I18n::Backend::Pluralization)
  I18n.backend = I18n::Backend::Chain.new(I18n::Backend::Simple.new, I18n.backend)

end

for Delayed Job we had extra check as

def in_delayed_job_process?
  executable_name = File.basename $0
  arguments = $*
  rake_args_regex = /\Ajobs:/
  (executable_name == 'delayed_job') || (executable_name == 'rake' && arguments.find{ |v| v =~ rake_args_regex })
end

After some serious searching and digging through both Delayed::Job source code and how we were using to setup its config, we started noticing some issues.

The first thing we found was that the problem did not turn up when delayed job workers were started using rake jobs:work task.

After looking at DelayedJob internals we found that the main difference between a rake task and a binstub was in the fork method that was invoked in the binstub version. The binstub version was being executed seamlessly using Daemons#run_process method and had a slightly different lifecycle of execution.

DelayedJob lifecycle

Let’s take a look into DelayedJob internals before proceeding. DelayedJob has systems of the hooks that can be used by plugin-writers and in our applications. All this events functionality is hidden in Delayed::Lifecycle class. Each worker has its own instance of that class.

So, which events exactly do we have here?

Job-related events:

:enqueue
:perform
:error
:failure
:invoke_job

Worker-related events:

:execute
:loop
:perform
:error
:failure

You can setup callbacks to be run on before, after or around events simply using Delayed::Worker.lifecycle.before, Delayed::Worker.lifecycle.after and Delayed::Worker.lifecycle.around methods.

The Solution

Let’s move on to our problem. It turned out that delayed job active record gem was closing all database connections in before_fork hook and reestablishing them in after_fork hook. It was clear that I18n-active-record did not play well with this, causing the issue at hand.

We looked into DelayedJob lifecycle and chose before :execute hook, which was executed after all DelayedJob ActiveRecord backend connections manipulations.

Finally the locales initializer for delayed_job workers was changed to match as below:

require 'i18n/backend/active_record'
Translation  = I18n::Backend::ActiveRecord::Translation

Delayed::Worker.lifecycle.before :execute do
  if (ActiveRecord::Base.connected? && Translation.table_exists?) || in_delayed_job_process?
    I18n.backend = I18n::Backend::ActiveRecord.new

    I18n::Backend::ActiveRecord.send(:include, I18n::Backend::Memoize)
    I18n::Backend::ActiveRecord.send(:include, I18n::Backend::Flatten)
    I18n::Backend::Simple.send(:include, I18n::Backend::Memoize)
    I18n::Backend::Simple.send(:include, I18n::Backend::Pluralization)

    I18n.backend = I18n::Backend::Chain.new(I18n::Backend::Simple.new, I18n.backend)
  end
end

This helped us to mitigate the connection errors, and connections stopped dying abruptly.

RedDotRubyConf 2014

RedDotRubyConf2014logo

I and Vipul recently gave a talk at RedDotRubyConf on ActiveRecord can’t do it? Arel can!. It was our first trip to Singapore and we enjoyed the conference as well as Singapore a lot.

RDRC2014 was awesome.

Day 1

We reached the venue in time for Koichi’s keynote on Ruby.Inspect. He talked about various things related to development of Ruby including Ruby team at Heroku, recent releases of Ruby and new syntax introduced in Ruby 2.1. He also talked about performance improvements including Generational GC - RGenGC and upcoming features in Ruby 2.2.

After that in second part of the talk, he talked about inspection tools available in Ruby. It was a deep technical part for me and something to learn about. The message he gave from the talk was to become low level engineer.

Second talk of the conf was from T.J. Schuck about solving one of the hardest problems. Storing and retrieving passwords in a secure way. He talked about how increasing improvements in hardware pose a challenge as even if you use proper algorithm it can be cracked with high computing machines. It was interesting to know about internals of storing passwords. I had never cared too much about it :)

After the coffee break, Brandon Keepers from Github gave talk on Tending Your Open Source Garden. Github is still on Rails 2.3 and Brandon is working on bringing it up to new version. His talk was a great advice for those who want to contribute to open source and community. I think this talk resonated well with the audience as most of the crowd was new and interested in open source contributions.

Gautam Rege from Josh Software gave talk on Dark Side of Ruby. We had attended this talk at GCRC so we left the hall after some time and did our one last practice. But i heard the feedback was very well for this talk.

After the lunch, Keith Pitt talked about Guide to Continuous Deployment with Rails. He talked about keeping everything related to deployment from CI to migrations in sync. One of the interesting thing that i came to know from this talk was how to enable zero downtime deployments on Heroku using preboot feature.

Benjamin Tan gave talk on Ruby + Elixir: Polyglotting FTW! after that. He talked about Elixir. This talk was about looking beyond Ruby and adding another tool to our skills. Benjamin also gave some demos including the last one in which he used sidekiq with Elixir. The actual work was done by Elixir workers. I will definitely give a shot to Elixir in the coming days.

After that we gave our talk on Arel. I was a bit nervous as it was my first talk. But it went well. We finished a bit early than expected. But there was tea break after our talk :). We got some good feedback from the attendees and especially beginners who had not used Arel before. Our slides are here

After the tea break, lightning talks started. First talk was by Hiroshi Shibata about how anyone can contribute to Ruby to make it better. He talked about how to submit issues, feature requests using Redmine. After that William Notowidagdo gave talk on Building REST API using Grape. With Rails and Rails API gem it has become easy to generate an API. But we have grape also as a lightweight tool. Sayani Basu talked about how to make a podcast with Jekyll and other tools in 5 minutes.

We are planning to start a podcast here in India about Ruby community so it was good to know about it.

Update: Checkout our podcast .

After these awesome lightning talks, the last session of Day 1 started. There were talks on Fluentd and Domain driven design. Both were good to know as something outside of daily routine. Konstantin Haase’s last talk of the day was **Meta** talk. He talked on abstraction and how it happens in our mind. Our mind affects what we see, like we see magenta color. Similarly abstraction happens in mind. I had to concentrate a lot in this talk to understand it. But it was worth it.

Andddd that ended the first day of the conf. It was exciting and we were looking forward to second day.

Day 2

Day 2 started with Brian Helmkamp’s talk on Docker. We missed the initial part of the talk. He talked about basics of Docker, how to deploying in container environment. He also discussed about deploying a Rails app using docker and how it makes very easy to deploy different parts of the system using docker very easily.

Zachary Scott gave next talk introducing Ruby Core team and how it works, how it collaborates, developer meetings, how anyone can contribute to MRI. We also had a Friday hug in this talk :) This talk combined with Hiroshi’s lightning talk on the first day was great insight into CRuby development.

After the break, Pioter Solnica gave an excellent talk on Convenience vs Simplicity. He talked about convenience offered by ActiveRecord may not be simple to understand. The things such as input conversion, validation are convenient to use as a developer but not necessarily simple to understand. He also discussed presenters, immutable data structures, Adamantium for creating immutable objects in Ruby. In the second part of the talk, he talked about relations and how they can be used in composing queries. He explained this idea using Ruby Object Mapper. It uses Axiom as underlying relational algebra instead of Arel. Its an interesting project to checkout.

After that our very own Anil Wadghule talked on Solid Design Principles in Ruby. His emphasis was on following designs than patterns. He also showed code examples and refactored them after applying principles. His talk was good insight into understanding what are these Solid principles and how they can be applied in real life.

We skipped the session after lunch and roamed around talking with people. We had an interesting discussion about hiring Ruby on Rails developers, interview processes etc.

Then lightning talks started. Sheng-Loong-Su talked first on using Algorithms for Trading. He talked about collecting data using feeder, preparing trading signals using strategy and making decision based on trading signals. One of the best talks of day 2 was by Grzegorz Witek on how he is traveling the world without getting burned out and still happily programming. He talked about his experiences in different countries being a Nomadic Programmer. It was one of the best inspirational talks according to me. The last lightning talk was about Using Vagrant for setting up Dev environment by Shuwei and Arathi.

Nicholas Simmons talked on experience of building a Single page web app and back again to normal app. He gave real life metrics from Shopify and showed problems faced with single page apps, batman.js and how moving back to a normal app helped them.

Then chocolate man from Belgium, Christophe Philemotte, gave talk on Safety Nets: Learn to code with confidence. His talk was about how we can prevent code in long term using testing, static analysis using tools such as flog, flay, rubocop for removing duplication, reducing complexity, fixing warnings. He also talked about importance of code review. His code is present here. He also gave us excellent chocolates from Belgium.

And the last keynote by Aaron Patterson. As always, it was full of everything - tech stuff, jokes, puns.

He talked on how he is making performance improvements in Active Record, link generations. He showed some graphs with performance of various database adapters tested on Rails versions ranging from 2.3 to 4 to master. He urged everyone to report performance issues to the core team so that they are addressed quickly. This is the app used for doing performance testing by him.

And that ended talks at RDRC. We had an awesome after party where we discussed with lots of people about Ruby, Rails as well as other stuff. We would like to thank Winston for inviting us to RedDotRubyConf.

Checkout some of the pictures from the conference here by Nestor.

I am already looking at RDRC 2015.

Six Year Old Optional / Keyword Arguments bug

I recently conducted a workshop about Contributing to Open-Source at first-ever Rubyconf Philippines. In its introductory talk, I spoke about how Aaron Patterson, fixed a 6 year old bug about Optional Arguments, that existed in Rails.

Bug in ruby

Let’s try a small program.

class Lab

  def day
    puts 'invoked'
    'sunday'
  end

  def run
    day = day
  end

end

puts Lab.new.run

What do you think would be printed on your terminal when you run the above program.

If you are using ruby 2.1 or below then you will see nothing. Why is that ? That’s because of a bug in ruby.

This is bug number 9593 in ruby issue tracker.

In the statement day = day the left hand side variable assignment is stopping the call to method day. So the method day is never invoked.

Another variation of the same bug

class Lab

  def day
    puts 'invoked'
    'sunday'
  end

  def run( day: day)
  end

end

puts Lab.new.run

In the above case we are using the keyword argument feature added in Ruby 2.0 . If you are unfamiliar with keyword arguments feature of ruby then checkout this excellent video by Peter Cooper.

In this case again the same behavior is exhibited. The method day is never invoked.

How this bug affects Rails community

You might be thinking that I would never write code like that. Why would you have a variable name same as method name.

Well Rails had this bug because rails has code like this.

def has_cached_counter?(reflection = reflection)
end

In this case method reflection never got called and the variable reflection was always assigned nil.

Fixing the bug

Nobu fixed this bug in ruby 2.2.0. By the way Nobu is also known as “ruby patch-monster” because of amount of patches he applies to ruby.

So this bug is fixed in ruby 2.2.0. What about the people who are not using ruby 2.2.0.

The simple solution is not to omit the parameter. If we change the above code to

def has_cached_counter?(reflection = reflection())
end

then we are explicitly invoking the method reflection and the variable reflection will be assigned the output of method reflection.

And this is how Aaron Patterson fixed six years old bug.

How to deploy jekyll site to heroku

jekyll is an excellent tool for creating static pages and blogs. Our BigBinary blog is based on jekyll. Deploying our blog to heroku took longer than I had expected. I am outlining what I did to deploy BigBinary blog to heroku.

Add exclude vendor to _config.yml

Open _config.yml and add following line at the very bottom.

exclude: ['vendor']

Add Procfile

Create a new file called Procfile at the root of the project with following content.

web: bundle exec jekyll build && bundle exec thin start -p$PORT -V
console: echo console
rake: echo rake

Add Gemfile

Add Gemfile at the root of the project.

source 'https://rubygems.org'

gem 'jekyll', '2.4.0'
gem 'rake'
gem 'foreman'
gem 'thin'
gem 'rack-contrib'

Add config.ru

Add config.ru at the root of the project with following content.

require 'rack/contrib/try_static'

use Rack::TryStatic,
    :root => "_site",
    :urls => %w[/],
    :try => ['.html', 'index.html', '/index.html']

run lambda { |env|
  return [404, {'Content-Type' => 'text/html'}, ['Not Found']]
}

Test on local machine first

Test locally by executing bundle exec jekyll serve.

Push code to heroku

Now run bundle install and add the Gemfile.lock to the repository and push the repository to heroku.

How to add additional directories to test

In a project we needed to write different parsers for different services. Rather than putting all those parsers in app/models or in lib we created a new directory. We put all the parsers in app/parsers .

We put all the tests for these parsers in test/parsers directory.

We can run tests parsers individually by executing rake test test/parsers/email_parser_test.rb. However when we run rake then tests in test/parsers are not picked up.

We added following code to Rakefile to make rake pickup tests in test/parsers.

# Adding test/parsers directory to rake test.
namespace :test do
  desc "Test tests/parsers/* code"
  Rails::TestTask.new(parsers: 'test:prepare') do |t|
    t.pattern = 'test/parsers/**/*_test.rb'
  end
end

Rake::Task['test:run'].enhance ["test:parsers"]

Now when we run rake or rake test then tests under test/parsers are also picked up.

Above code adds a rake task rake test:parsers which would run all tests under test/parsers directory.

We can see this task by execute rake -T test.

$ rake -T test
rake test         # Runs test:units, test:functionals, test:integration together
rake test:all     # Run tests quickly by merging all types and not resetting db
rake test:all:db  # Run tests quickly, but also reset db
rake test:parsers # Test tests/parsers/* code
rake test:recent  # Run tests for {:recent=>["test:deprecated", "test:prepare"]} / Deprecated; Test recent changes

Configuring Log Formatting in Rails

Ideally we should be logging an exception in Rails like this.

begin
  raise "Amount must be more than zero"
rescue => exception
  Rails.logger.info exception
end

Above code would produce one line log message as shown below.

Amount must be more than zero

In order to get backtrace and other information about the exception we need to handle logging like this.

begin
  raise "Amount must be more than zero"
rescue => exception
  Rails.logger.info exception.class.to_s
  Rails.logger.info exception.to_s
  Rails.logger.info exception.backtrace.join("\n")
end

Above code would produce following log message.

RuntimeError
Amount must be more than zero
/Users/nsingh/code/bigbinary_llc/wheel/app/controllers/home_controller.rb:5:in `index'
/Users/nsingh/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/actionpack-4.0.2/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/Users/nsingh/.rbenv/versions/2.0.0-p247

Now let’s look at why Rails logger does not produce detailed logging and what can be done about it.

A closer look at Formatters

When we use Rails.logger.info(exception) then the output is formatted by ActiveSupport::Logger::SimpleFormatter. It is a custom formatter defined by Rails that looks like this.

# Simple formatter which only displays the message.
class SimpleFormatter < ::Logger::Formatter
  # This method is invoked when a log event occurs
  def call(severity, timestamp, progname, msg)
    "#{String === msg ? msg : msg.inspect}\n"
  end
end

As we can see it inherits from Logger::Formatter defined by Ruby Logger . It then overrides call method which is originally defined as

#Format = "%s, [%s#%d] %5s -- %s: %s\n"
def call(severity, time, progname, msg)
  Format % [severity[0..0], format_datetime(time), $$, severity, progname,
    msg2str(msg)]
end

......
......

def msg2str(msg)
  case msg
  when ::String
    msg
  when ::Exception
    "#{ msg.message } (#{ msg.class })\n" <<
      (msg.backtrace || []).join("\n")
  else
    msg.inspect
  end
end

When exception object is passed to SimpleFormatter then msg.inspect is called and that’s why we see the exception message without any backtrace.

The problem is that Rails’s SimpleFormatter’s call method is a bit dumb compared to Ruby logger’s call method.

Ruby logger’s method has a special check for exception messages. If the message it is going to print is of class Exception then it prints backtrace also.In comparison SimpleFormatter just prints msg.inspect for objects of Exception class.

Configuring logger

This problem can be solved by using config.logger.

From Rails Configuring Guides we have

config.logger accepts a logger conforming to the interface of Log4r or the default Ruby Logger class. Defaults to an instance of ActiveSupport::Logger, with auto flushing off in production mode.

So now we can configure Rails logger to not to be SimpleFomatter and go back to ruby’s logger.

Let’s set config.logger = ::Logger.new(STDOUT) in config/application.rb and then try following code.

begin
  raise "Amount must be more than zero"
rescue => exception
  Rails.logger.info exception
end

Now above code produces following log message.

I, [2013-12-17T01:05:41.944859 #13537]  INFO -- : Amount must be more than zero (RuntimeError)
test_app/app/controllers/page_controller.rb:3:in `index'
/Users/sward/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/actionpack-4.0.2/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/Users/sward/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/actionpack-4.0.2/lib/abstract_controller/base.rb:189:in `process_action'
/Users/sward/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/actionpack-4.0.2/lib/action_controller/metal/rendering.rb:10:in `process_action'
...<snip>...

Sending log to STDOUT is also a good practice

As per http://12factor.net/logs, an application should not concern itself much with the kind of logging framework being used. The application should write log to STDOUT and logging frameworks should operate on log streams.

Displaying non repeating random records

For one of our clients we need to display random records from the database. That’s easy enough. We can use random() function.

Batch.published_and_featured.order('random()')
                            .paginate(per_page: 20, page: params[:page])

Here we are using PostgreSQL database but ,I believe, above query will also work on MySQL.

The problem here is that if the user clicks on next page then we will try to get next set of 20 random records. And since these records are truly random, sometimes the user might see the records which has already been seen in the first page.

The fix is to make it random but not truly random. It needs to be random with a seed.

Fix in MySQL

In MySQL we can pass seed directly to random() function.

Batch.published_and_featured.order('random(0.3)')
                            .paginate(per_page: 20, page: params[:page])

Fix in PostgreSQL

In PostgreSQL it is a little more cumbersome. We first need to set seed and then the subsequent query’s usage of random() will make use of seed value.

Batch.connection.execute "SELECT setseed(0.2)"
Batch.published_and_featured.order('random()')
                            .paginate(per_page: 20, page: params[:page])

Set seed value in before_action

For different user we should use different seed value and this value should be random. So we set the seed value in before_action.

def set_seed
  cookies[:random_seed] ||= SecureRandom.random_number
end

Now change the query to use the seed value and we are all set.

Active Record is still magical

WickedGoodRubyConf

I gave a talk at Wicked Good Ruby Conference conference. The conference was very well organized and I had a lot of fun meeting new people.

Confreaks has put out the video. Slides are below too. I’m sorry about the bad audio.

Boston in November is just awesome. I had a lot of fun driving around and enjoying the fall color.


Getting arguments passed to command

In previous blog we discussed ruby code where we used ps -ocommand. In this blog let’s discuss how to get arguments passed to a command.

What is the issue

In the referred blog we are trying to find if --force or -f argument was passed to the git push command.

The kernel knows the arguments that was passed to the command. So the only way to find that answer would be to to ask kernel what was the full command. The tool to deal with such issues is ps.

In order to play with ps command let’s write a simple ruby program first.

# sl.rb
puts Process.pid
puts Process.ppid
sleep 99999999

In terminal execute ruby sl.rb. In another terminal execute ps.

$ ps
  PID TTY           TIME CMD
82246 ttys000    0:00.51 -bash
87070 ttys000    0:00.04 ruby loop.rb a, b, c
82455 ttys001    0:00.40 -bash

So here I have two bash shell open in two different tabs in my terminal. First terminal tab is running s1.rb. The second terminal tab is running ps. In the second terminal we can see the the arguments that were passed to program s1.

By default ps lists all the processes belonging to the user executing the command and the processes started from the current terminal.

Option -p

ps -p87070 would show result only for the given process id.

$ ps -p 87070
  PID TTY           TIME CMD
87070 ttys000    0:00.04 ruby loop.rb a, b, c

We can pass more than on process id.

$ ps -o pid,command -p87070,82246
  PID COMMAND
82246 -bash
87070 ruby loop.rb a, b, c

Option -o

ps -o can be used to select the attributes that we want to be shown. For example I want only pids to be shown.

$ ps -o pid
  PID
82246
87070
82455

Now I want pid and command.

$ ps -o pid,command
  PID COMMAND
82246 -bash
87070 ruby loop.rb a, b, c
82455 -bash

I want result only for a certain process id.

$ ps -o command -p87070
COMMAND
ruby loop.rb a, b, c

Now we have the arguments that were passed to the command. This is the code that article was talking about.

For the sake of completeness let’s see a few more options.

Option -e

ps -e would list all processes.

$ ps -e
  PID TTY           TIME CMD
    1 ??         2:56.20 /sbin/launchd
   11 ??         0:01.90 /usr/libexec/UserEventAgent (System)
   12 ??         0:02.11 /usr/libexec/kextd
   14 ??         0:09.00 /usr/sbin/notifyd
   15 ??         0:05.81 /usr/sbin/securityd -i
   ........................................
   ........................................

Option -f

ps -f would list a lot more attributes including ppid.

$ ps -f
  UID   PID  PPID   C STIME   TTY           TIME CMD
  501 82246 82245   0  2:06PM ttys000    0:00.51 -bash
  501 87070 82246   0  4:54PM ttys000    0:00.04 ruby loop.rb a, b, c
  501 82455 82452   0  2:07PM ttys001    0:00.42 -bash

What is ppid

In previous blog we discussed ruby code where we used two things: ppid and ps -ocommand. In this blog let’s discuss ppid. ps -ocommand is discussed in the next blog.

Parent process id is ppid

We know that every process has a process id. This is usually referred as pid. In *nix world every process has a parent process. And in ruby the way to get the “process id” of the parent process is through ppid.

Let’s see it in action. Time to fire up irb.

irb(main):002:0> Process.pid
=> 83132
irb(main):003:0> Process.ppid
=> 82455

Now keep the irb session open and go to anther terminal tab. In this new tab execute pstree -p 83132

$ pstree -p 83132
-+= 00001 root /sbin/launchd
 \-+= 00151 nsingh /sbin/launchd
   \-+= 00189 nsingh /Applications/Utilities/Terminal.app/Contents/MacOS/Terminal -psn_0_45067
     \-+= 82452 root login -pf nsingh
       \-+= 82455 nsingh -bash
         \--= 83132 nsingh irb

If pstree is not available then you can easily install it using brew install pstree.

As you can see from the output the process id 83132 is at the very bottom of the tree. The parent process id is 82455 which belongs to “bash shell”.

In irb session when we did Process.ppid then we got the same value 82455.

Do not allow force push to master

At BigBinary we create a branch for every issue. We deploy that branch and only when it is approved that branch is merged into master.

Time to time we rebase the branch. And after rebasing we need to do force push to send the changes to github. And once in a while someone force pushes into master by mistake. We recommend to set push.default to current to avoid such issues but still sometimes force push does happen in master.

In order to prevent such mistakes in future we are using pre-push hook. This is a small ruby program which runs before any git push command. If you are force pushing to master then it will reject the push like this.

*************************************************************************
Your attempt to FORCE PUSH to MASTER has been rejected.
If you still want to FORCE PUSH then you need to ignore the pre_push git hook by executing following command.
git push master --force --no-verify
*************************************************************************

Requirements

pre-push hook was added to git in version 1.8.2. So you need git 1.8.2 or higher. You can easily upgrade git by executing brew upgrade git .

$ git --version
git version 1.8.2.3

Seting up hooks

In order for these hooks to kick in they need to be setup.

First step is to clone the repo to your local machine. Now open ~/.gitconfig and add following line.

[init]
  templatedir= /Users/neeraj/code/tiny_scripts/git-hooks

Change the value /Users/neeraj/code/tiny_scripts/git-hooks to match with the directory of your machine.

Making existing repositories aware of this hook

Now pre-push hook is setup. Any new repository that you clone will have the feature of not being able to force push to master.

But existing repositories do not know about this git-hook. To make existing repositories aware of this hook execute following command on all repositories.

$ git init
Reinitialized existing Git repository in /Users/nsingh/dev/projects/streetcommerce/.git/

Now if you look into the .git/hooks directory of your project you should see a file called pre-push.

$ ls .git/hooks/pre-push
.git/hooks/pre-push

It means this project is all set with pre-push hook.

New repositories

When you clone a repository then git init is invoked automatically and you will get pre-push already copied for you. So you are all set for all future repositories too.

How to ignore pre-push hook

To ignore pre-push hook all you need to do is

# Use following command to ignore pre-push check and to force update master.
git push master --force --no-verify

The hook is here .

How to keep your fork up-to-date

Let’s say that I’m forking repo rails/rails. After the repo has been forked to my repository I will clone it on my local machine.

git clone git@github.com:neerajdotname/rails.git

Now cd rails and execute git remote -v . This is what I see.

origin git@github.com:neerajdotname/rails.git (fetch)
origin git@github.com:neerajdotname/rails.git (push)

Now I will add upstream remote by executing following command.

git remote add upstream git@github.com/rails/rails.git

After having done that when I execute git remote -v then I see

origin git@github.com:neerajdotname/rails.git (fetch)
origin git@github.com:neerajdotname/rails.git (push)
upstream git://github.com/rails/rails.git (fetch)
upstream git://github.com/rails/rails.git (push)

Now I want to make some changes to the code. After all this is why I forked the repo.

Let’s say that I want to add exception handling to the forked code I have locally. Then I create a branch called exception-handling and make all your changes in this branch. The key here is to not to make any changes to master branch. I try to keep master of my forked repository in sync with the master of the original repository where I forked it.

So now let’s create a branch and I will put in all my changes there.

git checkout -b exception-handling

In the Gemfile I will use this code like this

gem 'rails', github: 'neerajdotname/rails', branch: 'exception-handling'

A month has passed. In the meantime rails master has tons of changes. I want those changes in my exception-handling branch. In order to achieve that first I need to bring my local master up-to-date with rails master.

I need to switch to master branch and then I need to execute following commands.

git checkout master
git fetch upstream
git rebase upstream/master
git push

Now the master of forked repository is in-sync with the master of rails/rails. Now that master is up-to-date I need to pull in the changes in master in my exception-handling branch.

git checkout exception-handling
git rebase master
git push -f

Now my branch exception-handling has my fix on top of rails master.

How to setup Pinch to Zoom for an image in RubyMotion

In this post we will see how to build “pinch to zoom” functionality to zoom in an image in RubyMotion.

First let’s add a UIViewController that is initialized with an image.

class ImageViewController < UIViewController
  def initWithImage(image)
    @image = image
  end
end

UIScrollView and UIImageView

Now, we will add a UIScrollView with frame size set to full screen size and some other properties as listed below.

scrollView = UIScrollView.alloc.initWithFrame(UIScreen.mainScreen.bounds)
scrollView.scrollEnabled = false
scrollView.clipsToBounds = true
scrollView.contentSize = @image.size
scrollView.minimumZoomScale = 1.0
scrollView.maximumZoomScale = 4.0
scrollView.zoomScale = 0.3

Create a new UIImageView and add it to the scrollView created above.

imageView = UIImageView.alloc.initWithImage(@image)
imageView.contentMode = UIViewContentModeScaleAspectFit
imageView.userInteractionEnabled = true
imageView.frame = scrollView.bounds

We are setting the image view’s content mode to UIViewContentModeScaleAspectFit. Content mode can be set to either UIViewContentModeScaleToFill, UIViewContentModeAspectFill or UIViewContentModeScaleAspectFit depending on what suits your app. By default, contentMode property for most views is set to UIViewContentModeScaleToFill, which causes the view’s contents to be scaled to fit the new frame size. This Apple doc explains this behavior.

We need to add the above imageView as a subview to our scrollView.

scrollView.addSubview(imageView)
self.view.addSubview(@scrollView)

This is how our controller looks with all the above additions.

class ImageViewController < UIViewController

  def initWithImage(image)
    @image = image
    scrollView = UIScrollView.alloc.initWithFrame(UIScreen.mainScreen.bounds)
    scrollView.scrollEnabled = false
    scrollView.clipsToBounds = true
    scrollView.contentSize = @image.size
    scrollView.minimumZoomScale = 1.0
    scrollView.maximumZoomScale = 4.0
    scrollView.zoomScale = 0.3
    scrollView.delegate = self

    imageView = UIImageView.alloc.initWithImage(@image)
    imageView.contentMode = UIViewContentModeScaleToFill
    imageView.userInteractionEnabled = true
    imageView.frame = scrollView.bounds
    init
  end

end

ScrollView delegate

We must set a delegate for our scroll view to support zooming. The delegate object must conform to the UIScrollViewDelegate protocol. This is the reason we are setting scrollView.delegate = self above. The delegate class must implement viewForZoomingInScrollView and scrollViewDidZoom methods.

def viewForZoomingInScrollView(scrollView)
  scrollView.subviews.first
end

def scrollViewDidZoom(scrollView)
  if scrollView.zoomScale != 1.0
    scrollView.scrollEnabled = true
  else
    scrollView.scrollEnabled = false
  end
end

These two methods added above allow the scrollView to support pinch to zoom.

Supporting orientation changes

There is one more thing to do if we want to support orientations changes. We need to add the following methods:

def shouldAutorotateToInterfaceOrientation(*)
  true
end

def viewDidLayoutSubviews
  @scrollView.frame = self.view.bounds
end

We have to set the scrollView’s frame to view bounds in viewDidLayoutSubviews so that the scrollView frame is resized when the device orientation changes.

That’s it. With all those changes now our app supports orientation change and now we are able to pinch and zoom images.

Fix image orientation issue in RubyMotion

I’m building an app using RubyMotion. When I take picture then it all looks good. However when the picture is posted on web then the orientation of the picture is different.

UIImage and UIImageOrientation

UIImage in iOS has a property called UIImageOrientation. Image orientation affects the way the image data is displayed when drawn. The api docs mention that by default, images are displayed in the up orientation. However, if the image has associated metadata (such as EXIF information), then this property contains the orientation indicated by that metadata.

After using UIImagePickerController to take an image using the iPhone camera, I was using BubbleWrap to send the image to a webserver. When the image is taken in landscape/portrait mode, then the image appeared fine when it is viewed in the browser. But, when the image is sent back via api and is shown on the iphone, the image is rotated by 90 degrees if the image is taken in portrait mode. In exif metadata, iOS incorrectly sets the orientation to UIImageOrientationRight .

Here is how I fixed the image orientation issue:

if image.imageOrientation == UIImageOrientationUp
  return_image = image
else
  UIGraphicsBeginImageContextWithOptions(image.size, false, image.scale)
  image.drawInRect([[0,0], image.size])
  normalized_image = UIImage.UIGraphicsGetImageFromCurrentImageContext
  UIGraphicsEndImageContext()
  return_image = normalized_image
end

First, we are checking the image orientation of the image we have in hand. If the image orientation is UIImageOrientationUp, we don’t have to change anything. Otherwise we are redrawing the image and returning the normalized image.

Visitor pattern and double dispatch in ruby

Let’s say that we have an AST that holds integer nodes. We want to print double the value of all nodes. We can do something like this

class IntegerNode
  def initialize(value)
    @value = value
  end

  def double
    @value * 2
  end
end

class Ast
  def initialize
    @nodes = []
    @nodes << IntegerNode.new(2)
    @nodes << IntegerNode.new(3)
  end

  def print_double
    @nodes.each do |node|
      puts node.double
    end
  end
end

ast = Ast.new
ast.print_double # => 4 6

Above solution works. Now let’s try to print triple the value. In order to do that we need to change class IntegerNode. And IntegerNode has knowledge of how to print triple value. Tomorrow if we have another node called FloatNode then that node will have knowledge about how to double and triple the value.

Nodes are merely storing information. And the representation of data should be separate from the data itself. So IntegerNode and FloatNode should not know about how to double and triple.

To take the data representation code out of nodes we can make use of visitor pattern . Visitor pattern uses double dispatch .

Before we look at “double dispatch” let’s first look at “single dispatch”.

Single dispatch

When we invoke a method in ruby we are using single dispatch. In single dispatch, method invocation is done based on a single criteria: class of the object. Most of the object oriented programming languages use single dispatch system.

In the following case method double is invoked solely based on the class of node.

node.double

Double dispatch

As the name suggests in the case of Double dispatch dispatching depends on two things: class of the object and the class of the input object.

Ruby inherently does not support “Double dispatch”. We will see how to get around that issue shortly. First let’s see an example in Java which support Double dispatch. Java supports method overloading which allows two methods with same name to differ only in the type of argument it receives.

class Node
   def double(Integer value); value *2; end
   def double(String value); Integer.parseInt(value) * 2; end
end

node.double(2)
node.double("51")

In the above case the method that would be invoked is decided based on two things: class of the object ( node ) and the class of the value (Integer or String). That’s why this is called Double dispatch.

In ruby we can’t have two methods with same name and different signature because the second method would override the first method. In order to get around that limitation usually the method name has class name. Let’s try to write above java code in ruby.

class Node
  def accept value
   method_name = "visit_#{value.class}"
   send method_name
  end

  def visit_Integer value
   value * 2
  end

  def visit_String value
    value.to_i * 2
  end
end

If the above code is not very clear then don’t worry. We are going to look at visitor pattern in ruby and that will make the above code clearer.

Visitor pattern

Now let’s get back to the problem of traversing the AST. This time we are going to use “Double dispatch” so that node information is separate from representation information.

In visitor pattern nodes define a method called accept. That method accepts the visitor and then that method calls visit on visitor passing itself as self.

Below is a concrete example of visitor pattern. You can see that IntegerNode has method accepts which takes an instance of visitor as argument. And then visit method of visitor is invoked.

class Node
  def accept visitor
    raise NotImpelementedError.new
  end
end

module Visitable
  def accept visitor
    visitor.visit self
  end
end

class IntegerNode < Node
  include Visitable

  attr_reader :value
  def initialize value
    @value = value
  end
end

class Ast < Node
  def initialize
    @nodes = []
    @nodes << IntegerNode.new(2)
    @nodes << IntegerNode.new(3)
  end

  def accept visitor
    @nodes.each do |node|
      node.accept visitor
    end
  end
end

class DoublerVisitor
  def visit subject
    puts subject.value * 2
  end
end

class TriplerVisitor
  def visit subject
    puts subject.value * 3
  end
end

ast = Ast.new
puts "Doubler:"
ast.accept DoublerVisitor.new
puts "Tripler:"
ast.accept TriplerVisitor.new

# =>
Doubler:
4
6
Tripler:
6
9

Above code used only IntegerNode. In the next example I have added StringNode. Now notice how the visit method changed. Now based on the class of the argument the method to dispatch is being decided.

class Node
  def accept visitor
    raise NotImpelementedError.new
  end
end

module Visitable
  def accept visitor
    visitor.visit(self)
  end
end

class IntegerNode < Node
  include Visitable

  attr_reader :value
  def initialize value
    @value = value
  end
end

class StringNode < Node
  include Visitable

  attr_reader :value
  def initialize value
    @value = value
  end
end

class Ast < Node
  def initialize
    @nodes = []
    @nodes << IntegerNode.new(2)
    @nodes << StringNode.new("3")
  end

  def accept visitor
    @nodes.each do |node|
      node.accept visitor
    end
  end
end

class BaseVisitor
  def visit subject
    method_name = "visit_#{subject.class}".intern
    send(method_name, subject )
  end
end

class DoublerVisitor < BaseVisitor
  def visit_IntegerNode subject
    puts subject.value * 2
  end

  def visit_StringNode subject
    puts subject.value.to_i * 2
  end
end

class TriplerVisitor < BaseVisitor
  def visit_IntegerNode subject
    puts subject.value * 3
  end

  def visit_StringNode subject
    puts subject.value.to_i * 3
  end
end

ast = Ast.new
puts "Doubler:"
ast.accept DoublerVisitor.new
puts "Tripler:"
ast.accept TriplerVisitor.new

# =>
Doubler:
4
6
Tripler:
6
9

Real world usage

Arel uses visitor pattern to build query tailored to the specific database. You can see that it has a visitor class for sqlite3, mysql and Postgresql.

You can read more about “double dispatch” in this article by Aaron Patterson.

Preload, Eagerload, Includes and Joins

Rails provides four different ways to load association data. In this blog we are going to look at each of those.

Preload

Preload loads the association data in a separate query.

User.preload(:posts).to_a

# =>
SELECT "users".* FROM "users"
SELECT "posts".* FROM "posts"  WHERE "posts"."user_id" IN (1)

This is how includes loads data in the default case.

Since preload always generates two sql we can’t use posts table in where condition. Following query will result in an error.

User.preload(:posts).where("posts.desc='ruby is awesome'")

# =>
SQLite3::SQLException: no such column: posts.desc:
SELECT "users".* FROM "users"  WHERE (posts.desc='ruby is awesome')

With preload where clauses can be applied.

User.preload(:posts).where("users.name='Neeraj'")

# =>
SELECT "users".* FROM "users"  WHERE (users.name='Neeraj')
SELECT "posts".* FROM "posts"  WHERE "posts"."user_id" IN (3)

Includes

Includes loads the association data in a separate query just like preload.

However it is smarter than preload. Above we saw that preload failed for query User.preload(:posts).where("posts.desc='ruby is awesome'"). Let’s try same with includes.

User.includes(:posts).where('posts.desc = "ruby is awesome"').to_a

# =>
SELECT "users"."id" AS t0_r0, "users"."name" AS t0_r1, "posts"."id" AS t1_r0,
       "posts"."title" AS t1_r1,
       "posts"."user_id" AS t1_r2, "posts"."desc" AS t1_r3
FROM "users" LEFT OUTER JOIN "posts" ON "posts"."user_id" = "users"."id"
WHERE (posts.desc = "ruby is awesome")

As you can see includes switches from using two separate queries to creating a single LEFT OUTER JOIN to get the data. And it also applied the supplied condition.

So includes changes from two queries to a single query in some cases. By default for a simple case it will use two queries. Let’s say that for some reason you want to force a simple includes case to use a single query instead of two. Use references to achieve that.

User.includes(:posts).references(:posts).to_a

# =>
SELECT "users"."id" AS t0_r0, "users"."name" AS t0_r1, "posts"."id" AS t1_r0,
       "posts"."title" AS t1_r1,
       "posts"."user_id" AS t1_r2, "posts"."desc" AS t1_r3
FROM "users" LEFT OUTER JOIN "posts" ON "posts"."user_id" = "users"."id"

In the above case a single query was done.

Eager load

eager loading loads all association in a single query using LEFT OUTER JOIN.

User.eager_load(:posts).to_a

# =>
SELECT "users"."id" AS t0_r0, "users"."name" AS t0_r1, "posts"."id" AS t1_r0,
       "posts"."title" AS t1_r1, "posts"."user_id" AS t1_r2, "posts"."desc" AS t1_r3
FROM "users" LEFT OUTER JOIN "posts" ON "posts"."user_id" = "users"."id"

This is exactly what includes does when it is forced to make a single query when where or order clause is using an attribute from posts table.

Joins

Joins brings association data using inner join.

User.joins(:posts)

# =>
SELECT "users".* FROM "users" INNER JOIN "posts" ON "posts"."user_id" = "users"."id"

In the above case no posts data is selected. Above query can also produce duplicate result. To see it let’s create some sample data.

def self.setup
  User.delete_all
  Post.delete_all

  u = User.create name: 'Neeraj'
  u.posts.create! title: 'ruby', desc: 'ruby is awesome'
  u.posts.create! title: 'rails', desc: 'rails is awesome'
  u.posts.create! title: 'JavaScript', desc: 'JavaScript is awesome'

  u = User.create name: 'Neil'
  u.posts.create! title: 'JavaScript', desc: 'Javascript is awesome'

  u = User.create name: 'Trisha'
end

With the above sample data if we execute User.joins(:posts) then this is the result we get

#<User id: 9, name: "Neeraj">
#<User id: 9, name: "Neeraj">
#<User id: 9, name: "Neeraj">
#<User id: 10, name: "Neil">

We can avoid the duplication by using distinct .

User.joins(:posts).select('distinct users.*').to_a

Also if we want to make use of attributes from posts table then we need to select them.

records = User.joins(:posts).select('distinct users.*, posts.title as posts_title').to_a
records.each do |user|
  puts user.name
  puts user.posts_title
end

Note that using joins means if you use user.posts then another query will be performed.

Set background and header for a form created using Formotion in RubyMotion

Formotion for Rubymotion makes it a breeze to create views with forms. I am building a rubymotion app and my login form uses formotion. I needed to set background color for my form and here is how you can set a background color for a form created using Formotion.

class LoginViewController < Formotion::FormController

  def viewDidLoad
    super
    view = UIView.alloc.init
    view.backgroundColor = 0x838E61.uicolor
    self.tableView.backgroundView = view
  end
end

After the login view is done loading, I’m creating a new UIView and setting its background color. Then this UIView object is set as the background view to formotion’s table view.

Setting header image

If you want to add some branding to the login form, you can add a image to the form’s header by adding the below code to viewDidLoad:

header_image = UIImage.imageNamed('header_image_name.png')
header_view = UIImageView.alloc.initWithImage(header_image)
self.tableView.tableHeaderView = header_view

We are creating a UIImageView and initializing it with the image we want to show in the header. Now, set the tableview’s tableHeaderView value to the UIImageView we created.

Cookies on Rails

Let’s see how session data is handled in Rails 3.2 .

If you generate a Rails application in 3.2 then ,by default, you will see a file at config/initializers/session_store.rb . The contents of this file is something like

Demo::Application.config.session_store :cookie_store, key: '_demo_session'

First thing this line is telling is to use cookie to store session information.

Second thing this line is telling is to use _demo_session as the key to store cookie data.

A single site can have cookies under different key. For example airbnb is using 14 different keys to store cookie data.

http://blog.bigbinary.com

airbnb cookies

Now let’s see how Rails 3.2.13 stores session information.

In my 3.2.13 version of Rails application I added following line to create session data.

session[:github_username] = 'neerajdotname'

Then I visit the action that executes above code. Now if I go and look for cookies for localhost:3000 then this is what I see .

demo session

As you can see I have only one cookie with key _demo_session .

The cookie has following data.

BAh7CEkiD3Nlc3Npb25faWQGOgZFRkkiJTgwZGFiNzhiYWZmYTc3NjU1ZmVmMGUxM2EzYmEyMDhhBjsAVEkiFGdpdGh1Yl91c2V
ybmFtZQY7AEZJIhJuZWVyYWpkb3RuYW1lBjsARkkiEF9jc3JmX3Rva2VuBjsARkkiMU1KTCs2dXVnRFo2R2NTdG5Kb3E2dm5Bcl
ZYRGJGbjJ1TXZEU0swamxyWU09BjsARg%3D%3D--b5bcce534ceab56616d4a215246e9eb1fc9984a4

Let’s open rails console and try to decipher this information.

content = 'BAh7CEkiD3Nlc3Npb25faWQGOgZFRkkiJTgwZGFiNzhiYWZmYTc3NjU1ZmVmMGUxM2EzYmEyMDhhBjsAVEkiFGdpdGh1Yl91c2V
ybmFtZQY7AEZJIhJuZWVyYWpkb3RuYW1lBjsARkkiEF9jc3JmX3Rva2VuBjsARkkiMU1KTCs2dXVnRFo2R2NTdG5Kb3E2dm5BclZYRGJGbjJ1T
XZEU0swamxyWU09BjsARg%3D%3D--b5bcce534ceab56616d4a215246e9eb1fc9984a4'

When the content is written to cookie then it is escaped. So first we need to unescape it.

> unescaped_content = URI.unescape(content)
=> "BAh7CEkiD3Nlc3Npb25faWQGOgZFRkkiJTgwZGFiNzhiYWZmYTc3NjU1ZmVmMGUxM2EzYmEyMDhhBjsAVEkiFGdpdGh1Yl91c2V
ybmFtZQY7AEZJIhJuZWVyYWpkb3RuYW1lBjsARkkiEF9jc3JmX3Rva2VuBjsARkkiMU1KTCs2dXVnRFo2R2NTdG5Kb3E2dm5BclZYRG
JGbjJ1TXZEU0swamxyWU09BjsARg==--b5bcce534ceab56616d4a215246e9eb1fc9984a4"

Notice that towards the end unescaped_content has -- . That is a separation marker. The value before -- is the real payload. The value after -- is digest of data.

> data, digest = unescaped_content.split('--')
=> ["BAh7CEkiD3Nlc3Npb25faWQGOgZFRkkiJTgwZGFiNzhiYWZmYTc3NjU1ZmVmMGUxM2EzYmEyMDhhBjsAVEkiFGdpdGh1Yl91c2V
ybmFtZQY7AEZJIhJuZWVyYWpkb3RuYW1lBjsARkkiEF9jc3JmX3Rva2VuBjsARkkiMU1KTCs2dXVnRFo2R2NTdG5Kb3E2dm5BclZYRGJ
GbjJ1TXZEU0swamxyWU09BjsARg==", "b5bcce534ceab56616d4a215246e9eb1fc9984a4"]

The data is Base64 encoded. So let’s unecode it.

> Marshal.load(::Base64.decode64(data))
=> {"session_id"=>"80dab78baffa77655fef0e13a3ba208a",
    "github_username"=>"neerajdotname",
    "_csrf_token"=>"MJL+6uugDZ6GcStnJoq6vnArVXDbFn2uMvDSK0jlrYM="}

So we are able to get the data that is stored in cookie. However we can’t tamper with the cookie because if we change the cookie data then the digest will not match.

Now let’s see how rails matches the digest.

In order to create the digest rails makes of use of config/initializer/secret_token.rb . In my case the file has following content.

Demo::Application.config.secret_token = '111111111111111111111111111111'

This secret token is used to create the digest.

> secret_token =  '111111111111111111111111111111'
> OpenSSL::HMAC.hexdigest(OpenSSL::Digest.const_get('SHA1').new, secret_token, data)
=> "b5bcce534ceab56616d4a215246e9eb1fc9984a4"

Notice that the result of above produces a value that is same as digest in earlier step. So if cookie data is tampered with then the digest match will fail. This is why it is absolute necessary that attacker should not be able to get access to secret_token value.

Did you notice that we can access the cookie data without needing secret_token. It means the data stored in cookie is not encrypted and anyone can see it. That is why it is recommended that application should not store any sensitive information in cookie .

In the previous example we used session to store and retrieve data from cookie. We can directly use cookie and that gives us a little bit more control.

cookies[:github_username] = 'neerajdotname'

Now if we look at cookie stored in browser then this is what we see.

update cookie

As you can see now we have two keys in our cookie. One created by session and the other one created by code written above.

Another thing to note is that the data stored for key github_username is not Base64encoded and it also does not have -- to separate the data from the digest. It means this type of cookie data can be tampered with by the user and the Rails application will not be able to detect that the data has been tampered with.

Now let’s try to sign the cookie data to make it tamper proof.

cookies.signed[:twitter_username] = 'neerajdotname'

Now let’s look at cookies in browser.

update cookies

This time we got data with another key twitter_username . Another thing to notice is that cookie data is signed and is tamper proof.

When we use session then behind the scene it uses cookies.signed. That’s why we end up seeing signed data for key _demo_session .

What happens when user tampers with signed cookie data.

Rails does not raise any exception. However when you try to access cookie data then nil is returned because the data has been tampered with.

Security should be on by default

session , by default, uses signed cookies which prevents any kind of tampering of data but the data is still visible to users. It means we can’t store sensitive information in session.

It would be nice if the session data is stored in encrypted format. And that’s the topic of our next discussion.

Rails 4 stores session data in encrypted format

If you generate a Rails application in Rails 4 then ,by default, you will see a file at config/initializers/session_store.rb . The contents of this file is something like

Demo::Application.config.session_store :cookie_store, key: '_demo_session'

Also you will notice that file at config/initializers/secret_token.rb looks like this .

Demo::Application.config.secret_key_base = 'b14e9b5b720f84fe02307ed16bc1a32ce6f089e10f7948422ccf3349d8ab586869c11958c70f46ab4cfd51f0d41043b7b249a74df7d53c7375d50f187750a0f5'

Notice that in Rails 3.2.x the key was secret_token. Now the key is secret_key_base .

session[:github_username] = 'neerajdotname'

cookies and site data

Cookie has following data.

RkxNUWo4NlBKakoyU1VqZWJIKzNaV0lQVVJwQjZhdUVTRnowVHppSVJ3Mk84TStoS1hndFZFNHlNaGw2RHBCc0ZiaEpsM0NtYTg4d
nptcjFaQWVJbUdOaFh5MVlCdWVmSHBMNWpKbkRKR0JrSU5KZFYwVjVyWTZ3aUNqSWxJM1RTMkQybEtPUFE5VDFsZVJyakx0dFh3PT
0tLTZ5NGIreU00Z0MyNnErS29SSGEyZkE9PQ%3D%3D--3f2fd67e4e7785933485a583720d29ba88bca15f

Let’s open rails console and try to decipher this information.

content = 'RkxNUWo4NlBKakoyU1VqZWJIKzNaV0lQVVJwQjZhdUVTRnowVHppSVJ3Mk84TStoS1hndFZFNHlNaGw2RHBCc0ZiaEpsM0NtYTg4d
nptcjFaQWVJbUdOaFh5MVlCdWVmSHBMNWpKbkRKR0JrSU5KZFYwVjVyWTZ3aUNqSWxJM1RTMkQybEtPUFE5VDFsZVJyakx0dFh3PT
0tLTZ5NGIreU00Z0MyNnErS29SSGEyZkE9PQ%3D%3D--3f2fd67e4e7785933485a583720d29ba88bca15f'

When the content is written to cookie then it is escaped. So first we need to unescape it.

unescaped_content = URI.unescape(content)
=> "RkxNUWo4NlBKakoyU1VqZWJIKzNaV0lQVVJwQjZhdUVTRnowVHppSVJ3Mk84TStoS1hndFZFNHlNaGw2RHBCc0ZiaEpsM0NtYTg4d
nptcjFaQWVJbUdOaFh5MVlCdWVmSHBMNWpKbkRKR0JrSU5KZFYwVjVyWTZ3aUNqSWxJM1RTMkQybEtPUFE5VDFsZVJyakx0dFh3PT 0tLTZ
5NGIreU00Z0MyNnErS29SSGEyZkE9PQ==--3f2fd67e4e7785933485a583720d29ba88bca15f"

Now we need secret_key_base value. And using that let’s build key_generator .

secret_key_base = 'b14e9b5b720f84fe02307ed16bc1a32ce6f089e10f7948422ccf3349d8ab586869c11958c70f46ab4cfd51f0d41043b7b249a74df7d53c7375d50f187750a0f5'
key_generator = ActiveSupport::KeyGenerator.new(secret_key_base, iterations: 1000)
key_generator = ActiveSupport::CachingKeyGenerator.new(key_generator)

Our MessageEncryptior needs two long random strings for encryption. So let’s generate two keys for encryptor.

secret = key_generator.generate_key('encrypted cookie')
sign_secret = key_generator.generate_key('signed encrypted cookie')
encryptor = ActiveSupport::MessageEncryptor.new(secret, sign_secret)

Now we can finally decipher the data.

data =  encryptor.decrypt_and_verify(unescaped_content)
puts data
=> neerajdotname

As you can see we need the secret_key_base to make sense out of cookie data. So in Rails 4 the session data will be encrypted ,by default.

Rails4 will transparently will upgrade cookies from unencrypted to encrypted cookies. This is a brilliant example of trivial choices removed by Rails.