Voice based phone verification using twilio

In my previous blog post, I talked about how to do phone verification using SMS and now in this blog post I’m going to talk about how to do voice based phone verification using Twilio.

Requirement

Let’s change the requirement a bit this time.

  • After the user signs up, send an SMS verification. If SMS verification goes well then use that phone number for future use.
  • But if the user’s phone number doesn’t support the SMS feature, then call on user’s phone number for the verification.
  • In the call, ask the user to press 1 from keypad to complete the verification. If that user presses the 1 key, then mark the phone number as verified.

Step 1: Make a call for phone verification

We already handled SMS verification, so let’s add call related changes in PhoneVerificationService class.

class PhoneVerificationService
  attr_reader :user, :verification_through

  VERIFICATION_THROUGH_SMS  = :sms
  VERIFICATION_THROUGH_CALL = :call

  def initialize options
    @user                 = User.find(options[:user_id])
    @verification_through = options[:verification_through] || VERIFICATION_THROUGH_SMS
  end

  def process
    if verification_through == VERIFICATION_THROUGH_SMS
      perform_sms_verification
    else
      make_call
    end
  end

  private

  def from
    #phone number given by twilio
    Settings.twilio_number_for_app
  end

  def to
    "+1#{user.phone_number}"
  end

  def body
    "Please reply with this code '#{user.phone_verification_code}'" <<
    "to verify your phone number"
  end

  def send_sms
    Rails.logger.info "SMS: From: #{from} To: #{to} Body: \"#{body}\""

    twilio_client.account.messages.create ( from: from, to: to, body: body)
  end

  def make_call
    Rails.logger.info "Call: From: #{from} To: #{to}"

    twilio_client.account.calls.create( from: from, to: to, url: callback_url)
  end

  def perform_sms_verification
    begin
      send_sms
    rescue Twilio::REST::RequestError => e
      return make_call if e.message.include?('is not a mobile number')
      raise e.message
    end
  end

  def callback_url
    Rails.application.routes.url_helpers
      .phone_verifications_voice_url(host: Settings.app_host,
                                     verification_code: user.phone_verification_code)
  end

  def twilio_client
    @twilio ||= Twilio::REST::Client.new(Settings.twilio_account_sid,
                                         Settings.twilio_auth_token)
  end
end

In PhoneVerificationService class we have added some major changes:

  1. We have defined one more attribute reader verification_through and in initialize method we have set it.
  2. We have created two new constants VERIFICATION_THROUGH_SMS & VERIFICATION_THROUGH_CALL.
  3. We have changed process method and now we check which verification process should be taken place based on verification_through attribute.
  4. We also have added new methods perform_sms_verification, make_call and callback_url.

Let’s go through these newly added methods.

perform_sms_verification:

In this method, first we try to send an SMS verification. If SMS verification fails with error message like ‘is not a mobile number’ then in the rescue block, we make a phone verification call or we raise the error.

make_call:

In this method, we create an actual phone call and pass the all required info to twilio client object.

callback_url:

In this method, we set the callback_url which is required for Twilio for making a call. When we call the user for verification, our app needs to behave like a human and should ask to the user to press 1 to complete the verification (i.e. In the form of TwiML). This callback_url needs be to set in our app.

Step 2: Add voice action in phone verification controller

For callback_url, add route for voice action in config/routes.rb

post 'phone_verifications/voice' => 'phone_verifications#voice'

Add voice action and required code for it in PhoneVerificationsController.

class PhoneVerificationsController < ApplicationController
  skip_before_filter :verify_authenticity_token
  after_filter :set_header

  HUMAN_VOICE = 'alice'

  def voice
    verification_code = params[:verification_code]

    response = Twilio::TwiML::Response.new do |r|
      r.Gather numDigits: '1',
               action: "/phone_verifications/verify_from_voice?verification_code=#{verification_code}",
               method: 'post' do |g|

        g.Say 'Press 1 to verify your phone number.', voice: HUMAN_VOICE
      end
    end

    render_twiml response
  end

  def verify_from_message
    user = get_user_for_phone_verification
    user.mark_phone_as_verified! if user

    render nothing: true
  end

  private

  def get_user_for_phone_verification
    phone_verification_code = params['Body'].try(:strip)
    phone_number            = params['From'].gsub('+1', '')

    condition = { phone_verification_code: phone_verification_code,
                  phone_number: phone_number }

    User.unverified_phones.where(condition).first
  end

  def set_header
    response.headers["Content-Type"] = "text/xml"
  end

  def render_twiml(response)
    render text: response.text
  end
end

In this voice method, we have set up the Twilio response. When a call is made to the user, this response will get converted into robotic human voice. render_twiml method which sets twilio response in text form is required for Twilio APIs. Set up the response header in the set_header method which gets called in after_filter method.

Step 3: Set request URL for voice phone verification in Twilio

In voice action method, we have setup the request url in the action key of Twilio response object, that also needs to be set in your Twilio account. So when, user replies back to the call query, Twilio will make a request to our app and adds its own some values as parameters to the request. Using those parameters we handle the actual phone verification in the app.

Open twilio account and under NUMBERS section/tab, click on your Twilio number. Then in Voice section, add request URL with HTTP POST method. Add URL like this.

http://your.ngrok.com/phone_verifications/verify_from_voice/

We need Ngrok to expose local url to external world. Read more about it in my privous blog post.

Step 4: Add verify_from_voice action in phone verification controller

First add route for this action in config/routes.rb

post "phone_verifications/verify_from_voice" => "phone_verifications#verify_from_voice

Add this method, in your PhoneVerificationsController.

def verify_from_voice
    response = Twilio::TwiML::Response.new do |r|
      if params['Digits'] == "1"
        user = get_user_for_phone_verification
        user.mark_phone_as_verified!

        r.Say 'Thank you. Your phone number has been verified successfully.',
               voice: HUMAN_VOICE
      else
        r.Say 'Sorry. Your phone number has not verified.',
               voice: HUMAN_VOICE
      end
    end

    render_twiml response
  end

Modify private method get_user_for_phone_verification to support voice verification in PhoneVerificationsController.

def get_user_for_phone_verification
    if params['Called'].present?
      phone_verification_code = params['verification_code']
      phone_number            = params['To']
    else
      phone_verification_code = params['Body'].try(:strip)
      phone_number            = params['From']
    end
    condition = { phone_verification_code: phone_verification_code,
                  phone_number: phone_number.gsub('+1', '') }

    User.unverified_phones.where(condition).first
  end

In the verify_from_voice method, we get parameters Digits, To & verification_code from Twilio request. Using these parameters, we search for user in the database. If we get the proper user then we mark user’s phone number as verified phone number.

Gotcha with after_commit callback in Rails

after_commit callback in Rails is triggered after the end of transaction.

For eg. I have a Post model for which the number of lines of the content are calculated in the after_commit callback:

class Post < ActiveRecord::Base
  after_commit :calculate_total_lines, if: -> (post) { post.previous_changes.include?(:content) }

  def calculate_total_lines
    update! total_lines: content.split("\n").length
  end
end
post = Post.create! content: "Lets discuss Rails 5.\n", author: 'Prathamesh'
assert_equal 1, post.total_lines

Now lets wrap the creation of post inside a transaction block:

Post.transaction do
  post = Post.create! content: "Lets discuss Rails 5.\n", author: 'Prathamesh'
  assert_equal 1, post.total_lines
end

The test will fail now:

#   1) Failure:
# BugTest#test_within_transaction [after_commit_test.rb:45]:
# Expected: 1
#   Actual: nil

Why? Lets recall. after_commit callback will get executed after the end of transaction.

So until all the code inside transaction is completed, the callback is not going to get executed.

Here is a gist with complete test.

Next time you are using an after_commit callback and a transaction, make sure that code inside the transaction is not dependent on the result of the callback.

Blue border around JWPLAYER video

Latest versions of JWPlayer(6.9 onwards) add blue border around the video when it is in focus.

JWPlayer blue border

This is because of the CSS class jwplayer-tab-focus.

The blue border around currently selected video allows to identify which instance of JWPlayer is in focus.

But with a single JWPlayer instance, it can be annoying.

To remove this blue border, we can override the default JWPlayer CSS as follows.

.jw-tab-focus:focus {
    outline: none;
}

To keep all the overridden CSS in once place, we can add this change in a separate file such as jwplayer_overrides.css.

Phone verification using SMS via Twilio

In this blog post, I’m going to show you how to do phone verification using SMS via Twilio. We will be using Twilio gem.

Requirement

  • When a user signs up , we want to send an SMS to that user with a random string to verify user’s phone number.
  • If that user replies back with that code, then verify user’s phone number in the app. Once the phone number is verified, then we can use that phone number for future use.

Step 1: Create required columns in users table

Let’s create phone_number, phone_verification_code and phone_verified columns in the users table.

 $ bin/rails generate migration AddPhoneAttributesToUsers phone_number:string phone_verification_code:string phone_verified:boolean

Then run the migration.

Step 2: Add phone number field in registration form

Add phone_number field in the registration form.

  <%= f.text_field :phone_number %>

Step 3: Send an SMS with verification code

When a user submits registration form, we need to send SMS if phone number is present. To handle this, add following code in User model.

class User < ActiveRecord::Base
  scope :unverified_phones,  -> { where(phone_verified: false) }

  before_save :set_phone_attributes, if: :phone_verification_needed?
  after_save :send_sms_for_phone_verification, if: :phone_verification_needed?

  def mark_phone_as_verified!
    update!(phone_verified: true, phone_verification_code: nil)
  end

  private

  def set_phone_attributes
    self.phone_verified = false
    self.phone_verification_code = generate_phone_verification_code

    # removes all white spaces, hyphens, and parenthesis
    self.phone_number.gsub!(/[\s\-\(\)]+/, '')
  end

  def send_sms_for_phone_verification
    PhoneVerificationService.new(user_id: id).process
  end

  def generate_phone_verification_code
    begin
     verification_code = SecureRandom.hex(3)
    end while self.class.exists?(phone_verification_code: verification_code)

    verification_code
  end

  def phone_verification_needed?
    phone_number.present? && phone_number_changed?
  end
end

We have added 2 major changes in the user model,

  • set_phone_attributes method is set in before_save callback.
  • send_sms_for_phone_verification method is set in after_save callback.

In set_phone_attributes method, we are setting up the phone attributes mainly sanitizing phone number and generating unique phone verification code. In send_sms_for_phone_verification method, we send SMS to the user. Creation of SMS for the phone verification message is handled in PhoneVerificationService class.

class PhoneVerificationService
  attr_reader :user

  def initialize(options)
    @user = User.find(options[:user_id])
  end

  def process
    send_sms
  end

  private

  def from
    # Add your twilio phone number (programmable phone number)
    Settings.twilio_number_for_app
  end

  def to
    # +1 is a country code for USA
    "+1#{user.phone_number}"
  end

  def body
    "Please reply with this code '#{user.phone_verification_code}' to
    verify your phone number"
  end

  def twilio_client
    # Pass your twilio account sid and auth token
    @twilio ||= Twilio::REST::Client.new(Settings.twilio_account_sid,
                                         Settings.twilio_auth_token)
  end

  def send_sms
    Rails.logger.info "SMS: From: #{from} To: #{to} Body: \"#{body}\""

    twilio_client.account.messages.create(
      from: from,
      to: to,
      body: body
    )
  end
end

In PhoneVerificationService class, we have defined user as an attribute reader and in initialize method, we have set the user object to it. Now if you see process method, it does lots of stuff for us.

Lets go through each method.

  • from - In this method, we set up the twilio phone number e.g. programmable number.
  • to - In this method, we set the phone number to which we want to send an SMS.
  • body - In this method, we build text message with verification code.
  • twilio_client - It creates twilio client based on twilio account sid and auth token.
  • send_sms - And last, it sends SMS to the user.

Step 4: Set request URL for phone verification in Twilio

As of now the system has the capability to send SMS to a user. Now we need to add capability to receive the SMS and to match the verification code.

First, we need to set up a request URL in Twilio account.

Open twilio account and under NUMBERS section/tab, click on your Twilio number. Then in Messaging section, add request URL with HTTP POST method. Add URL like this.

http://your.ngrok.com/phone_verifications/verify_from_message/

But to make it work, we need our Rails app on the public internet. There are two options for this:

  1. Deploy Rails app to your VPS or PaaS of choice.
  2. Use a tunneling service to take the server running on your development machine and make it available at an address on the public internet.

I’m going to use Ngrok tunneling service for the purpose of this blog. You can check this blog for more about its usage.

Step 5: Create phone verification controller

We need one more controller, which will handle phone verification when request comes from Twilio. When a user replies back with a verification code, it will trigger request URL through Twilio API. To handle that request, let’s add phone verifications controller.

  $ bin/rails generate controller phone_verifications

Add new route in config/routes.rb

post "phone_verifications/verify_from_message" => "phone_verifications#verify_from_message"

Add following code to the PhoneVerificationsController.

class PhoneVerificationsController < ApplicationController
  skip_before_action :verify_authenticity_token

  def verify_from_message
    user = get_user_for_phone_verification
    user.mark_phone_as_verified! if user

    render nothing: true
  end

  private

  def get_user_for_phone_verification
    phone_verification_code = params['Body'].try(:strip)
    phone_number            = params['From'].gsub('+1', '')

    condition = { phone_verification_code: phone_verification_code,
                  phone_number: phone_number }

    User.unverified_phones.where(condition).first
  end
end

In this controller, we added skip_before_action :verify_authenticity_token, this is because we need to allow twilio request in our Rails app which is actually an outside request (e.g. not secured request). This means we have disabled CSRF detection for this controller.

Now look at the verify_from_message method, in this method we take phone verification code and phone number from params hash. Using those data, we find the user from the database. Once we get the user, then we mark user’s phone number as a verified phone number.

Finally we are set to send business level text messages to the verified phone number.

This blog has more infomation about how to make secure twilio requests.

Author information in jekyll blog

BigBinary’s blog is powered by jekyll. In every blog we display author’s name, author’s twitter handle, author’s github id and author’s avatar. In this blog I’m going to discuss how we collect all that information in a simple manner.

We create a directory called _data in the root folder. This directory has a single file called authors.yml which in our case looks like this.

vipulnsward:
  name: Vipul
  avatar: http://bigbinary.com/assets/team/vipul.jpg
  github: vipulnsward
  twitter: vipulnsward
neerajdotname:
  name: Neeraj Singh
  avatar: http://bigbinary.com/assets/team/neeraj.jpg
  github: neerajdotname
  twitter: neerajdotname

We do not need to do anything to load authors.yml . It is automatically loaded by jekyll.

When we create a blog then the top of the blog looks like this.

---
layout: post
title: How to deploy jekyll site to heroku
categories: [Ruby]
author_github: neerajdotname
---

Notice the last line where we have put in the author’s github id. That’s the identifier we use to pull in author’s information.

In order to display author’s name we have following code in the layout.


<span class="author-name">
  {{ site.data.authors.[page.author_github].name }}
</span>

Similary to display author’s twitter handle and github id we have following code.


<a href="www.twitter.com/{{site.data.authors.[page.author_github].twitter}}">
  <i class="ico-twitter"></i>
</a>
<a href="www.github.com/{{site.data.authors.[page.author_github].github}}">
  <i class="ico-github"></i>
</a>

Now the blog will display the author information and all this information is nicely centralized in one single file.

How to obtain current time from a different timezone in Selenium IDE using javascript.

I had encountered a problem in one of our projects where it was required to obtain the current time from a different timezone and then use it. I am in IST (Indian Standard Timezone - 5.50) and was required to get the time in EST (Eastern Standard Timezone + 5.00)

These are the set of commands I used to achieve that.

Command : storeEval
Target  : var d = new Date(); 
          d.setTime(new Date( (d.getTime() + (d.getTimezoneOffset() * 60000))            + (3600000 * (-5) ) )); 
          d.toLocaleTimeString(); 
Value   : timeEst 

Command : echo 
Target  : ${timeEst}
Value   :

storeEval command is used to store the value in the variable timeEst after evaluating the javascript code.

var d = new Date(); a new date object is created.

d.getTime() method returns the number of milliseconds between midnight of January 1, 1970 and the specified date (as is present in the variable “d”).

d.getTimezoneOffset() method returns the time difference between UTC (Coordinated Universal Time) time and local time from the variable “d”, in minutes. For example, If our time zone is GMT-5.50, -330 will be returned.

Since getTime() is in milliseconds and getTimezoneOffset() is in minutes; getTimezoneOffset() is multiplied with 60000 to make it into milliseconds.

The expression 3600000 * (- 5) is required to make the time from UTC to EST. The difference between UTC and EST is -5 hours. So, to make the hours to milliseconds we need to multiply with 3600000 (60 min x 60 sec x 1000 millisec).

d.setTime() method is used to set the time of variable “d” after evaluating the expression.

With the above, the variable “d” is set with the date and time of the EST time.

d.toLocaleTimeString() method returns the time portion of a Date object as a string, using locale conventions.

The time string obtained above is stored in the variable timeEst.

echo ${timeEst}, the time string is displayed in the console.


Using est time

Selenium javascript est time


Using est date

Selenium javascript est date


2014 - Year in Community Engagement

At BigBinary our team loves to engage with the community as well as help out in different ways. We love contributing to OpenSource, speak and attend different conferences, and help out in community meetups, organizing conferences, and events like RailsGirls .

Short summary

In year 2014 we presented at following 8 conferences across 6 countries.

  • RubyConf Goa, India
  • RubyConf Philippines
  • RedDotRubyConf Singapore
  • DeccanRubyConf Pune, India
  • Madion+Ruby Wisconsin, USA
  • RubyConf Brazil
  • RubyKaigi Tokyo, Japan
  • Golden Gate Ruby Conference San Francisco, USA

Rails and Ruby Conferences

At the start of our travel, we visited RubyConfIndia 2014. The conference took place at an amazing beach resort in Goa. The two days of conference were full of fun and interactions with the best ruby people around India. During the conference our team announced launch of Ruby India to help spread ideas and experiments from the Ruby Indian Community, as well as highlight content from people.

Soon after, I visited Philippines, to conduct a workshop on “Contributing to Rails” at RubyConf Philippines. It was amazing to meet the growing Philippines community. I was happy to spend time with some amazing Rubyists, like Nick Sutterer, Zachary Scott, Konstantin Hasse, Koichi Sasada San, PJ Hagerty, Andre Arko and so on.

After that I and Prathamesh went to Singapore to speak at RedDotRubyConf. We spoke on Arel and ActiveRecord. “RedDorRubyConf” was our first joint talk together at a conference. Again we met a lot of awesome people like Piotr Solnica, Grzegorz Witek, Yinquan Teo, Sayanee Basu, Chinmay Pendharkar and Winston Teo Yong Wei. We also visited Marina Sand Bay and Sentosa Island.

Back in Pune, we hosted the first ever DeccanRubyConf. Our team was busy working on tasks right from building the website, inviting speakers, planning, and other arrangements. The conference had good talks and some really useful workshops. it was a fun one day conference, with attendance of over 170+ people.

The conference also saw our team announce the launch of RubyIndia Podcast, which does regular podcast interviews with notable people from the Ruby Community and Indian Community.

Prathamesh and I, then left on around a one and a half month travel, to attend and speak at multiple conferences.

We started with Madison+Ruby, in Madison, WI. After several missed flights, and a storm, we visited our first US conference after a travel of 48 hours. MadisonRuby was a conference like no other. Several topics touched the humane side of Ruby and the community. We spoke on ‘Building an own ORM using ARel’. Set in the cultural town of Madison, we immensely enjoyed the cheese-curds, farmers markets and game night arranged by the Conf team. A huge thanks to Jim and Jennifer Remsik, for hosting such an amazing event. Thanks also to Scott Ruttencutter for giving us space to work from his office and giving us a tour of the state capital.

We then visited Sao Paulo, Brazil for RubyConf Brazil and presented a talk on Building an ORM using ARel. It was a pleasure to meet Fabio Akita and the CodeMiner team. We made friends with Celso Fernandes and Paulo who were kind enough to help us around, since in Brazil primarily portuguese is spoken. We also met Rafael Franca and Carlos Antonio da Silva who have helped us a lot with Rails issue tracker.

Next Prathamesh headed to RubyKaigi, being held in Tokyo, Japan. He presented on Fixtures in Rails. He met Matz, creator of Ruby on his first day in Japan. Mostly all the core Ruby contributors attended RubyKaigi. He got to interact with Koichi Sasada San, Jonan Scheffler, Akira Matsuda, Godfrey Chan, Richard Schneeman and lot of awesome Rubyists. He also met with his JRuby Core Tom Enebo for the first time. Thanks to Yasuo Honda, Michael Reinsch for helping with Japanese food.

From Brazil, I first visited Miami, and was happy to visit The Lab Miami, WynCode and interact with Rubyists from Miami.

Before heading to San Francisco, for GoGaRuco, I was able to make a quick stop in Boston and visit AlterConf Boston. The theme of the conference was around diversity in tech and gaming industry.

My latest conference was in the amazing city on San Francisco. I presented about ‘Building an ORM’, at GoGaRuCo, which incidentally was the last ever GoGaRuCo. The conference taking place in San Francisco, saw an amazing turnout of crowd. I was able to interact with Sarah Allen, Yehuda Katz, Pat Allen, Sarah Mei, Ryan Davis. I spent most of the time along with Nathan Long, Randy Coulman, and Nathan’s friend Michael Gundlach, who is the creator of popular plugin Adblock. I also ran into Chris Eppstein, creator of compass. All around it was one of the most amazing interactions I had in a conference.

2014, was an amazing year for our team. Together we presented or were part of 8 conferences, launched RubyIndia Newsletter as well as the RubyIndia Podcast, started with 6 new video series on topics from ReactJS to Rubymotion to Selenium, published numerous blogs, and contributed to a number of OpenSource projects.

2015, starts with our team presenting at GardenCityRuby Conf. We hope to get more such chances to interact and help out the community. Onwards to a new year!