Verifying PubSub Services from Rails using Redis

Let’s say that we have a Service that reads and writes to Redis. We have BaseRedisService for managing connection, RedisWriterService to write to Redis and RedisReaderService to read from Redis.

require 'redis'

 # Base class to manage connection
 class BaseRedisService
   REDIS_HOST = '127.0.0.1'

   def connection
     if !(defined?(@@connection) && @@connection)
       @@connection =  Redis.new({ host: REDIS_HOST })
     end
     @@connection
   end

 end
# Class to write to Redis
 class RedisWriterService <  BaseRedisService
   attr_reader :key, :value

   def initialize key, value
     @key, @value = key, value
   end

   def process
     connection.set key, value
   end

 end
# Class to read from Redis
class RedisReaderService < BaseRedisService
  attr_reader :key

  def initialize key
    @key = key
  end

  def process
    connection.get key
  end

end

A test for the above mentioned case might look like this.

require 'test_helper'

class RedisPubSubServiceTest < ActiveSupport::TestCase

  def test_writing_and_reading_value_using_redis
    value = 'Vipul A M'

    RedisWriterService.new('name', value).process
    assert_equal value, RedisReaderService.new('name').process
  end

end

But now, we need to add PubSub to the service and verify that the service sends proper messages to Redis. For verifying from such a service, the service would need to listen to messages sent to Redis. Problem is that Redis listens in a loop. We would need to explicitly release control from listening and allow our tests to go ahead and verify some scenario.

Lets look at how these services would look.

class RedisPublisherService < BaseRedisService
  attr_reader :options, :channel

  def initialize channel, options
    @channel     = channel
    @options = options
  end

  def process
    connection.publish(options, channel)
  end

end

and our Subscriber looks like this.

class RedisSubscriberService < BaseRedisService
  attr_reader :channel

  def initialize channel
    @channel = channel
  end

  def process
    connection.subscribe(channel) do |on|
      on.message do |channel, message|
        puts message
      end
    end
  end

end

Notice that we don’t have control over returning value from the loop easily. Right now we just print on receiving a new message.

Now, lets start persisting our messages to some array in our Service. Also we will start exposing this from a thread variable so that it could be accessed from outside of execution of this listen loop.

class RedisSubscriberService < BaseRedisService
  attr_reader :channel, :messages, :messages_count

  def initialize channel, messages_count = 5
    @channel        = channel
    @messages       = []
    @messages_count = messages_count
  end

  def process
    connection.subscribe(channel) do |on|
      on.message do |channel, message|
        messages.unshift message
        Thread.current[:messages] = messages
      end
    end
  end

end

We now have a way to access message state from the service to read any messages received by it. Lets say we define a new SubscriberService from a Thread, we could read messages like this.

subscriber = Thread.new { RedisSubscriberService.new('payment_alerts').process }
# Print first message from messages received
puts subscriber[:messages].first

Armed with this, we can now define a set of Rails helpers to use in our Rails tests.

module SubscriberHelpers
  THREAD_PROCESS_TIMEOUT = 0.5

  def setup_subscriber channel = 'test_channel'
    @subscriber = Thread.new { RedisSubscriberService.new(channel).process }
  end

  def teardown_subscriber
    @subscriber.kill
  end

  def with_subscriber
    yield
    @subscriber.join THREAD_PROCESS_TIMEOUT
  end

  def message_from_subscription
    @subscriber[:messages].first
  end
end

Notice the with_subscriber method. It executes some code passed to it, then passes method execution to the subscriber process to read any messages sent and store onto messages store.

The count of the variable THREAD_PROCESS_TIMEOUT, can be experimented to set to a value that suites the system that’s being verified.

In our tests, we can now verify PubSub as-

require 'test_helper'

class RedisPubSubServiceTest < ActiveSupport::TestCase
  include SubscriberHelpers

  def setup
    setup_subscriber
  end

  def teardown
    teardown_subscriber
  end

  def test_writing_and_reading_back_values_from_pub_sub
    value = 'Vipul A M'

    with_subscriber do
      RedisPublisherService.new('test_channel', value).process
    end

    assert_equal value, message_from_subscription
  end
end

Summary

We took a look at how PubSub based services can be verified by using threads and exposing messages from them, for verification. These can be tailored to support any similar PubSub service like Redis, and can be used to easily verify values being published from our services from Rails tests.

Voice based phone verification using twilio

In my previous blog post, I talked about how to do phone verification using SMS and now in this blog post I’m going to talk about how to do voice based phone verification using Twilio.

Requirement

Let’s change the requirement a bit this time.

  • After the user signs up, send an SMS verification. If SMS verification goes well then use that phone number for future use.
  • But if the user’s phone number doesn’t support the SMS feature, then call on user’s phone number for the verification.
  • In the call, ask the user to press 1 from keypad to complete the verification. If that user presses the 1 key, then mark the phone number as verified.

Step 1: Make a call for phone verification

We already handled SMS verification, so let’s add call related changes in PhoneVerificationService class.

class PhoneVerificationService
  attr_reader :user, :verification_through

  VERIFICATION_THROUGH_SMS  = :sms
  VERIFICATION_THROUGH_CALL = :call

  def initialize options
    @user                 = User.find(options[:user_id])
    @verification_through = options[:verification_through] || VERIFICATION_THROUGH_SMS
  end

  def process
    if verification_through == VERIFICATION_THROUGH_SMS
      perform_sms_verification
    else
      make_call
    end
  end

  private

  def from
    #phone number given by twilio
    Settings.twilio_number_for_app
  end

  def to
    "+1#{user.phone_number}"
  end

  def body
    "Please reply with this code '#{user.phone_verification_code}'" <<
    "to verify your phone number"
  end

  def send_sms
    Rails.logger.info "SMS: From: #{from} To: #{to} Body: \"#{body}\""

    twilio_client.account.messages.create ( from: from, to: to, body: body)
  end

  def make_call
    Rails.logger.info "Call: From: #{from} To: #{to}"

    twilio_client.account.calls.create( from: from, to: to, url: callback_url)
  end

  def perform_sms_verification
    begin
      send_sms
    rescue Twilio::REST::RequestError => e
      return make_call if e.message.include?('is not a mobile number')
      raise e.message
    end
  end

  def callback_url
    Rails.application.routes.url_helpers
      .phone_verifications_voice_url(host: Settings.app_host,
                                     verification_code: user.phone_verification_code)
  end

  def twilio_client
    @twilio ||= Twilio::REST::Client.new(Settings.twilio_account_sid,
                                         Settings.twilio_auth_token)
  end
end

In PhoneVerificationService class we have added some major changes:

  1. We have defined one more attribute reader verification_through and in initialize method we have set it.
  2. We have created two new constants VERIFICATION_THROUGH_SMS & VERIFICATION_THROUGH_CALL.
  3. We have changed process method and now we check which verification process should be taken place based on verification_through attribute.
  4. We also have added new methods perform_sms_verification, make_call and callback_url.

Let’s go through these newly added methods.

perform_sms_verification:

In this method, first we try to send an SMS verification. If SMS verification fails with error message like ‘is not a mobile number’ then in the rescue block, we make a phone verification call or we raise the error.

make_call:

In this method, we create an actual phone call and pass the all required info to twilio client object.

callback_url:

In this method, we set the callback_url which is required for Twilio for making a call. When we call the user for verification, our app needs to behave like a human and should ask to the user to press 1 to complete the verification (i.e. In the form of TwiML). This callback_url needs be to set in our app.

Step 2: Add voice action in phone verification controller

For callback_url, add route for voice action in config/routes.rb

post 'phone_verifications/voice' => 'phone_verifications#voice'

Add voice action and required code for it in PhoneVerificationsController.

class PhoneVerificationsController < ApplicationController
  skip_before_filter :verify_authenticity_token
  after_filter :set_header

  HUMAN_VOICE = 'alice'

  def voice
    verification_code = params[:verification_code]

    response = Twilio::TwiML::Response.new do |r|
      r.Gather numDigits: '1',
               action: "/phone_verifications/verify_from_voice?verification_code=#{verification_code}",
               method: 'post' do |g|

        g.Say 'Press 1 to verify your phone number.', voice: HUMAN_VOICE
      end
    end

    render_twiml response
  end

  def verify_from_message
    user = get_user_for_phone_verification
    user.mark_phone_as_verified! if user

    render nothing: true
  end

  private

  def get_user_for_phone_verification
    phone_verification_code = params['Body'].try(:strip)
    phone_number            = params['From'].gsub('+1', '')

    condition = { phone_verification_code: phone_verification_code,
                  phone_number: phone_number }

    User.unverified_phones.where(condition).first
  end

  def set_header
    response.headers["Content-Type"] = "text/xml"
  end

  def render_twiml(response)
    render text: response.text
  end
end

In this voice method, we have set up the Twilio response. When a call is made to the user, this response will get converted into robotic human voice. render_twiml method which sets twilio response in text form is required for Twilio APIs. Set up the response header in the set_header method which gets called in after_filter method.

Step 3: Set request URL for voice phone verification in Twilio

In voice action method, we have setup the request url in the action key of Twilio response object, that also needs to be set in your Twilio account. So when, user replies back to the call query, Twilio will make a request to our app and adds its own some values as parameters to the request. Using those parameters we handle the actual phone verification in the app.

Open twilio account and under NUMBERS section/tab, click on your Twilio number. Then in Voice section, add request URL with HTTP POST method. Add URL like this.

http://your.ngrok.com/phone_verifications/verify_from_voice/

We need Ngrok to expose local url to external world. Read more about it in my privous blog post.

Step 4: Add verify_from_voice action in phone verification controller

First add route for this action in config/routes.rb

post "phone_verifications/verify_from_voice" => "phone_verifications#verify_from_voice

Add this method, in your PhoneVerificationsController.

def verify_from_voice
    response = Twilio::TwiML::Response.new do |r|
      if params['Digits'] == "1"
        user = get_user_for_phone_verification
        user.mark_phone_as_verified!

        r.Say 'Thank you. Your phone number has been verified successfully.',
               voice: HUMAN_VOICE
      else
        r.Say 'Sorry. Your phone number has not verified.',
               voice: HUMAN_VOICE
      end
    end

    render_twiml response
  end

Modify private method get_user_for_phone_verification to support voice verification in PhoneVerificationsController.

def get_user_for_phone_verification
    if params['Called'].present?
      phone_verification_code = params['verification_code']
      phone_number            = params['To']
    else
      phone_verification_code = params['Body'].try(:strip)
      phone_number            = params['From']
    end
    condition = { phone_verification_code: phone_verification_code,
                  phone_number: phone_number.gsub('+1', '') }

    User.unverified_phones.where(condition).first
  end

In the verify_from_voice method, we get parameters Digits, To & verification_code from Twilio request. Using these parameters, we search for user in the database. If we get the proper user then we mark user’s phone number as verified phone number.

Gotcha with after_commit callback in Rails

after_commit callback in Rails is triggered after the end of transaction.

For eg. I have a Post model for which the number of lines of the content are calculated in the after_commit callback:

class Post < ActiveRecord::Base
  after_commit :calculate_total_lines, if: -> (post) { post.previous_changes.include?(:content) }

  def calculate_total_lines
    update! total_lines: content.split("\n").length
  end
end
post = Post.create! content: "Lets discuss Rails 5.\n", author: 'Prathamesh'
assert_equal 1, post.total_lines

Now lets wrap the creation of post inside a transaction block:

Post.transaction do
  post = Post.create! content: "Lets discuss Rails 5.\n", author: 'Prathamesh'
  assert_equal 1, post.total_lines
end

The test will fail now:

#   1) Failure:
# BugTest#test_within_transaction [after_commit_test.rb:45]:
# Expected: 1
#   Actual: nil

Why? Lets recall. after_commit callback will get executed after the end of transaction.

So until all the code inside transaction is completed, the callback is not going to get executed.

Here is a gist with complete test.

Next time you are using an after_commit callback and a transaction, make sure that code inside the transaction is not dependent on the result of the callback.

Blue border around JWPLAYER video

Latest versions of JWPlayer(6.9 onwards) add blue border around the video when it is in focus.

JWPlayer blue border

This is because of the CSS class jwplayer-tab-focus.

The blue border around currently selected video allows to identify which instance of JWPlayer is in focus.

But with a single JWPlayer instance, it can be annoying.

To remove this blue border, we can override the default JWPlayer CSS as follows.

.jw-tab-focus:focus {
    outline: none;
}

To keep all the overridden CSS in once place, we can add this change in a separate file such as jwplayer_overrides.css.

Phone verification using SMS via Twilio

In this blog post, I’m going to show you how to do phone verification using SMS via Twilio. We will be using Twilio gem.

Requirement

  • When a user signs up , we want to send an SMS to that user with a random string to verify user’s phone number.
  • If that user replies back with that code, then verify user’s phone number in the app. Once the phone number is verified, then we can use that phone number for future use.

Step 1: Create required columns in users table

Let’s create phone_number, phone_verification_code and phone_verified columns in the users table.

 $ bin/rails generate migration AddPhoneAttributesToUsers phone_number:string phone_verification_code:string phone_verified:boolean

Then run the migration.

Step 2: Add phone number field in registration form

Add phone_number field in the registration form.

  <%= f.text_field :phone_number %>

Step 3: Send an SMS with verification code

When a user submits registration form, we need to send SMS if phone number is present. To handle this, add following code in User model.

class User < ActiveRecord::Base
  scope :unverified_phones,  -> { where(phone_verified: false) }

  before_save :set_phone_attributes, if: :phone_verification_needed?
  after_save :send_sms_for_phone_verification, if: :phone_verification_needed?

  def mark_phone_as_verified!
    update!(phone_verified: true, phone_verification_code: nil)
  end

  private

  def set_phone_attributes
    self.phone_verified = false
    self.phone_verification_code = generate_phone_verification_code

    # removes all white spaces, hyphens, and parenthesis
    self.phone_number.gsub!(/[\s\-\(\)]+/, '')
  end

  def send_sms_for_phone_verification
    PhoneVerificationService.new(user_id: id).process
  end

  def generate_phone_verification_code
    begin
     verification_code = SecureRandom.hex(3)
    end while self.class.exists?(phone_verification_code: verification_code)

    verification_code
  end

  def phone_verification_needed?
    phone_number.present? && phone_number_changed?
  end
end

We have added 2 major changes in the user model,

  • set_phone_attributes method is set in before_save callback.
  • send_sms_for_phone_verification method is set in after_save callback.

In set_phone_attributes method, we are setting up the phone attributes mainly sanitizing phone number and generating unique phone verification code. In send_sms_for_phone_verification method, we send SMS to the user. Creation of SMS for the phone verification message is handled in PhoneVerificationService class.

class PhoneVerificationService
  attr_reader :user

  def initialize(options)
    @user = User.find(options[:user_id])
  end

  def process
    send_sms
  end

  private

  def from
    # Add your twilio phone number (programmable phone number)
    Settings.twilio_number_for_app
  end

  def to
    # +1 is a country code for USA
    "+1#{user.phone_number}"
  end

  def body
    "Please reply with this code '#{user.phone_verification_code}' to
    verify your phone number"
  end

  def twilio_client
    # Pass your twilio account sid and auth token
    @twilio ||= Twilio::REST::Client.new(Settings.twilio_account_sid,
                                         Settings.twilio_auth_token)
  end

  def send_sms
    Rails.logger.info "SMS: From: #{from} To: #{to} Body: \"#{body}\""

    twilio_client.account.messages.create(
      from: from,
      to: to,
      body: body
    )
  end
end

In PhoneVerificationService class, we have defined user as an attribute reader and in initialize method, we have set the user object to it. Now if you see process method, it does lots of stuff for us.

Lets go through each method.

  • from - In this method, we set up the twilio phone number e.g. programmable number.
  • to - In this method, we set the phone number to which we want to send an SMS.
  • body - In this method, we build text message with verification code.
  • twilio_client - It creates twilio client based on twilio account sid and auth token.
  • send_sms - And last, it sends SMS to the user.

Step 4: Set request URL for phone verification in Twilio

As of now the system has the capability to send SMS to a user. Now we need to add capability to receive the SMS and to match the verification code.

First, we need to set up a request URL in Twilio account.

Open twilio account and under NUMBERS section/tab, click on your Twilio number. Then in Messaging section, add request URL with HTTP POST method. Add URL like this.

http://your.ngrok.com/phone_verifications/verify_from_message/

But to make it work, we need our Rails app on the public internet. There are two options for this:

  1. Deploy Rails app to your VPS or PaaS of choice.
  2. Use a tunneling service to take the server running on your development machine and make it available at an address on the public internet.

I’m going to use Ngrok tunneling service for the purpose of this blog. You can check this blog for more about its usage.

Step 5: Create phone verification controller

We need one more controller, which will handle phone verification when request comes from Twilio. When a user replies back with a verification code, it will trigger request URL through Twilio API. To handle that request, let’s add phone verifications controller.

  $ bin/rails generate controller phone_verifications

Add new route in config/routes.rb

post "phone_verifications/verify_from_message" => "phone_verifications#verify_from_message"

Add following code to the PhoneVerificationsController.

class PhoneVerificationsController < ApplicationController
  skip_before_action :verify_authenticity_token

  def verify_from_message
    user = get_user_for_phone_verification
    user.mark_phone_as_verified! if user

    render nothing: true
  end

  private

  def get_user_for_phone_verification
    phone_verification_code = params['Body'].try(:strip)
    phone_number            = params['From'].gsub('+1', '')

    condition = { phone_verification_code: phone_verification_code,
                  phone_number: phone_number }

    User.unverified_phones.where(condition).first
  end
end

In this controller, we added skip_before_action :verify_authenticity_token, this is because we need to allow twilio request in our Rails app which is actually an outside request (e.g. not secured request). This means we have disabled CSRF detection for this controller.

Now look at the verify_from_message method, in this method we take phone verification code and phone number from params hash. Using those data, we find the user from the database. Once we get the user, then we mark user’s phone number as a verified phone number.

Finally we are set to send business level text messages to the verified phone number.

This blog has more infomation about how to make secure twilio requests.

Author information in jekyll blog

BigBinary’s blog is powered by jekyll. In every blog we display author’s name, author’s twitter handle, author’s github id and author’s avatar. In this blog I’m going to discuss how we collect all that information in a simple manner.

We create a directory called _data in the root folder. This directory has a single file called authors.yml which in our case looks like this.

vipulnsward:
  name: Vipul
  avatar: http://bigbinary.com/assets/team/vipul.jpg
  github: vipulnsward
  twitter: vipulnsward
neerajdotname:
  name: Neeraj Singh
  avatar: http://bigbinary.com/assets/team/neeraj.jpg
  github: neerajdotname
  twitter: neerajdotname

We do not need to do anything to load authors.yml . It is automatically loaded by jekyll.

When we create a blog then the top of the blog looks like this.

---
layout: post
title: How to deploy jekyll site to heroku
categories: [Ruby]
author_github: neerajdotname
---

Notice the last line where we have put in the author’s github id. That’s the identifier we use to pull in author’s information.

In order to display author’s name we have following code in the layout.


<span class="author-name">
  {{ site.data.authors.[page.author_github].name }}
</span>

Similary to display author’s twitter handle and github id we have following code.


<a href="www.twitter.com/{{site.data.authors.[page.author_github].twitter}}">
  <i class="ico-twitter"></i>
</a>
<a href="www.github.com/{{site.data.authors.[page.author_github].github}}">
  <i class="ico-github"></i>
</a>

Now the blog will display the author information and all this information is nicely centralized in one single file.

How to obtain current time from a different timezone in Selenium IDE using javascript.

I had encountered a problem in one of our projects where it was required to obtain the current time from a different timezone and then use it. I am in IST (Indian Standard Timezone - 5.50) and was required to get the time in EST (Eastern Standard Timezone + 5.00)

These are the set of commands I used to achieve that.

Command : storeEval
Target  : var d = new Date();
          d.setTime(new Date( (d.getTime() + (d.getTimezoneOffset() * 60000))            + (3600000 * (-5) ) ));
          d.toLocaleTimeString();
Value   : timeEst

Command : echo
Target  : ${timeEst}
Value   :

storeEval command is used to store the value in the variable timeEst after evaluating the javascript code.

var d = new Date(); a new date object is created.

d.getTime() method returns the number of milliseconds between midnight of January 1, 1970 and the specified date (as is present in the variable “d”).

d.getTimezoneOffset() method returns the time difference between UTC (Coordinated Universal Time) time and local time from the variable “d”, in minutes. For example, If our time zone is GMT-5.50, -330 will be returned.

Since getTime() is in milliseconds and getTimezoneOffset() is in minutes; getTimezoneOffset() is multiplied with 60000 to make it into milliseconds.

The expression 3600000 * (- 5) is required to make the time from UTC to EST. The difference between UTC and EST is -5 hours. So, to make the hours to milliseconds we need to multiply with 3600000 (60 min x 60 sec x 1000 millisec).

d.setTime() method is used to set the time of variable “d” after evaluating the expression.

With the above, the variable “d” is set with the date and time of the EST time.

d.toLocaleTimeString() method returns the time portion of a Date object as a string, using locale conventions.

The time string obtained above is stored in the variable timeEst.

echo ${timeEst}, the time string is displayed in the console.


Using est time

Selenkum javascript EST time


Using est date

Selenkum javascript EST date