Avoid exception for dup on Integer

This blog is part of our Ruby 2.4 series.

Prior to Ruby 2.4, if we were to dup an Integer, it would fail with a TypeError.

> 1.dup
TypeError: can't dup Fixnum
	from (irb):1:in `dup'
	from (irb):1

This was confusing because Integer#dup is actually implemented.

> Integer.respond_to? :dup
=> true

However, if we were to freeze an Integer it would fail silently.

> 1.freeze
=> 1

Ruby 2.4 has now included dup-ability for Integer as well.

> 1.dup
=> 1

In Ruby, some object types are immediate variables and therefore cannot be duped/cloned. Yet, there was no graceful way of averting the error thrown by the sanity check when we attempt to dup/clone them.

So now Integer#dup functions exactly the way freeze does – fail silently and return the object itself. It makes sense because nothing about these objects can be changed in the first place.

Deploying Rails applications on Kubernetes cluster with Zero downtime

This post assumes that you have basic understanding of Kubernetes terms like pods and deployments.

Problem

We deploy Rails applications on Kubernetes frequently and we need to ensure that deployments do not cause any downtime. When we used Capistrano to manage deployments it was much easier since it has provision to restart services in the rolling fashion.

Kubernetes restarts pods directly and any process already running on the pod is terminated. So on rolling deployments we face downtime until the new pod is up and running.

Solution

In Kubernetes we have readiness probes and liveness probes. Liveness probes take care of keeping pod live while readiness probe is responsible for keeping pods ready.

This is what Kubernetes documentation has to say about when to use readiness probes.

Sometimes, applications are temporarily unable to serve traffic. For example, an application might need to load large data or configuration files during startup. In such cases, you don’t want to kill the application, but you don’t want to send it requests either. Kubernetes provides readiness probes to detect and mitigate these situations. A pod with containers reporting that they are not ready does not receive traffic through Kubernetes Services.

It means new traffic should not be routed to those pods which are currently running but are not ready yet.

Using readiness probes in deployment flow

Here is what we are going to do.

  • We will use readiness probes to deploy our Rails app.
  • Readiness probes definition has to be specified in pod spec of deployment.
  • Readiness probe uses health check to detect the pod readiness.
  • We will create a simple file on our pod with name health_check returning status 200.
  • This health check runs on arbitrary port 81.
  • We will expose this port in nginx config running on a pod.
  • When our application is up on nginx this health_check returns 200.
  • We will use above fields to configure health check in pod’s spec of deployment.

Now let’s build test deployment manifest.

---
apiVersion: v1
kind: Deployment
metadata:
  name: test-staging
  labels:
    app: test-staging
  namespace: test
spec:
  template:
    metadata:
      labels:
        app: test-staging
    spec:
      containers:
      - image: <your-repo>/<your-image-name>:latest
        name: test-staging
        imagePullPolicy: Always
       env:
        - name: POSTGRES_HOST
          value: test-staging-postgres
        - name: APP_ENV
          value: staging
        - name: CLIENT
          value: test
        ports:
        - containerPort: 80
      imagePullSecrets:
        - name: registrykey

This is a simple deployment template which will terminate pod on the rolling deployment. Application may suffer a downtime until the pod is in running state.

Next we will use readiness probe to define that pod is ready to accept the application traffic. We will add the following block in deployment manifest.

  readinessProbe:
    httpGet:
      path: /health_check
      port: 81
    periodSeconds: 5
    successThreshold: 3
    failureThreshold: 2

In above rediness probe definition httpGet checks the health check.

Health-check queries application on the file health_check printing 200 when accessed over port 81. We will poll it for each 5 seconds with the field periodSeconds.

We will mark pod as ready only if we get a successful health_check count for 3 times. Similarly, we will mark it as a failure if we get failureThreshold twice. This can be adjusted as per application need. This helps deployment to determine if the pod is in ready status or not. With readiness probes for rolling updates, we will use maxUnavailable and maxSurge in deployment strategy.

As per Kubernetes documentation.

maxUnavailable is a field that specifies the maximum number of Pods that can be unavailable during the update process. The value can be an absolute number (e.g. 5) or a percentage of desired Pods (e.g. 10%). The absolute number is calculated from percentage by rounding down. This can not be 0.

and

maxSurge is field that specifies The maximum number of Pods that can be created above the desired number of Pods. Value can be an absolute number (e.g. 5) or a percentage of desired Pods (e.g. 10%). This cannot be 0 if MaxUnavailable is 0. The absolute number is calculated from percentage by rounding up. By default, a value of 25% is used.

Now we will update our deployment manifests with two replicas and the rolling update strategy by specifying the following parameters.

  replicas: 2
  minReadySeconds: 50
  revisionHistoryLimit: 10
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 50%
      maxSurge: 1

This makes sure that on deployment one of our pods is always running and at most 1 more pod can be created while deployment.

We can read more about rolling-deployments here.

We can add this configuration in original deployment manifest.

apiVersion: v1
kind: Deployment
metadata:
  name: test-staging
  labels:
    app: test-staging
  namespace: test
spec:
  replicas: 2
  minReadySeconds: 50
  revisionHistoryLimit: 10
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 50%
      maxSurge: 1
  template:
    metadata:
      labels:
        app: test-staging
    spec:
      containers:
      - image: <your-repo>/<your-image-name>:latest
        name: test-staging
        imagePullPolicy: Always
       env:
        - name: POSTGREs_HOST
          value: test-staging-postgres
        - name: APP_ENV
          value: staging
        - name: CLIENT
          value: test
        ports:
        - containerPort: 80
        readinessProbe:
          httpGet:
            path: /health_check
            port: 81
          periodSeconds: 5
          successThreshold: 3
          failureThreshold: 2
      imagePullSecrets:
        - name: registrykey

Let’s launch this deployment using the command given below and monitor the rolling deployment.

$ kubectl apply -f test-deployment.yml
deployment "test-staging-web" configured

After the deployment is configured we can check the pods and how they are restarted.

We can also access the application to check if we face any down time.

$ kubectl  get pods
    NAME                                  READY      STATUS  RESTARTS    AGE
test-staging-web-372228001-t85d4           1/1       Running   0          1d
test-staging-web-372424609-1fpqg           0/1       Running   0          50s

We can see above that only one pod is re-created at the time and one of the old pod is serving the application traffic. Also, new pod is running but not ready as it has not yet passed the readiness probe condition.

After sometime when the new pod is in ready state, old pod is re-created and traffic is served by the new pod. In this way, our application does not suffer any down-time and we can confidently do deployments even at peak hours.

Rails 5.1 returns unmapped timezones from ActiveSupport::TimeZone.country_zones

This blog is part of our Rails 5.1 series.

The ActiveSupport::TimeZone class serves as wrapper around TZInfo::TimeZone class. It limits the set of zones provided by TZInfo to smaller meaningful subset and returns zones with friendly names. For example, TZInfo gem returns “America/New_York” whereas Active Support returns “Eastern Time (US & Canada)”.

ActiveSupport::TimeZone.country_zones method returns a set of TimeZone objects for timezones in a country specified as 2 character country code.

# Rails 5.0
>> ActiveSupport::TimeZone.country_zones('US')

=> [#<ActiveSupport::TimeZone:0x007fcc2b9b3198 @name="Hawaii", @utc_offset=nil, @tzinfo=#<TZInfo::DataTimezone: Pacific/Honolulu>>, #<ActiveSupport::TimeZone:0x007fcc2b9d9ac8 @name="Alaska", @utc_offset=nil, @tzinfo=#<TZInfo::DataTimezone: America/Juneau>>, #<ActiveSupport::TimeZone:0x007fcc2ba03a08 @name="Pacific Time (US & Canada)", @utc_offset=nil, @tzinfo=#<TZInfo::DataTimezone: America/Los_Angeles>>,...]

In Rails 5.0, the country_zones method returns empty for some countries. This is because ActiveSupport::TimeZone::MAPPING supports only a limited number of timezone names.

>> ActiveSupport::TimeZone.country_zones('SV') // El Salvador

=> []

Rails 5.1 fixed this issue. So now if the country is not found in the MAPPING hash then a new ActiveSupport::TimeZone instance for the country is returned.

>> ActiveSupport::TimeZone.country_zones('SV') // El Salvador

=> [#<ActiveSupport::TimeZone:0x007ff0dab83080 @name="America/El_Salvador", @utc_offset=nil, @tzinfo=#<TZInfo::DataTimezone: America/El_Salvador>>]

Difference between type and type alias in Elm

This blog is part of our Road to Elm series.

What is the differnece between type and type alias.

Elm FAQ has an answer to this question. However I could not fully understand the answer.

This is my attempt in explaining it.

What is type

In Elm everything has a type. Fire up elm-repl and you will see 4 is a number and “hello” is a String.

> 4
4 : number

> "hello"
"hello" : String

Let’s assume that we are working with users records and we have following attributes of those users.

  • Name
  • Age
  • Status (Active or Inactive)

It’s pretty clear that “Name” should be of type “String” and “Age” should be of type “number”.

Let’s think about a moment what is the type of “Status”. What is “Active” and “Inactive” in terms of type.

Active and Inactive are two valid values of Status. In other programming languages we might represent Status as an enum.

In Elm we need to create a new type. And that can be done as shown here.

type Status = Active | Inactive

Second thing we are doing is that we are stating that the valid values for this new type are Active and Inactive.

When I discussed this code with my team members they asked me to show where is Active and Inactive defined. Good question.

The simple answer is that they are not defined anywhere. They do not need to be defined. What needs definition is the new type that is being created.

What makes understanding it a bit hard for people coming from Ruby, Java and such background is that these people (including me) are looking at Active and Inactive as a class or a constant which is not the right way to look at.

Active and Inactive are the valid values for type Status.

> Active
-- NAMING ERROR ----------

Cannot find variable `Active`

3|   Active
     ^^^^^^

As you can see repl is not sure what Active is.

We can solve this by pasting following code in repl.

type Status = Active | Inactive

Now we can run the same code again. This time no error.

> Active
Active : Repl.Status

What is type alias

Let’s see a simple application which just prints name and age of a single user.

Here is the code. I’m posting screenshot of the same below with certain part highlighted.

code without type alias

As you can see { name : String, age : Int } is repeated at four different places. In a bigger application it might get repeated more often.

This is what type alias does. It removes repetition. It removes verbosity.

As the name suggests this is just an alias. Note that type creates a new type whereas type alias is literally saving keystrokes. type alias does not create a new type.

Now if you read the FAQ answer again then hopefully it will make morse sense now.

Here is modified code using type alias.

Why use type alias Username : String

While browsing Elm code in general, I came across following code.

type alias Username = String

Question is what does code like this buy us. All it does is that instead of String I can now type Username.

First let’s see how it might be used.

Let’s assume that we have a function which returns Status of a user for the given username.

The function might have implementation as shown below.

getUserStatus username =
  make_db_call_and_return_user_status

Now let’s think about what the type annotation (rubyist think of it as method signature ) of function getUserStatus might look like.

It takes username as input and returns user record.

So the type annotation might look like

getUserStatus : String -> Status

This works. However the issue is that String is not expressive enough. It can be made more expressive if the signature were

getUserStatus : Username -> Status

Now that we know about type alias all we need to do is

type alias Username = String

This makes code more expressive.

No recursion with type alias

An example of where we might need recursion is while designing commenting system. A comment can have sub-comments. However since type alias is just a substituation and recursion does not work with it.

> type alias Comment = { message : String, responses : List Comment }

This type alias is recursive, forming an infinite type!

2| type alias Comment = { message : String, responses : List Comment }
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
When I expand a recursive type alias, it just keeps getting bigger and bigger.
So dealiasing results in an infinitely large type! Try this instead:

    type Comment
        = Comment { message : String, responses : List Comment }

This is kind of a subtle distinction. I suggested the naive fix, but you can
often do something a bit nicer. So I would recommend reading more at:
<https://github.com/elm-lang/elm-compiler/blob/0.18.0/hints/recursive-alias.md>

Hint for Recursive Type Aliases discusses this issue in greater detail and it also has solution to the problem of recursion.

Dual role of type alias as constructor and type

Let’s say that we have following code.

type alias UserInfo =
    { name : String, age : Int }

Now we can use UserInfo as a constructor to create records.

> type alias UserInfo = { name : String, age : Int }
> sam = UserInfo "Sam" 24
{ name = "Sam", age = 24 } : Repl.UserInfo

In the above case we used UserInfo as a constructor to create new user records. We did not use UserInfo as a type.

Now let’s see another function.

type alias UserInfo =
    { name : String, age : Int }


getUserAge : UserInfo -> Int
getUserAge userinfo =
    userinfo.age

In this case UserInfo is being used in type annotation as type and not as as constructor.

Which one to use type or type alias

Both of them serve different purpose. Let’s see an example.

Let’s say that we have following code.

type alias UserInfo =
    { name : String, age : Int }

type alias Coach =
    { name : String, age : Int, sports : String }

Now let’s write a function that gets age of the given userinfo.

getUserAge : UserInfo -> Int

getUserAge UserInfo =
    UserInfo.age

Now let’s create two types of users.

sam = UserInfo "Sam" 24
charlie = Coach "Charlie" 52 "Basketball"

Now let’s try to get age of both of these people.

getUserAge sam
getUserAge charlie

Here is the compelete version if you want to run it.

Please note that elm-repl does not support type annotation so you can’t test this code in elm-repl.

The main point here is that since we used type alias, function getUserAge works for both UserInfo as well as Coach. It would be a stretch to say that this sounds like “duck typing in Elm” but it comes pretty close.

Yes Elm is staticly typed language and it enforces type. However the point here is the type alias is not exactly a type.

So why did this code work.

It worked because of Elm’s support for pattern matching for records.

As mentioned earlier type alias is just a shortcut for typing the verbose version. So let’s expand the tye type annotation of getUserAge.

If we were not using type alias UserInfo then it might have looked like as shown below.

getUserAge : { name : String, age : Int } -> Int

Here the argument is a record. Here is official guide on Records. While dealing with records Elm looks at the argument and if that argument is a record and has all the matching attributes then Elm will not complain because of its support for pattern matching.

Since Coach has both name and age attribute getUserAge charlie works.

You can test it by removing the attribute age from Coach and then you will see that Compiler will complain.

In summary if we want strict type enforcement then we should go for type. If we need something so that we do not need to type all the attributes all the type and we want pattern matching then we should go for type alias.

In Ruby 2.4, IPAddr#== and IPAddr#<=> do not throw exception for objects that can't be converted to IPAddr

This blog is part of our Ruby 2.4 series.

In Ruby, IPAddr#== method is used to check whether two IP addresses are equal or not. Ruby also has IPAddr#<=> method which is used to compare two IP addresses.

In Ruby 2.3, behavior of these methods was inconsistent. Let’s see an example.

# Ruby 2.3

>> IPAddr.new("1.2.1.3") == "Some ip address"
=> IPAddr::InvalidAddressError: invalid address

But if the first argument is invalid IP address and second is valid IP address, then it would return false.

# Ruby 2.3

>> "Some ip address" == IPAddr.new("1.2.1.3")
=> false

The <=> method would raise exception in both the cases.

# Ruby 2.3

>> "Some ip address" <=> IPAddr.new("1.2.1.3")
=> IPAddr::InvalidAddressError: invalid address

>> IPAddr.new("1.2.1.3") <=> "Some ip address"
=> IPAddr::InvalidAddressError: invalid address

In Ruby 2.4, this issue is fixed for both the methods to return the result without raising exception, if the objects being compared can’t be converted to an IPAddr object.

# Ruby 2.4

>> IPAddr.new("1.2.1.3") == "Some ip address"
=> false

>> "Some ip address" == IPAddr.new("1.2.1.3")
=> false

>> IPAddr.new("1.2.1.3") <=> "Some ip address"
=> nil

>> "Some ip address" <=> IPAddr.new("1.2.1.3")
=> nil

This might cause some backward compatibility if our code is expecting the exception which is no longer raised in Ruby 2.4.

Rails 5.1 has dropped dependency on jQuery from the default stack

This blog is part of our Rails 5.1 series.

Rails has been dependent on jQuery for providing the unobtrusive JavaScript helpers such as data-remote, data-url and the Ajax interactions. Every Rails application before Rails 5.1 would have the jquery-rails gem included by default.

The jquery-rails gem contains the jquery-ujs driver which provides all the nice unobtrusive features.

But now JavaScript has progressed well such that we can write the unobtrusive driver which Rails needs using just plain vanilla JavaScript.

That’s what has happened for the 5.1 release. The jquery-ujs driver has been rewritten using just plain JavaScript as part of a GSoC project by Dangyui Liu.

Now that the unobtrusive JavaScript driver does not depend on jQuery, new Rails applications also need not depend on jQuery.

So, Rails 5.1 has dropped jQuery as a dependency from the default stack.

The current jquery-based approach would still be available. It’s just that it’s not part of the default stack. You will need to manually add the jquery-rails gem to newly created 5.1 application and update the application.js to include the jquery-ujs driver.

It’s worth noting that rails-ujs only supports IE 11+. Visit the Desktop Browser Support section of Basecamp to see the full list of all the supported browsers.

Browsers support without jQuery

We saw some discussion about which all browsers are supported without jQuery. We decided to test it outselves on a plain vanilla CRUD Rails app. We tested “adding”, “editing” and “deleting” of a resource.

We found that all three operations (adding, editing and deleting) to be working in following cases.

  • Win 7 - IE 9
  • Win 7 - IE 10
  • Win 7 - IE 11
  • Win 8 - IE 10
  • Win 8.1 - IE 11
  • Win 10 - IE 14 Edge
  • Win 10 - IE 15 Edge
  • Win 10 - Firefox 53
  • Win 10 - Chrome 58
  • Win 10 - Safari 5.1
  • Mac Siera - Safari 10.1
  • Mac Sierra - Firefox 53
  • Mac Sierra - Chrome 58

API change for the event handlers

rails-ujs driver has changed the signature of the event handler functions to just pass one event object instead of event, data, status and xhr as in the case of jquery-ujs driver.

Check the documentation for the rails-ujs event handlers for more details.

Ruby 2.4 has deprecated toplevel constants TRUE, FALSE and NIL

This blog is part of our Ruby 2.4 series.

Ruby has top level constants like TRUE, FALSE and NIL. These constants are just synonyms for true, false and nil respectively.

In Ruby 2.4, these constants are deprecated and will be removed in future version.

# Ruby 2.3

2.3.1 :001 > TRUE
 => true
2.3.1 :002 > FALSE
 => false
2.3.1 :003 > NIL
 => nil
# Ruby 2.4

2.4.0 :001 > TRUE
(irb):1: warning: constant ::TRUE is deprecated
 => true
2.4.0 :002 > FALSE
(irb):2: warning: constant ::FALSE is deprecated
 => false
2.4.0 :003 > NIL
(irb):3: warning: constant ::NIL is deprecated
 => nil

Managing Rails tasks such as 'db:migrate' and 'db:seed' on Kubernetes while performing rolling deployments

This post assumes that you have basic understanding of Kubernetes terms like pods and deployments.

Problem

We want to deploy a Rails application on Kubernetes. We assume that the assets:precompile task would be run as part of the Docker image build process.

We want to run rake tasks such as db:migrate and db:seed on the initial deployment, and just db:migrate task on each later deployment.

We cannot run these tasks while building the Docker image as it would not be able to connect to the database at that moment.

So, how to run these tasks?

Solution

We assume that we have a Docker image named myorg/myapp:v0.0.1 which contains the source code for our Rails application.

We also assume that we have included database.yml manifest in this Docker image with the required configuration needed for connecting to the database.

We need to create a Kubernetes deployment template with the following content.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: myapp
spec:
  template:
    spec:
      containers:
      - image: myorg/myapp:v0.0.1
        name: myapp
        imagePullPolicy: IfNotPresent
        env:
        - name: DB_NAME
          value: myapp
        - name: DB_USERNAME
          value: username
        - name: DB_PASSWORD
          value: password
        - name: DB_HOST
          value: 54.10.10.245
        ports:
        - containerPort: 80
      imagePullSecrets:
        - name: docker_pull_secret

Let’s save this template file as myapp-deployment.yml.

We can change the options and environment variables in above template as per our need. The environment variables specified here will be available to our Rails application.

To apply above template for the first time on Kubernetes, we will use the following command.

$ kubectl create -f myapp-deployment.yml

Later on, to apply the same template after modifications such as change in the Docker image name or change in the environment variables, we will use the following command.

$ kubectl apply -f myapp-deployment.yml

After applying the deployment template, it will create a pod for our application on Kuberentes.

To see the pods, we use the following command.

$ kubectl get pods

Let’s say that our app is now running in the pod named myapp-4007005961-1st7s.

To execute a rake task, for e.g. db:migrate on this pod, we can run the following command.

$ kubectl exec myapp-4007005961-1st7s                              \
          -- bash -c                                               \
          'cd ~/myapp && RAILS_ENV=production bin/rake db:migrate'

Similarly, we can execute db:seed rake task as well.

If we already have an automated flow for deployments on Kubernetes, we can make use of this approach to programmatically or conditionally run any rake task as per the needs.

Why not to use Kubernetes Jobs to solve this?

We faced some issues while using Kubernetes Jobs to run migration and seed rake tasks.

  1. If the rake task returns a non-zero exit code, the Kubernetes job keeps spawning pods until the task command returns a zero exit code.

  2. To get around the issue mentioned above we needed to unnecessarily implement additional custom logic of checking job status and the status of all the spawned pods.

  3. Capturing the command’s STDOUT or STDERR was difficult using Kubernetes job.

  4. Some housekeeping was needed such as manually terminating the job if it wasn’t successful. If not done, it will fail to create a Kubernetes job with the same name, which is bound to occur when we perform later deployments.

Because of these issues, we choose not to rely on Kubernetes jobs to solve this problem.

Ruby 2.4 allows to customize suffix of the rotated log files

This blog is part of our Ruby 2.4 series.

In Ruby, The Logger class can be used for rotating log files daily, weekly or monthly.

daily_logger = Logger.new('foo.log', 'daily')

weekly_logger = Logger.new('foo.log', 'weekly')

monthly_logger = Logger.new('foo.log', 'monthly')

At the end of the specified period, Ruby will change the file extension of the log file as follows:

foo.log.20170615

The format of the suffix for the rotated log file is %Y%m%d. In Ruby 2.3, there was no way to customize this suffix format.

Ruby 2.4 added the ability to customize the suffix format by passing an extra argument shift_period_suffix.

# Ruby 2.4

logger = Logger.new('foo.log', 'weekly', shift_period_suffix: '%d-%m-%Y')

Now, suffix of the rotated log file will use the custom date format which we passed.

foo.log.15-06-2017

Ruby 2.4 added Hash#transform_values and its destructive version from Active Support

This blog is part of our Ruby 2.4 series.

It is a common use case to transform the values of a hash.

{ a: 1, b: 2, c: 3 } => { a: 2, b: 4, c: 6 }

{ a: "B", c: "D", e: "F" } => { a: "b", c: "d", e: "f" }

We can transform the values of a hash destructively (i.e. modify the original hash with new values) or non-destructively (i.e. return a new hash instead of modifying the original hash).

Prior to Ruby 2.4, we need to use following code to transform the values of a hash.

# Ruby 2.3 Non-destructive version

> hash = { a: 1, b: 2, c: 3 }
 #=> {:a=>1, :b=>2, :c=>3}

> hash.inject({}) { |h, (k, v)| h[k] = v * 2; h }
 #=> {:a=>2, :b=>4, :c=>6}

> hash
 #=> {:a=>1, :b=>2, :c=>3}

> hash = { a: "B", c: "D", e: "F" }
 #=> {:a=>"B", :c=>"D", :e=>"F"}

> hash.inject({}) { |h, (k, v)| h[k] = v.downcase; h }
 #=> {:a=>"b", :c=>"d", :e=>"f"}

> hash
 #=> {:a=>"B", :c=>"D", :e=>"F"}
# Ruby 2.3 Destructive version

> hash = { a: 1, b: 2, c: 3 }
 #=> {:a=>1, :b=>2, :c=>3}

> hash.each { |k, v| hash[k] = v * 2 }
 #=> {:a=>2, :b=>4, :c=>6}

> hash
 #=> {:a=>2, :b=>4, :c=>6}

> hash = { a: "B", c: "D", e: "F" }
 #=> {:a=>"B", :c=>"D", :e=>"F"}

> hash.each { |k, v| hash[k] = v.downcase }
 #=> {:a=>"b", :c=>"d", :e=>"f"}

> hash
 #=> {:a=>"b", :c=>"d", :e=>"f"}

transform_values and transform_values! from Active Support

Active Support has already implemented handy methods Hash#transform_values and Hash#transform_values! to transform hash values.

Now, Ruby 2.4 has also implemented Hash#map_v and Hash#map_v! and then renamed to Hash#transform_values and Hash#transform_values! for the same purpose.

# Ruby 2.4 Non-destructive version

> hash = { a: 1, b: 2, c: 3 }
 #=> {:a=>1, :b=>2, :c=>3}

> hash.transform_values { |v| v * 2 }
 #=> {:a=>2, :b=>4, :c=>6}

> hash
 #=> {:a=>1, :b=>2, :c=>3}

> hash = { a: "B", c: "D", e: "F" }
 #=> {:a=>"B", :c=>"D", :e=>"F"}

> hash.transform_values(&:downcase)
 #=> {:a=>"b", :c=>"d", :e=>"f"}

> hash
 #=> {:a=>"B", :c=>"D", :e=>"F"}
# Ruby 2.4 Destructive version

> hash = { a: 1, b: 2, c: 3 }
 #=> {:a=>1, :b=>2, :c=>3}

> hash.transform_values! { |v| v * 2 }
 #=> {:a=>2, :b=>4, :c=>6}

> hash
 #=> {:a=>2, :b=>4, :c=>6}

> hash = { a: "B", c: "D", e: "F" }
 #=> {:a=>"B", :c=>"D", :e=>"F"}

> hash.transform_values!(&:downcase)
 #=> {:a=>"b", :c=>"d", :e=>"f"}

> hash
 #=> {:a=>"b", :c=>"d", :e=>"f"}

Using prettier and rubocop in Ruby on Rails application to format JavaScript, CSS and Ruby files

Recently we started using prettier and rubocop to automatically format our code on git commit. Here is how we got started with setting up both prettier and rubocop in our Ruby on Rails applications.

Generate package.json

If you don’t already have a package.json file then execute the following command to create a package.json file with value {}.

echo "{}" > package.json

Install prettier

Now execute following command to install prettier.

npm install --save-dev lint-staged husky prettier

# Ignore `node_modules`
echo "/node_modules" >> .gitignore

Add scripts & ignore node_modules

Now open package.json and replace the whole file with following content.

{
  "scripts": {
    "precommit": "lint-staged"
  },
  "lint-staged": {
    "app/**/*.{js,es6,jsx}": [
      "./node_modules/prettier/bin/prettier.js --trailing-comma es5 --write",
      "git add"
    ]
  },
  "devDependencies": {
    "husky": "^0.13.4",
    "lint-staged": "^3.6.0",
    "prettier": "^1.4.2"
  }
}

Note that if you send pull request with your changes and circleCI or such tools run npm install then downgrade husky to ^0.13.4 and that will solve the problem.

In Ruby on Rails applications third party vendor files are stored in vendor folder and we do not want to format JavaScript code in those files. Hence we have applied the rule to run prettier only on files residing in app directory.

Here at BigBinary we store all JavaScript files using ES6 features with extension .es6. Hence we are running such files through prettier. Customize this to match with your application requirements.

Note that “precommit” hook is powered by husky. Read up “husky” documentation to learn about “prepush” hook and other features.

Commit the change

git add .
git commit -m "Added support for prettier for JavaScript files"

Execute prettier on current code

./node_modules/prettier/bin/prettier.js --single-quote --trailing-comma es5 --write "{app,__{tests,mocks}__}/**/*.{js,es6,jsx,scss,css}"

We want more

We were thrilled to see prettier format our JavaScript code. We wanted more of it at more places. We found that prettier can also format CSS files. We changed our code to also format CSS code. It was an easy change. All we had to do was change one line.

Before : "app/**/*.{js,es6,jsx}"

After : "app/**/*.{js,es6,jsx,scss,css}"

Inspired by prettier we welcomed rubocop

Now that JavaScript and CSS files are covered we started looking at other places where we can get this productivity gain.

Since we write a lot of Ruby code we turned our attention to rubocop.

It turned out that “rubocop” already had a feature to automatically format the code.

Open package.json and change lint-staged section to following

"app/**/*.{js,es6,jsx,scss,css}": [
  "./node_modules/prettier/bin/prettier.js --single-quote --trailing-comma es5 --write",
  "git add"
],
"{app,test}/**/*.rb": [
  "bundle exec rubocop -a",
  "git add"
]

Open Gemfile and add following line.

group :development do
  gem "rubocop"
end

The behavior of rubocop can be controlled by .rubocop.yml file. If you want to get started with the rubocop file that Rails uses then just execute following command at the root of your Rails application.

wget https://raw.githubusercontent.com/rails/rails/master/.rubocop.yml

Open the downloaded file and change the TargetRubyVersion value to match with the ruby version the project is using. ` value to match with the ruby version the project is using.

Execute rubcop in all ruby files.

bundle install
bundle exec rubocop -a "{app}/**/*.rb"

Code is changed on git commit and not on git add

We notice that some people were a bit confused when git add did not format the code.

Code is formatted when git commit is done.

npm install is important

It’s important to note that users need to do npm install for all this to work. Otherwise prettier or rubocop won’t be activated and they will silently fail.

Full package.json file

After all the changes are done then package.json should be like as shown below.

{
  "scripts": {
    "precommit": "lint-staged"
  },
  "lint-staged": {
    "app/**/*.{js,es6,jsx,scss,css}": [
      "./node_modules/prettier/bin/prettier.js --trailing-comma es5 --write",
      "git add"
    ],
    "{app,test}/**/*.rb": [
      "bundle exec rubocop -a",
      "git add"
    ]
  },
  "devDependencies": {
    "husky": "^0.13.4",
    "lint-staged": "^3.6.0",
    "prettier": "^1.4.2"
  }
}

Rails 5.1 adds delegate_missing_to

This blog is part of our Rails 5.1 series.

When we use method_missing then we should also use respond_to_missing?. Because of this code becomes verbose since both method_missing and respond_to_missing? need to move in tandem.

DHH in the issue itself provided a good example of this verbosity.

class Partition
  def initialize(first_event)
    @events = [ first_event ]
  end

  def people
    if @events.first.detail.people.any?
      @events.collect { |e| Array(e.detail.people) }.flatten.uniq
    else
      @events.collect(&:creator).uniq
    end
  end

  private
    def respond_to_missing?(name, include_private = false)
      @events.respond_to?(name, include_private)
    end

    def method_missing(method, *args, &block)
      @events.public_send(method, *args, &block)
    end
end

He proposed to use a new method delegate_missing_to. Here is how it can be used.

class Partition
  delegate_missing_to :@events

  def initialize(first_event)
    @events = [ first_event ]
  end

  def people
    if @events.first.detail.people.any?
      @events.collect { |e| Array(e.detail.people) }.flatten.uniq
    else
      @events.collect(&:creator).uniq
    end
  end
end

Why not SimpleDelegator

We at BigBinary have used SimpleDelegator. However one issue with this is that statically we do not know to what object the calls are getting delegated to since at run time the delegator could be anything.

DHH had following to say about this pattern.

I prefer not having to hijack the inheritance tree for such a simple feature.

Why not delegate method

Delegate method works. However here we need to white list all the methods and in some cases the list can get really long. Following is a real example from a real project.

delegate :browser_status, :browser_stats_present?,
         :browser_failed_count, :browser_passed_count,
         :sequential_id, :project, :initiation_info,
         :test_run, success?,
         to: :test_run_browser_stats

Delegate everything

Sometimes we just want to delegate all missing methods. In such cases method delegate_missing_to does the job neatly. Note that the delegation happens to only public methods of the object being delegated to.

Check out the pull request for more details on this.

Using Kubernetes Configmap with configuration files for deploying Rails applications

This post assumes that you have basic understanding of Kubernetes terms like pods and deployments.

We deploy our Rails applications on Kubernetes and frequently do rolling deployments.

While performing application deployments on kubernetes cluster, sometimes we need to change the application configuration file. Changing this application configuration file means we need to change source code, commit the change and then go through the complete deployment process.

This gets cumbersome for simple changes.

Let’s take the case of wanting to add queue in sidekiq configuration.

We should be able to change configuration and restart the pod instead of modifying the source-code, creating a new image and then performing a new deployment.

This is where Kubernetes’s ConfigMap comes handy. It allows us to handle configuration files much more efficiently.

Now we will walk you through the process of managing sidekiq configuration file using configmap.

Starting with configmap

First we need to create a configmap. We can either create it using kubectl create configmap command or we can use a yaml template.

We will be using yaml template test-configmap.yml which already has sidekiq configuration.

apiVersion: v1
kind: ConfigMap
metadata:
  name: test-staging-sidekiq
  labels:
    name: test-staging-sidekiq
  namespace: test
data:
  config: |-
    ---
    :verbose: true
    :environment: staging
    :pidfile: tmp/pids/sidekiq.pid
    :logfile: log/sidekiq.log
    :concurrency: 20
    :queues:
      - [default, 1]
    :dynamic: true
    :timeout: 300

The above template creates configmap in the test namespace and is only accessible in that namespace.

Let’s launch this configmap using following command.

$ kubectl create -f  test-configmap.yml
configmap "test-staging-sidekiq" created

After that let’s use this configmap to create our sidekiq.yml configuration file in deployment template named test-deployment.yml.

---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: test-staging
  labels:
    app: test-staging
  namespace: test
spec:
  template:
    metadata:
      labels:
        app: test-staging
    spec:
      containers:
      - image: <your-repo>/<your-image-name>:latest
        name: test-staging
        imagePullPolicy: Always
       env:
        - name: REDIS_HOST
          value: test-staging-redis
        - name: APP_ENV
          value: staging
        - name: CLIENT
          value: test
        volumeMounts:
            - mountPath: /etc/sidekiq/config
              name: test-staging-sidekiq
        ports:
        - containerPort: 80
      volumes:
        - name: test-staging-sidekiq
          configMap:
             name: test-staging-sidekiq
             items:
              - key: config
                path:  sidekiq.yml
      imagePullSecrets:
        - name: registrykey

Now let’s create a deployment using above template.

$ kubectl create -f  test-deployment.yml
deployment "test-pv" created

Once the deployment is created, pod running from that deployment will start sidekiq using the sidekiq.yml mounted at /etc/sidekiq/config/sidekiq.yml.

Let’s check this on the pod.

deployer@test-staging-2766611832-jst35:~$ cat /etc/sidekiq/config/sidekiq_1.yml
---
:verbose: true
:environment: staging
:pidfile: tmp/pids/sidekiq_1.pid
:logfile: log/sidekiq_1.log
:concurrency: 20
:timeout: 300
:dynamic: true
:queues:
  - [default, 1]

Our sidekiq process uses this configuration to start sidekiq. Looks like configmap did its job.

Further if we want to add one new queue to sidekiq, we can simply modify the configmap template and restart the pod.

For example if we want to add mailer queue we will modify template as shown below.

apiVersion: v1
kind: ConfigMap
metadata:
  name: test-staging-sidekiq
  labels:
    name: test-staging-sidekiq
  namespace: test
data:
  config: |-
    ---
    :verbose: true
    :environment: staging
    :pidfile: tmp/pids/sidekiq_1.pid
    :logfile: log/sidekiq_1.log
    :concurrency: 20
    :queues:
      - [default, 1]
      - [mailer, 1]
    :dynamic: true
    :timeout: 300

Let’s launch this configmap using following command.

$ kubectl apply -f  test-configmap.yml
configmap "test-staging-sidekiq" configured

Once the post is restarted, it will use new sidekiq configuration fetched from the configmap.

In this way, we keep our Rails application configuration files out of the source-code and tweak them as needed.

Rails 5.1 adds support for limit in batch processing

This blog is part of our Rails 5.1 series.

Before Rails 5.1, we were not able to limit the number of records fetched in batch processing.

Let’s take an example. Assume our system has 20 users.

 User.find_each{ |user| puts user.id }

The above code will print ids of all the 20 users.

There was no way to limit the number of records. Active Record’s limit method didn’t work for batches.

 User.limit(10).find_each{ |user| puts user.id }

The above code still prints ids of all 20 users, even though the intention was to limit the records fetched to 10.

Rails 5.1 has added support to limit the records in batch processing.

 User.limit(10).find_each{ |user| puts user.id }

The above code will print only 10 ids in Rails 5.1.

We can make use of limit in find_in_batches and in_batches as well.

 total_count = 0

 User.limit(10).find_in_batches(batch_size: 4) do |batch|
   total_count += batch.count
 end

 total_count
#=> 10

Rails 5.1 does not share thread_mattr_accessor variable with subclass

This blog is part of our Rails 5.1 series.

Rails 5.0 provides mattr_accessor to define class level variables on a per thread basis.

However, the variable was getting shared with child classes as well. That meant when a child class changed value of the variable, then its effect was seen in the parent class.

class Player
  thread_mattr_accessor :alias
end

class PowerPlayer < Player
end

Player.alias = 'Gunner'
PowerPlayer.alias = 'Bomber'

> PowerPlayer.alias
#=> "Bomber"

> Player.alias
#=> "Bomber"

This isn’t the intended behavior as per OOPS norms.

In Rails 5.1 this problem was resolved. Now a change in value of thread_mattr_accessor in child class will not affect value in its parent class.

class Player
  thread_mattr_accessor :alias
end

class PowerPlayer < Player
end

Player.alias = 'Gunner'
PowerPlayer.alias = 'Bomber'

> PowerPlayer.alias
#=> "Bomber"

> Player.alias
#=> "Gunner"

Rails 5.1 introduced assert_changes and assert_no_changes

This blog is part of our Rails 5.1 series.

Rails 5.1 has introduced assert_changes and assert_no_changes. It can be seen as a more generic version of assert_difference and assert_no_difference.

assert_changes

assert_changes asserts the value of an expression is changed before and after invoking the block. The specified expression can be string like assert_difference.

@user = users(:john)
assert_changes 'users(:john).status' do
  post :update, params: {id: @user.id, user: {status: 'online'}}
end

We can also pass a lambda as an expression.

@user = users(:john)
assert_changes -> {users(:john).status} do
  post :update, params: {id: @user.id, user: {status: 'online'}}
end

assert_changes also allows options :from and :to to specify initial and final state of expression.

@light = Light.new
assert_changes -> { @light.status }, from: 'off', to: 'on' do
  @light.turn_on
end

We can also specify test failure message.

@invoice = invoices(:bb_client)
assert_changes -> { @invoice.status }, to: 'paid', 'Expected the invoice to be marked paid' do
  @invoice.make_payment
end

assert_no_changes

assert_no_changes has same options and asserts that the expression doesn’t change before and after invoking the block.

Forward ActiveRecord::Relation#count to Enumerable#count if block given

This blog is part of our Rails 5.1 series.

Let’s say that we want to know all the deliveries in progress for an order.

The following code would do the job.

class Order
  has_many :deliveries

  def num_deliveries_in_progress
    deliveries.select { |delivery| delivery.in_progress? }.size
  end

end

But usage of count should make more sense over a select, right?

class Order
  has_many :deliveries

  def num_deliveries_in_progress
    deliveries.count { |delivery| delivery.in_progress? }
  end

end

However the changed code would return count for all the order deliveries, rather than returning only the ones in progress.

That’s because ActiveRecord::Relation#count silently discards the block argument.

Rails 5.1 fixed this issue.

module ActiveRecord
  module Calculations

    def count(column_name = nil)
      if block_given?
        to_a.count { |*block_args| yield(*block_args) }
      else
        calculate(:count, column_name)
      end
    end

  end
end

So now, we can pass a block to count method.

Rails 5.1 has introduced Date#all_day helper

Sometimes, we want to query records over the whole day for a given date.

>> User.where(created_at: Date.today.beginning_of_day..Date.today.end_of_day)

=> SELECT "users".* FROM "users" WHERE ("users"."created_at" BETWEEN $1 AND $2) [["created_at", 2017-04-09 00:00:00 UTC], ["created_at", 2017-04-09 23:59:59 UTC]]

Rails 5.1 has introduced a helper method for creating this range object for a given date in the form of Date#all_day.

>> User.where(created_at: Date.today.all_day)

=> SELECT "users".* FROM "users" WHERE ("users"."created_at" BETWEEN $1 AND $2) [["created_at", 2017-04-09 00:00:00 UTC], ["created_at", 2017-04-09 23:59:59 UTC]]

We can confirm that the Date#all_day method returns the range object for a given date.

>> Date.today.all_day

=> Sun, 09 Apr 2017 00:00:00 UTC +00:00..Sun, 09 Apr 2017 23:59:59 UTC +00:00

Binding irb - Runtime Invocation for IRB

This blog is part of our Ruby 2.4 series.

It’s very common to see a ruby programmer write a few puts or p statements, either for debugging or for knowing the value of variables.

pry did make our lives easier with the usage of binding.pry. However, it was still a bit of an inconvenience to have it installed at runtime, while working with the irb.

Ruby 2.4 has now introduced binding.irb. By simply adding binding.irb to our code we can open an IRB session.

class ConvolutedProcess
  def do_something
    @variable = 10

    binding.irb
    # opens a REPL here
  end
end

irb(main):029:0* ConvolutedProcess.new.do_something
irb(#<ConvolutedProcess:0x007fc55c827f48>):001:0> @variable
=> 10

Using Kubernetes Persistent volume to store persistent data

In one of our projects we are running Rails application on Kubernetes cluster. It is proven tool for managing and deploying docker containers in production.

In kubernetes containers are managed using deployments and they are termed as pods. deployment holds the specification of pods. It is responsible to run the pod with specified resources. When pod is restarted or deployment is deleted then data is lost on pod. We need to retain data out of pods lifecycle when the pod or deployment is destroyed.

We use docker-compose during development mode. In docker-compose linking between host directory and container directory works out of the box. We wanted similar mechanism with kuberentes to link volumes. In kubernetes we have various types of volumes to use. We chose persistent volume with AWS EBS storage. We used persistent volume claim as per the need of application.

As per the Persistent Volume’s definition (PV) Cluster administrators must first create storage in order for Kubernetes to mount it.

Our Kubernetes cluster is hosted on AWS. We created AWS EBS volumes which can be used to create persistent volume.

Let’s create a sample volume using aws cli and try to use it in the deployment.

aws ec2 create-volume --availability-zone us-east-1a --size 20 --volume-type gp2

This will create a volume in us-east-1a region. We need to note VolumeId once the volume is created.

$ aws ec2 create-volume --availability-zone us-east-1a --size 20 --volume-type gp2
{
    "AvailabilityZone": "us-east-1a",
    "Encrypted": false,
    "VolumeType": "gp2",
    "VolumeId": "vol-123456we7890ilk12",
    "State": "creating",
    "Iops": 100,
    "SnapshotId": "",
    "CreateTime": "2017-01-04T03:53:00.298Z",
    "Size": 20
}

Now let’s create a persistent volume template test-pv to create volume using this EBS storage.

kind: PersistentVolume
apiVersion: v1
metadata:
  name: test-pv
  labels:
    type: amazonEBS
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteMany
  awsElasticBlockStore:
    volumeID: <your-volume-id>
    fsType: ext4

Once we had template to create persistent volume, we used kubectl to launch it. Kubectl is command line tool to interact with Kubernetes cluster.

$ kubectl create -f  test-pv.yml
persistentvolume "test-pv" created

Once persistent volume is created you can check using following command.

$ kubectl get pv
NAME       CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS      CLAIM               REASON    AGE
test-pv     10Gi        RWX           Retain          Available                                7s

Now that our persistent volume is in available state, we can claim it by creating persistent volume claim policy.

We can define persistent volume claim using following template test-pvc.yml.

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-pvc
  labels:
    type: amazonEBS
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi

Let’s create persistent volume claime using above template.

$ kubectl create -f  test-pvc.yml

persistentvolumeclaim "test-pvc" created

After creating the persistent volume claim, our persistent volume will change from available state to bound state.

$ kubectl get pv
NAME       CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS     CLAIM               REASON    AGE
test-pv    10Gi        RWX           Retain          Bound      default/test-pvc              2m

$kubectl get pvc
NAME        STATUS    VOLUME    CAPACITY   ACCESSMODES   AGE
test-pvc    Bound     test-pv   10Gi        RWX           1m

Now we have persistent volume claim available on our Kubernetes cluster, Let’s use it in deployment.

Deploying Kubernetes application

We will use following deployment template as test-pv-deployment.yml.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: test-pv
  labels:
    app: test-pv
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: test-pv
        tier: frontend
    spec:
      containers:
      - image: <your-repo>/<your-image-name>:latest
        name: test-pv
        imagePullPolicy: Always
        env:
        - name: APP_ENV
          value: staging
        - name: UNICORN_WORKER_PROCESSES
          value: "2"
        volumeMounts:
        - name: test-volume
          mountPath: "/<path-to-my-app>/shared/data"
        ports:
        - containerPort: 80
      imagePullSecrets:
        - name: registrypullsecret
      volumes:
      - name: test-volume
        persistentVolumeClaim:
          claimName: test-pvc

Now launch the deployment using following command.

$ kubectl create -f  test-pvc.yml
deployment "test-pv" created

Once the deployment is up and running all the contents on shared directory will be stored on persistent volume claim. Further when pod or deployment crashes for any reason our data will be always retained on the persistent volume. We can use it to launch the application deployment.

This solved our goal of retaining data across deployments across pod restarts.