In one of our projects
we are running Rails application on
It is proven tool for managing and deploying docker containers in production.
In kubernetes containers are managed using
and they are termed as
deployment holds the specification of pods.
It is responsible to run the pod with specified resources.
When pod is restarted or deployment is deleted then data is lost on pod.
We need to retain data out of pods lifecycle
when the pod or deployment is destroyed.
We use docker-compose during development mode.
In docker-compose linking between host directory and container directory works out of the box.
We wanted similar mechanism with kuberentes to link volumes.
In kubernetes we have various types of
We used persistent volume claim as per the need of application.
We can define persistent volume claim using following template test-pvc.yml.
Let’s create persistent volume claime using above template.
the persistent volume claim, our
persistent volume will change from available state to bound state.
Now we have persistent volume claim available on our Kubernetes cluster,
Let’s use it in deployment.
Deploying Kubernetes application
We will use following deployment template as test-pv-deployment.yml.
Now launch the deployment using following command.
Once the deployment is up and running all the contents on shared directory will
be stored on persistent volume claim.
Further when pod or deployment crashes for
any reason our data will be always retained
on the persistent volume.
We can use it to launch the application deployment.
This solved our goal of retaining data across deployments across pod
A Tempfile object
creates a temporary file
with a unique filename.
It behaves just like
a File object,
and therefore we
can perform all the
usual file operations
Why Tempfile when we can use File
These days it is common
to store file on services like S3.
Let’s say that we have a users.csv
file on S3.
Working with this file remotely is
In such cases it is desirable
to download the file on local machine
After the work is done then
file should be deleted.
Tempfile is ideal for such cases.
Basename for tempfile
If we want to create a temporary file
then we needed to pass parameter to it
prior to Ruby 2.3.
As we can see above the generated file name
begins with “bigbinary” word.
Since Tempfile ensures that the generate filename
always be unique
the point of passing the argument is meaningless.
Ruby doc calls this passing “basename”.
So in Ruby 2.3.0 it was
that the basename parameter
was meaningless for
and an empty string
will be the default value.
But the same was not implemented for Tempfile#create.
in Ruby 2.4.
So now the basename parameter
is set to
empty string by default,
to keep it
consistent with the
lstrip and rstrip methods
which can be used to remove
leading and trailing whitespaces
respectively from a string.
Ruby also has strip method
which is a combination
of lstrip and rstrip
and can be used to remove both,
leading and trailing whitespaces,
from a string.
Prior to Ruby 2.4,
the rstrip method was optimized for performance,
but the lstrip and strip were
In Ruby 2.4,
String#lstrip and String#strip
methods too have been
to get the performance benefit
of String#rstrip .
Let’s run following snippet
in Ruby 2.3 and Ruby 2.4
to benchmark and compare
the performance improvement.
Result for Ruby 2.3
Result for Ruby 2.4
From the above results,
we can see that in Ruby 2.4,
String#lstrip is around 14x faster
String#strip is around 6x faster.
String#rstrip as expected,
has nearly the same performance
as it was already optimized
in previous versions.
Performance remains same for multi-byte strings
Strings can have
single byte or
Lé Hello World is a
because of the presence of
é which is a
Let’s do performance
Lé hello world
Result for Ruby 2.3
Result for Ruby 2.4
As we can see,
the performance for
multi-byte strings is
almost the same across
Ruby 2.3 and Ruby 2.4.
The optimization introduced is
related to how the
strings are parsed
to detect for whitespaces.
Checking for whitespaces
in multi-byte string
requires an additional
adds an initial condition
to check if the string
is a single byte string,
and if so, processes it separately.
In most of the cases,
the strings are single byte
so the performance improvement
would be visible and helpful.
Consider the following file
which needs to be read in Ruby.
We can use the
to get the lines in an array.
As we can see,
the lines in the array
have a \n, newline character,
which is not skipped
while reading the lines.
The newline character
needs to be chopped
in most of the cases.
Prior to Ruby 2.4,
it could be done in
the following way.
Since it was a
Ruby team decided to
an optional parameter
to the readlines method.
So the same
can now be achieved
in Ruby 2.4
in the following way.
methods also have been
modified to accept
an optional chomp flag.
In Ruby, Object#equal? method is used to compare two objects by their
identity, that is, the two objects are exactly the same or not.
Ruby also has Object#eql? method which returns true if two objects
have the same value.
We can see that object ids of the objects are not same.
In ruby, Set
does not allow duplicate items in its collection.
To determine if two items are equal or not in a Set
ruby uses Object#eql? and not Object#equal?.
So if we want to add two different objects
with the same values in a set,
that would not have been possible prior to Ruby 2.4 .
Starting with Ruby 2.4
usage of Fixnum and Bignum constants
How to know if a number is Fixnum, Bignum or Integer?
We don’t have to worry
about this change
most of the times
in our application code.
But libraries like Rails
use the class of numbers
for taking certain decisions.
These libraries need to support
both Ruby 2.4
and previous versions of Ruby.
Easiest way to know
whether the Ruby version
is using integer unification or not
is to check class of 1.
Look at PR #25056
to see how Rails is handling this case.
Similarly Arel is
both Ruby 2.4 and previous versions of Ruby.
This is a story of how we found and fixed memory leak in
We use shoryuken to process SQS messages inside of docker containers. A while
back we noticed that memory was growing without bound. After every few days, we
had to restart all the docker containers as a temporary workaround.
Since the workers were inside of a docker container we had limited tools.
So we went ahead with the UNIX way of investigating the issue.
First we noticed that the number of threads inside the worker was high, 115 in
our case. shoryuken boots up all the worker threads at
The proc filesystem exposes a lot of useful information of all the running
processes. ``/proc/[pid]/task` directory has information about all the threads
of a process.
Some of the threads with lower ID’s were executing syscall 23
These threads were waiting
for a message to arrive in the SQS queue, but most of the threads were
executing syscall 202
At this point we had an idea about the root cause of the memory leak -
it was due to the worker
starting a lot of threads which were not getting terminated.
We wanted to know
how and when these threads are started.
Ruby 2.0.0 introduced tracepoint,
which provides an interface to a lot of
internal ruby events like when a exception is raised, when a method is called or
when a method returns, etc.
We added the following code to our workers.
Executing the ruby workers with tracing enabled revealed that a new
Celluloid::Thread was being created before each method was processed and that
thread was never terminated. Hence the number of zombie threads in the worker
was growing with the number messages processed.
Unfortunately tracepoint didn’t pinpoint the place where the thread was started,
hence we added a couple of puts statements to investigate the issue futher.
After a lot of debugging, we were able to find that a new thread was started to
increase the visibility time
of the SQS message in a shoryuken middleware when
auto_visibility_timeout was true.
The fix was to
the thread after the work is done.