Often while developing a Rails application you may look to have one of these
to boost the performance.
Along with these, Rails 5 now provides a way of caching a collection of records,
thanks to the introduction of the following method:
What is collection caching?
Consider the following example where we are fetching a collection of all users belonging to city of Miami.
Here @users is a collection of records and is an object of class ActiveRecord::Relation.
Whether the result of the above query would be same depends on following
The query statement doesn’t change. If we change city name from
“Miami” to “Boston” then result might change.
No record is deleted. The count of records in the collection should be same.
No record is added. The count of records in the collection should be same.
implemented caching for a collection of records .
Method cache_key was added to ActiveRecord::Relation which
takes into account many factors including
query statement, updated_at column value and the count of the records in collection.
We have object @users of class ActiveRecord::Relation.
Now let’s execute cache_key method on it.
Let’s try to understand each piece of the output.
users represents what kind of records we are holding.
In this example we have collection of records of class User.
Hence users is to illustrate that we are holding users records.
query- is hardcoded value and it will be same in all cases.
is a digest of the query statement that will be executed.
In our example it is MD5( "SELECT "users".* FROM "users" WHERE "users"."city" = 'Miami'")
3 is the size of collection.
is timestamp of the most recently updated record in the
collection. By default, the timestamp column considered is updated_at
hence the value will be the most recent updated_at value in the collection.
Let’s see how to use cache_key to actually cache data.
In our Rails application, if we want to cache records of users belonging to “Miami”
then we can take following approach.
What if your table doesn’t have updated_at column?
Previously we mentioned that cache_key method uses
updated_at column. cache_key also provides an option of
passing custom column as a parameter
then the highest value of that column among the records in the collection will be considered.
For example if your business logic considers
a column named last_bought_at in products table as a factor to decide caching, then you can use the following code.
Edge cases to watch out for
Before you start using cache_key there are some edge cases to watch
Consider you have an application where there are
5 entries in users table with city Miami.
Using limit puts incorrect size in cache key if collection is not loaded.
If you want to fetch three users belonging to city “Miami” then you would execute following query.
Here users contains only three records
hence the cache_key has 3 for size of collection.
Now let’s try to execute same query without fetching the records first.
You can see that the count in the cache is 5 this time even though we have set a limit to 3.
This is because the implementation of
executes query without limit
to fetch the size of the collection.
Cache key doesn’t change when an existing record from a collection is replaced
I want 3 users in the descending order of ids.
Above statement will give us
users with ids [5, 4, 3].
Now let’s remove the user with id = 3.
Note that cache_key both users1 and users2 is exactly same.
This is because none of the parameters that affect the cache key is changed
i.e., neither the number of records,
nor the query statement,
nor the timestamp of the latest record.
a discussion undergoing
about adding ids of the collection records as part of the cache key.
This might help solve the problems discussed above.
Using group query gives incorrect size in the cache key
Just like limit case discussed above cache_key behaves differently
when data is loaded and when data is not loaded in memory.
Let’s say that we have two users with first_name “Sam”.
First let’s see a case where collection is not loaded in memory.
In the above case, the size is 1 in cache_key.
For the system mentioned above, the sizes that you will get shall either be 1 or 5.
That is, it is size of an arbitrary group.
Now let’s see when collection is first loaded.
In the above case, the size is 2 in cache_key.
You can see that the count in the cache key here
is different compared to that where the collection was unloaded
even though the query output in both the cases will be exactly same.
In case where the collection is loaded,
the size that you get is equal to the total number of groups.
So irrespective of what the records in each group are,
we may have possibility of having the same cache key value.
The advantage is that we do not need to restart the
server manually if we want to turn caching “on” or “off”.
It is internally taken care by the dev_cache method
that is executed when rails dev:cache is executed.
You can see in the source code that tmp/restart.txt is being touched.
Please note that this feature is not supported by unicorn, thin and
webrick. My guess is that DHH wants this feature because his team uses
pow and pow restarts when tmp/restart.txt is touched.
He also created an issue for
spring to watch tmp/restart.txt
Disabling development cache
Execute the same command that was used to enable caching. If caching was
previously enabled then it will be turned “off” now.
By default PostgreSQL is configured to be bound to “localhost”.
As we can see above port 5432 is bound to 127.0.0.1. It means any
attempt to connect to the postgresql server from outside the machine will be refused.
We can try hitting the port 5432 by using telnet.
In order to fix this issue we need to find postgresql.conf. In
different systems it is located at different place. I usually search for
Open postgresql.conf file and replace line
Now restart postgresql server.
Here we can see that “Local Address” for port 5432 has changed to 0.0.0.0.
Let’s try to connect to remote postgresql server using “psql”.
In order to fix it, open pg_hba.conf and add following entry at the
The second entry is for IPv6 network.
Do not get confused by “md5” option mentioned above. All it means is
that a password needs to be provided. If you want client to allow
collection without providing any password then change “md5” to “trust”
and that will allow connection unconditionally.
Restart postgresql server.
You should be able to see list of databases.
Now we are able to connect to postgresql server remotely.
Please note that in the real world you should be using extra layer of
security by using “iptables”.
In Rails 4 some commands start with rails and some commands start with
rake. This could be quite confusing for people new to Rails. Let’s see
Our task is to write a database migration and then to run that migration.
Above command creates a migration. Now we need to run that migration.
As you can see first command starts with rails and second command
starts with rake.
In order to consolidate them we can either use rails for everything or
we can use rake for everything.
Choosing rails command over rake command
Some favor using rake over rails.
But an important feature missing in Rake
is the ability to pass arguments.
In order to execute the above command using rake
we will have to pass console and development
as arguments. We can pass these values using environment variables.
That would mean adding additional code in Rake task to fetch right
values and then only we will be able to invoke command
rails console development.
Rails 5 enables executing rake commands with rails
Rails core team
to have consistency by enabling rails command to support
everything that rake does.
For example in Rails 5 commands like db:migrate, setup, test etc which are part of rake command in Rails 4 are
now being supported by rails command.
However you can still choose to use rake to run those commands similar to how
they were run in Rails 4.
This is because Rails community has
introduced Rake Proxy
instead of completely moving the command options from rake to rails.
What happens internally is that when rails db:migrate command is executed, Rails checks if
db:migrate is something that rails natively supports or not.
In this case db:migrate is not natively supported by rails,
so Rails delegates the execution to Rake via Rake Proxy.
If you want to see all the commands that is supported by rails in
Rails 5 then you can get a long list of options by executing rails
More improvements in pipeline
In Rails 4, the routes are usually searched like this.
Since nested forms are not allowed browser will accept the top most
level form. In this case that happens to be the form created by the
hacker. When this form is submitted then “authenticity_token” is also
submitted and Rails will do its check and will say everything is looking
good and thus hacker will be able to hack the site.
Rails 5 fixes the issue by generating a custom token for a form
Rails request-response cycle is very easy to understand.
A request hits the app, a route is matched to a controller action from routes.rb,
and finally controller action processes the request and renders HTML or JSON based
on the type of the request.
But sometimes we want to render our HTML or JSON response outside of this request-response
For example let’s say user is allowed to download PDF version of a
report on web. This can be done using request-response cycle. We also
need to send a weekly report to managers and the email should have the
report as an attachment. Now we need to generate the same PDF but since
emails are sent using background job the request-response cycle is
Rails 5 has this feature baked in.
Let’s say we have a OrdersController and
we want to render individual order outside of controller.
Fire up rails console and execute following command.
This will render app/views/orders/show.html.erb with @order set to
Order.last. Instance variables can be set using
assignsin the same way
we use them in controller actions.
Those instance variables will be passed to
the view that is going to be rendered.
Rendering partials is also possible.
This will render app/views/orders/_form.html.erb and will pass order as local
Say I want to render all orders, but in JSON format.
Even rendering simple text is possible.
Similar to text, we can also use render file and render template.
A typical web request carries it’s own environment with it. We usually handle
this environment using request.env in controllers.
Certain gems like devise depends on env hash for information such as warden token.
So when we are rendering outside of controller,
we need to make sure that
the rendering happens with correct environment.
Rails provides a default rack environment for this purpose.
The default options used
can be accessed through renderer.defaults.
Internally, Rails will build a new Rack environment based on these options.
Customizing the environment
We can customize environment using method renderer.
Let’s say that we need “method as post” and we want
“https to be true” for our background job processing.
Now that we have our custom renderer we can use it to generate view.
Overall this is a nice feature which enables reuse of existing code.
Recently we came across following error
when mobile app was trying to upload images using backend API.
The backend API was developed using Ruby on Rails and was powered using NGINX HTTP server along with unicorn application server.
Notice the the http response received was 502 which is documented in
RFC as shown below.
10.5.3 502 Bad Gateway
The server, while acting as a gateway or proxy, received an invalid response from the upstream server it accessed in attempting to fulfill the request.
Debugging the problem
From the first look it seemed that the client_max_body_size parameter for nginx was too less,
which might cause the request to fail if the uploaded filesize is greater than the allowed limit.
But that was not the case with our NGINX configuration.
The NGINX was configured to accept file size upto 250 megabytes, while the file being uploaded was around 6 megabytes.
In our Rails code we are forcing all traffic to use HTTPS by having production.rb to have following setting.
What this setting does is, it forces HTTPS by redirecting HTTP requests to their HTTPS counterparts.
So any request accessing http://domain.com/path will be redirected to https://domain.com/path
Forcing all traffic to HTTPS still does not explain the vague error sendfile() failed (32: Broken pipe) while sending request to upstream and why the HTTP 502 error which means Bad Gateway
Here is how our nginx.conf is configured.
NGINX is configured to accept both HTTP and HTTPS requests. This is a
pretty standard setting so that application can support both
HTTP and HTTPS.
What it means is that nginx will accept both HTTP and HTTPS request.
It would not force HTTP request to be HTTPS.
It would forward HTTP request to rails backend api.
Here is what happens when you upload a file
When we upload a file then NGINX takes that request and passes that
request to Rails. NGINX expects a response from Rails only when
NGINX is done sending the whole request.
Let’s see what happens when a user visits login page using HTTP. NGINX
takes the request and passes the request to Rails. When NGINX is done
handing over the request to Rails then NGINX expects a response from
Rails. However in this case rather than sending 200 Rails sends a
redirect over HTTPS. So now NGINX takes up the request again over HTTPS
and then hands over request to Rails. Rails processes the request and
returns 200. Everything works out.
Now let’s see what happens when a file is uploaded over HTTP.
User uploads a file over HTTP. NGINX takes up the request and starts
sending the file to Rails. More details about the file upload process
can be see at RFC. The data is
sent to server in chunks. When first chunk is sent to server then server
notices that data is being sent over HTTP and it immediately sends a
response that data should be sent over HTTPS.
In the meantime on the client side , client is still pushing rest of the
data to the server. Client expects a response from the server only when
it is done pushing all the data. However in this case server sends a
response when client is not expecting it.
NGINX at this time is all confused and thinks something has gone wrong
with the gateway and returns HTTP/1.1 502 Bad Gateway and aborts the
upload operation. And that how we get the 502 error.
So next time you see an error like this make sure that you are using
https to upload the file if server is enforcing https.
We at BigBinary adopted ReactJS pretty early. Early in the year we
published series of videos titled Learn ReactJS in steps
which takes a “Hello World” app into a full TODO application using
ReactJS in incremental steps.
If you run bin/rails -h in a Rails 5 app, you will see a new command for running tests.
Before Rails 5, we had to use bin/rake test to run tests.
But in Rails 5, we can use bin/rails test.
It is not just replacement of old rake task but it is backed by a test runner
based on maxitest.
Let’s see what bin/rails test can do.
Running a single test
Now it is possible to run a single test out of the box using the
line number of the test.
Rails will intelligently run the test from user_test.rb having line number 27.
Note that the line number 27 does not need to be first line of the test.
Below is an example.
In the above case test_valid_user can be run as long as line number
provide is between 22 and 30.
Running multiple tests
You can also pass multiple test paths and Rails will run all of those tests.
It is also possible to run all tests from a directory.
Improved failure messages
When a test fails, Rails displays a command which can be used to just run
the failed test.
I can simply copy
bin/rails test test/controllers/posts_controller_test.rb:15
and rerun the failing test.
By default when a test fails then rails reports about the test failures
and then moves on to the next test. If you want to stop running the test
when a test fails then use option -f.
Defer test output until the end of the full test run
By default when a test fails then rails prints F and then details
about the failure like what assertion failed and how to re run the test
If you want to have a clean output of . and F and would like all the
test failures report to come at the every end then use option -d.
It is also possible to see the verbose output using -v switch.
It shows time required to run each test.
This would help in detecting slow running tests.
Better backtrace output
By default when an error is encountered while running the test
then the output does not contain full stacktrace.
This makes debugging little bit difficult.
Now we can use -b switch,
which will display complete backtrace of error message.
Switch -s to provide your own seed
Now we can also provide our own seed using -s switch.
Switch -n to run matching tests
Switch -n will run tests matching
the given string or regular expression pattern.
Now by default we will get colored output.
No need to add additional gem to colored output.
With all these awesome features, testing Rails 5 apps has definitely become
a better experience.
Rails has shipped all these features within
the framework itself so you don’t have to use multiple gems and libraries to achieve
all of these things.