Let’s say that the route definition looks like this.
The task at hand is to develop a new programming language which will understand the rules of the route definitions. Since this language deals with routes let’s call this language Poutes . Well Pout sounds better so let’s roll with that.
It all begins with scanner
rexical is a gem which generates scanner generator. Notice that rexical is not a scanner itself.
It will generate a scanner for the given rules. Let’s give it a try.
Create a folder called pout_language and in that folder create a file called pout_scanner.rex . Notice that the extension of the file is .rex .
Before we proceed any further, let’s compile to make sure it works.
While doing gem install do not do gem install rex . We are installing gem called rexical not rex .
Time to add rules
Now it’s time to add rules to our pout.rex file.
Let’s try to develop scanner which can detect difference between integers and strings .
Regenerate the scanner .
Now let’s put the scanner to test . Let’s create pout.rb .
You will get the error undefined method tokenize’ for # (NoMethodError)` .
To fix this error open pout_scanner.rex and add inner section like this .
Regenerate the scanner by executing rex pout_scanner.rex -o pout_scanner.rb . Now let’s try to run pout.rb file.
So this time we got some result.
Now let’s test for a string .
So the scanner is rightly identifying string vs integer. We are going to add a lot more testing so let’s create a test file so that we do not have to keep changing the pout.rb file.
Tests and Rake file
This is our pout_test.rb file.
And this is our Rakefile file .
Also let’s change the pout_scanner.rex file to return an array instead of puts statements . The array contains information about what type of element it is and the value .
With all this setup now all we need to do is write test and run rake .
Tests for integer
I added following test and it passed.
However following test failed .
Test is failing with following message
Notice that in the error message before 123 there is a space. So the scanner does not know how to handle space. Let’s fix that.
Here is the updated rule. We do not want any action to be taken when a space is detected. Now test is passing .
Back to routing business
Now that we have some background on how scanning works let’s get back to business at hand. The task is to properly parse a routing statement like /page/:id(/:action)(.:format) .
Test for slash
The simplest route is one with / . Let’s write a test and then rule for it.
And here is the .rex file .
Test for /page
Here is the test for /page .
And here is the rule that was added .
Test for /:page
Here is test for /:page .
And here are the rules .
Test for /(:page)
Here is test for /(:page) .
And here is the new rule
We’ll stop here and will look at the final set of files
This is Rakefile .
This is pout_scanner.rex .
This is pout_test.rb .
How scanner works
Here we used rex to generate the scanner. Now take a look that the pout_scanner.rb . Here is that file . Please take a look at this file and study the code. It is only 91 lines of code.
If you look at the code it is clear that scanning is not that hard. You can hand roll it without using a tool like rex . And that’s exactly what Aaron Patternson did in Journey . He hand rolled the scanner .
In this blog we saw how to use rex to build the scanner to read our routing statements . In the next blog we’ll see how to parse the routing statement and how to find the matching routing statement for a given url .
Neeraj Singhon February 01, 2013