Ruby and Ajax

LUG Programming Course, 25th February 2008
After our brief, but intense, first look at Ruby, we can start using this excellent scripting language to create our first web application. We previously used the Prototype and script.aculo.us libraries to dynamically modify an XHTML page, now we'll use Ajax via the script.aculo.us Ajax.Autocompleter to 'talk' to a simple Ruby web application.
This lesson will introduce you to WEBrick, the Ruby web server, HTML forms, the ERB templating system, and Ajax.
Despite the amount of code involved, the lesson went quite quickly, and I managed to finish with about twenty minutes to spare. We used that time to go over some of the Ruby code again. Everyone seemed to enjoy the results of running their first web application. One student even started adding his own text files, which I found gratifying, because that was what the application was designed for – any number of plain text files.

The Word Search Web Application

Well don't get too excited, this isn't going to get Google worried. Our objective in this lesson is to use the WordCount class from the previous lesson so that the user can type a word, and then choose one of the text files that contains that word.
To make things a little more interesting, we want the results list to show the title and word count for each text file, we want to use Ajax to reduce the bandwidth to the server and increase responsiveness, and we also want to fall back to traditional HTTP requests and responses if JavaScript is not available.
A Little Architecture
Before we dive into the code, we'll have to spend a little time thinking about how we want to accomplish this task.
Obviously we need to learn how to use the Ajax.Autocompleter class in script.aculo.us. There is an example in the /tests/functional/ folder of the downloaded code, called ajax_autocompleter.html.
Further we'll need to write some Ruby to create our server application. This means we'll need to learn a little about the WEBrick server, and Ruby's templating system, called ERB. For a brief overview of WEBrick, there's the Gnome's Guide to WEBrick, by Yohanes Santoso. The class documentation provides plenty of examples for ERB.
Finally, our two paragraph specifications require a little more from our WordCount class than it currently handles – we need the count for a specific word. The WordCount class has this information, it's just that there's no method to get at it. We'll also need a class that parses all five text files, and which effectively becomes the search engine, and a class for the results.
Before we start our project, let's take a look at the folder layout:
app/
public/
  scripts/
  styles/
  text/
This is a little more complicated than our previous examples. We've added an app/ folder which contains our Ruby code. Under the public/ folder we've added a sub-folder text/ containing the text files. Why go to all this trouble? The reason is security. We don't want a user to have access to our web application script files, only to the public static files, or to the results that our application sends back. Although our application will only be run localhost (this is to say, on our own computer), I'm trying to get you into the habit of keeping security in mind – it's a big bad world out there.
The Search Classes
We'll build our web application in two files, one for the server (server.rb), and one for the search classes (search.rb). Of course, we'll also be using our word_count.rb file from the last lesson, which we'll copy into the app/ folder.
As always, you can download the source code for this lesson, via the link given at the end of this discussion. First then, let's take a look at our 'search engine' in search.rb.
01 require 'pathname'
02 require 'word_count'
03
04 # Add a method to our existing class
05 class WordCount
06
07   attr_accessor :url
08
09   # Return the count for a specific word
10   def word_count(word)
11     (count = @word_hash[word]) ? count : 0
12   end
13 end
14
We need to add a couple of methods to our original WordCount class. Since Ruby is a dynamic language, it allows us to reopen existing class definitions (including built-in classes like String), and make the necessary changes. Since these changes are specifically for this application, we prefer not to change the original code.
Lines 1 and 2 use the Kernel.require method to load external scripts into memory. We'll see this in more detail in our server application code.
Line 7 adds a (read and write) accessor method for the url of the text file. We'll need this, because the URL is different from the file path we use to parse the file.
Lines 10 to 12 adds a new method word_count which returns the word count for a specified word. The ternary operator is used to ensure that a numeric value of 0 is returned when the @word_hash does not contain the word.
15 # Holds the result of a word search
16 class Result
17
18   attr_reader :url, :title, :count
19
20   def initialize(wc, count)
21     @url = wc.url
22     @title = wc.title
23     @count = count
24   end
25
26   # Compare two results for ordering
27   def <=>(other)
28     return @title <=> other.title if @count == other.count
29     other.count <=> @count
30   end
31 end
32
Next comes the Result class. It's job is to contain the result for each word found, that is to say the URL, title, and word count.
Lines 27 to 30 define the comparison operator for this class. Ruby, different to JavaScript, allows for operator overloading. The <=> method (the comparison or 'spaceship' method) is used, among other things, by Array.sort. We can overload this operator to order our results by highest word count (line 29), or by title when the word counts are identical (line 28).
33 # The search engine
34 class SearchEngine
35
36   attr_reader :files
37
38   def initialize(path, remove, add)
39     @files = []
40     path = Pathname.new(path)
41     path.each_entry do |file|
42       file = path + file
43       if file.file? && file.readable?
44         url = file.to_s
45         wc = WordCount.new(url, WordCount::MIN)
46         wc.parse
47         wc.url = url.index(remove) == 0 ? add + url[remove.length..-1] : url
48         @files << wc
49       end
50     end
51   end
52
The search engine has two important jobs. First it has to parse all the text files, then it has to produce results for a specified word. Lines 38 to 51 take care of the file parsing. To do that we make heavy use of the Pathname class, as well as our modified WordCount class.
We check every file in the path (line 41), accepting readable files (line 43), which we put into a WordCount instance and parse (lines 45 and 46), and then add to the @files array (line 48). Line 47 massages the file path into the correct web application URL, removing the remove string from the start of the URL, and suffixing the add string. Using a range and array notation in a String, such as url[remove.length..-1], is the Ruby equivalent of the JavaScript substring method.
53   # Return an array of the matching results
54   def search(word)
55     results = []
56     if word.length > 0
57       @files.each do |wc|
58         count = wc.word_count(word)
59         results << Result.new(wc, count) if count > 0
60       end
61     end
62     results.sort
63   end
64 end
The search method produces an array of Result instances for the specified word. Line 57 to 60 retrieve the word count for each file parsed (as long as the word is not the empty string, tested on line 56). If the word count is greater than 0, then we create a new Result object, and add it to the array (line 59).
The array is then sorted and returned to the caller (line 62).
The Server Application
With our helper classes in place, we can now look at the server application itself, server.rb.
01 #!/usr/local/bin/ruby -w
02
03 require 'erb'
04 require 'webrick'
05 require 'search'
06
While our program has immediate access to all the built-in classes of Ruby, we frequently need to use other library code stored in external files. Ruby provides a mechanism for loading external files, which may be Ruby code or compiled binary libraries, via the Kernel.require method. If you leave off the extension, Ruby will search its library paths first for files with the .rb extension, then binary files with .so or .dll extensions. It is normal Ruby practice to leave off the extension. The application's directory also forms part of the paths that Ruby searches for external files, which means that we can also easily include our own code, on line 5.
07 # The Search Web Application
08 class SearchServer < WEBrick::HTTPServer
09
10   def initialize(config = {})
11     super(config)
12
We'll create a SearchServer class based on the WEBrick::HTTPServer class. We'll leave WEBrick to do most of the hard work, but we still need our own initialize method. Here we simply use more or less the same parameters as the super class, and then call the super class initialize method, on line 11.
Now that we have initialised the super class, we can get on with our own initialisation.
13     # HTML template
14     @html = ERB.new <<-xXx
15 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
16 <html xmlns='http://www.w3.org/1999/xhtml'>
17   <head>
18     <meta http-equiv="Content-Type" content='text/html; charset=UTF-8' />
19     <link rel="stylesheet" href="/styles/search.css" type="text/css" media="screen" charset="utf-8" />
20     <script src="/scripts/prototype.js" type="text/javascript" charset="utf-8"></script>
21     <script src="/scripts/effects.js" type="text/javascript" charset="utf-8"></script>
22     <script src="/scripts/controls.js" type="text/javascript" charset="utf-8"></script>
23     <script src="/scripts/search.js" type="text/javascript" charset="utf-8"></script>
24     <title>LUG Programming Course, Lesson 7</title>
25   </head>
26   <body>
27     <p class='title'>Example of Ruby and Ajax</p>
28     <form id='searchForm' action='/search'>
29       <p class='search'>
30         <label for="search">Search word:</label>
31         <input type="text" id="search" name="search" maxlength='20' value='<%= word %>' />
32       </p>
33       <div id="results" class="autocomplete" style="display: block;">
34         <p>Results:</p>
35         <%= links %>
36       </div>
37     </form>
38   </body>
39 </html>
40 xXx
41
Lines 14 to 40 creates our first template. This is practically identical to the normal static XHTML file contents, except for lines 31 and 35. There we have used the ERB expression delimiters <%= ... %>, to inject two Ruby variables, word and links. The first will contain the word submitted by the form, and the second will contain the results of the word search.
Another novelty are the <form> and <input> elements (lines 28 and 31). The <form> element defines or contains all of the <input> elements which make up a parameterised request. When the form is submitted to the URL defined by action, the parameters are sent to the server as name-value pairs.
42     # HTML/Ajax template
43     @ajax = ERB.new <<-xXx
44 <ul>
45 <% results.each do |r| %>
46   <li><a href='<%= r.url %>' type='text/plain'><%= r.title %> (<%= r.count %>)</a></li>
47 <% end %>
48 <% if results.length == 0 && is_html %>
49   <li>No matches found, sorry.</li>
50 <% end %>
51 </ul>
52 xXx
53
This is the second, and more complex template in our application. The <% ... %> delimiters allow us to add Ruby statements inside the template. Lines 45 to 47 produce <li> elements for each of the Result objects. Lines 48 to 50 add a <li> element explaining that no matches were found, though this only happens if there are no results and the is_html flag is true.
In fact, we'll use this template both for the Ajax responses and for the HTML form submission responses. In the first case, we don't need to make any apologies, as the list will simply disappear, but in the case of our HTML form, we have already produced a “Results” title for our list, so we have to show something.
54     # Search engine initialisation
55     @engine = SearchEngine.new('../public/text', '../public', '')
56     @engine.files.each { |file| logger.info("Loaded #{file.to_s} URL: #{file.url}") }
57
Now we can build the search engine. We set the remove parameter to '../public' and the add parameter to '', so that all the text file URLs will map to '/text/...'. Just to make sure, we log the parsed results, so we can see that the right thing is happening via the server console on line 56.
58     # Define the server mount points
59     mount('/', WEBrick::HTTPServlet::FileHandler, '../public')
60     mount_proc('/search') { |req, resp| search(req, resp, true) }
61     mount_proc('/word')   { |req, resp| search(req, resp, false) }
62   end
63
We use the mount method to map the public/ folder containing our static files. This tells the server that all server requests beginning with '/' should be handled as files, and that the FileHandler should look in the folder '../public'.
We use the mount_proc method to specify the procedure or code block that responds to a specific server URL. Here we map the HTML form URL (/search) and the Ajax request URL (/word) to a search method that we'll define in a moment. Note that, for HTML form requests, the third parmeter is true, and for Ajax requests it is false.
64   # Define the dynamic (HTML form or Ajax) word search response
65   def search(request, response, is_html)
66     word = request.query['search'] || ''
67     word.strip!
68     results = @engine.search(word)
69     response['Content-Type'] = 'text/html'
70     links = @ajax.result(binding)
71     if is_html
72       response.body = @html.result(binding)
73       logger.info "HTML Word: '#{word}' Results: #{results.inspect}"
74     else
75       response.body = links
76       logger.info "Ajax Word: '#{word}' Results: #{results.inspect}"
77     end
78   end
79 end
80
In the search method the is_html flag is used to modify the response, either sending back the XHTML fragment using the @ajax template, when is_html is false, or the entire XHTML document using the @html template, when is_html is true.
We retrieve the word from the request query, using the short circuit technique to guarantee a string value (line 66). The parameter name, search, matches the <input> element name attribute value. We remove any leading and trailing whitespace, and then call the search @engine for the search results on line 68.
Next, we prepare the response, and use the Kernel.binding method to bind our local parameters and variables (is_html, word, and results) to the @ajax template. Calling the template result method returns a String, with all the expressions resolved.
Next we test if the is_html flag is true on line 71. If it is, we use the @html template (which incorporates the @ajax template result via the links local variable) to create the response.body, otherwise the response.body will consist of the result of the @ajax template, previously stored in the links local variable. In both cases, to be sure that the engine is doing the right thing, we send the results to the server logger (line 73 and 76), using the Object.inspect method to expand the contents of the array.
81 # Create the server
82 server = SearchServer.new(:Port => 8086)
83
84 # trap signals to invoke the shutdown procedure cleanly
85 ['INT', 'TERM'].each do |signal|
86   trap(signal) { server.shutdown }
87 end
88
89 # Start the server
90 server.start
Finally we create the server, specifying port 8086, instead of the usual port 80, to avoid collisions with other web servers on your computer, such as IIS or Apache. Then we take a code snippet from the Gnome's Guide to WEBrick to trap shutdown (Ctrl+C), and start the server.
The JavaScript File
Thanks to the all the code we have written in Ruby, and more importantly, to the script.aculo.us library, the JavaScript code in search.js is simplicity itself.
01 /*
02  * Word Search
03  */
04
05 document.observe('dom:loaded', function() {
06   new Ajax.Autocompleter('search', 'results', '/word', {
07     method: 'get',
08     minChars: 2,
09     updateElement: function(item) { /* no update */ }
10   });
11 });
We must specify the input element (search), output element (results), and the submission URL (/word). The important options here are minChars and updateElement. We set minChars to 2 because that is also the minimum word length that we parsed in the server.
Normally Ajax.Autocompleter will fill the form input element with the value selected from the autocompletion list. In our case this isn't either necessary nor desired. Our list contains links to the text files, which we expect the user to click. To remove this default behaviour, we provide an empty function for updateElement.
The CSS and HTML Files
The CSS file, search.css, is relatively simple, as shown below.
01 body {
02   background-color: #d8ffd8;
03   font: 12pt Verdana, Helvetica, sans-serif;
04 }
05 
06 p, div, ul, li {
07   margin: 0.1em 0.25em;
08   padding: 0;
09 }
10 
11 input {
12   font-family: 'Courier New', Courier, monospace;
13   width: 35ex;
14 }
15
The first section sets style rules for the elements in the page. In particular, we set a monospace font for the <input> element.
16 .title {
17   font-size: 14pt;
18 }
19
20 .search {
21   margin: 1.0em 0.25em;
22 }
23
We use two style classes for the page title, and to distance the input field from the title and the results list.
24 .autocomplete {
25   background-color: #d8f8d8;
26   border: 1px solid #4c4c4c;
27   width: 40ex;
28 }
29
30 .autocomplete ul li {
31   font-size: 10pt;
32   list-style-type: none;
33   margin: 0.25em;
34 }
35
36 .autocomplete ul li:hover {
37   background-color: #e8f8e8;
38}
The final section of our style sheet defines the appearance and behaviour of the autocompletion list.
Our application actually 'lives' at the URL /search, but our users will probably be more used to simply typing the root URL /, which WEBrick converts into a request for /index.html. This means that we also need to supply a simple static XHTML file which redirects the user to our application:
1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
2 <html xmlns='http://www.w3.org/1999/xhtml'>
3   <head>
4     <meta http-equiv="refresh" content="0;url=/search" />
5   </head>
6   <body></body>
7 </html>
The redirect is actioned by the <meta> element on line 4. This causes the page to be refreshed after 0 seconds, using the URL /search.

Running the Application.

To be able to run the application we need to open a console in the apps/ folder. Start the server by typing server.rb. At this point we can reference the root URL of our application in a browser by typing http://localhost:8086/. If all goes well, you should see something like this:
LUGPC7Images/firefox1.png
Now type in flowers in the input box, and you should see the following:
LUGPC7Images/firefox2.png
The console output will also look like the following picture. Note the 'Ajax Word ...' log messages, which refers to data sent in response to the Ajax requests.
LUGPC7Images/cmd1.png
If you type flowers reasonably slowly, you will see the list appear once you've typed flow, it then disappears for flowe and reappears for flower and flowers.
Functional Degradation
The specifications also required that our application still work for browsers that have disabled JavaScript, or that have no JavaScript interpreter (such as some mobile telephone browsers).
Switch off JavaScript in your browser, then refresh the page – you may want to clear the cache first. The result should be:
LUGPC7Images/firefox3.png
Now type flowers in the input box, then press Enter. The result should be:
LUGPC7Images/firefox4.png
As you can see from the picture above, the value of the input element search now also appears in the page URL. The console output will look like the following picture. Note that the log now shows only 'HTML Word ...' messages, as no Ajax requests can now be sent.
LUGPC7Images/cmd2.png

Source Files

All the source files for this lesson, including the Aptana project file, can be found in the LUGPC7.zip archived file, distributed under the GNU Lesser General Public License.

What's Next?

Next we'll take a pause to review some of the important features of Ruby, and take a high level look at Ruby on Rails.