Prototype and script.aculo.us

LUG Programming Course, 4th February 2008
The third lesson showed some of the power available in the JavaScript language, but also kept clear of the uneven surfaces that exist in currently available browsers. The plain fact of the matter is that browsers behave differently, and they are not perfect. Unless you want to become a specialised consultant, most web programmers don't have the time, energy, or resources to understand, and find solutions to these problems.
This fourth lesson makes use of two popular open source JavaScript libraries (Prototype and script.aculo.us) which take care of smoothing out the browser bumps, and create some very professional results, with very little code required by the developer.
This time I managed to recover the part of last week's lesson that I didn't finish, and get though this lesson with about 15 minutes to spare. Unfortunately, I lost about a third of the class along the way. I spent half an hour going over the concepts of the Document Object Model, and anonymous functions, but we'll have to look at these again in the next lesson, which will be a review of the things we have seen up to now.

The Todo List

We'll use Prototype and script.aculo.us to produce a todo list. In fact we want two lists; the “to do” list, and the “done” list. Both lists will be provided as JavaScript objects – actually, we'll use JSON (JavaScript Object Notation) for them both. We'll use the libraries to render the lists as HTML markup, and to change the list order, and swap items from one list to the other. We want the user to be able to change the list order, and swap items, by simply dragging the item around or between the lists.
Obviously, you will need the script.aculo.us library, and we'll be using version 1.8.1 for this course. This library also includes version 1.6.0.1 of the prototype library. The full prototype library, including unit tests, can be downloaded using Subversion, though this isn't necessary for our lesson, and also requires Ruby to be installed to be able to build the distribution library.
The script.aculou.us library also comes with a fairly comprehensive set of unit tests. These serve a dual purpose, apart from testing that the library and its dependencies work, it also demonstrates some of the effects available. We'll be building on the test/functional/sortable4_test.html example in the functional test suite (test/run_functional_tests.html).
Our specification is reasonably lean (just the first paragraph of this section), we've got a reasonable example, but there are still some building blocks that we'll require:
  • Create the HTML page
  • Create the styles
  • Make use of events to update the lists
  • Make the lists draggable and droppable
  • Define our JSON objects
  • Convert our JSON objects to HTML
  • Keep the JSON objects synchronised with the HTML lists
Developers Start your IDEs
We'll create a new project in Aptana Studio, which we'll call SortableLists, and it will have the familiar folder pattern:
public/
  scripts/
  styles/
Take lib/prototype.js, src/builder.js,src/dragdrop.js and src/effects.js from the script.aculo.us download and put them in public/scripts/. Create a CSS file called lists.css in the public/styles/ folder, create a JavaScript file called lists.js in the public/scripts/ folder, and finally create an HTML file called lists.html in the public/ folder.
Next, we'll modify the lists.html file so that it has some, er, lists:
01 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
02 <html xmlns='http://www.w3.org/1999/xhtml'>
03   <head>
04     <meta http-equiv="Content-Type" content='text/html; charset=UTF-8' />
05     <link rel="stylesheet" href="styles/lists.css" type="text/css" media="screen" charset="utf-8" />
06     <script src="scripts/prototype.js" type="text/javascript" charset="utf-8"></script>
07     <script src="scripts/builder.js" type="text/javascript" charset="utf-8"></script>
08     <script src="scripts/effects.js" type="text/javascript" charset="utf-8"></script>
09     <script src="scripts/dragdrop.js" type="text/javascript" charset="utf-8"></script>
10     <script src="scripts/lists.js" type="text/javascript" charset="utf-8"></script>
11     <title>LUG Programming Course, Lesson 4</title>
12   </head>
The <head> of the document isn't new to us. It just contains links to our CSS and JavaScript files, and a title. The <body>, however, contains our first visible markup. Although there are a large number of elements available to us, we're just going to use five of them. To break our page up into divisions (think panels), we use the <div> element. All the titles use paragraph elements, <p>. The lists are made up of unordered lists <ul>, which contain the list items <li>. Our fifth element <pre>, is for preserving the formatting of output from our (yet to be written) JavaScript code:
13   <body>
14     <p class='title'>Example of prototype/script.aculo.us usage</p>
15     <div id='todo'>
16       <p>Todo list:</p>
17       <ul id='todoList'>
18         <li id='todo_4'>Using Professional JavaScript Libraries</li>
19         <li id='todo_5'>JavaScript Refresher Lesson</li>
20         <li id='todo_6'>Ruby Basics Lesson</li>
21         <li id='todo_7'>Ruby on Rails Basics Lesson</li>
22         <li id='todo_8'>Ajax Basics Lesson</li>
23         <li id='todo_9'>Ruby Refresher Lesson</li>
24         <li id='todo_10'>Party Time, Questions and Maybe Some Answers</li>
25       </ul>
26     </div>
27     <div id='done'>
28       <p>Done list:</p>
29       <ul id='doneList'>
30         <li id='todo_1'>Welcome and Software Installation Lesson</li>
31         <li id='todo_2'>JavaScript Basics Lesson</li>
32         <li id='todo_3'>Advanced JavaScript Lesson</li>
33       </ul>
34     </div>
35     <pre id="feedback">...</pre>
36   </body>
37 </html>
What? Too much to type? Alright, I know this isn't a typing course, so just pick up the text file, and rename it lists.html. Now click on the browser preview and lets take a look at what our page looks like:
LUGPC4Images/aptana-1.png
Ok, no design awards for 20th century style, we'll make things a little better in a moment using CSS.
The world of XHTML elements is roughly divided into two; block elements and inline elements. We've only used block elements in our XHTML page, that is to say, elements which break up the flow of text. Every paragraph of this tutorial is a block. Inline elements, such as <img> for images, do not change the flow of text. In that last sentence, I used a <span> inline element to change the font for <img> to monospace.
Two other things to note is that I have used the class and id attributes on some of the elements. The class attribute allows me to create a specific style for a set of elements. More than one element can have the same class name, and the class attribute can specify more than one class name, separated by whitespace. We'll see more of this in a moment.
The id attribute is a powerful helper for JavaScript that has to examine the document content. An id attribute value is much like a JavaScript variable name, and must also be unique within the document. I've created an identifier for each list, todoList and doneList, so that we can access them very efficiently from JavaScript, as we'll see a little later on.
You can learn more about XHTML by checking out the World Wide Web Consortium, they have a brief tutorial, and a big, boring but authoritative specification.
Moving on, we need to make that web page look a little more attractive. CSS rules syntax is quite similar to JavaScript syntax, you start with a comma separated list of selectors, then the style properties cosily wrapped up in curly braces. The style properties are name/value pairs, which have a colon (:) as the separator, and semicolon (;) as the terminator. Comments can also be added, but use the /* comment */ format only.
Again, you can learn more about CSS by checking out the World Wide Web Consortium, they have a brief tutorial, another big, boring but authoritative specification, and a home page with other information.
To keep things simple, we'll stick to simple selectors. If we use a name, we are referring to an element name, such as body, div or ul. If the name is prefixed with a dot (.), such as .todo, then we're referring to a class name, which we previously placed in a class attribute in our XHTML. If the name is prefixed with a hash (#), then we're referring to an identifier, again, which we previously placed in an id attribute in our XHTML.
Open lists.css and add the following code, which I'll explain as we go along:
01 body {
02   font: 12pt Verdana, Helvetica, sans-serif;
03   width: 34em;
04 }
05
Units of measure in CSS come in two flavours, relative and absolute. Relative units can be a percentage (such as 125%), em, ex or px. An em is equivalent to the size of the biggest character in the font (usually the letter m), an ex is equivalent to the size of the average character in the font (typically the letter x), and a px is a pixel. Absolute units are in, cm, mm, pt and pc. An in is an inch (2.54 cm), a cm is a centimeter, a mm is a millimeter, a pt is a printers point (1/72 of an inch), and a pc is a printers pica (1/6 of an inch).
As a side note, always try to avoid using px (pixel) units for anything except image sizes. It's the easiest to use, requiring little or no thought. But how large is your user's monitor? Today, or in two years time?
06 .title {
07   font-size: 14pt;
08   margin: 0.15em;
09 }
10
11 #todo, #done {
12   float: left;
13   width: 16.5em;
14 }
15
The float property is interesting in that it allows us to 'float' blocks on the page, which would otherwise be rendered one below the other. We use it here to get our two lists to appear side by side.
16 p {
17   margin: 0.2em;
18 }
19 
20 ul {
21   list-style: none none;
22   font-size: 11pt;
23   margin: 0;
24   padding: 0;
25 }
26
Unordered lists usually have bullets or other graphics before each item. Using the list-style property, we can change this behaviour. In our case, we get rid of them altogether.
27 li {
28   border: 1px solid;
29   margin: 0.2em;
30   padding: 0.2em;
31   cursor: move;
32  }
33
For each <li> element we set the cursor to move, so that the user will understand that it can be dragged.
34 #todoList li {
35   background-color: #ecacac;
36 }
37
38 #doneList li {
39   background-color: #acecac;
40 }
41
The last two style rules have slightly more complicated selectors. Basically, they change the background colour of all <li> elements that are descendants of the two identifiers todoList and doneList.
42 pre {
43   clear: both;
44   font-size: 8pt;
45 }
The clear property can be used to reset any previous float property values. This guarantees that our <pre> element will appear below the two lists.
Still too much to type? Just pick up the text file, and rename it lists.css. Now we'll take another look:
LUGPC4Images/aptana-2.png
That certainly looks a little better, though I still don't think it'll win any design awards.
One cautionary note about syntax errors in CSS files must be made. The browser won't tell you if there are any. It will either ignore the syntactically incorrect rule, or one or more of the syntactically incorrect properties. In total silence.
This can be the cause of much grief and waste of time. If you think the page rendering is incorrect, check out what Firebug tells you. If you can't find the style, it probably got erased due to a syntax error. Firebug, fortunately, reports such syntax errors.
From Static to Dynamic
Now we'll add the JavaScript code to make the two lists sortable and exchangeable. Because we're using a library, we don't really need to write much code. First we create a helper object to manage the lists, and more importantly, the feedback from changes made to the lists:
01 /*
02  * Sortable Lists
03  */
04 var SortableLists = {
05   lists: ["todoList", "doneList"],
06
07   updated: function (list) {
08     // does nothing for now...
09   },
10
Our variable SortableLists has an array containing the identifiers of our lists, and a method to handle an update of the list (list reordered, or an item was added or removed), which does very little for now.
11   createSortables: function (event) {
12     var lists = SortableLists.lists;
13     lists.each(function (id) {
14       Sortable.create(id, {
15         dropOnEmpty: true,
16         containment: lists,
17         constraint: false,
18         onUpdate: SortableLists.updated
19       });
20     });
21   }
22 };
23
Our final helper function is responsible for making our two lists sortable. The method receives an event object (which it doesn't actually make use of), and then, with the help of some Prototype and script.aculo.us methods, adds the sortable magic to the list elements. I'll leave the discussion about events till a little later, meanwhile we've got a few functions and methods here which you'll need a little help with.
Prototype arrays have many more methods available, including several known as Enumerable. These provide powerful mechanisms for iterating collections. We can 'walk' through our lists array using the each() method, line 13. It basically hands each item in the array as a parameter to a function, which the programmer supplies. The function itself (lines 13 to 19) is anonymous, because it is only used in one place. We don't need to clutter an object with function names that only get used once. This is a recurring theme in the Prototype library. See the Prototype documentation for more information.
Inside our anonymous function, on line 14, we call the script.aculo.us Sortable.create() method, for each list in our lists array. This method uses powerful magic to make some part of the XHTML document sortable. It accepts one required parameter (the identifier of the element to make sortable), and a host of options. Since a programmer may want to change none, one, or half a dozen of these options, the library uses a very smart technique – a map. So, in the map, we can specify just the options we wish to change. All the others will maintain their default values. By the way, documentation of script.aculo.us is available from their wiki. Typing Sortable.create in the search form (yes, it does use auto-completion, which is available from the library itself), yields the relative documentation page.
The options we use tell the Sortable.create method that:
  • We want to be able to drop on an empty list (line 15); dropOnEmpty: true
  • The user can only drop on our two lists (line 16); containment: SortableLists.lists
  • There are no (other) constraints (line 17); constraint: false
  • When the order changes, or when items are added or removed, call our helper function (line 18); onUpdate: SortableLists.updated
One important point is that Sortable makes use of a simple naming convention within the identifiers in each <li> element in order to move the element around.
The only remaining piece in the puzzle is to make sure that our SortableLists.createSortables() method gets called.
24 document.observe("dom:loaded", SortableLists.createSortables);
Still tired of typing? OK, grab the text file then. Remember to rename it to lists.js.
We use a Prototype method document.observe() to add a function which will be called when an event happens. In this case the event is dom:loaded, which means that the DOM (Document Object Model) has been fully loaded into memory. Nota bene: We pass the function as the parameter, we don't actually call the function - document.observe() will do that at the right time.
Until our XHTML document is loaded into a browser, it's really just a piece of text. When loaded into the browser, it becomes a hierarchical set of objects, bristling with properties and methods, which we can then manipulate programmatically. That's the DOM. But we've got to be careful not to touch it until it's complete, and that's what the dom:loaded event tells us.
Finally, we can check out that everything works. Try it out in Firefox. Grab an item, drag it around, see what happens. I personally am very impressed, because I know how hard this is to do. Well, it's true that we loaded 6,000 lines of code from the two libraries, but we only had to write 26 lines of our own code (at least, up to now). How long would it have taken to write code to create that much dynamic action? One week? One month?
But, there are some problems. Move all the done elements to the to do list. Now try moving one back again. Oops, it can't go back. This is because the done list has zero height, so the drop zone is never triggered (see the Firebug information, below). We'll fix this in our final version.
LUGPC4Images/ff2-1.png
Now try it out in some other browsers. Oops, this is what happens in Opera 9.1:
LUGPC4Images/opera9.png
It happens when you move the top item down. It doesn't happen when you move any other item. The Error Console doesn't give any consolation either.
Welcome to the exciting world of the web developer!
So, we need to try some experimentation. After a couple of hours (really), I found out that the problem goes away if you add a small margin and padding to the ul element of each list – 0.05em will do. Fine, so we check everything works again, but, look what happens now in Internet Explorer 7:
LUGPC4Images/ie7-1.png
Hmm. There is a recurring theme here – problems with the first <li>. Empty drop zone, Opera problem, IE problem. We'll cure all these problems in our final version by supplying a non sortable first <li>.
Are we done now? Well, let's try something a little more rigorous. Try enlarging the page (press Ctrl++ a few times). Perfect in Firefox 2, same with Opera 9.1, but look at this mess in Internet Explorer 7, during the drag process:
LUGPC4Images/ie7-2.png
The positioning has clearly gone horribly wrong. The item being dragged seems to move under the done list, and script.aculo.us hasn't done its usual magic shifting the correct red items out of the way.
The item actually ended up in the right list, more or less in the right position. Sigh.
Do we want to find the culprit? I think that would take a lot more work, though my money's on that particular browser. Of course, the others have their problems too.
This is, in my humble opinion, a fairly unlikely situation, so I'm going to pretend that it doesn't happen. Life is full of compromises.

Final Version

The final step is to fix the first <li> problem, remove our fixed list from the XHTML markup, and build it from a JavaScript object. This is a slightly convoluted exercise, but allows us to experiment with dynamic DOM manipulation ourselves.
To do this, we'll have to modify our little trio; lists.js, lists.css and lists.html files.
We'll start with the lists.html file, and fix our first <li> problem at the same time. Only the body changes, as follows:
13   <body>
14     <p class='title'>Example of prototype/script.aculo.us usage</p>
15     <div id='todo'>
16       <ul id='todoList'>
17         <li class='fixed'>Todo List:</li>
18       </ul>
19     </div>
20     <div id='done'>
21       <ul id='doneList'>
22         <li class='fixed'>Done List:</li>
23       </ul>
24     </div>
25     <pre id="feedback">...</pre>
26   </body>
27 </html>
Basically what we are doing here is to remove the lists (they'll be created dynamically by our JavaScript), but more importantly, we've moved the list titles into the lists themselves. This way, we'll have a fixed first item, which cures the first item problem nicely.
Next we'll look at our lists.css file, again I'll just highlight the changed style rules:
18 ul {
19   list-style: none none;
20   font-size: 11pt;
21   margin: 0.2em;
22   padding: 0.2em;
23 }
24
25 li {
26   border: 1px solid #000000;
27   margin: 0.2em;
28   padding: 0.2em;
29 }
30
Our first two style rules define the styles for generic <ul> and <li> elements. The only real difference is that we've added a little margin and padding to the <ul> element.
31 li.fixed {
32   font-size: 12pt;
33 }
34
35 li.sortable {
36   cursor: move;
37 }
38
We have two types of <li> element; fixed (for the titles) and sortable (for the sortable list).
39 #todoList li.fixed {
40   color: #bc2c2c;
41 }
42
43 #doneList li.fixed {
44   color: #2cbc2c;
45 }
46
These two style rules set the font colour for the titles of each list.
47 #todoList li.sortable {
48   background-color: #ecacac;
49 }
50
51 #doneList li.sortable {
52   background-color: #acecac;
53 }
The last two style rules set the background colours for the sortable items of each list.
A Touch of JSON
We'll finish our final example by modifying the lists.js file.
Add the following lines:
07   doneList: [1, 2, 3],
08
09   todoList: [4, 5, 6, 7, 8, 9, 10],
10
11   items: {
12     1:  "1. Welcome and Software Installation Lesson",
13     2:  "2. JavaScript Basics Lesson",
14     3:  "3. More Advanced JavaScript Lesson",
15     4:  "4. Using Professional JavaScript Libraries",
16     5:  "5. JavaScript Refresher Lesson",
17     6:  "6. Ruby Basics Lesson",
18     7:  "7. Ruby on Rails Basics Lesson",
19     8:  "8. Ajax Basics Lesson",
20     9:  "9. Ruby Refresher Lesson",
21     10: "10. Party Time, Questions and Answers"
22   },
23
Here we create three objects; two arrays and a map, which are also JSON objects. This will come in handy when we start working with Ajax and a back-end server, but that's still a little way off.
There isn't anything special here, the two arrays hold the identifiers of the items done, and the identifiers of the items to do, respectively. The map holds all the items, and uses the identifier as the key. Think of this as a database table, or spreadsheet, where the identifier value represents the database record number, or row number of the spreadsheet.
Data Synchronisation
Previously, we had a updated() method which was called when the list order was changed, or items were added or deleted from the list. At the time, it didn't do very much, but now it's going to become a key component, because it will be responsible for keeping our data model synchronised to the view model that the browser user sees.
So what will we have to do to synchronise the data? We'll have to modify the ordered list of identifiers on every updated() call.
24   updated: function (list) {
25     var id = list.getAttribute('id');
26     var array = SortableLists[id];
27     array.length = 0;
28     Sortable.sequence(list).each(function (value) {
29       array.push(parseInt(value));
30     });
31     // Debugging code
32     $('feedback').innerHTML = id + "<br /> " +
33       " todo: " + SortableLists.todoList.inspect() + "<br /> " +
34       " done: " + SortableLists.doneList.inspect();
35   },
36
Our updated() method expects the list element as its only parameter. It gets the identifier of the list, and then gets the associated ordered array for that list (lines 25 to 26). Then it recreates the ordered list, by walking though the <li> elements given by the Sortable.sequence() script.aculo.us method, which returns all valid identifiers as strings. To get the data model identifier then, we simply convert the identifier to an integer and add it to the ordered array (line 29).
The debugging code simply creates a string for the list being modified, and the values of the todo and the done ordered lists.
Obviously, once we're convinced that the code works, we can comment out lines 32 to 34.
Using Sortable.sequence()
We should be a little careful here, because Sortable.sequence() is an internal method, and is not documented in the Wiki. It could change in future versions of script.aculo.us, breaking our code. Caveat Emptor.
Finally, we modify our existing createSortables() method so that it creates the lists first.
37   createSortables: function (event) {
38     var lists = SortableUtils.lists;
39
40     // create the lists
41     lists.each(function (name) {
42       var list = SortableUtils[name];
43       var container = $(name);
44       var attrs = { id: 'todo', 'class': 'sortable' };
45       list.each(function (id) {
46         var text = SortableUtils.items[id];
47         attrs.id = 'todo_' + id;
48         container.insert({ bottom: Builder.node('li', attrs, text) });
49       });
50     });
51
The interesting part here is on lines 40 to 49, which creates the lists in the DOM. First we create a set of <li> elements for each list (lines 44 to 48), using our now familiar each() method. The script.aculo.us library provides a helper object Builder, which creates DOM nodes for us. All we have to supply is the node name, a map of attributes, and the text to go into the node as contents (line 47). We then insert this newly created node into the <ul> element (container), using the Prototype Element.insert() method. Note that, since the element must go to the bottom of the list, we specify the bottom parmeter.
52     // make them sortable
53     lists.each(function (id) {
54       Sortable.create(id, {
55         dropOnEmpty: true,
56         containment: lists,
57         constraint: false,
58         only: 'sortable',
59         onUpdate: SortableUtils.updated
60       });
61     });
62   }
63 };
The code to make our lists sortable has hardly changed. We have, however, added the only option, which tells script.aculo.us to only make <li> elements sortable that have the sortable class name. This will leave our fixed title <li> elements, er, fixed.
Finally, fire up your favourite browser (Firefox, of course), and try it out. You'll see the debugging text at the bottom of the page, once you move an item. Check that it is correct.

Source Files

All the source files for this lesson, including the Aptana project file can be found in the LUGPC4.zip archived file, distributed under the GNU Lesser General Public License.

What's Next?

Next we'll retrace our steps a little, to make sure that you understand these new concepts.