beta

SQL by created on

2009-01-20T09:16:00.001-08:00

Model.find(:all, :conditions => ["date(created_on) = date ( ? )", Date.today.to_s(:db) ])

Views and Boolean query methods

2009-01-19T13:19:00.001-08:00

Code smell: a Boolean query method in your views like a has? Or is?
Solution : move it to a helper or better yet the model

Using github pages

2009-01-16T21:15:00.000-08:00

Started using github pages for this blog. Made a couple of posts.

Find out how easy it is to blog on github

Also released accordion.js that is much simpler and completed tested. Check it out here.
accordion.js announcement

Encapsulation

2009-01-04T10:04:00.000-08:00

The idea of block structure originated with the programming language Algol 60. It appears in most advanced programming languages and is an important tool for helping to organize the construction of large programs. - SICP

* Block Structuring is a way for a procedure to have internal definitions that are local to that procedure. This prevents name collision at minimal and provides simplification. Along with this it also allows the user to unclutter their minds and think in terms of details they are interested in. Splitting a program into smaller procedures and internalising a lot of details leads to good *encapsulation* and better understanding of the overall program.

Inspite of it being such an old idea why do I still come across code that has 300 lines of if_else, case_switch statements in a single big method.

CruiseControl Monitor

2008-12-26T06:49:00.000-08:00

Just put CruiseControl Monitor project on github. It is very 0.1 but all the current features work. Thanks to the team in Pune who showed me this wonderful but extremely skinny application that brings lot of visibility. Special thanks to Nigel Fernandes. Extra special thanks to Thomas Czarniecki from the Australia office who was so open to sharing his project and allowing me to be in the driver seat.

This kind of a project already exists for CruiseControl.NET which can do much more than just CC Monitor. But there was nothing in the Java or Ruby.

From the Readme file:

DESCRIPTION--

CC Monitor as the name suggests allows you to monitor multiple Continuous Integration environments. Currently it can parse cc_tray compatible xml for CruiseControl.
CC Monitor collects all the build information and puts it on a dashboard. CC Monitor is different that the dashboard that comes with Cruisecontrol. With CC Monitor you can actually monitor different Cruisecontrol environments just by specifying the location of the cc_tray.xml file.

CC Monitor provides a big visible display which can be projected to the whole development room. Having a big visible display of Cruise builds can help the team monitor the health of the code all the time and they do not need to look at their local CruiseControl Monitors.

Currently it can display information that is part of the xml file. Apart from that it also displays the progress over a period of time to indicate how healthy the application has been.

Features to come:

1. Adding graphical display for health of application over time
2. Resetting graphs daily
3. Adding audible indications to the monitor
4. Better graphics to indicate the status of the build
5. Marking fix owners for broken builds
and many more....

USAGE--

To use just download and change the project url in controller/main.rb.

Type 'rake monitor:clean' - to setup the database
then type
'rake monitor:start'

PS: It uses amazing ramaze.

Quote for the day

2008-10-07T20:29:00.001-07:00

Maven .... What is it good for?.. Absolutely nothing..

MySQL and Unsigned integer in Rails

2008-10-07T20:07:00.000-07:00

Rails does not allow you to create unsigned integer in an intuitive way.
For example

t.integer user_id, :unsigned => true

will not work.

To make it work you will have to do http://dev.rubyonrails.org/ticket/5219

t.column user_id, 'integer unsigned'

or use http://brentrubyrails.blogspot.com/2007/11/assigning-custom-data-types-to-new.html

or use the patch.

On Linux you may get a /tmp/mysql.sock not found error to fix that locate mysql.sock(or mysqld.sock)
which should typically be found in /var/run/mysqld/mysqld.sock or /var/local/lib/mysql.sock or some place around. (whereis 'mysql.sock' should be of help here.).
Make sure you add a 'socket: /var/run/mysqld/mysqld.sock' entry in your database.yml for every instance of the database located on the
local machine.

On the network we are in we seem to get an 'EMULTIHOP' error accessing the local instance of mysql.
The fix for that is to add a 'host: 127.0.0.1'. I think this error appears only on networks that have something weird in their DNS.

Launchy, Windows, Environment Variables etc.

2008-09-11T20:12:00.000-07:00

I am working on a project that is JRuby, Javascript, Java, Maven, Ant, Rake and lots of css fun. But the even more fun part of it is that we are developing on Windows. It is really heartening to find how, Windows and application development on Windows is getting more and more annoying.

To ease the pain I introduced a few tools on Windows.

ProcExp - www.sysinternals.com - is a very good task manager replacement. It is a free download and very light weight. It allows you to look at all processes(like doing a 'ps' or a 'top' on *nix) including system processes. You can kill a single process or a process tree. The benefit of looking at the tree is, you know how windows applications are spawning processes and how much memory/cpu they are eating up.
Launchy for windows - Inspired by QuickSilver on the MacOSX - this is a decent application launcher. It works similar to QuickSilver is finding and launching applications. Worth a look and download if you are still a fan of 'Quick launch toolbar' and now you have run out of task bar space to add more applications to be on single click. This way you will also use the keyboard more often and take your hand off the mouse.

Now the reason I am giving a special mention to both these applications is as follows :

When you set Environment variables on Windows by navigating as follows 'My Computer ->Right Click -> Properties -> Advanced -> Environment Variables OR Control Panel -> System -> Advanced -> Environment Variables(please let me know if you can do this without touching your mouse - that would be just brilliant) - we all know that you need to restart the Command Prompt for it to be able to load these new directories that you have added to the path. Except in this situation :

When you have Launchy and if you launch command prompt through launchy. Restarting Command Prompt will not help in this case, because the command prompt that was launched by Launchy receives its context from Launchy. And if Launchy was started before you made the changes to the Environment Variables, Launchy will pass on this old context to Command Prompt. In this case you will have to restart Launchy so that the Environment Variables are updated and all the applications that are started by Launchy will have the updated Environment variables.
I could trace this because of ProcExp - as it showed the tree of processes and their parent-child relationship.

So if you have Launchy - restart Launchy after you change Environment Variables on Windows.

Another thing that I was hurt by and my colleague Chris Kozak* rightly attributed to being a windows problem is - when you add a new Environment Variable - it is not available till you restart Windows. Whosoever said that a registry was a good idea and doing it Windows style is better must have been looking at the stars.....

I am glad that I have a *nix system on my personal machine. Its just more logical.

*No developer Chris Kozaks were harmed in the making of this post.

Testing randomness in your application

2008-08-13T19:36:00.000-07:00

I was involved in writing an application which involved pairing participants up for activities. One thing we had to limit was that every participant was paired with 4 other participants. That way the burden would be even on all participants and the distribution of activities would be fair.

Another aspect of this story was to make sure we could do this repeatedly and ensure that a participant is not paired with the same one in the next cycle.(We wanted to come up with a way to do this automatically every so often.)

When we went ahead and analysed this story we found out that, pairing similar to this was being done manually where someone would pick names and assign them. Fairness was verified manually.

This process of throwing pieces of paper and picking names worked for small groups and it became a nightmare once we wanted to do this multiple times. Often we would come across repeated pairs and confusion about what is fair and what is not. The randomness of the whole thing was arbitrary and error prone.

Having known all that we starting designing this story. Figuring out the tests was our next task. We started writing tests but all we ended up with was something that said "test_random_numbers_are_generated" and the body of the test could only go about white box testing the fact that the randomization method was called. Writing different variations gave us something we already knew about. We also went about testing the math behind randomization - which we dropped after initial attempts - since it was clearly testing implementation.

Intrigued with the idea of using random numbers the inherent difficulty of testing random numbers Paulo(who I was working with then) asked how could we test randomization - its almost untestable - he said.

Stepping back and looking at the problem from a the business requirement looking at the desired behavior we started rethinking about our tests and moving away from testing random numbers. What we found was the application as such never cared about randomization. All that the administration was interested in was with the fairness of the pairing. Another thing that was implicit in this requirement was that a person cannot be paired with self. Randomness was our implementation detail and it was how we were thinking about implementing this. Also we did not even need the random numbers provided by ruby. All we needed was

def test_association_with_parity
parity = 2
set = [1,2,3,4,5,6,7,8,9,10]
relations = Relation.associate_with_parity(set, parity)
relations.each do |index, value|
 assert_equal parity, value.size
 assert_equal false, value.include?(index)
end
end

def test_should_ensure_parity_with_no_duplicates
User.randomize 
has_no_duplicates(some_user.pairs)
end

In both these tests the acceptance criteria do not care about randomness of the result. Also it does not matter how this randomness if at all is implemented. What matters is the fairness of the pairing and that there is no one who pairs with self.

In summary I would say that if you are thinking about testing randomness - your application may need another look.

Try asking the question - Does my domain care about the randomness(implementation)? - More often than not your answer would be NO and you will be able to figure out better functional tests for your application. Once you start thinking in terms of behavior of the system instead of the implementation, you will end up with better tests.

Java 5.0 and Junit 4

2008-07-29T07:35:00.000-07:00

While working on some Java code and working with Java 5.0 and Junit4.4 I started by writing some examples and tests.

I would say it was a very pleasant experience working on the new JUnit with Java 5. Parts of code became a lot clearer and easy to read. Lot of clutter was hidden behind more aptly named annotations(For the critics of everything annotated - "Right now we are just talking about good things").

I ran across a lot of issues when using Junit4(most of it was my ignorance.) Some of the following links helped me understand how Junit4 was different.
Junit FAQ
Junit 4 Basics
What's new in Junit 4.4
NoClassDefFoundError with Junit 4.4 (This was the first error I had to deal with when running tests using the command line or ant)
Converting tests from Junit 3.8 to JUnit 4.4

When using Junit4 on command line the following JUnit3 command

java -classpath lib\junit.jar;build\prod\com\MyProject\;build\test\com\MyProject junit.textui.TestRunner com.MyProject.MyTest

becomes

java -classpath lib/junit-4.4.jar;build org.junit.runner.JUnitCore com.MyProject.MyTest

Note the change of runner class in Junit4. Happy Testing with Junit4.

Installing jRuby on Mac OS X

2008-06-20T23:08:00.000-07:00

I was trying to install and run jruby on my mac(Jruby 1.1.2). After downloading and unzipping the installer I tried to run jruby.
But all I was getting was an exception

Exception in thread "main" java.lang.NoClassDefFoundError: org/jruby/Main

Googling did not yield anything useful.
So I started debugging and I found that the CLASSPATH had not been specified correctly in the
jruby file.
All I was getting was an empty string for the CLASSPATH which is why it was not able to find the jruby.jar(which contains the org.jruby.Main class)

To fix this I added

CLASSPATH=./../lib/jruby.jar

before CLASSPATH was being used in the jruby file.

Looks like CLASSPATH has not been defined in the jruby.bat file too. So jruby won't work on Windows either.

Hope this tip helps someone.

Somethings I did not expect from a Mac

2008-03-13T22:55:00.000-07:00

I got a Mac (the new and shiny Mac Book Pro with 2.4Ghz processor and 2GB RAM and 120GB harddisk). I was very
excited and feeling very cool about my new possesion(thanks to a couple of secret angels in the world).
I was becoming excited about doing things differently(or doing them more intuitively as it were), with my Mac.
And so I did - it started when I began experiencing the sleekness of the product.

Its weight way under the bulkiness of my old Dell Latitude D620.
The keyboard has just the number of keys you normally use - also the multi-function function keys (especially the F9-F12 set.)
The comfort of using one click trackpad and the clever use of programmability to add on the right click functionality of the track pad. - I am more of a mouse user though.
The presence of the apple key(ohhh it so much looks and sometimes works like the Windows key on my Dell- but more about that in a bit.)
MacOSX packed with the power of Unix under the hood - thats just brilliant - Now anything I want to setup is just a hack/script away. - Best thing to happen to Operating Systems since Windows(just kidding...)
Placement of speakers as well as the camera is just right.

Some pain points on my Mac (or how Mac missed the bus on some of the key aspects of usability for a developer)

Leopard when launched in December had this bug - the keyboard freezes occasionally. For about 2 months the only solutions were - wait till Mac unfreezes the keyboard on its own or try restarting it(Windows contributed this elixir to software - if it does not work reboot) Thankfully the newest firmware release is supposed to take care of that(fingers crossed). (Another fix I heard was to press on the area between the space bar and the track pad with your thumb for a bit - let me know if it works) - latest update is that on some machines the keyboard firmware update did not fix this. So cross your fingers.
Finder does not show you the hidden files - thanks to all the programmers out there who wrote TinkerTool which helps you do just that. (whereas on windows there is a setting for this - how thoughtful of you Bill)
The applications on Mac are under the Applications folder as Applications/appname.app/Contents/MacOS/app...(isnt this a mac.. having a macOS directory under each one of these is just redundant - especially when you install this application through the dmg installer.)
The Menu items are not mapped to keys - Alt+F+O does not open a new file or Alt+F does not take me to the File menu ...
An important annoyance that comes to mind when using a Mac - they mapped the apple key to do what Control key does on any other OS - Windows, linux, Solaris -
all respond to Ctrl+C and Ctrl+V for copying and pasting. Why would Mac want to be different? Why are they not following the convention?

My Iphone
My iPhone was something that I bought just after I got the Mac and I am very proud to say that I have an unlocked iphone with the latest firmware(1.1.3 - I know the unlocking software works with 1.1.4 but I haven't had the time to update) thanks to Zibri(http://www.hackthatifone.com/forum/index.php?topic=42.0), thanks to GeoHotz(http://iphonejtag.blogspot.com/) and thanks to all those who strived days and nights to get all the iphones to unlock and make numerous iphone owners proud.

I am very happy with the unlocked iphone especially with the OSX under the hood.
All the software that I can install on it(when I last checked I could put VNC and ssh on it.. ) - hackers prevail.
The usability is splendid (thanks Steve Jobs) - except the well known annoyances (viz. cannot forward messages, cannot send messages to a list, no zoom/video on camera) - apparently
there are softwares available which can fix all of these in some way.
The biggest problem is with the Camera part - where I need to use iPhoto to download pictures.
I have found out since then that on Windows the iPhone shows up as a disk and pictures and songs can be transferred
like on a USB key. - But on a Mac I cannot do it. The iphone is not mounted as a volume either.How annoying???????

I am sure even with all this I will still prefer my Mac. But the Mac by no means is the best in usability. I am going to collect all these annoyances(maybe there can be a annoyanceswithMac.com to compete with annoyances.org ) and keep an updated list.

Using Verizon EVDO card on Windows and Linux

2008-01-04T14:24:00.000-08:00

I am using the Verizon Wireless EVDO card on my Dell D620 and it has been a life saver a number of times. It connects and keeps the connection without any hiccups. What I did not like was the bundled software that came with it(which required me to enter so much information about the card and configure it to be used). The bundled software also adds features(that u don't like) like monitoring and controlling all of your wireless networks - thus taking over control of the other wireless options on my machine - 802.11g and bluetooth.
It also adds a useless(irritating) feature of compressing data transfer with Venturi.

I was trying to use the card on my linux partition and found out that all that the EVDO card does is adds itself as a modem for dialup using PPP. I found quick and easy instructions for using EVDO on Ubuntu Linux. I can do all this on linux through a shell command.
For getting the same smooth usability there is a little more work required on windows.

Windows does detect the card as a Modem(Curitel PC Card) and can be easily configured
to use PPP dialing.

Here is how you do it.

From Network Configuration

Select "Create a new connection"
Choose "Connect to the Internet"
Choose "Setup my connection manually"
Choose "Connect using a dial-up modem"
Choose "Modem - Curitel PC Card(COM4)"
"Enter an ISP name" (this is your name for the connection and has nothing to do with the ISP)
Phone number to enter would be "#777"
Uncheck "Use this account name and password when anyone connects to the internet from this computer"
Uncheck "Make this the default connection"
Click "Finish"

The newly created connection starts dialing immediately and will
will open a prompt to enter Username and Password -

Click "Properties" on the prompt

Verify that
In the "General" tab

"Modem - Curitel PC Card(COM4)" is checked
Phone number shows as "#777"
Check "Show icon in notification area when connected"

In "Options" tab

Check "Display progress while connecting"
Uncheck "Prompt for username, password, certificate, etc."
Uncheck "Prompt for phone number"

In Networking ensure

"Type of dialup server I am calling" is set to "PPP: Windows 95/98/NT, Internet"

"This connection uses following items" shows Internet Protocol (TCP/IP) as checked, QoS Packet Scheduler as checked, Deterministic network enhancer as checked

Now from you Network Configuration double click to connect using these connection settings.
You can create a shortcut and save it on the desktop for faster access.(Standard Windows operating procedure)

You should see the balloon in the notification area(system tray) indicating that you are connected.

rails - save! broke the contract - mocking to rescue

2007-09-01T20:35:00.000-07:00

Detail descriptions of mockist and classicist are on Martin Fowler's blog.

Consider

   class Person < ActiveRecord::Base
       #has name, address, home, business, cell phone numbers
      .... more object behaviour
   end

The intended behaviour is to do some system specific tasks incognito - lets say send an email to all his friends.

So we alias the save method as follows

   alias :save_with_notification, :save

   def save_with_notifications
      send_emails
      save
   end

This works well for save. But when somewhere we call save! it all breaks - i.e. our save_with_notifications is not called and nobody gets notified.

The root cause(as analysed by David Vollbracht (not this one)) of this problem was this piece of rails code

   def save
      create_or_update
   end

   # Attempts to save the record, but instead of just returning

   # false if it couldn't happen, it raises a

   # RecordNotSaved exception

   def save!

      create_or_update || raise(RecordNotSaved)

   end

save! does not call save.

On further investigation I found functional tests around the base class for active_record.

But what was missed was a good unit test with mocking like so

   test "save! should successfully save object by calling save" do
      base = Base.new
      base.expects(:save).returns(true)
      assert_nothing_raised base.save!
   end

   test "save! should throw on exception if save fails" do
      base = Base.new
      base.expects(:save).returns(false)
      assert_raises RecordNotSaved, base.save!
   end

Here is a place where mocking does a much better job at testing a specific contract that is a convention and was missed due to black box testing.

This is not the only way to fix this problem though.

The real problem is that the requirements did not capture the fact of aliasing save or save! methods.

Testing this scenario functionally is perfectly possible though the testing could get more involving.

The tests in that case would make sure that aliasing of save method does not break save!.

But the above mocked tests are as simple as it gets and they reflect the simpler(obvious?) underlying requirement of save! calling save.

Note: Formatting code on blogger sucks

:partials are objects - too..

2007-05-09T06:14:00.001-07:00

My recent adventures with ruby and rails have been on an actual application that will be used by 1000s of users. This application seems to suit perfectly for the design of rails. The good thing is that there are not many database transactions, mostly the users are expected to read and search for stuff on this website.

Many parts of this application are reusable. What changes most of the times is the way the parts look like when they are appear in different parts of the website. Also some times the details of those parts behave in different ways on different views(ajax/non-ajax, popup/tab-module, associated/unassociated, etc.).

This behaviour actually make them perfect candidates to be partials so that they can be programmed once and then the view can overlay and change what and how it shows up.

There are some things that really impress me about ruby - especially the part where everything is an object and the duck typing. Also having started doing real object oriented programming since I joined ThoughtWorks ( the other way of doing "OOP" is where I have a Universe class which knows about everything - that was so easy to do in those days), my mindset of trying to find objects has changed the way I write software.

With ruby - when used correctly - all these concepts fit perfectly. Ruby by design allows you excellent encapsulation - a very important feature of OO Design( eg. the duck typing - u only need to know whether it quacks like a duck - no internals are exposed when responding to a message). Rails tries to take these constructs in ruby one level further to the web application level, where the views also can be treated as objects(another important layer that can be treated like an object is the database layer using ActiveRecord).

What I like about the views in rails is that the views can be directly rendered from the actions(no clumsy xml mapping required). Also, the flexibility of using partials in my rhtml code.

A little bit about :partials - partials are something similar to 'include' tags in html where one can create parts of a webpage and bring them together using 'include file'. Creating multiple html files helps reuse of contents on a website - so that one can show a consistent header menu for example all through the website - and also keeps the html code DRY.

:partial builds on this idea. In rails one can create an rhtml file(which is actually interpretable ruby) that rails automagically handles for you to create html content. What can further be done is separate code that one wants to reuse in a file that starts with an underscore eg. _photos.rhtml.

There are multiple benefits of this breaking of code

code is now more readable - when u see a partial that you are not concerned about u can just ignore it and focus on the task u are doing(say u have _photos and _videos partials and you are currently working on the videos part - you dont have to worry about what the _photos partial is doing)

code is much more unit testable - rspec is excellent at testing :partials (My colleagues Mike and Jake can help you with that)

code is much more testable as a whole - mocking out partials at a time can greatly simplify testing - again rspec is your friend.

code is DRY

In your 'my multimedia page' (multimedia.rhtml) can look like
<% @some_variables %>
<% some_other_variables %>
render :partial => 'path/to/photos/photos.rhtml'
<% some_more_ruby_code %>

when _photo.rhtml looks like
<% @some_variable1 manipulation %>Tags: partial, ruby, rails, encapsulation, refactoring, rspec
<% @some_variable2 manipulation %>

The beauty here is the simplicity with which the _photos.rhtml can be separated. What rails does more is that the '@some_variables' that showed up in multimedia.rhtml is automatically available to the :partial. Isn't that awesome one would say - and it is for a start.
But when there are many of these rhtml file that render more of these partials and all the variables are now the @variables one can get lost. Not only that the application is now getting too open - Let me explain.
With a tree of these partials multimedia -> photos -> photo -> photo_detail, if we follow the above pattern, the @variables from multimedia would now be available in photo_detail and this is when everything starts falling apart. Now when you refactor photo_detail you have to worry about how the @variables.
This becomes more painful if you want to split the photo_detail and use the split portions together and separately at the same time. Then the problem of every :partial knowing too much about every other :partial - soon becoming a hairball so big that even cats will choke on them.
To prevent this from happening one should think about :partials as objects - this also goes well with the ruby philosophy of everything being an object.
So when I say :partials = objects all the laws of encapsulation apply.
Having said that the multimedia.rhtml now looks like so
<% @some_variables %>
<% some_other_variables %>
render :partial => 'path/to/photos/photos.rhtml', :locals => {:photo_variable => @some_variable1, :photo_caption => @some_variable2, :photo_credit => some_other_variable1 }
<% some_more_ruby_code %>

Notice that now the :partial can only care about the local variables that are passed to them it can be simplified to look like
<% photo_variable manipulation %>
<% photo_caption manipulation %>
<% photo_credit manipulation %>

Notice how all the @ signs disappeared in the :partial. Also note how I could pass in a local variable from the rhtml to the :partial.

This resolves a few issues :
- Unit testing becomes a breeze - Since the :partial is encapsulated testing the :partial in isolation is easy.
- Reusing the :partial becomes easy - No more grand_parent to grand_son global variables
- refactoring within the :partial - not a problem - because you can write tests for this new functionality and refactor to create new partials - total insulation from the rest of the code.
- refactoring the caller of the :partial - even better since now you do not have to be afraid of breaking something because you removed an @variable.

This technique did help us refactor and increase our testing levels drastically.
Hope it helps you too.

Firefox, MySQL, Rails quirks

2007-04-10T18:54:00.001-07:00

Firefox

Firefox 2.0 has a new feature. It can remember sessions and reopen/reconnect to the same urls if Firefox was closed abruptly.

This helps a lot when you have to restart your Windows machine every so often. This feature becomes annoying especially when you want to automatically open Firefox, run Selenium tests and then close it.

Since this closing of the browser is considered abrupt(even though the tests are done the browser process is killed when we automate with Selenium) and the next time you open firefox - a popup appears asking you whether you want to 'Restore Session'. For the automation programs it is almost impossible to detect this pop-up and interact with it.

Firefox allows you to disable this feature and much more.

The trick is to type 'about:config' in your Firefox address box. It shows you all the parameters you can set(including the ones for your plugins/addons).

There is a cool filter which lets you narrow down your search for a parameter and change it. The one I am referring to for 'Restore Session' is 'browser.sessionstore.enabled'

Rails 1.2 and '.' separators

Starting Rails1.2 routes.rb makes as reserved and you cannot use a '.' in your routes.

Enter :requirements - The workaround to using '.' in your route elements.

For example:

map.connect '/:controller/:action/:id', :controller => 'wiki_pages', 
                                            :action => 'show',
                                            :requirements => {:id => /.*/}

This allows :id to contain '.'s. A nice thing to note is that it supports any regexp.(ref : http://forum.textdrive.com/viewtopic.php?pid=116742)

MySQL and Windows

MySQL goes away(ref : http://dev.mysql.com/doc/refman/5.0/en/gone-away.html) literally a number of times during the day on Windows. The solutions mentioned in this document

seem reasonable and logical but do not necessarily work. What worked this time is a machine restart. I also tried tweaking and setting all timeout values to a very large number but the problem does not completely go away.

Roderick wrote this MySQL-Ruby-Windows thing. Worth a try if it can fix the windows timeout issue.

 Tags: firefox, rails, mysql, windows, dot separator, dot, selenium

Automated Testing and Continuous Integration with Selenium - How to do it out of the box...

2007-03-27T21:34:00.001-07:00

After struggliing with running Selenium with Cruisecontrol I hit on a new solution today and am now able to run Selenium and post results more consistently than before. This way of doing it has the advantages that I did not need to run a rails app to collect results. I have managed to do it with Webrick. Since it was my first time working with Webrick it took me lot longer than I thought.

To start with, here is the problem I was facing. I was trying to do the Cruisecontrol with Selenium all over again on a fresh setup. Once I installed Cruisecontrol and Java(u have to have Java - I had ruby already). I was ready to run the same batch file I had. But I was not able to post the results. Installing a rails application with all the required gems seemed painful and an overkill for the purpose. So I started poking around with WEBrick. Some documentation on WEBrick website got me started and I could send a request to the server. But what I was not able to do was read the request parameters. All the documentation I looked up talked about the same kinds of examples as on the WEBrick website. They all mentioned and showed with an example how to handle simple requests and responses. Printing out the request did not help as it did not have parameters anywhere. On further digging I found some code in the PickAxe book (edition 2 page 248)

where the example spelled out exactly where the query object was 'req.query'. That was where one will find the parameters hash that Selenium populates when it posts results.

Here is a reference of the code that achieved the desired result.

require 'webrick'

if ARGV.empty?
  p "Usage: ruby post_results.rb browser_name"
  p "Example: ruby post_results.rb iexplore" 
  exit(0)
end

@browser = ARGV[0]
url = APPLICATION_UNDER_TEST #"http://www.yourapp.com"
testSuite=PATH_OF_TESTSUITE_RELATIVE_TO_SELENIUM
resultsURL= "http://localhost:2000/postResults" #the server that gathers the results..
testRunner="#{url}/selenium/core/TestRunner.html?test=#{testSuite}&auto=true&resultsUrl=#{resultsURL}"
command = "#{@browser} \"#{testRunner}\""

system "start #{command}" #DOS batch command start starts the command in a new thread

include WEBrick

s = HTTPServer.new( :Port => 2000 )

# HTTPServer#mount_proc(path){req, res ...}
# You can mount also a block by `mount_proc'.
# This block is called when GET or POST.
def save_to_file(params)
Dir.mkdir("results") unless FileTest.exist?("results")
Dir.chdir("results")
Dir.mkdir(@browser) unless FileTest.exist?(@browser)
Dir.chdir(@browser)
results_file_name = "selenium-results.html"

save to file as before

end

s.mount_proc("/postResults"){|req, res|
  params = req.query
  results_file_name = save_to_file(params)
  s.shutdown
  system "TASKKILL /F /IM #{@browser}.exe" #KILL the browser since selenium is done.
}

trap("INT"){ s.shutdown }
s.start

Automated Testing and Continuous Integration with Selenium

2007-03-02T20:53:00.001-08:00

For those who are closely following - this post has been edited for clarity of content and language ;) - ohh and to fit your screen

Selenium is a nice and slim tool for testing. If I may start with the history it was built by Jason Huggins and Paul Gross* from ThoughtWorks and then opensourced on http://www.openqa.com/selenium (List of all the contributors can be found there too).There are 2 versions of Selenium - Selenium Remote Control and Selenium core. Selenium core is the essence of Selenium where as Selenium RC packages the core to enable you to write tests in the language of your choice. There is a recorder (and much more) with Firefox which allows you to point and click and create your tests in any of the languages that Selenium RC supports.

All these niceties made Selenium the choice of testing tool at my current project.

Since it was designed by developers we also thought that Selenium was easy to automate. But as it turned out not so much. Here is the issues we faced and what we did to fix it.

We tried to run Selenium in IE just to realise that we have a same origin policy problem - we had our tests open http://localhost/ and then navigate to http://location.localhost/. Yes you guessed it our domain name changed during the test run. This is what Selenium cannot handle. As per the Same Origin Policy your javascript(Selenium tests/testrunner) cannot run/access content that is on a different domain, protocol or port. If you still would like to use Selenium you have to use the experimental modes (chrome for Firefox or hta for IE) which are really beta and flaky.

So having thrown that out of consideration we had to run tests from the same machine as the application under test(AUT).

We decided to go with Selenium core.(since Selenium RC for some reason stripped our pages of css - ugh?)

To do this we installed Selenium core in our application code ie in the public directory(since ours is a ruby app - for a regular java app in apache tomcat for example webapps would be the equivalent location). Then we could run TestRunner with our TestSuite and fire the tests.

The next step was to automate all this testing so that it runs periodically and runs as part of a build so that your application is functionally tested continuously.

Some details you would like to note before we dig deeper.

Selenium core just runs as a url :

Our URL was as follows

http://localhost:3000/selenium/core/TestRunner.html?test=../tests/TestSuite.html&auto=true&resultsUrl=http://localhost:3000/postResults?browser=ie6

This URL does a few things -

it specifies a test suite TestSuite.html
that the tests are going to run automatically (basically as soon as you point your browser to the above URL that is)
they are going to post results to a results URL.

All this is well documented. What is not documented clearly is that one needs to write a POST handler (our POST handler is at http://localhost:3000/postResults). This could be the hard one if a non-developer is trying to set this thing up.

I have written a quick and dirty one in ruby which I will post here for reference

class PostResultsController < ApplicationController
def index
browser_name = params["browser"]
time = Time.now
timestamp = "#{time.month}/#{time.day}/#{time.year} #{time.hour}:#{time.min}:#{time.sec}"
results_file_name = "public/results/#{browser_name}(#{timestamp})-results.html"

f = File.new(results_file_name, "w+")
# f.write(params)

f.write("<html><LINK href='selenium-test.css' type=text/css rel=stylesheet><body>")
result = params["result"]
f.write("<table>")

f.write("<tr>")
f.write("<td>Result : #{result}</td>")
numTestTotal = params["numTestTotal"]
f.write("<td>Tests Total : #{numTestTotal}</td>")
f.write("</tr>")

f.write("<tr>")
totalTime = params["totalTime"]
f.write("<td>Total Time : #{totalTime}secs.</td>")
f.write("</tr>")

f.write("<tr>")
num_test_passes = params["numTestPasses"]
f.write("<td>Tests Passed : #{num_test_passes}</td>")
num_command_passes = params["numCommandPasses"]
f.write("<td>Commands Passed : #{num_command_passes}</td>")
f.write("</tr>")

f.write("<tr>")
numTestFailures = params["numTestFailures"]
f.write("<td>Tests Failed : #{numTestFailures}</td>")
numCommandFailures = params["numCommandFailures"]
f.write("<td>Commands Failed : #{numCommandFailures}</td>")
numCommandErrors = params["numCommandErrors"]
f.write("<td>Commands Errored : #{numCommandErrors}</td>")
f.write("</tr>")
f.write("</table>")
totalTests = params["numTestTotal"].to_i - 1
for i in 1..totalTests
index_str = 'testTable.' + i.to_s
testTable = params[index_str]
f.write(testTable)
end
f.write("</body></html>")
f.close
end

end

Another thing that I did not allude to was the browser=ie6 parameter on the results URL. This was another requirement for our scripts -that they be run in 6 different browsers and post results for us to look at. That means I have to know for each test what I am running against and this extra parameter on the resultsURL would be nice to have.

To make sure Selenium understood it I had to change selenium-test-runner.js as follows

add the following to HtmlTestRunnerControlPanel prototype

getBrowser: function() {
return this._getQueryParameter("browser");
},

and then use it in TestResult prototype like so

//added getBrowser so that we can run tests on a browser and send the browser name with results.
var browser = this.controlPanel.getBrowser();

If this is not enough Selenium also adds a constraint on how you automate using cruisecontrol. The only way to do that is to fire the browser for test with the URL like above. So cruisecontrol has to take the help of a batch file to automate this - which is less than ideal.

Cruisecontrol by design waits for the build tasks to complete(in this case the batch file). But as the tests run inside the browser Cruise has no way to know after they have finished as browser does not change state.

We had to do something like this in our batch file

set PATH = %PATH%;path_to_ie

start iexplore.exe selenium_url_like_above

rem approx wait of 60secs hack

ping -n 61 127.0.0.1 > nul

taskkill /F /IM iexplore.exe

This batch file above will set the path so that IE can be run. then 'start' the browser in a separate process then wait for 60 secs and then kill IE. Once IE is killed the batch process is over and cruise can wakeup.

You can automate running Selenium tests for any browser as shown above. Just make sure your ping interval is set to appropriate time so that the browser is closed only after the tests are done not before.

(A good reference for batch files and command prompt automation is - http://www.allenware.com/icsw/icswidx.htm. I found the info for 'start' and the hack for sleep using ping here)

2 more hacks are on my mind but I will post in a later discussion.

Phew! This has been my first blog post in a long time. Thanks for the patience.

Don't show errors - Prevent them

2006-12-16T10:30:00.000-08:00

One of my colleagues Josh Evnin recently did a User Centered Design study for the application we are building at a client.

We were trying to build a web application where the user interacts with the system using the well known html forms.
We were doing the traditional form validation where the user is required to enter information/select the required fields.
When the user clicks submit the application does a validation routine and displays errors next to the error fields.

This is one way to make sure that no erratic information enters the application. I would call this method as 'trial and error'.

With the new UCD that Josh came up with - he gave a new direction of thought to error handling and error perception to the user.

The philosophy was why let the user make an error and submit in the first place.
Why not just stop the user from submitting ? - (by being more proactive) - that way the user already knows something is missing and he needsto fill out/select the required fields or else he will not be able to proceed.

Essentially what my UCD expert wants to tell me is why let the user get into error conditions at all.
I as a developer think -Let's do preventive design and minimise the number of round trips to the server and make sure only valid data gets to the server in the first place.

This saves me not only the time to build code for all the errors that the server gets and also adds confusion to my user. (Also I should mention that all the analysts on the project had their own opinions about the error text that should go on the page - but that is a funnier discussion in its own right)

The new design of the above page would look and behave as shown below.

Notice the difference.
As a developer I don't have to use server side validation - which at most times is desired - also with the different types of validation required today the number/string validation that comes standard with frameworks is hardly of any use. So I write my own regex based validation in javascript anyway.

As a user - this is what is more pleasing because I know that I have not completed this page correctly till the submit button is clickable.

Some sugar around training/informative messages on the screen make the screen more friendly to the user.

Get your Agile UI/UCD/Human Computer Interaction earlier in the game... and make friends with him/her..- It can save u a lot of code and make the application look slick....

How to write a blog

2006-11-22T17:35:00.000-08:00

I follow Seth's blog. He has a short post about writing a blog post.

who are we creating software for?

2006-11-03T19:41:00.000-08:00

who are the users of an apple imac, an ipod, a bluetooth phone, a laptop with a dual core processor? - do all of these have to be designed for your granma?

if you glance over the types of users for these hitech devices - or applications - it is the techno savvy college going/graduated individuals. Not many granmas actually own a computer or would like to own an ipod to listen to music. If your granma is anything like my granma (I bet she is) she would be happy with a radio or a record player or a TV that plays decent programs.

so why should we build everything that is good for my granma. I think I would build each of the above based on what my intelligent user - who is different if slightly so in each case - needs. An Ipod - as it is rightly so - is for the active user who travels to college, to work, is in the gym for 2 hours a day etc.
A laptop with dual core is useless for viewing pictures or checking email - that my granma does.

Point to be noted - Don't think about your granma when u design something