Apache Benchmarking with Compression

We belatedly noticing that we didn't turn gzip compression on for our Apache web server. After turning it on, we ran the Apache benchmarking tool and didn't notice any improvement. To get the ab tool to accept compression you need to have it pass the correct header as follows:

ab -n 100 -H "Accept-Encoding: gzip, deflate" http://......

It was quite simple. On our local network we didn't see any improvement since everything is fast. From home and on smart phones, you do see a difference.

Sorting Alphabetically in Java

I've recently had to fix alphabetical sorting in two different Java-based web applications and know we have the problem in other places. It's a pain that this comes up all of the time and there's really nothing to prevent it in the future except programmer diligence. The basic problem is that Strings in Java are sorted lexicographically by default (usually using String's compareTo() method). This is almost never what you want shown in your UI, but when you are writing code it's easy to just call Collections.sort() and expect it to do what you want.
Once you know you have this problem, there's a few ways to do the sorting:

• Do a case-insensitive sort on the Strings. This is very common and is basically a left-over pattern for handling this issue that shouldn't be done. You see this in Java code and JDBC/SQL code where a toUpperCase() or toLowerCase() function is called. A nuance of this is that it's technically random whether "de Lucca" or "De Lucca" comes first since when sorting they both look like "de lucca".
• Use one of Java's language-aware sorting methods such as Arrays.sort(array, Collator.getInstance()). This puts lower case letters before upper case, but on a per-letter basis so that capital Z is not before lower case 'a'. See the sorting examples below.
• Use a SQL Function to properly sort if your database supports it. For example, Oracle's NLSSORT() method. Note that you can't rely on SQL's order by clause by itself because it will also sort text values by their ascii code. And to make matters worse...some of our code did use the NLSSORT() function, but it put the results into a Java TreeSet which resorted them by their ascii code :(
• Consistently use the natural ordering of Strings and get your users to start thinking in terms of ascii and unicode.

Here's some simple code to show the differences between the above techniques. The output is in the table below.

import java.text.*;
import java.util.*;

public class coll {
  public static void main(String args[]) {

    String [] a1 = { "def", "daf", "de la Sol", "De la Sol", "de Battista", "DeBatista", "de Lucca", "deBattista", "Deloitte", "Das", "Deep" };

    String [] a2 = { "def", "daf", "de la Sol", "De la Sol", "de Battista", "DeBatista", "de Lucca", "deBattista", "Deloitte", "Das", "Deep" };

    String [] a3 = { "def", "daf", "de la Sol", "De la Sol", "de Battista", "DeBatista", "de Lucca", "deBattista", "Deloitte", "Das", "Deep" };

    Arrays.sort(a1);
    Arrays.sort(a2, Collator.getInstance());
    Arrays.sort(a3, new Comparator<String>() {
          public int compare(String s1, String s2) {
            if (s1 == null) {
              if (s2 == null) return 0;
              return 1;
            } else if (s2 == null) {
              return -1;
            }
            return s1.toUpperCase().compareTo(s2.toUpperCase());
          } 
        });

    for (int i=0; i < a1.length; i++) {
      System.out.printf("%-15s %-15s %-15s\n", a1[i], a2[i], a3[i]);
    }
  }
}

Here's the example output:

Natural Order (by ASCII code)	using a Collator	Case-insensitive Ordering
Das	daf	daf
De la Sol	Das	Das
DeBatista	DeBatista	de Battista
Deep	deBattista	de la Sol
Deloitte	de Battista	De la Sol
daf	Deep	de Lucca
de Battista	def	DeBatista
de Lucca	de la Sol	deBattista
de la Sol	De la Sol	Deep
deBattista	Deloitte	def
def	de Lucca	Deloitte

Fix for Yahoo! Mail Crashing Firefox

This post may not seem like it's related to Software Development, but if you rely on a working browser to get your job done, problems with them or small features in them, can really impact your productivity. Here's a solution the problem I was having.

A few weeks ago, Firefox was frequently crashing on me. The culprit seemed to be either the Yahoo! toolbar or Yahoo! Mail. After searching around, I removed a .dll associated with the Yahoo! toolbar, but that didn't work. I then completely removed the Yahoo! toolbar, but that still didn't work. Finally the folks at Firefox put out a new update and I thought that may address the issue, but no still luck. It was now easy to find friends experiencing the same problem and tons of questions on Yahoo! Anwsers about this.

The problem was clearly related to Yahoo! Mail's web UI now and Firefox would crash whenever I logged out or closed the tab that Yahoo! Mail was in. After living with this for a while, things got worse, my Firefox download dialog stopped showing entries. After searching for a solution to that issue, I found a link on Mozilla's site that recommended removing mimeTypes.rdf file used by Firefox. That didn't solve the problem. I then found another link that recommended deleting Firefox's downloads.rdf file. Finally after exiting Firefox, deleting both of these .rdf files and restarting firefox, I no longer am having issues with Firefox and Yahoo! Mail.

Here's the location of the .rdf files on my machine:

c:/Documents and Settings/username/Application Data/Mozilla/Firefox/Profiles/eak5tu9x.default/downloads.rdf

c:/Documents and Settings/username/Application Data/Mozilla/Firefox/Profiles/eak5tu9x.default/mimeTypes.rdf

I hope this helps someone else out there.

Recursively Adding Files With Subversion

This script is for people who use the command line version of Subversion (SVN). I often use SVN's command line tools in Cygwin rather than in my IDE, eclipse. This is just Bash script that runs the svn stat command, pulls out all of the files whose status line starts with a question mark, and runs the svn add command on them.

You need to be cautious, if you have set up your svn:ignore property or your SVN config file to ingore temporary files this will blindly add them. After that, you can use the svn revert filename command to un-add them.

###
#.Name
#   svnadd
#.What
#   recursively finds all of the new files via the 'svn stat' command
#   and runs the 'svn add' command on each one.
###

case "$1" in
    -t) SVN_CMD="echo svn"
        ;;
    -h)
        echo Usage: $(basename 0) [-t|-h]
        exit 1;
        ;;
esac

: ${SVN_CMD:=svn}

for f in $(svn stat | awk '/^?/ {print $2}')
do
    $SVN_CMD add "$f"
done

Find and Grep with Subversion and Cygwin

Subversion (SVN) creates .svn directories to store information. These get in the way of the Unix/Cywin find command. Here's a Bash shell script that will ignore all of the .svn directories for you.

#!/bin/bash
#
#.Name
#   sfind
#.Title
#   Find Command for Subversion Workspaces
#.Author
#   Don Mitchell (10/2005)
#.What
#   Strips out the .svn directories when using the Unix find command.
#

if [ $# -lt 2 ]
then
    echo Usage: $(basename $0) [path] [expression...]
fi

path="$1"
shift

find "${path}" ! -wholename '*/.svn/*' -a ! -wholename '*~' \
    -a ! -wholename "*/.*.swp" -a "$@"

I put this script in my PATH and then add a few aliases to make it easier to invoke.

alias sf='sfind'

This alias just abbreviates the actual name I gave the shell script. I should have just renamed the shell script. Example: sf . -name "*Foo*.java"

alias sff='sfind . -type f -print0 | xargs -0 grep'

This alias, sff, combines grep with find, grepping for the specified pattern in all files under the current directory. Example: sff SOME_CONSTANT_NAME

alias sfg='sfind . -name "*.java" -print0 | xargs -0 grep'

This alias, does the same thing as my sff alias, but it only greps in Java files and ignores everything else.

• Example: sfg someMethodName


• Example: sfg -l someMethodName # list just the file names containing that mehtod

alias sfn='sfind . -name '

This alias is another quick shortcut, but I rarely use it. For example, sfn "*Dao.java" will list all of the files under the current directory that end in Dao.java.

JavaOne Critique

As always, I'm enjoying the JavaOne conference this year. It's only the second day, but my usual criticism of the presentations holds true. The presenters love to show flashy demos and rarely if ever cover the gory details of how you manage the source code, build your project in a test environment, deploy to production, and manage developers using a mix of IDEs. In particular how do you handle the differences between development, test, and production environments: different databases, hostnames, mail servers, and other external resources that must be configured.

This is a minor point, but I would expect a developer's conference to talk about the development life cycle and environment. These issues get too little coverage.

Supporting multiple browsers

http://browsershots.org/

This is an interesting site I just found out about. I'm not sure how accurate the results are and if you are really paying attention to details in your page styling, then this is too slow to be usable. However, it still gives you a quick idea of how your basic web pages look and might highlight a browser that they look bad in.

You're still better off looking at what browsers your users use and then testing with them directly.