PHP and Tomcat 5

I've been recently looking at integrating PHP with Tomcat. So far I've been unsuccessful at finding a solution that works on all of our development platforms, in particular, finding a php-cgi implementation for Mac OS X has been hard.

On Windows XP, PHP/Java Bridge has worked well. Here's a couple of useful links:

http://php-java-bridge.sourceforge.net/pjb/index.php

http://php-java-bridge.sourceforge.net/pjb/FAQ.html

JSP Best Practices

I'm thinking of giving a talk on JSP best practices. We recently had a couple developers give an excellent talk on CSS and best practices and many of the concepts have corollaries in the underlying JSP rendering of a page. So I thought I start to gather some thoughts in this blog posting:

1. Your JSP page should be valid XML


2. Avoid Scriptlets and JSP directives <% ... %>


3. Use ${fn:escapeXml(...)} frequently, even if you know the data is safe


4. Use <c:url> tag properly, including it's nested <c:param> tag so that URLs are properly encoded.


5. Do not put your URL parameters within the <c:url> tag, put them in nested <c:param> tags.


6. Use the proper and most recent standard JSTL taglibs. Globally replace all old JSTL taglib declarations so you don't have a mix of them floating around.


7. Don't store your own copy of the standard JSTL tld files, they will inevitably end up out of sync with the JAR file you are using.


8. Validate the rendered HTML content


9. Open your JSP page in an XML editor to validate it


10. Differences between <% include %>, <js:include>, and <c:import>


11. Page scope for <c:forEach> tag variables and how to pass them to included or imported pages


12. Parameterizing include/imported JSP fragments


13. Naming standards for JSP page and fragments


14. Using a main page template


15. Using value objects so JSP fragments can be reused


16. Precompile your JSPs

Oracle Text Indexing - escaping user input

Here's a class passed to me for escaping keywords and dealing with how to pass user input to Oracle's text indexing functions. In particular the CONTAINS operator.

// from oracle web site.
package foo;

import java.util.Vector;

/**
 * Escapes search phrases so they can be safely passed to Oracle's SQL
 * CONTAINS operator.
 * 

 * WARNING: it appears that this class is not thread safe and that
 * it maintains state between calls and thus a new object should be created
 * between each use rather than reusing the same object.
 */
public class QueryTranslator {

  // XXX make thread safe and change so that reqWords and notWords get cleared
  // out.  If that's done this we don't have to create a new instance of this
  // object each time we want to translate something.

  private Vector reqWords = new Vector();

  private Vector notWords = new Vector();

  public String translate(String input) {
    if (input.indexOf('{') > -1 || input.indexOf('}') > -1) {
      throw new IllegalArgumentException("'}' and '{' should not appear in input");
    }
    processString(input);
    if (this.reqWords.size() == 0) {
      throw new IllegalArgumentException("no 'required' words in query: " + input);
    }
    String translatedQuery = getQuery();
    if (translatedQuery.indexOf("()") > -1 
        || translatedQuery.indexOf("{}") > -1
        || translatedQuery.indexOf("\\}") > -1) {
      throw new IllegalArgumentException("can't construct a valid oracle text query from: " + input);
    }
    return translatedQuery;
  }

  private void addWord(final String word, final boolean isRequired) {

    if (isRequired) {
      this.reqWords.add(word);
    } else {
      this.notWords.add(word);
    }
  }

  public void processString(final String input) {
    int p = 0;
    int startWord;
    String theWord;

    this.reqWords = new Vector();
    this.notWords = new Vector();

    while (true) { // Loop over all words

      startWord = p;
      while (p < input.length() && input.charAt(p) != ' ') {
        // Check for quoted phrase
        if (input.charAt(p) == '"') { // Quote - skip to next or end
          p++; // skip the actual quote
          while (p < input.length() && input.charAt(p) != '"') {
            p++;
          }
          if (p < input.length()) {
            p++; // Skip the final quote if found
          }
        } else {
          p++;
        }
      }

      // Got a word. Check for required/not wanted flags (+-)

      theWord = input.substring(startWord, p);

      // CY bug 11825, don't process zero length string
      if (theWord.length() > 0) {
        // CY changed this to required from optional to make it AND
        // logic
        boolean isRequired = true;

        if (theWord.charAt(0) == '+' && theWord.length() > 1) {
          isRequired = true;
          theWord = theWord.substring(1);
        }

        else if (theWord.charAt(0) == '-' && theWord.length() > 1) {
          isRequired = false;
          theWord = theWord.substring(1);
        }

        // Replace * wild cards with %

        theWord = theWord.replace('*', '%');

        if (!"%".equals(theWord)) {
          addWord(theWord, isRequired);
        }

      }
      p++;
      if (p >= input.length()) {
        break;
      }
    }
  }

  // Get word gets a single word from the "words" vector,
  // surrounds it in braces (to avoid reserved words)
  // and attaches a WITHIN clause if appropriate.

  private String getWord(final Vector words, final int pos) {
    // here I added stuff for handling the wildcard, which doesn't work if
    // in {}
    String word = words.elementAt(pos);
    if (word.indexOf('%') > -1) {
      word =  word.replaceAll("[\\W&&[^%]]", "");
      if ("%".equals(word)) {
        return "";
      }
      return word;
    }
    if (word.lastIndexOf('\\') == word.length() - 1) {
      word = word.substring(0, word.length() - 1);
    }
    return "${".concat( word) + '}';
  }

  // getQuery returns a formatted, ready-to-run ConText query.
  // In order to satisfy the altavista syntax, we have to generate
  // the following query:

  // ( req1 & req2 & ... reqN)
  // | ( (req1 & req2 & .. reqN)*10*10
  // & (req1, req2 , ... reqN , opt1 , opt2 , ... optN) )
  // NOT (not1 | not2 | ... notN)

  public String getQuery() {
    StringBuffer sb = new StringBuffer();
    // String tempString = "";

    String boolOp = ""; // AND, OR, NOT operator
    int reqCount; // Count of required words
    int notCount; // Count of not wanted words
    int i; // Loop control

    boolOp = "";
    reqCount = this.reqWords.size();
    notCount = this.notWords.size();

    if (this.reqWords.size() > 0) {
      // Required words - first time

      sb.append("((");
      for (i = 0; i < reqCount; i++) {
        sb.append(boolOp).append(getWord(this.reqWords, i));
        boolOp = " & ";
      }
    }

    if (reqCount > 0) {
      sb.append(")) ");
    }

    if (notCount > 0) {
      boolOp = " NOT ";
    } else {
      boolOp = "";
    }

    for (i = 0; i < notCount; i++) {
      sb.append(boolOp).append(getWord(this.notWords, i));
      boolOp = " NOT ";
    }
    return sb.toString();
  }

  public static void main(String args[]) {

    if (args.length != 1) {
      System.out.println("java " + QueryTranslator.class.getName()
          + " search_phrase");
      System.exit(1);
    }

    System.out.println("Orginal Phrase:    " + args[0]);
    System.out.println("Translated Phrase: "
        + new QueryTranslator().translate(args[0]));
  }
}

Aggregation of Social Content

This is an interesting article on aggregation of others' content.

http://www.web-strategist.com/blog/2009/03/24/breakdown-twitter-federated-media-and-microsofts-sponsored-aggregation/

RequestDispatcher

I've been running into some strange behavior in legacy code dealing with how RequestDispatcher objects are gotten when the incoming URL has invalid characters in the query string portion of the URL. This page didn't help much, but it was interesting reading.

http://www.roseindia.net/javacertification/wcd-guide/machanism.shtml

My problem seems to be caused by crawlers my parsing our pages and not properly unescaping XHTML entities such as & that they encounter in links or image URLs.

Apache Benchmarking with Compression

We belatedly noticing that we didn't turn gzip compression on for our Apache web server. After turning it on, we ran the Apache benchmarking tool and didn't notice any improvement. To get the ab tool to accept compression you need to have it pass the correct header as follows:

ab -n 100 -H "Accept-Encoding: gzip, deflate" http://......

It was quite simple. On our local network we didn't see any improvement since everything is fast. From home and on smart phones, you do see a difference.

Sorting Alphabetically in Java

I've recently had to fix alphabetical sorting in two different Java-based web applications and know we have the problem in other places. It's a pain that this comes up all of the time and there's really nothing to prevent it in the future except programmer diligence. The basic problem is that Strings in Java are sorted lexicographically by default (usually using String's compareTo() method). This is almost never what you want shown in your UI, but when you are writing code it's easy to just call Collections.sort() and expect it to do what you want.
Once you know you have this problem, there's a few ways to do the sorting:

• Do a case-insensitive sort on the Strings. This is very common and is basically a left-over pattern for handling this issue that shouldn't be done. You see this in Java code and JDBC/SQL code where a toUpperCase() or toLowerCase() function is called. A nuance of this is that it's technically random whether "de Lucca" or "De Lucca" comes first since when sorting they both look like "de lucca".
• Use one of Java's language-aware sorting methods such as Arrays.sort(array, Collator.getInstance()). This puts lower case letters before upper case, but on a per-letter basis so that capital Z is not before lower case 'a'. See the sorting examples below.
• Use a SQL Function to properly sort if your database supports it. For example, Oracle's NLSSORT() method. Note that you can't rely on SQL's order by clause by itself because it will also sort text values by their ascii code. And to make matters worse...some of our code did use the NLSSORT() function, but it put the results into a Java TreeSet which resorted them by their ascii code :(
• Consistently use the natural ordering of Strings and get your users to start thinking in terms of ascii and unicode.

Here's some simple code to show the differences between the above techniques. The output is in the table below.

import java.text.*;
import java.util.*;

public class coll {
  public static void main(String args[]) {

    String [] a1 = { "def", "daf", "de la Sol", "De la Sol", "de Battista", "DeBatista", "de Lucca", "deBattista", "Deloitte", "Das", "Deep" };

    String [] a2 = { "def", "daf", "de la Sol", "De la Sol", "de Battista", "DeBatista", "de Lucca", "deBattista", "Deloitte", "Das", "Deep" };

    String [] a3 = { "def", "daf", "de la Sol", "De la Sol", "de Battista", "DeBatista", "de Lucca", "deBattista", "Deloitte", "Das", "Deep" };

    Arrays.sort(a1);
    Arrays.sort(a2, Collator.getInstance());
    Arrays.sort(a3, new Comparator<String>() {
          public int compare(String s1, String s2) {
            if (s1 == null) {
              if (s2 == null) return 0;
              return 1;
            } else if (s2 == null) {
              return -1;
            }
            return s1.toUpperCase().compareTo(s2.toUpperCase());
          } 
        });

    for (int i=0; i < a1.length; i++) {
      System.out.printf("%-15s %-15s %-15s\n", a1[i], a2[i], a3[i]);
    }
  }
}

Here's the example output:

Natural Order (by ASCII code)	using a Collator	Case-insensitive Ordering
Das	daf	daf
De la Sol	Das	Das
DeBatista	DeBatista	de Battista
Deep	deBattista	de la Sol
Deloitte	de Battista	De la Sol
daf	Deep	de Lucca
de Battista	def	DeBatista
de Lucca	de la Sol	deBattista
de la Sol	De la Sol	Deep
deBattista	Deloitte	def
def	de Lucca	Deloitte