Tomcat Mime-type Settings

I recently had a user complain that a 3rd party web application that we support was returning garbage when people try to download Microsoft Office 2007 document types in IE7 (it works fine in Firefox). This particular 3rd party application runs in an Apache Tomcat servlet container. I just updated the server's web.xml file to have the following mime-type mappings:

<mime-mapping>
  <extension>docm</extension>
  <mime-type>application/vnd.ms-word.document.macroEnabled.12</mime-type>
</mime-mapping>
<mime-mapping>
  <extension>docx</extension>
  <mime-type>application/vnd.openxmlformats-officedocument.wordprocessingml.document</mime-type>
</mime-mapping>
<mime-mapping>
  <extension>dotm</extension>
  <mime-type>application/vnd.ms-word.template.macroEnabled.12</mime-type>
</mime-mapping>
<mime-mapping>
  <extension>dotx</extension>
  <mime-type>application/vnd.openxmlformats-officedocument.wordprocessingml.template</mime-type>
</mime-mapping>
<mime-mapping>
  <extension>potm</extension>
  <mime-type>application/vnd.ms-powerpoint.template.macroEnabled.12</mime-type>
</mime-mapping>
<mime-mapping>
  <extension>potx</extension>
  <mime-type>application/vnd.openxmlformats-officedocument.presentationml.template</mime-type>
</mime-mapping>
<mime-mapping>
  <extension>ppam</extension>
  <mime-type>application/vnd.ms-powerpoint.addin.macroEnabled.12</mime-type>
</mime-mapping>
<mime-mapping>
  <extension>ppsm</extension>
  <mime-type>application/vnd.ms-powerpoint.slideshow.macroEnabled.12</mime-type>
</mime-mapping>
<mime-mapping>
  <extension>ppsx</extension>
  <mime-type>application/vnd.openxmlformats-officedocument.presentationml.slideshow</mime-type>
</mime-mapping>
<mime-mapping>
  <extension>pptm</extension>
  <mime-type>application/vnd.ms-powerpoint.presentation.macroEnabled.12</mime-type>
</mime-mapping>
<mime-mapping>
  <extension>pptx</extension>
  <mime-type>application/vnd.openxmlformats-officedocument.presentationml.presentation</mime-type>
</mime-mapping>
<mime-mapping>
  <extension>xlam</extension>
  <mime-type>application/vnd.ms-excel.addin.macroEnabled.12</mime-type>
</mime-mapping>
<mime-mapping>
  <extension>xlsb</extension>
  <mime-type>application/vnd.ms-excel.sheet.binary.macroEnabled.12</mime-type>
</mime-mapping>
<mime-mapping>
  <extension>xlsm</extension>
  <mime-type>application/vnd.ms-excel.sheet.macroEnabled.12</mime-type>
</mime-mapping>
<mime-mapping>
  <extension>xlsx</extension>
  <mime-type>application/vnd.openxmlformats-officedocument.spreadsheetml.sheet</mime-type>
</mime-mapping>
<mime-mapping>
  <extension>xltm</extension>
  <mime-type>application/vnd.ms-excel.template.macroEnabled.12</mime-type>
</mime-mapping>
<mime-mapping>
  <extension>xltx</extension>
  <mime-type>application/vnd.openxmlformats-officedocument.spreadsheetml.template</mime-type>
</mime-mapping>

Resending Email (.eml) filesh

I recently had to resend a bunch of old email files and I wanted to preserve the original sent date on them.

I couldn't find an add-on for Thunderbird to do this. I did see an add-on that allows you set a future date for an email, but not a past date. I also couldn't find any other simple program or Cygwin command to use. So I just did it with Java and it ended up being rather simple.

The key thing here was preserving the sent date. I tried setting the clock on my machine back in time, but that doesn't really work because I'd have to reset it to a different date and time for each email and that's just too much manual work. My internet connection also doesn't like it when my machine's time is way off from reality.

So my solution was to first, in Thunderbird (my email client), save all of the emails I wanted to resend as .eml files in a folder. Then I wrote this simple program using Java Mail's API to read in the .eml file and to send the email again preserving all of the fields, except the TO address, which I changed based on my needs.

Here's the program:

import java.io.*;
import java.util.*;

import javax.mail.*;
import javax.mail.internet.*;

public class Eml {

  public static void main(String args[]) throws Exception {

    String host = "smtp.foo.edu";
    String from = "phu@foo.edu";
    String to = "bar1@foo.edu, bar2@foo.edu";

    Properties props = System.getProperties();
    props.setProperty("mail.smtp.host", host);
    props.put("mail.transport.protocol", "smtp");

    Session mailSession = Session.getDefaultInstance(props, null);
    mailSession.setDebug(true);

    File emlFile = new File(args[0]);
    InputStream source = new FileInputStream(emlFile);

    MimeMessage message = new MimeMessage(mailSession, source);

    System.out.println("--------------");
    System.out.println("To: " + Arrays.asList(message.getAllRecipients()));
    System.out.println("From: " + message.getFrom()[0]);
    System.out.println("Subject: " + message.getSubject());
    System.out.println("Sent Date: " + message.getSentDate());
    System.out.println("Body: " + message.getContent());

    message.setRecipients(Message.RecipientType.TO, to);
    Transport.send(message);

    System.out.println("Message sent....");
    System.out.println("--------------");
  }
}

It takes one argument, the name of the file. I used a simple Cygwin for command to loop through the *.eml files and run this program for each one. I could have added that logic to this Java program, but it was just as quick and easy to do it this way.

There's no error handling here and I used wildcards in the import lines which I usually avoid doing, but this works. I ran with it Java 5.

PHP and Tomcat 5

I've been recently looking at integrating PHP with Tomcat. So far I've been unsuccessful at finding a solution that works on all of our development platforms, in particular, finding a php-cgi implementation for Mac OS X has been hard.

On Windows XP, PHP/Java Bridge has worked well. Here's a couple of useful links:

http://php-java-bridge.sourceforge.net/pjb/index.php

http://php-java-bridge.sourceforge.net/pjb/FAQ.html

JSP Best Practices

I'm thinking of giving a talk on JSP best practices. We recently had a couple developers give an excellent talk on CSS and best practices and many of the concepts have corollaries in the underlying JSP rendering of a page. So I thought I start to gather some thoughts in this blog posting:

1. Your JSP page should be valid XML


2. Avoid Scriptlets and JSP directives <% ... %>


3. Use ${fn:escapeXml(...)} frequently, even if you know the data is safe


4. Use <c:url> tag properly, including it's nested <c:param> tag so that URLs are properly encoded.


5. Do not put your URL parameters within the <c:url> tag, put them in nested <c:param> tags.


6. Use the proper and most recent standard JSTL taglibs. Globally replace all old JSTL taglib declarations so you don't have a mix of them floating around.


7. Don't store your own copy of the standard JSTL tld files, they will inevitably end up out of sync with the JAR file you are using.


8. Validate the rendered HTML content


9. Open your JSP page in an XML editor to validate it


10. Differences between <% include %>, <js:include>, and <c:import>


11. Page scope for <c:forEach> tag variables and how to pass them to included or imported pages


12. Parameterizing include/imported JSP fragments


13. Naming standards for JSP page and fragments


14. Using a main page template


15. Using value objects so JSP fragments can be reused


16. Precompile your JSPs

Oracle Text Indexing - escaping user input

Here's a class passed to me for escaping keywords and dealing with how to pass user input to Oracle's text indexing functions. In particular the CONTAINS operator.

// from oracle web site.
package foo;

import java.util.Vector;

/**
 * Escapes search phrases so they can be safely passed to Oracle's SQL
 * CONTAINS operator.
 * 

 * WARNING: it appears that this class is not thread safe and that
 * it maintains state between calls and thus a new object should be created
 * between each use rather than reusing the same object.
 */
public class QueryTranslator {

  // XXX make thread safe and change so that reqWords and notWords get cleared
  // out.  If that's done this we don't have to create a new instance of this
  // object each time we want to translate something.

  private Vector reqWords = new Vector();

  private Vector notWords = new Vector();

  public String translate(String input) {
    if (input.indexOf('{') > -1 || input.indexOf('}') > -1) {
      throw new IllegalArgumentException("'}' and '{' should not appear in input");
    }
    processString(input);
    if (this.reqWords.size() == 0) {
      throw new IllegalArgumentException("no 'required' words in query: " + input);
    }
    String translatedQuery = getQuery();
    if (translatedQuery.indexOf("()") > -1 
        || translatedQuery.indexOf("{}") > -1
        || translatedQuery.indexOf("\\}") > -1) {
      throw new IllegalArgumentException("can't construct a valid oracle text query from: " + input);
    }
    return translatedQuery;
  }

  private void addWord(final String word, final boolean isRequired) {

    if (isRequired) {
      this.reqWords.add(word);
    } else {
      this.notWords.add(word);
    }
  }

  public void processString(final String input) {
    int p = 0;
    int startWord;
    String theWord;

    this.reqWords = new Vector();
    this.notWords = new Vector();

    while (true) { // Loop over all words

      startWord = p;
      while (p < input.length() && input.charAt(p) != ' ') {
        // Check for quoted phrase
        if (input.charAt(p) == '"') { // Quote - skip to next or end
          p++; // skip the actual quote
          while (p < input.length() && input.charAt(p) != '"') {
            p++;
          }
          if (p < input.length()) {
            p++; // Skip the final quote if found
          }
        } else {
          p++;
        }
      }

      // Got a word. Check for required/not wanted flags (+-)

      theWord = input.substring(startWord, p);

      // CY bug 11825, don't process zero length string
      if (theWord.length() > 0) {
        // CY changed this to required from optional to make it AND
        // logic
        boolean isRequired = true;

        if (theWord.charAt(0) == '+' && theWord.length() > 1) {
          isRequired = true;
          theWord = theWord.substring(1);
        }

        else if (theWord.charAt(0) == '-' && theWord.length() > 1) {
          isRequired = false;
          theWord = theWord.substring(1);
        }

        // Replace * wild cards with %

        theWord = theWord.replace('*', '%');

        if (!"%".equals(theWord)) {
          addWord(theWord, isRequired);
        }

      }
      p++;
      if (p >= input.length()) {
        break;
      }
    }
  }

  // Get word gets a single word from the "words" vector,
  // surrounds it in braces (to avoid reserved words)
  // and attaches a WITHIN clause if appropriate.

  private String getWord(final Vector words, final int pos) {
    // here I added stuff for handling the wildcard, which doesn't work if
    // in {}
    String word = words.elementAt(pos);
    if (word.indexOf('%') > -1) {
      word =  word.replaceAll("[\\W&&[^%]]", "");
      if ("%".equals(word)) {
        return "";
      }
      return word;
    }
    if (word.lastIndexOf('\\') == word.length() - 1) {
      word = word.substring(0, word.length() - 1);
    }
    return "${".concat( word) + '}';
  }

  // getQuery returns a formatted, ready-to-run ConText query.
  // In order to satisfy the altavista syntax, we have to generate
  // the following query:

  // ( req1 & req2 & ... reqN)
  // | ( (req1 & req2 & .. reqN)*10*10
  // & (req1, req2 , ... reqN , opt1 , opt2 , ... optN) )
  // NOT (not1 | not2 | ... notN)

  public String getQuery() {
    StringBuffer sb = new StringBuffer();
    // String tempString = "";

    String boolOp = ""; // AND, OR, NOT operator
    int reqCount; // Count of required words
    int notCount; // Count of not wanted words
    int i; // Loop control

    boolOp = "";
    reqCount = this.reqWords.size();
    notCount = this.notWords.size();

    if (this.reqWords.size() > 0) {
      // Required words - first time

      sb.append("((");
      for (i = 0; i < reqCount; i++) {
        sb.append(boolOp).append(getWord(this.reqWords, i));
        boolOp = " & ";
      }
    }

    if (reqCount > 0) {
      sb.append(")) ");
    }

    if (notCount > 0) {
      boolOp = " NOT ";
    } else {
      boolOp = "";
    }

    for (i = 0; i < notCount; i++) {
      sb.append(boolOp).append(getWord(this.notWords, i));
      boolOp = " NOT ";
    }
    return sb.toString();
  }

  public static void main(String args[]) {

    if (args.length != 1) {
      System.out.println("java " + QueryTranslator.class.getName()
          + " search_phrase");
      System.exit(1);
    }

    System.out.println("Orginal Phrase:    " + args[0]);
    System.out.println("Translated Phrase: "
        + new QueryTranslator().translate(args[0]));
  }
}

Aggregation of Social Content

This is an interesting article on aggregation of others' content.

http://www.web-strategist.com/blog/2009/03/24/breakdown-twitter-federated-media-and-microsofts-sponsored-aggregation/

RequestDispatcher

I've been running into some strange behavior in legacy code dealing with how RequestDispatcher objects are gotten when the incoming URL has invalid characters in the query string portion of the URL. This page didn't help much, but it was interesting reading.

http://www.roseindia.net/javacertification/wcd-guide/machanism.shtml

My problem seems to be caused by crawlers my parsing our pages and not properly unescaping XHTML entities such as & that they encounter in links or image URLs.