Printing directly to a network printer by Frank Worsley

Recently I had to print from a mobile Java application that was being developed for both BlackBerry and Android. Printing from Java SE on the desktop is quite easy by using either the Java Print Service API or the older java.awt.print API for printing Swing/AWT components. Unfortunately neither of these APIs is available in the mobile Java environments for BlackBerry or Android.

Instead I figured it should be possible to create the print data on my own and send it directly to a network printer. It turns out this was a bit trickier than I expected, so I decided to write this blog post about it. At the end of this post I also include some useful links to other reference material.

Getting Started

The first question I had was which port on the network printer to use. Port 9100 is the standard port for the "RAW" print protocol. This port appears to be used by all network printers. Many printers also support the Internet Printing Protocol on port 631. Some printers even support the LPD printer protocol on port 515. The LPD protocol is quite old and is documented in RFC 1179

I was a bit confused initially since in my research port 9100 was often referred to as the port for the "RAW" print "protocol". It turns out there is no "protocol" here, it is indeed simply the raw print data that can be sent directly to this port and the printer will interpret it for printing. Therefore port 9100 is most likely what you want to use, unless you need some of the features provided by the other protocols. I decided to print directly to port 9100.

The next question is, what "raw" print data should be sent to the printer? Printers expect print data in a Page Description Language (PDL). There are two common PDL languages in use today. One is PostScript from Adobe and the other is the Printer Command Language (PCL) from HP. There are also some other manufacturer specific languages such as UFRII from Canon, but they only work with printers from that manufacturer.

PostScript has a lot of history and there are several Java libraries available for generating PostScript code. I was under the assumption that all network laser printers today support PostScript, so I started with that. The problem was that sending PostScript to our office printer resulted in it simply printing the PostScript source as plain text. Not good!

This led me to discover a third printer language that is in common use today: Printer Job Language (PJL) which was also developed by HP. PJL sits above languages like PostScript and PCL. It can be used to set printer parameters and then switch the printer into the desired page description language (usually either PostScript or PCL).

After discovering PJL I modified my prototype to first send some PJL commands to switch the printer into PostScript mode. Unfortunately this still did not work on any of the printers I tried. They either printed nothing at all or the PostScript code as plain text. After more research I discovered that in fact PostScript is not commonly supported. You usually have to purchase a printer add-on for PostScript support. So it turns out that in the end there is no way around using PCL. This is where the fun really started since PCL is quite cryptic and there are no existing PCL generation libraries for Java that I could find.

Next I am going to explain how to send a simple PJL/PCL print job directly to a network printer on port 9100.

PJL Commands

The first thing you need to send to the printer are some PJL commands. 

All PJL commands start with an escape sequence: 

<ESC>%–12345X

Here <ESC> is the code for the ESC key (decimal 27, hex 1B) and the others are regular ASCII characters.

This escape sequence has to be included at the start of all PJL command sequences. After the escape sequence each PJL command appears as plain text and is terminated by CR-LF characters. Here are some basic PJL commands to switch the printer into PCL mode:

<ESC>%–12345X@PJL JOB NAME = "My Print Job Name" <CR><LF>
@PJL ENTER LANGUAGE = PCL <CR><LF>

Notice that there is no newline between the escape sequence and the first PJL command. This is intentional. Newlines only appear after the first PJL command. The PCL print data is sent directly after the ENTER LANGUAGE command.

After sending the PCL print data the printer has to be switched back to PJL mode so that we can end the print job. This is done by again sending the PJL escape sequence followed by the end of job command:

<ESC>%–12345X @PJL EOF <CR><LF>
<ESC>%–12345X

Notice that the final thing we send is the PJL escape sequence one more time, without anything after it (no newlines). Therefore a complete PJL command sequence would look like this:

<ESC>%–12345X@PJL JOB NAME = "My Print Job Name" <CR><LF>
@PJL ENTER LANGUAGE = PCL <CR><LF>
<PCL COMMANDS HERE>
<ESC>%–12345X @PJL EOF <CR><LF>
<ESC>%–12345X

That was pretty easy. PJL basically forms an envelope around PCL (or PostScript). There are a few other PJL commands, most importantly the SET command that is used to configure various printer options. This is not needed unless you want to switch the page size, change the number of copies, etc. More information can be found in the HP PJL Technical Reference Manual.

PCL Commands

The trickier part is sending the print data in the form of PCL commands. All PCL commands start with the <ESC> character followed by the command identifier. Note that PCL commands are not terminated by newline characters. Any text that follows a PCL command is simply printed as text on the page, unless it is preceded by the <ESC> character and recognized as another PCL command. Any newlines within the PCL commands are interpreted for printing.

The first PCL command you have to know is the reset command: 

<ESC>E

In general this is the first command you send to reset the printer and start a new page. For example, to start a new page and print "Hello World" at the top of the page you would send this:

<ESC>EHello World

Notice that there is no space between the PCL command and the "Hello World" text. It's not needed!

Moving Around The Page

Now, what if we want to print "Hello World" somewhere else on the page? In this case you can use the PCL move commands:

<ESC>*p150X
<ESC>*p150Y
Hello World

This moves 150 units along the X and Y axes respectively. Notice that no newline characters appear between the commands. I write out the text using newlines for readability, but they don't appear in the actual commands sent to the printer.

Since both move commands use the *p command prefix we can actually combine them into a single command, like this:

<ESC>*p150x150Y

Notice that the "x" is in lowercase. When commands with the same command prefix are combined, then only the last command suffix is spelled in uppercase. In this case "Y" is last, so it is spelled in uppercase and the "x" in-between is in lowercase.

So far so good. Now you are probably wondering, what are the units that I am moving around with? This depends on the printer defaults, but we can explicitly specify it using some handy PCL commands:

<ESC>&u150D
<ESC>*t150R

These two commands set the dots-per-inch (DPI) resolution of text and graphics operations respectively. In this case I am setting the resolution to 150 DPI. That means when I move 150 units I am moving 1 inch on the page along each axis. Nice!

Let's See Some Colors

Wouldn't it be nice to print in color? This requires a couple of PCL commands to be set up correctly. First we need to tell the printer to use the RGB color space:

<ESC>*v6W

This selects the RGB color space with 8-bits per pixel. Next we need to specify the color we want to use:

<ESC>*v255a0b0C

Notice that there are three components to this command: red (a), green (b) and blue (c). In this case I am selecting the color red with no green or blue. Finally I need to store the selected color in a color palette and select it for use:

<ESC>*v0i0S

This command first assigns the color to palette index 0 (the i command) and then selects that palette (the S command).

Very nice. If I were to write some text or draw something it would now come out in red. You can even combine all of these commands into a single command string since they all use the *v prefix:

<ESC>*v6w255a0b0c0i0S

Note that the "W" command to select the RGB color space is really only needed once and could be sent at the beginning right after the reset command. If you are printing a large document it can significantly reduce the document size to not include extra commands when not needed. That's also why you combine commands into one string, so you don't have to include the <ESC> character all the time.

Let's Draw Some Graphics

Now that we can draw text and use colors, how about some graphics? Here are the basic commands to fill a box on the page:

<ESC>*c300a150b5P

This fills a box 300 dots (2 inches) wide along the horizontal axis and 150 dots (1 inch) high along the vertical axis. The "5P" command at the end selects the current color palette. So in our case the box would be filled in red.

Using these commands you can also draw box outlines. You would simply fill very narrow boxes for each side of the box outline that you are drawing. The narrow box you are filling in effect becomes a line and four lines make ... another box! Makes sense, right?

Putting It All Together

I've provided a quick overview of the most useful commands and how they all fit together. There are many more PCL commands to select font family, size, style, etc. and also draw boxes in different fill patterns. You can find a reference of all these commands in the HP PCL5 Technical Reference Manual

PCL includes a number of common fonts. If you want to use other font families you have to first upload them to the printer, which is something that I didn't need to do. It's also possible to upload bitmap images or TIFF images to the printer, again something that I didn't get into. All of that is explained in the reference manual.

Once I had enough experience with the basic PCL commands I put together a nice Java class similar to the "java.awt.Graphics" class. The constructor accepts an OutputStream as a parameter and the class will write all PCL commands to the output stream. It provides methods to start a new document, set the font/color, draw text, draw lines, etc.

    While it's pretty basic this class does everything I needed. I could draw text with different styles and also draw little line and bar graphs. I can't share the code for this here, but hopefully my post gives you enough ideas to implement it yourself.

    More Resources

    Here are a few more useful resources:

    A better way to detect Java Webstart by Frank Worsley

    So, just as I wrote that I wouldn't be posting much in future, I thought I would quickly post this little tidbit. There are various ways described online that you can use to detect if a client has Java WebStart installed. The problem is all of them require you to mix in VBScript for Internet Explorer and then use Javascript for the other browsers. There is a way you can support Internet Explorer and still use only Javascript. Check it out:

    function jwsInstalled() {
      // For Internet Explorer.
      if (navigator.userAgent.indexOf('MSIE') > -1) {
        try {
          var jws = new ActiveXObject('JavaWebStart.isInstalled');
          return true;
        } catch (e) {
          return false;
        }
      }
    
      // Firefox is happy with "x-java-jnlp-file". For Chrome and Safari
      // this does not work, instead I just check for "x-java-vm".
      // If they have a recent JVM installed, then they usually also have
      // Java WebStart installed.
      return navigator.mimeTypes &&
             navigator.mimeTypes.length &&
             (navigator.mimeTypes['application/x-java-jnlp-file'] != null ||
              navigator.mimeTypes['application/x-java-vm'] != null);
    }
    

    I use this to popup a little dialog and tell users to download a new JRE if Java WebStart is not installed. It works pretty well. And yes, it's 2009 and I'm still using Java WebStart ... although I do have a good reason for it. :-)

    Open a file in the default application using the Windows command line (without JDIC) by Frank Worsley

    Quite a few people have asked me about this in the past. If you have a file how can you open it in the default associated application without querying the registry or using some other Windows API? Or if you program in Java how can you do it without using JDIC?

    The easiest way to do this is using the "start" command. For example to open the file "readme.txt" in the default text editor you would do this:

    C:\>start readme.txt

    You can also use start to open folders or follow shortcuts:

    C:\>start "My Shortcut" // note that you don't need .lnk at the end

    This will open the target of the "My Shortcut" shortcut. If the shortcut points to a folder it will open a Windows Explorer window for it, if the shortcut points to a document it will open it in the default application and if the shortcut is for a program it will launch the program.

    The trick is that "start" isn't an executable. It is a built-in command of the Windows command line interpreter "cmd.exe". In Java (and other languages) if you try to create a process using the "start" command this will fail since there is no "start.exe" executable in the system.

    Instread you have to invoke "start" through the "cmd.exe" interpreter. This can be done using the /C flag:

    cmd /c "start readme.txt"

    This can be run successfully in Java using Runtime.exec() or a ProcessBuilder. Simply calling "start" directly would fail. Note that this limitation is also true for many other Windows commands. If something fails to invoke you should always try running it using "cmd /c".

    Backpacking with Diabetes by Frank Worsley

    A lot of people have asked me about my experiences backpacking in Asia being a Type 1 Diabetic. I thought I would write this blog entry about it since others might find it useful too.

    For background info, I traveled to Hong Kong, Vietnam, Cambodia and Thailand for seven and a half weeks in total. I've previously also traveled for six weeks in Europe, two weeks in Egypt, and many other one to two week business trips throughout Europe and North America. While some of my trips were quite long, they were not anything like the 6 months or multi-year trips that many of the other people you meet on the road are taking.

    Planning Ahead - The first thing I do before going on a trip is to carefully plan how much insulin and how many test strips I will need. I prefer to always bring all my supplies with me. Even if going for a few months this really isn't a problem. At room temperature insulin will keep for around two months and most of the time on the road you will find fridges.

    I always plan for many test strips since I find I test more often on the road. This is mostly because life isn't as "regulated" as it would be at home and I'm less confident in my insulin doses. For my Asia trip - even with generous planning - my test strips just lasted to the end and I went to the pharmacy the first day back in Canada to get more. Therefore I really don't think you can bring too many test strips.

    For insulin I plan ahead using my normal daily dosages as a baseline. In general I always use less insulin when traveling, so this way I will end up with a comfortable surplus of insulin at the end of the trip. I suspect I use less insulin when traveling since I am much more active. Hot climates seem to result in me using even less insulin, possibly because of the influence of the sun, because I eat less when it is hot, or simply because I'm even more active in a sunny climate. Note that the reduced insulin usage is only for pleasure trips. For business related trips I find I use more insulin, probably because I eat too much restaurant food, sit in too many meetings, get less exercise and am more stressed. Yes, business travel really isn't that good for you.

    I also always bring a good supply of sugar tablets (Dextrosol) and energy bars (Clif Bars). These work very well when low on sugar. I recommend Clif Bars over other brands since they keep very well in all climates. They don't freeze solid when it's cold and don't melt when it is hot. Also, they generally taste quite good. Dextrosol tables are always good to have, although they tend to get soggy in humid climates.

    Caring for your Supplies on the Road - One of my main concerns going to hot places was keeping my insulin cool. However, I've never had a problem with this. Almost all hostels or guest houses I have stayed at had either a shared fridge or a fridge in the room. Especially in Asia it is very common for the guest houses to have a mini-fridge in the room. That means I can easily leave my insulin in the fridge during the day. Basically the only time your insulin isn't refrigerated is while you are traveling from one place to another. This means that even for longer trips it shouldn't be a problem bringing your insulin, since it will last a long time when refrigerated.

    I have also never had a problem with theft of any kind while on the road. This may just be lucky, but in general I think leaving your medical supplies in a shared fridge is fine. I don't think anyone would have any interest in stealing that. Still, for safety I do always keep one vial of each insulin type and extra test strips in my backpack. When I need to refill my pen I use a vial from the backpack and then replace it with a new one from the refrigerated supply. This ensures that even if my main supplies get lost or stolen I still have enough to last for at least a week.

    Getting more Supplies on the Road - This is an interesting problem I've never had to deal with. In general I would assume this is only a problem in the developing world. In developed countries you should be able to get whatever you need, although the expenses will probably be out of your pocket. Even in the developing world the pharmacies in the bigger cities seem to be well stocked, although they might not have the exact brand/type you are used to. For example, in Thailand I checked in a pharmacy in Samui and they had blood glucose meters and test strips, but only for one type of meter. Samui is a smaller town, so I bet in Bangkok I could have gotten what I wanted.

    I try to save the sugar tables and energy bars I bring along for emergency situations and use local food instead. Coca Cola seems to be readily available in even remote places. Carrying around a can/bottle of coke to use in low sugar situations is a very easy solution. Buying fruit at the market is always possible, but it doesn't keep very well in a backpack. Finally each country usually has some sort of local snack that you can buy at the markets. For example in Asia one can always find various nut-based snack bars that are held together by caramelized sugar. These keep very well and work great for snacking or when low on sugar.

    What else to bring - Bringing a dry bag is a good idea when spending time on or near the water. It's great for carrying the insulin pen, test kit and food, for example when kayaking or boating.

    Buying one storage bag for all supplies is a good idea. I bought a nice first aid kit bag at MEC and use it for all my test strips, insulin, needles, sugar tablets, etc. Then I can just take the whole bag and put it in the fridge or put it in my luggage. It holds everything together and keeps it organized.

    Also, make sure you bring a letter from your doctor explaining that you need your medical supplies in carry on luggage. Since 9/11 I've always carried one with me but have so far only needed it once: in Hong Kong before flying back to Vancouver.

    Embedded Tomcat Class Loading Trickery by Frank Worsley

    Recently I've embedded Tomcat directly within an application. The idea is that web application extensions to the system can easily share the core of the platform and the root Spring application context. This has worked well up until yesterday when I ran into some weird class loading issues with Tomcat.

    Once issue was that if an application included a JAR file that is also included in the core platform, Tomcat would still load the class from the system classloader, instead of loading it from the WAR file using the Tomcat web app classloader.

    For example, this is a problem when using the Wicket web framework. Wicket will load a class for a page from inside one of the Wicket core classes using getClass().getClassLoader().loadClass(XYZ). However, since Wicket was loaded using the system class loader it cannot see the web app's classes and this will result in a ClassNotFoundException.

    Another problem was using CGLib. When CGLib tried to instantiate a class using ClassLoader.defineClass() it would result in a NoClassDefFoundError. Again, the problem here is that CGLib was loaded from the system class loader and cannot resolve classes from the web app's WAR file.

    It took a while to find out why this is happening. According to the Tomcat 5 Class Loader HOW-TO, it should always load classes from the web app itself before loading them from the system class loader, except for special case classes. Looking at the middle of the page however, it does say that the system class loader is used first. Unfortunately for the longest time I was looking at the Tomcat 4 Class Loader HOW-TO and on that page it still indicates that the web app class loader is always used first.

    It turns out the system class loader is in fact always used first. Specifically line 1267 of WebappClassLoader in the Tomcat 5.5.17 sources. However, if you use the normal Catalina startup script it resets the system class path to only include a minimal set of classes, so in that case it would not find application specific classes using the system class loader. Therefore running Tomcat normally it would always end up using the WebappClassLoader.

    To get the same behaviour when embedding Tomcat in your own application, you have to create a bootstrap class. You only include the bootstrap class in the class path when loading your application. The bootstrap class then creates a URLClassLoader to load in the rest of your application classes. For example:

    public class Bootstrap {
      public static void main(String args[]) {
        String root = args[0];
        try {
          List classpath = new ArrayList();
          classpath.add(new File(root + File.separator + "conf" + File.separator).toURL());
          addJarFileUrls(new File(root + File.separator + "libs"), classpath);
    
          ClassLoader cl = new URLClassLoader(classpath.toArray(new URL[0]));
    
          // Set the proper classloader for this thread.
          Thread.currentThread().setContextClassLoader(cl);
    
          // Use reflection to load a class to normally load the rest of the app.
          // Reflection will use the Thread's context class loader and therefore pick up
          // the rest of our libraries.
          Class appClass = cl.loadClass("com.zinepal.frank.Application");
          Object app = appClass.newInstance();
    
          Method m = app.getClass().getMethod("start", new Class[0]);
          m.invoke(app, new Object[0]);
        } catch (Exception ex) {
          ex.printStackTrace();
          System.exit(1);
        }
      }
    
      /**
       * Add JAR files found in the given directory to the list of URLs.
       * @param root The directory to recursively search for JAR files.
       * @param jarUrls The list to add URLs to.
       */
      private static void addJarFileUrls(File root, List jarUrls) throws MalformedURLException {
        File[] children = root.listFiles();
        if (children == null) {
          return;
        }
    
        for (int i = 0; i < children.length; i++) {
          File child = children[i];
          if (child.isDirectory() && child.canRead()) {
            addJarFileUrls(jarUrls, child);
          } else if (child.isFile() && child.canRead() && 
                     child.getName().toLowerCase().endsWith(".jar")) {
            jarUrls.add(child.toURL());
          }
        }
      }
    }
    

    And in a separate class file that is included in one of the JAR files which we dynamically add to the URLClassPath above:

    public class Application {
      public void start() {
        // Do whatever you want here.
        // Then initialize embedded Tomcat.
      }
    }
    

    After adding this bootstrap class to the application everything worked fine.