Dec 17 2010

AppBundler and a Christmas Surprise

Christmas Snow Simulator

For the past few months I’ve been working on an open source UI toolkit and scenegraph called Amino. While everything I’ve shown so far is traditional 2D desktop features, I have always planned to have 3D support as well. Since Christmas is coming up I thought it would be fun to build a simple snowflake simulator with true 3D graphics.


If you check out the source to Amino you’ll find this simulator under src/org/joshy/gfx/test/threedee/Xmas.java . The code is pretty simple, actually. It creates a bunch of particle objects which are subclasses of TransformNode, the base for all 3D transforms. When particles are created they get initialized with a position, falling velocity, angular speed, and snowflake image. Then I just drop them into a Group inside the scene and turn the crank on every repaint. Pretty standard stuff, but also quite easy to do with Amino since it handles all of the actual OpenGL code for you.

The simulator is fun, and it’s a good test for the 3D scenegraph, but the real magic of this demo isn’t what it does, but how it’s deployed. I built it with a new tool called AppBundler, which can generate a Windows EXE, a Mac .app bundle, a WebStart JNLP, and an executable jar all from a single codebase.

Java Deployment Problems

One of the long time problems with desktop Java has been deployment. Double click jars are feature limited. Applets crash all the time. JNLPs sometimes work once and then get their caches mixed of date. And not only has Java app deployment typically been unreliable, but the available install mechanisms always result in an app that feels different than native apps.

When I started working on Leonardo and Amino I knew this would be a problem, so I decided to fix this once and for all; or at least as much as possible with a stock JRE and specific use cases. The result of my work is AppBundler, now in it’s first public alpha release.

Introducing AppBundler

AppBundler is open source (BSD & LGPL, we use jSmooth for the Windows support). At its core, it’s pretty simple. AppBundler processes a small XML descriptor file into a variety of executable formats. Give it the name of your app and a list of support jars, then AppBundler can produce a native Windows app and a native Mac app, as well as JNLPs and executable jars. There are plenty of tools which do parts of this already. What makes AppBundler different is reliability and ease of use.

When I first started designing AppBundler I began with the following requirements, in priority order:

  1. Reliablity: Every output executable must work 100% of the time. It doesn’t matter how many features or how efficient your deployment system is if it fails 5% of time. Only 100% will do.
  2. Ease of use: This really is a simple task so the tool should be simple too. AppBundler assumes standard conventions, auto-detects likely problems and automates common tasks. Also, we should have only one tool to target all outputs.
  3. Native Features: What’s the point of making a desktop app if you don’t take advantage of local features. That means AppBundler must make things that look and feel as native as possible: by default. Throw a coupe of icon images at it. It’ll figure out the right one to use.

How to use AppBundler

First, create an XML file describing your app. It must have the name, a list of jars, and indicate which is the main jar.

<?xml version="1.0" encoding="UTF-8"?>
<app name="XmasSim">
    <jar name="amino-core.jar" main-class="org.joshy.gfx.test.threedee.Xmas"/>
    <jar name="XMLLib.jar"/>
    <jar name="commons-codec-1.4.jar"/>
    <jar name="apache-mime4j-0.6.jar"/>
    <jar name="commons-logging-1.1.1.jar"/>
    <jar name="httpclient-4.0.1.jar"/>
    <jar name="httpcore-4.0.1.jar"/>
    <jar name="httpcore-nio-4.0.1.jar"/>
    <jar name="httpmime-4.0.1.jar"/>
    <jar name="parboiled-"/>
    <!-- jogl -->
    <native name="jogl"/>

Now call AppBundler from the command line or your ant build script. You must pass --file= for the XML file, then --jardir= for each directory where your jars are stored. Finally use --target= for the target platform. You can use mac, win, jnlp, onejar, or all. Then AppBundler will do the right thing. Here’s an example:

<target name="build-xmassim" depends="build-core">
        classname="com.joshondesign.appbundler.Bundler" fork="true">
        <arg value="--file=xmassim-bundler.xml"/>
        <arg value="--target=onejar"/>
        <arg value="--outdir=dist/"/>
        <arg value="--jardir=build/jars/"/>
        <arg value="--jardir=lib/"/>

What about Native Libraries

Native Libraries have always been extra tricky. You can’t simply set a java.library.path variable from within your application because the VM has already set up the paths. Instead you must pass it from outside, but executable jars don’t let you set VM properties. Then you have to deal with the multitude of platform specific files. To solve these problems I had to be a bit tricky. Here’s how it works.

First, your native libs must be stored in a specific structure already. In one of your jar dirs you must have a directory with the name of your lib. In the example above this is ‘jogl’. Inside of this dir should be all of your jar files and a directory called ‘native’. Inside native is a directory for each platform: mac, win, and linux. Inside those are the native libraries for each platform. Once your libs are in this structure AppBundler can process them correctly for each platform.

Currently Windows and JNLPs aren’t supported with native libs, but support will be coming soon. Mac .app bundles have the native libs stored inside the bundle in a known location rather than installing them in some system location. This way one app can never collide with another. (while less efficient this is far more reliable). Windows EXE files simply reference the native libs in a ‘lib’ directory stored next to the EXE.

Finally the executable jars. This are a special case where I had to be extra tricky. AppBundler builds the final jar by importing the classes from all support jars into a single file, then it adds all native jars for all platforms. Finally it creates a little stub launcher which will decompress the native libs to a temp directory, then invoke the real app. This still won’t work, however, because java.library.path isn’t set correctly.

To handle this last challenge the stub will actually fork a second java process to run the real app, giving us a chance to set up the environment as we need. Is it a hack? Absolutely. But does it work? Heck yeah!. Hacks are fine in this case because you would never ship an app like this anyway. Executable jars are used when you want to quickly and easily send a single file app to someone else to test. AppBundler is now by far the easiest way to do that.

What AppBundler doesn’t do.

To achieve my goals for AppBundler I had to give up on a few things. First: don’t support applets at all. The problems with applets can only be addressed by the JRE itself, which is beyond the scope of what I can do. I’ll leave that to the talented engineers at Oracle.

Next I assume apps have full access to the system. There is a place for sandboxed apps, but those are moving towards HTML 5. By focusing on apps that will be downloaded and run from the desktop we can ensure tighter integration with the system.

Finally, use hacks if necessary. This is deployment code. We don’t care if it’s pretty. We just care that it works. As long as the hacks are hidden then we can fix’em later.

Next Steps

This is just the first alpha release of AppBundler. I need your help to test it and add more features. In the future I’d like to support:

  • Pack200 and more zip compression
  • More error handling: Tell if your main class really has a main method. Auto detect the main class in a jar. Trace all classes used and figure out if something is missing.
  • Build a real Ant task
  • simplify the commandline usage by assuming more defaults. It should be possible to do: java -jar appbundler.jar org.joshy.app.Start to find all jars in the lib, build, and dist directories, the produce output for all platforms.
  • Detailed documentation
  • Linux support

And most especially, Linux support. I don’t really know what you guys want. Spit out a .DEB or .RPM file? Shell scripts? Auto-inject into a repo somewhere? I’m willing to build whatever makes the most sense for linux end users.

If you are interested, please join the Amino dev list and start asking questions. If you want to play around with AppBundler, you can check out the sources from the google code project.


Skip to comment form

  1. tbee

    Hey Josh,

    I wrote a replacement for onejar myself, because of all kinds of issues in setting things up (for example passing a -javaagent parameter to the JVM), because onejar works with the classloader trick and not with a second VM. I’ve got it setup and working now, complete with a Maven plugin to build a appjar file from.

    But apparently there is a way to get a second JVM with onejar? Did I miss that?

  2. Rob Juurlink

    AppBundler looks very promising, I will give it a try!

    Some questions though. Is a zipped app really enough native for Mac? Shouldn’t Mac OS X apps be deployed in dmg format?

    What about an automatic update mechanism? Not sure about it, but should that also be the responsibility of the part that starts the application?

  3. admin

    Auto updates are a part of AppBundler because that really felt like a separate problem. I have some ideas for it, but it’ll have to wait for a big.

    Yes, a zipped .app is fine for Mac. That’s the advice I’ve gotten from an Apple person.

    For the second JVM I detect the location of ‘java’ then invoke it again with the correct environment. How did you set java.library.path with a classloader trick?

  4. Walter Laan

    The trick I know from JDIC uses reflection to have the (Sun/Oracle) ClassLoader check the system property again.

    See https://jdic.dev.java.net/source/browse/jdic/trunk/src/jdic/src/share/classes/org/jdesktop/jdic/init/JdicManager.java?view=markup

  5. Rossi

    This is great,
    I will have a look at appbuilder when time permits.
    It is so amazing. You create so many cool things faster than I can look at it.

    For me as a (K)Ubuntu user a deb package would be the preferred way for Linux.

    I think what onejar does is install an own class loader and overwrite the Classloader.findLibrary(…) method.
    We do the same trick to load the correct native libraries from our appserver on demand for the client platform.

    Have fun,
    - Rossi

Comments have been disabled.