Ant’s IO System

I have long been intending to write a little tutorial on Ant’s IO system but I’ve been putting it off in favour of other tasks. I decided to write it up an a blog entry for now …

Many people probably wonder what I mean by Ant’s IO system and might even be surprised to find that Ant has an IO system. For the most part it works in the background and you don’t need to be aware of it. It can become necessary to understand Ant’s IO system when you try to use Ant tasks outside of the main Ant environment.

Why an IO System?
The need for an IO system arises from three Ant capabilities – the ability to run non Ant-aware tasks through a task adapter, the ability to run <java> tasks within the Ant VM and the ability to run tasks in parallel.

A task that is designed for Ant generates output by using the task’s log method to log messages. These messages appear with the familar [java] style adornment typical of Ant’s output. In older versions of Ant, a task which instead wrote to System.out would write directly to the console without the adornment. It was sometimes difficult to know which task generated such output and it wasn’t easy to read when the rest of the build output was indented and labeled.

In most cases, a task that generated direct output was considered to be badly behaved but two of the cases I identified above are clear exceptions. A task which is not aware of Ant and which is executed using Ant’s TaskAdapter mechanism has no access to the Ant log system. Similarly a java class executed with the <java> task without forking does not know about the Ant log system and in most cases expects to write to System.out and System.err without issue. In early versions of Ant, the output of the <java> task would change if the task was forked, with the output from the forked version being logged through the log system as it was captured from the external process.

The Ant IO system, therefore, is designed to associate any output sent to System.out and System.err with the task that generated the output and to log the output accordingly. That requirement is relatively simple but it becomes a little more complicated due to Ant’s ability to run multiple tasks concurrently using the <parallel> task container.

Output Demultiplexing
The first part of the IO system is setup by Ant’s command line environment . Ant’s Main class installs its own ouput streams into System.out and System.err. These streams, which are instances of DemuxOutputStreams, demultiplex the single global output stream into a stream per thread. The thread output is accumulated within the DemuxOutputStream. When an end-of-line is seen, the buffer is flushed or the buffer is closed, the accumulated output is forwarded to the Ant Project object. The forwarding occurs on the thread with which the output was associated.

Project’s job is to figure out the task that is currently active on each thread. For most threads that’s not too hard. As each task starts, it fires a build event and Project records the task which is currently associated with that thread. Project also provides a method to allow a task to be registered against a new thread. For arbitrary Java code, however, threads can be created without any knowledge that Ant is involved. To associate a task with any new threads, Ant uses ThreadGroups. ThreadGroups allow you to navigate the Thread parent-child hierarchy and Project uses this to navigate up the hierarchy until it comes to a thread for which it knows the current task. When Ant’s parallel task is creating new threads it adds them to a new ThreadGroup.

Output Handling
If Project can’t find a task associated with the thread on which output is generated, the output is logged at the project level. System.out is logged at INFO level and System.err is logged at WARN level. If a task can be determined, the output is forwarded to the task to be processed by the task.

Most tasks don’t need to do anything specific with their output and so the default implementation in the Task class is to log the output as a message with the appropriate level. A few tasks do need, however, to do some additional processing. The <ant> and <antcall> tasks need to forward the output onto the sub-project instance currently executing to allow it to forward it onto one of its tasks. The other tasks that do special processing are those which can capture output. For example, the <java> task can capture output to be stored into a file or an Ant property. In this case the output is passed to a Redirector instance to allow the output to be routed appropriately. If Ant did not have its IO system, the ability to capture output into a file or property would only work when the java task was forked.

Input
For a long time, Ant did not do anything about input in Java tasks. It simply was not connected and any attempt to read would hang the VM. Not too flash. Now a similar system to that used for output has been put in place with a DemuxInputStream passing any input requests onto the current task. By default the Task base class reads from whatever input is configured for the project, which defaults to System.in. For unattended builds, you can use the -noinput flag whereupon any attempt to read input in Ant will cause an end-of-file result to be given to the read.

Caveats
When you run an Ant task, expecially the Java task, outside of the normal Ant command line environment, none of the IO processing described above will happen. Any output generated by a task or java invocation will just go through to System.out. If you are running Ant tasks in this way it is up to you to decide how to handle this. Don’t blindly use Ant’s Main class to run Ant in an embedded environment. Main assumes it is running from a command line and that it owns the VM. Changing System.out, System.err and System.in is not something you should do in other environments. If you do so, you may affect the operation of the system in which you are running. You may also find a lot of additional output appearing in your Ant build output.