What are the advanced tools in Java

23.3 Programming with the Tools API

Tools like the Java compiler javac, but also jarsigner, javadoc, xjc, javap and other command line tools are written in Java only. Some tools provide an API for developers so that they can be extended and accessed by programs. This is interesting for tool providers who, for example, want to extend the compiler or adjust the output of the javadoc tool.

23.3.1 Java tools implemented in Java

All programs written in Java are located in an extra Java archive called tools.jar, which is located in the lib directory of the JDK. The JDK only delivers the class files, however, the sources with Javadoc have to be obtained from the Mercurial Repository [154] (for Java 8 http://hg.openjdk.java.net/jdk8/jdk8/langtools/ provides access They can also be downloaded in one go in different archive formats, including ZIP.), because they are not included in the usual src.zip from the JDK.

The normal executable programs like javadoc or javah (under Windows) all have the same size around 15 KiB, since they do nothing other than call the Java runtime environment with tools.jar in the class path and then forward it to the corresponding method of the program. Instead of calling javadoc, the goal is the same.

23.3.2 Address tools from your own Java programs

There are two options for addressing a tool directly from a Java program:

  • The public class provides an API to get access to the compiler and to call the javadoc tool, with which the Java documentation can be created or the Javadoc comments can be processed. The problem: only two tools are accessible, no other tools like javap, xjc, ...

  • Calls to the entry-level tool classes, for example. Since all tools have such a class, a direct call from a Java program is always possible, but not without problems, since developers make themselves dependent on the internal package structures here.

23.3.3 API documentation of the tools

The public class is part of Java SE and is therefore documented, as is access to the compiler and the javadoc tool. An example of translating Java sources is shown in Chapter 18, "Dynamic Translation and Scripting Languages."

The other JDK tools that developers can address from Java also have API documentation, which is not part of the Java SE standard documentation, but is in the documentation ZIP under docs \ jdk \ api. All details can be read there, such as how the Java compiler is addressed or how programs that build linked HTML documents, so-called doclets, from the Java API documentation, for example, are written. Oracle does not define a public API for every tool. For example, it is not intended to intervene in the creation process of javah, and Oracle also documents this property in the sources as follows:

“This is NOT part of any supported API. If you write code that depends on this, you do so at your own risk. This code and its internal interfaces are subject to change or deletion without notice. "

23.3.4 Own doclets

The javadoc tool has the task of reading out the Java source code and extracting the Javadoc comments. What then happens to the data is the job of a doclet. The standard doclet from Oracle generates the familiar structure on linked HTML files, but you can also write your own doclets, for example to check links contained in Javadoc comments for accessibility or to generate UML diagrams from the type relationships.

The Doclet API is not part of the Java standard library and is not included in the class path. To write your own doclets, tools.jar must first be added to the class path. Part of the Java archive is the package that declares types and methods with which visited packages, classes, methods etc. and their Javadoc texts can be queried.

In the following example we want to write a small doclet that outputs classes, methods and constructors that carry the tag (or what is actually wrong). This makes it easy to determine what has been added in version Java 8. Doclets are usually called from the command line and passed to the javadoc tool. Our program simplifies this by calling the tool directly via Java with the appropriate parameters. tools.jar must be in the class path and the documentation unpacked at the specified location.

Listing 23.2 com / tutego / tools / javadoc / SinceJava8FinderDoclet.java

package com.tutego.tools.javadoc;

import java.util. *;
import java.util.function.Predicate;
import com.sun.javadoc. *;
import com.sun.tools.javadoc.Main;

public class SinceJava8FinderDoclet {

public static boolean start (RootDoc ​​root) {
for (ClassDoc clazz: root.classes ())
processClass (clazz);
return true;

private static void processClass (ClassDoc clazz) {
Predicate isJava18 = tag -> tag.text (). Matches ("(1 \.)? 8");

if (Arrays.stream (clazz.tags ("since")) .anyMatch (isJava18))
System.out.printf ("New type% s% n", clazz);

for (MethodDoc method: clazz.methods ())
if (Arrays.stream (method.tags ("since")) .anyMatch (isJava18))
System.out.printf ("New method% s% n", method);

for (ConstructorDoc constructor: clazz.constructors ())
if (Arrays.stream (constructor.tags ("since")) .anyMatch (isJava18))
System.out.printf ("New constructor% s% n", constructor);

for (FieldDoc field: clazz.fields ())
if (Arrays.stream (field.tags ("since")) .anyMatch (isJava18))
System.out.printf ("New attribute% s% n", field);

public static void main (String [] args) {
String [] params = {
"-quiet", "-XDignore.symbol.file",
"-doclet", SinceJava8FinderDoclet.class.getName (),
"-sourcepath", "C: / Program Files / Java / jdk1.8.0 / src /",
// "java.lang", // java.lang only
"-subpackages", "java: javax" // Everything recursively under java. * and javax. *
Main.execute (params);

Our method calls the JDK doclet program via and passes its own doclet class via parameters - the arguments of remind of the command line parameters. The main doclet program in turn calls our method - the same thing would also happen if the doclet were called from outside via javadoc. Ours runs over all types identified and hands over the responsibility for processing the innards. The metadata come via various types. One pulls out the tag test, another use of the new Java 8 streams would not make the program clearer.

The program uses a few tricks to focus the output on the essentials. The switch switches off the usual charging status, which leads to outputs such as

Loading source files for package java.lang ...
Loading source files for package java.applet ...
Loading source files for package java.awt ...


The switch, in turn, suppresses messages like this one:

C: \… \ src \ java \ lang \ Class.java: 57: warning: Unsafe is internal proprietary API and may ¿
be removed in a future release
import sun.misc.Unsafe;

The messages land on the channel so that they can also be sent in an output-discarding stream in order to suppress them.

23.3.5Access the compiler AST of a class

Access to the Java compiler is standardized via the Java compiler API, but all internals, such as the actual representation of the program code, are hidden. The compiler API abstracts everything via interfaces, and so developers only come into contact with and - all interfaces from the package. To go a little deeper, there is a trick: classes implement interfaces, and when a program adapts the interface type to the specific class type, there are usually more methods available. So it can be cast on one, and then a method is available for. The method returns an enumeration of. This puts us directly in the internal representation that the compiler has built up from program code. It's called the Abstract Syntax Tree (AST). The visitor pattern can be used to walk this tree. provides a method to which we pass one with callback methods. The method then calls our visitor as each node expires:

Listing 23.3 com / tutego / tools / javac / PrintAllMethodNames.java

package com.tutego.tools.javac;

import java.io. *;
import java.net. *;
import javax.tools. *;
import javax.tools.JavaCompiler.CompilationTask;
import com.sun.source.tree. *;
import com.sun.source.util.TreeScanner;
import com.sun.tools.javac.api.JavacTaskImpl;

public class PrintAllMethodNames {

final static TreeScanner methodPrintingTreeVisitor = new TreeScanner () {
@Override public Void visitCompilationUnit (CompilationUnitTree unit, Void arg) {
System.out.println ("Package:" + unit.getPackageName ());
return super.visitCompilationUnit (unit, arg);
@Override public Void visitClass (ClassTree classTree, Void arg) {
System.out.println ("Class:" + classTree.getSimpleName ());
return super.visitClass (classTree, arg);
@Override public Void visitMethod (MethodTree methodTree, Void arg) {
System.out.println ("Method:" + methodTree.getName ());
return super.visitMethod (methodTree, arg);

public static void main (String [] args) throws IOException, URISyntaxException {
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler ();
StandardJavaFileManager fileManager = ¿
compiler.getStandardFileManager (null, null, null);
URI filename = ¿
PrintAllMethodNames.class.getResource ("PrintAllMethodNames.java") .toURI ();
Iterable fileObjects = fileManager.getJavaFileObjects (new File (filename));
CompilationTask task = compiler.getTask (null, null, null, null, null, fileObjects);

JavacTaskImpl javacTask = (JavacTaskImpl) task;

for (CompilationUnitTree tree: javacTask.parse ())
tree.accept (methodPrintingTreeVisitor, null);

One has many methods, we are only interested in starting a compilation unit for the package name, for all classes and methods. But we could also get information about all annotations or loops. The output is:

Package: com.tutego.tools.javac
Class: PrintAllMethodNames
Method: visitCompilationUnit
Method: visitClass
Method: visitMethod
Method: main

The second entry for the class name is empty because the anonymous class has no name.

How did you like the Openbook? We always look forward to your feedback. Please send us your feedback as an e-mail to [email protected]