GraalVM: Native command line tools for Linux and Windows written in Java

GraalVM: Native command line tools for Linux and Windows written in Java

Siegfried Steiner
Siegfried Steiner
10 min read

Categories

Tags

Thanks to the GraalVM, you may compile your Java command line tools1 to native executables, directly running on MS Windows or GNU Linux without any JVM being installed or bundled. As a side effect, the resulting *.exe and *.elf files start up lightening fast, which makes your commands feasible drop ins for FaaS applications2.

In the past I coded some command line tools in Java which are quite useful for me and which I ought to be useful for others3. I used various approaches to get those tools up and running on different machines, either requiring a JVM being installed already or by having a JVM bundled within the deliverable. Another approach is using the native-image tool provided by the GraalVM in order to compile a self contained native executable running stand alone as is. As of the relative slow startup times of JVM driven applications (e.g. launching your application by calling java -jar ...), the former approach was not optimal for command line tools executed using bash, CMD, PowerShell or the like. The latter approach harnessing the native-image tool now is very much suitable for command line tools as start up times (and therewith responsiveness) are lightening fast.

Deliverables

As said, there are some approaches I was playing around to get some more or less ease of use experience when providing my command line tool deliverables3. Here I will focus on the native approach, which means providing a fully self contained native executable, having all it requires in one file (for each targeted platform we need a dedicated executable). Besides this approach, there are few more approaches such launcher, bundle or installer deliverables4.

Toolchain

For building native images with the GraalVM, we need an up and running build toolchain for creating native images on MS Windows and on GNU Linux respectively2. Next, we require a fat JAR file dropping out of our build process, containing all the resources, third party libraries and dependencies your application requires to run when being invoked via java -jar .... This can be achieved in several ways: As I am using Maven for build automation, I decided to go with the Maven Shade Plugin - I wont’t use any heavy weight frameworks such as Spring Boot5 for my command line tools! As reflection is a native image’s natural enemy, we may need to gather some runtime information beforehand (for tweaking the native image configuration afterwards) by executing the fat JAR in a special way with the GraalVM. Finally we are ready to build our native executable application (e.g. *.exe or *.elf).

If not mentioned otherwise, shell scripts *.sh proposed to be invoked are intended to be issued in a bash alike shell on GNU Linux or within a GNU userland such as Cygwin or Git BASH on MS Windows. Plain commands may also be issued using CMD on MS Windows.

GraalVM & Maven

First of all, an according GraalVM needs to be installed, either by directly using the downloads from graalvm.org or by using SDKMAN!6. Make sure you have set the JAVA_HOME and the GRAALVM_HOME environment variables correctly and your GraalVM is on your system’s path7. You may verify your installation by invoking java --version in a terminal:

As we use Java with a version >= 16, make sure your GraalVM is chosen accordingly.

openjdk 17.0.5 2022-10-18
OpenJDK Runtime Environment GraalVM CE 22.3.0 (build 17.0.5+8-jvmci-22.3-b08)
OpenJDK 64-Bit Server VM GraalVM CE 22.3.0 (build 17.0.5+8-jvmci-22.3-b08, mixed mode, sharing)

The printed version information should contain the string “GraalVM” and a version >= 16 somewhere in the output.

Having your GraalVM in place, we need to add GraalVM’s native-image tooling issuing the following command in a terminal:

gu install native-image

Maven is expected to be installed already8, though when using SDKMAN!6, it is easily installed as of sdk install maven 3.8.6.

Compilers

In addition we need to install some platform specific compiler suites for GNU Linux and MS Windows accordingly. Setting up the compilers and stuff on GNU Linux is straight forward, on MS Windows some few more steps are required.

GNU Linux

On GNU Linux, install the common build tools such as GCC, glibc as well as zlib.

Debian

For Debian based distributions (such as Ubuntu):

sudo apt install build-essential libz-dev zlib1g-dev

Arch Linux

For Arch Linux (such as Manjaro Linux):

sudo pacman -Sy base-devel

RedHat

For RedHat based distributions (such as CentOS):

yum groupinstall 'Development Tools'

For other GNU Linux distributions, make sure that the according GCC toolchain is installed.

MS Windows

On MS Windows, you got to get hold of the Visual Studio Build Tools as well as of the Windows SDK. The easiest way (as of my opinion) is to install the Visual Studio Community Edition:

  1. Download Visual Studio Community Edition
  2. Start the Visual Studio Community Edition installer
  3. Choose category Desktop development with C++
  4. Make sure all the C/C++ related item’s are selected
  5. Make sure the Windows 10 SDK is selected
  6. Select only the English language pack, remove(!) other selected language packs!9
  7. Press Installand wait till done

From now on, for building native executables with Maven, use the x64 Native Tools Command Prompt (for Maven on the command line to see Visual Code C/C++ tools), you’ll find it in the MS Windows Start Menu (you may need to use the search function).

Archetypes

Below find a selection of Maven Archetypes of which we will use one in the next section to get our Maven project with some sample Java code up and running:

For a jump start into developing Java driven command line tools, I created some fully pre-configured Maven Archetypes available on Maven Central. Those Maven Archetypes already provide means to directly create native executables, bundles as well as launchers and support out of the box command line argument parsing as well as out of the box property file handling.

Please adjust my.corp with your actual Group-ID and myapp with your actual Artifact-ID:

refcodes-archetype-alt-c2

Use the refcodes-archetype-alt-c2 archetype to create a bare metal command & control (CLI) driven Java application:

“mvn archetype:generate [...]” 
mvn archetype:generate \
  -DarchetypeGroupId=org.refcodes \
  -DarchetypeArtifactId=refcodes-archetype-alt-c2 \
  -DarchetypeVersion=3.3.5 \
  -DgroupId=my.corp \
  -DartifactId=myapp \
  -Dversion=0.0.1

refcodes-archetype-alt-cli

Use the refcodes-archetype-alt-cli to create a bare metal command line interface (CLI) driven Java application:

“mvn archetype:generate [...]” 
mvn archetype:generate \
  -DarchetypeGroupId=org.refcodes \
  -DarchetypeArtifactId=refcodes-archetype-alt-cli \
  -DarchetypeVersion=3.3.5 \
  -DgroupId=my.corp \
  -DartifactId=myapp \
  -Dversion=0.0.1

refcodes-archetype-alt-csv

Use the refcodes-archetype-alt-csv archetype to create a bare metal CSV (CLI) driven Java application:

“mvn archetype:generate [...]” 
mvn archetype:generate \
  -DarchetypeGroupId=org.refcodes \
  -DarchetypeArtifactId=refcodes-archetype-alt-csv \
  -DarchetypeVersion=3.3.5 \
  -DgroupId=my.corp \
  -DartifactId=myapp \
  -Dversion=0.0.1

refcodes-archetype-alt-decoupling

Use the refcodes-archetype-alt-decoupling archetype to create a bare metal dependency injection and inversion of control (IoC) driven Java application breaking up dependencies between components or modules of a software system:

“mvn archetype:generate [...]” 
mvn archetype:generate \
  -DarchetypeGroupId=org.refcodes \
  -DarchetypeArtifactId=refcodes-archetype-alt-decoupling \
  -DarchetypeVersion=3.3.5 \
  -DgroupId=my.corp \
  -DartifactId=myapp \
  -Dversion=0.0.1

refcodes-archetype-alt-eventbus

Use the refcodes-archetype-alt-eventbus archetype to create a bare metal event driven driven Java service:

“mvn archetype:generate [...]” 
mvn archetype:generate \
  -DarchetypeGroupId=org.refcodes \
  -DarchetypeArtifactId=refcodes-archetype-alt-eventbus \
  -DarchetypeVersion=3.3.5 \
  -DgroupId=my.corp \
  -DartifactId=myapp \
  -Dversion=0.0.1

refcodes-archetype-alt-filter

Use the refcodes-archetype-alt-filter archetype to create a bare metal command line interfaceCLI) driven Pipes and Filters Java application (the pipes are provided by the shell, your application will be the filter to interact with other UNIX filters via pipes):

“mvn archetype:generate [...]” 
mvn archetype:generate \
  -DarchetypeGroupId=org.refcodes \
  -DarchetypeArtifactId=refcodes-archetype-alt-filter \
  -DarchetypeVersion=3.3.5 \
  -DgroupId=my.corp \
  -DartifactId=myapp \
  -Dversion=0.0.1

refcodes-archetype-alt-rest

Use the refcodes-archetype-alt-rest to create a bare metal REST driven Java application within just one source code file:

“mvn archetype:generate [...]” 
mvn archetype:generate \
  -DarchetypeGroupId=org.refcodes \
  -DarchetypeArtifactId=refcodes-archetype-alt-rest \
  -DarchetypeVersion=3.3.5 \
  -DgroupId=my.corp \
  -DartifactId=myapp \
  -Dversion=0.0.1

Project

Having set up the GraalVM alongside the build toolchain, we now create a Maven project for a Pipes and Filters Java application using the refcodes-archetype-alt-filter archetype:

Please adjust my.corp with your actual Group-ID and myapp with your actual Artifact-ID:

mvn archetype:generate \
  -DarchetypeGroupId=org.refcodes \
  -DarchetypeArtifactId=refcodes-archetype-alt-filter \
  -DarchetypeVersion=3.3.5 \
  -DgroupId=my.corp \
  -DartifactId=myapp \
  -Dversion=0.0.1

In the root folder of the newly created project you find the pom.xml alongside all required source code and configuration files as of the common Maven project layout.

You may have to invoke chmod +x *.sh on your project’s root folder to make the scripts executable!

Building

For building a native image, fire up a terminal and change cd into the root folder of your newly created project. Building is as easy as this:

Make sure that when using MS Windows, go for the x64 Native Tools Command Prompt!

mvn clean package -P native-image

The -P native-image argument tells Maven to use a profile called native-image found in the project’s pom.xml: Here the Native Maven Plugin is configured alongside the Maven Shade Plugin, resulting in a native executable found below the target folder (suffixed with either .exe on MS Windows or .elf on GNU Linux) after successfully building.

The resulting .exe and .elf files are fully self contained!

Launching

Launching is as easy as this:

Linux

On GNU Linux:

./target/myapp-native-x86_64-0.0.1.elf

Windows

On MS Windows:

target\myapp-native-x86_64-0.0.1.exe

Tweaking

In case you use additional dependencies in your project which bring their own JNI libraries or use reflection, then tweaking your project’s native image configuration may be required.

Building or running your native executable may fail due to some issues of missing resources, undetected reflective access or referencing types required by JNI: In such cases you will have to tweak your native configuration!

For gathering additional information when building the native image, you may dry-run your fat JAR in a terminal to get insights for your according native image’s configuration and required tweaks:

mvn clean package

When having built the fat JAR, we will gather runtime information for tweaking by running your JAR accordingly in a terminal:

java -agentlib:native-image-agent=config-output-dir=target -jar target/myapp-0.0.1.jar

This will create (amongst others) the following files below your target folder, after exiting or aborting (<CTRL>+C) the launched instance:

  • jni-config.json: Denotes required Java types provided by JNI mechanisms.
  • reflect-config.json: Denotes required Java types usually instantiated or modified at runtime by reflection.
  • resource-config.json: Denotes required native libraries being invoked by JNI, resources such as images or configurations files.

Given that we go for my.corp as Group-ID and myapp as Artifact-ID, the counterparts of those files actually used when building the project are found below the folder ./src/main/resources/META-INF/native-image/my.corp/myapp. In case building or running your native executable fails, you may use the information generated below the target folder and compare (diff) those files with the according files found below the folder ./src/main/resources/META-INF/native-image/my.corp/myapp.

The step of tweaking your project’s native image configuration may be the most time consuming one, not because it is complicated, furthermore because it involves tiresome repetitive steps of building your native image, tweaking your configuration, running your native image, tweaking your configuration, and so on …

JavaFX and Android

In addition to Java command line tools, you can also build native JavaFX powered applications. Sample applications include funcodes-pixgrid as well as funcodes-watchdog applications also found in the Downloads section of this site. A hybrid running on the desktop as well as on Android is described in the blog post Write one run anywhere: An Android game using JavaFX and the GraalVM. The funcodes-testbild app is sample on how to build native Android apps using JavaFX.

Further reading

The refcodes-archetype collection provides easy means do build launcher, bundle or installer deliverables in addition to native deliverables. See Java deliverables as executable bundle and the launcher trick for detailed insights on how building launcher, bundle or installer deliverables is achieved.

See also

  1. Java command line tools, aka public static void main( String[] args){ ... }. ↩

  2. On macOS the approach may be very similar to the one on GNU Linux, though I cannot try out as of lack of according Hardware. ↩ ↩2

  3. The command line tools can be found in the Downloads section of this site. ↩ ↩2

  4. For deliverables such as launcher, bundle or installer see Java deliverables as executable bundle and the launcher trick. ↩

  5. There are various Maven plugins for frameworks such as Spring Boot creating framework specific fat JAR files. ↩

  6. As I often switch forth and back between different JVM and GraalVM installations, I preferably use SDKMAN!(e.g. sdk install java 22.3.r17-grl, verify the available candidates via sdk list java). ↩ ↩2

  7. The basic setup of your GraalVM is described in this Quick Start Guide. ↩

  8. Maven may be either installed directly following the installation guide or by using SDKMAN! (e.g. sdk install maven 3.8.6, verify the available candidates via sdk list maven)6. ↩

  9. It seems that on Windows the GraalVM’s native-image tool “greps” Visual Studio’s compiler output text to identify compiler versions. ↩

LinkedIn Reddit