jlink’ed Spring Boot, Packages vs Folders, King Kong

nipafx news #84–31st of January 2021

Hi everyone,

finally the newsletter is back to coding, so no yapping — I’ll get right to it.

I send this newsletter out some Sundays. Or other days. Sometimes not for weeks. But as an actual email. So, subscribe!

Creating a Spring Boot application image with jlink

I’ve recently started a small side project (a calendar view of the year) and picked Spring Boot (and Maven and React) for that. I would have loved to ship it as a self-contained application image, meaning a Java runtime image with just the needed JDK modules plus my dependencies and my code. Here’s how I failed in five different ways.

The intended way to create such an image is to have your code and your dependencies as modules (say in a folder mods) and then use jlink to create the image with a command like this:

jlink
	--module-path mods
	--add-modules dev.nipafx.calendar
	--output app

If you’ve ever used Spring Boot, you may immediately spot the first problem: By default, Spring Boot creates a fat JAR (i.e. a JAR with all your code and dependencies). That was only the first in a series of problems, though…

Proper module-info.java

My first try was to create my own module-info.java, but that gets moved into the fat JAR's root without further changes. That means it still lists the Spring modules as dependencies, but since they're now included in the JAR, they won't be found on any module path and thus jlink refuses to create the image.

There are hacky ways around that (creating empty JARs with the right module names), but I suspect this would lead to the same problems as the next approach.

Using Moditect on fat JAR

With Moditect, I can inject a module declaration into the fat JAR. That’s actually pretty easy because there are no external dependencies. That step worked, but the next one failed.

Unfortunately, jlink refuses to create the image because of the BOOT-INF folder that Spring Boot creates:

Error: java.lang.IllegalArgumentException:
	BOOT-INF.classes.dev.nipafx.calendar.spring:
	Invalid package name: 'BOOT-INF' is not a Java identifier

(I expect the approach above would fail at this step as well even if the problem with the module descriptor could be solved.)

moditect/moditect

Using Moditect on all JARs

I considered configuring Spring Boot to not create a fat JAR. My idea was to then use Moditect to create a module descriptor for all (transitive) dependencies of the app and thus be able to use jlink on all of them. This may work, but I would have to configure the module declarations for all dependencies. That’s not only a lot of work, it also strikes me as very unmaintainable given that dependencies may change frequently.

That said, we do have a proof of concept that uses this approach. Ihor Herasymenko watched me struggle with this on stream (I did all the experiments there) and decided to go all in and implement this with Gradle. The build tool switch was motivated by Ihor wanting to use a specific plugin, but I’m not sure whether that ended up in his currently proposed solution. Either way, it works! :) But build.gradle contains 300+ lines of module declarations, so I wouldn't use this in practice.

Using Moditect on shaded JAR

Andy Wilkinson (of Spring Boot fame) recommended to not create the default fat JAR but instead a shaded one because then there won’t be a BOOT-INF folder. One would assume a shaded JAR has some disadvantages (otherwise, why would Spring Boot go out of their way to do something else), but I was willing to give it a go.

Support application images with jlink · Issue #24889 · spring-projects/spring-boot

Initially, I didn’t get this to work because the BOOT-INF folder was still in there, but when retracing my steps just now I couldn't reproduce that. I think when first trying this out I may have misconfigured the build. Anyway, even with the correct configuration that now produces a perfectly valid shaded JAR (runs successfully with java -jar), jlink still refuses to do its job:

Error: java.lang.IllegalArgumentException:
	META-INF.versions.9.org.apache.logging.log4j.util:
	Invalid package name: 'META-INF' is not a Java identifier

What the actual fuck, shouldn’t jlink be able to handle the god-damn META-INF folder?! (Can you tell that I'm frustrated?) This can't be right, I'm sure there's another mistake hiding somewhere, but I won't be digging deeper into that tonight (I'm writing this on Saturday evening).

Creating an app pseudo-image

Gunnar Morling, the creator of Moditect, had an interesting proposal: Why not dump the entire idea of including app code as modules in the image and instead start with a custom runtime image (i.e. an image that contains only JDK modules, preferably just the ones needed for the app) and then just dump the JAR(s) in there. Create a simple script (the launcher created by jlink isn’t anything else anyway) and ship it!

In theory, you can use jdeps to determine which JDK modules an app depends on, but my initial efforts were once again thwarted and at that point I was quick to give up. Instead I just used trial and error and the command line to put things together and indeed, this works:

# create runtime image
jlink
	--output app
	--add-modules java.desktop,java.naming
# copy app into image
mkdir app/jars
cp target/$FAT_JAR app/jar/app.jar
# launch
app/bin/java -cp app/jar/app.jar $MAIN_CLASS

Gunnar mentioned that he’d like Moditect to be able automate that and we decided that it’s time to get to know his tool a bit better and then give it a shot to implement this feature. So we’ll do that on February 16th, 1800 UTC — maybe I’ll see you there.

Schedule For Streams, Talks, Courses - nipafx

Java packages aren’t folders

It pops up over at the live stream every now and then and so I thought I should just write it down:

Java packages aren’t folders!

We may have gotten used to thinking of packages as folders (I surely have) and IDEs are on board with that fiction (they create packages and source files accordingly and allow displaying packages as folders — some by default), but they’re not! For the compiler, packages are just namespaces used to apply visibility and to avoid name collisions — it couldn’t care less about folders.

In case you don’t believe me, try it out for yourself:

• pick a random project of yours
• make sure you have no local changes
• copy all source files into the root folder (or any other) — here’s how to do that on Linux in one fell swoop:

find src/main/java/ -name '*.java' -exec mv -t src/main/java {} +

• run your build tool — successfully
• revert (unless you like it that way ;) )

So, packages aren’t folders and, very importantly, there’s no such thing as a “subpackage”. Packages don’t contain one another, there’s no hierarchy — that’s a folder thing. And while I really don’t like it for navigational purposes, a flat package view as Eclipse uses by default (or used to when I still used it) is actually a more accurate representation of packages than the hierarchical one.

Consistency is king — simplicity is King Kong

The other day, Lukas Eder (the SQL/JOOQ guy) asked on Twitter whether… and then I… and then Steven… Ah, screw it, read for yourself (I slightly edited and rearranged my tweets for your convenience).

Lukas:

SQL is probably the hardest word to use in camelCase / PascalCase scenarios.

SQLType or SqlType?
MySQLType or MySqlType?
sQLVar or sqlVar?
mySQLVar or mySqlVar?

Me:

At some point I decided to treat all abbreviations as words, so I’d write:

SqlType
MySqlType
sqlVar
mySqlVar

IIRC I got this from .NET, although (more IIRC), they only recommend it for 3+ letters, so it would be CDPlayer, but I just go with CdPlayer to keep it simple. It’s a bit weird at first, but the consistency changes that quickly (at least it did for me).

Steven Dick:

Pick the rule that works for you (or your team) and stick to it. Consistency is more important that the specific rule chosen. I follow the same rule as you

Me:

Consistency is king, but simplicity is King Kong: Consistently following complex rules is very tough, so if in doubt pick the simpler rule just because applying it consistently is much easier.

I sometimes have a hard time accepting non-ideal structures or outcomes (where ideal means how I like it best), but over time I’m getting better at letting that slide and preferring simplicity.

In some cases, this code style or naming convention or design pattern or cutlery arrangement leads to really weird results? That’s unfortunate and my common reaction was to expand the rules to account for that, to make the result better. But if these are rules that everybody in the team should follow, then expanding them, making them more complex, runs the risk of making the overall outcome worse.

Quick maths with totally made up numbers:

• simple rule that covers 90% of cases and people follow it correctly 95% of the time ~> 90% * 95% = ~85% of cases are ok
• complex rule that covers 99% of cases and people follow it correctly 80% of the time ~> 99% * 80* = ~79% of cases are ok

So if — in good Pareto fashion — 20% of the rules get you 80% of the results, you have to add unproportionally many rules to theoretically (!) improve your outcome. Theoretically because adding so many rules may make it way harder for people to implement them correctly in practice.

In other words, consistently applying good rules may lead to better results than erratically applying great rules (consistency is king) and such consistency is more likely if the rules are simple (simplicity is King Kong).

(Btw, I didn’t come up with the X is king, Y is King Kong thing — I’m not nearly clever enough for that. I first saw it in Åsa Liljegren’s blog post 4 years of constant mob programming — shout out to her!)

4 years of constant mob programming

Project showcase — rerunner-jupiter

Over at JUnit Pioneer, we have an extension that retries failing test runs a few times, marking the test as failed if all runs fail and as passed otherwise. Hilarious, right, who would use such an abomination? You’d be surprised! Not only are there users, there’s an entire project focused on just that with lots more features: rerunner-jupiter (not surprisingly for JUnit Jupiter, commonly referred to as JUnit 5).

With rerunner-jupiter, you can configure…

• the number of repetitions
• the exceptions upon which to retry
• a minimum number of successes (e.g. 4 out of 10)
• a delay between retries

That’s pretty neat and if you’re in the unfortunate situation to feel like you could use this extension, why not give in to the darkness and try it out?

artsok/rerunner-jupiter

Shots

Here’s what I published recently:

• Sorting A React App Into Java’s Folder Structure (post)
• JDK News #1 — Pattern Matching Quartet (video)

And here are a few random but pretty cool things that others put out there:

• performance of new method Stream::toList
• boot performance of jlink’ed+AppCDS’ed images
• who profits from the GameStop trades?
• the tragedy of droids in Star Wars
• 25-minute fresh file speedrun of Hades

so long … Nicolai

PS: Don’t forget to subscribe or recommend! :)

Image by Lothar Dieterich from Pixabay