I would need to configure the maven-failsafe-plugin:

     <configuration>
            <trimStackTrace>false</trimStackTrace>
            <systemPropertyVariables>
                <root.route>${project.artifactId}-${project.version}</root.route>
                <!-- this is where the driver-binary-downloader-maven-plugin puts them -->
                <webdriver.chrome.driver>webdriver/${os.name}/googlechrome/64bit/chromedriver</webdriver.chrome.driver>
                <!-- Similarly for other browsers -->
            </systemPropertyVariables>
        </configuration>

This is in order to make it work in Linux, OSX, Windows.
The problem, though, is that the downloader uses its very own names, such as
"linux" in small letters when "Linux" is the ${os.name}.

I would love to see this corrected so that one pom.xml is good for all three supported operating systems.

I am very confused:
selenium-standalone-server-plugin
is the same as
driver-binary-downloader-maven-plugin
?

yes it's the same plugin, I decide the name wasn't very accurate before I published it to maven, but I never got around to changing the repo name (something I must do).

The plugin stores the location of the binary that it downloaded in a maven variable that you can use to configure your maven-failsafe-plugin properties, so your configuration should look like this:

<configuration>
    <trimStackTrace>false</trimStackTrace>
    <systemPropertyVariables>
        <root.route>${project.artifactId}-${project.version}</root.route>
        <!--Set properties passed in by the driver binary downloader-->
        <webdriver.chrome.driver>${webdriver.chrome.driver}</webdriver.chrome.driver>
        <webdriver.gecko.driver>${webdriver.gecko.driver}</webdriver.gecko.driver>
        <!-- etc... -->
    </systemPropertyVariables>
</configuration>

This maven property that it sets will be for driver binary for the current OS

The code is here:

https://github.com/Ardesco/selenium-standalone-server-plugin/blob/master/src/main/java/com/lazerycode/selenium/SeleniumServerMojo.java#L273

Originally I did try setting environmental variables, but Maven starts new JVM's for each phase so you can't automatically share the the current list of env variables between them and it didn't just work unfortunately.

I see. Now, I have a project with JUnit4 here where this setting works and one with JUnit5 where it doesn't. Is this a known incompatibility? The problem seems to be with the failsafe plugin not propagating the <webdriver.chrome.driver>path</webdriver.chrome.driver> in its systemPropertyVariables at all.

It's not a known incompatibility, and I'm surprised to be honest. If the maven-failsafe-plugin is failing to create system properties passed into the JVM it should fail no matter what version of jUnit you are using, and it should work whatever libraries you are using.

Are you using the maven-failsafe-plugin for both your jUnit4 and jUnit5 projects, or are you using the maven-surefire-plugin for one of them and consequently setting the system properties in the wrong phase?