Freitag, 27. September 2013

Lightweight release builds with Apache Maven

This article is a follow-up of http://diplingfh.blogspot.ch/2013/02/how-has-our-release-process-matured.html as I have enhancements to the version.override feature I presented in this article the first time.

For quick readers here a short howto:
  • Download maven-core-extensions-0.3.2.jar and store it under $M2_HOME/lib/ext
  • Configure your root pom.xml according to [1]
  • execute mvn clean install -Dversion.override
See https://github.com/rotscher/emerging for more information.

Some more explanation


It's not the main purpose to replace the maven-release-plugin. But still, for doing continuous delivery during development phase it's not convenient:

We need a build with a unique version
And I don't mean unique SNAPSHOTs as this generated version might be different for each module inside a multimodule project. And there is no 100% safety that I really reference the latest SNAPSHOT! And this timestamp-buildId construct is ok for machines but not for humans (my tester answers my "which-version-is-deployed-question" with 2.2.3-201309261313-1).

When the build- and deployment process is started a unique version is generated and used throughout the toolchain (E.g. as build parameter in Jenkins, or input parameter to a deployment tool, or mentioned in a status report).

We don't want the pom version updated and committed daily
The developers don't like that all pom files are updated and they have to build from scratch to get new SNAPSHOT artifacts into the local maven repository.

we don't need a tag daily
The unique version mentioned above (and the revision from version control, too) is stored where ever it makes sense:
  • display-name of web.xml (this is visible in Tomcat manager)
  • somewhere in the (web) application and then visible for all users
  • in a database table
  • in all MANIFEST.MF
With all that information it's easy to track down problems during the development (when done in short cycles ;-). And it's even possible to reproduce a build if necessary at all.

The build must fail fast
How many times did your release process failed because you still had configuration errors? And how good do you feel about your release when you finally fixed all that errors? I mean, you just released something different than what you tested.
This is mainly the problems of SNAPSHOT versions. With the version.override feature you detect such configuration errors very fast!

The release plugin is not handy
In the last phase of a development iteration it should be possible to deliver as fast as possible. The release plugin builds three times and commits twice: this takes time and the chance of failures is high.
How many times did you fixed the scm connection configuration in your root pom?
And ever tried to execute the release plugin in an automated way? E.g. automatically triggered at 9pm?

Final thoughts
To me, SNAPSHOT versions are ok on local developer machines but it's subobtimal when SNAPSHOTs are in a remote repository, or even on a Jenkins with shared local repository (but having private repositories is not always possible due to infrastructure limitations).
This approach here is as easy as executing normal SNAPSHOT builds (also on local developer machines!), is not changing anything in the version control (and no risk of failures with the version control), but is behaving like release:perform. And it still leaves the choice to use all options in the development process:

mvn clean install (after each commit during the day)
mvn clean install -Dversion.override (once per day)
mvn release:prepare release:perform (when day x arrives  ;-)


[1] version.override custom install plugin configuration
<profiles>
    <profile>
        <id>version.override</id>
        <activation>
            <property>
                <name>version.override</name>
            </property>
        </activation>
        <build>
            <pluginManagement>
                <plugins>
                    <!-- deactivate the default install plugin -->
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-install-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>default-install</id>
                                <phase>none</phase>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </pluginManagement>
            <plugins>
                <!-- configure the custom install plugin for lightweight release builds -->
                <plugin>
                    <groupId>ch.rotscher.maven.plugins</groupId>
                    <artifactId>install-custom-version-plugin</artifactId>
                    <version>0.3.1</version>
                    <executions>
                        <execution>
                            <id>custom-install</id>
                            <phase>install</phase>
                            <goals>
                                <goal>install</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

Mittwoch, 20. Februar 2013

Maven - From POM to HOM...

...and other ideas for a future maven releases


Bored by all the Ant-vs.-Maven-vs.-Gradle articles (sorry for all build tools not mentioned here), I'd like to give some ideas how Maven could be improved.

Sometimes, I swear about Maven when I don't get to where I actually wanted to go to. Or, I don't like the idea to run mvn clean install and it's deploying something to a database which means it's not at all doing for what Maven is actually designed for. But as I know what Maven can and what it can't and knowing that no build tool is perfect I don't want to change it.

So, here is a list of ideas which can be summarized as follows: ideas in paragraph 1 and 2 optimize the current implementation, paragraph 3 are ideas extending Maven for supporting continuous delivery.
Before I really start:
  • I assume that projects follow the existing maven conventions, e.g. having a hiearchical folder structure.
  • There exist no proof of concept if any of this ideas can be implemented.

1 - More convention over configuration...

... which would reduce the lines of configuration

 

no explicit parent declaration

the parent is one directory above, why do I have to declare that on five lines?

no artifactId declaration

Convention for the artifactId could be the folder name, recursivly concatenated with the parent folders (e.g. parent-child, parentA-parentB-child).

no version declaration for internal dependencies

dependencies which are modules of the reactor are internal dependencies, and normally have the same version. In other words: the ${project.verison} should be declared once in the root pom of the project.

reduce the amount of pom.xml

not all modules must have a pom.xml, especially intermediate folders being itself parent of submodules dont' always need a pom.xml. The file should be optional, taking groupId and version from the parent, artifactId is based on the foldername.

2 - Make the POM more human readable...

...the current one is ok for machines

 

Why not have attributes in the POM? Instead of configuring
    <dependency>
        <groupId>commons-logging</groupId>
        <artifactId>commons-logging</artifactId>
        <version>1.1.1</version>
        <optional>true</optional>
    </dependency>


the dependency could be configured with attributes*
    <dependency groupId="commons-logging" artifactId="commons-logging" 
                version="1.1.1" optional="true" />

*in a normal editor this would be a one liner

I would leave the current (stable) POM, but humans should configure the modules in a hom.xml, human (friendly) object model, which is tranformed to a POM at runtime. This could be easily done in a generic manner whith XSLT: all attributes are transformed to child elements.

3 - New features...

...we're not just building software, it also gets deployed!

 

version.override

It should be possible to override the version when a build is started
(e.g. mvn deploy -Dversion.override=1.2.3)
This would help creating "releases" in a lightweight manner without having any snapshots and is helpful when doing continuous delivery where a unique version is generated outside of maven and is reused for later deployments to different stages.

more lifecycles

apart from the existing build lifecycle which creates jars, ears and so on there should be more follow-up lifecycles
  • dist lifecycle, for creating a distribution
  • stage lifecycle, for deployment (not 'mvn deploy') to an environment (app server, database etc)
mvn clean install dist stage would (locally) build all modules, create a dist and deploy the dist to a (local) app server

Of course, the dist lifecycle can be integrated in the existing build lifecycle (normally with the assembly plugin) but it should already be better integrated at all. Like with the ear type: Normally, the convention is enough, no or only little additional configuration is necessary. Compared to the assembly descriptor files. Why not have a packaging type like zip/tar (and rpm, deb, exe etc.)?

That's it...

...for now


Next thing for me would be to play around with the one (hom.xml) or other idea (mvn stage) to see what is feasible.