Diagnosing Maven Dependency Problems

Background:

Maven provides dependency management.  You specify the immediate dependencies of each project, and Maven works out the transitive dependencies.

A transitive dependency list includes indirect dependencies.  E.g. if A depends on B and B depends on C, A's transitive dependency list is B and C.

Problem:

After regular maintenance, the Maven projects do not satisfy all dependencies at runtime..

In my case, Maven failed to make available org.apache.commons.io.FileUtils of the commons-io package.  E.g. 

 mvn -pl :cloud-client-ui jetty:run  
 ...  
 INFO [ConfigurationServerImpl] (Timer-2:null) SSL keystore located at /root/github/cshv3/client/target/generated-webapp/WEB-INF/classes/cloud.keystore  
 Exception in thread "Timer-2" java.lang.NoClassDefFoundError: org/apache/commons/io/FileUtils  
     at com.cloud.server.ConfigurationServerImpl.getBase64Keystore(ConfigurationServerImpl.java:453)  
 ...  
 Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.FileUtils  
     at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)  
 ...  

However, commons-io is a known dependency, which can be demonstrated by using the mvn dependency:tree command.  E.g.

 root@mgmtserver:~/github/cshv3/client# mvn dependency:tree  
 [INFO] Scanning for projects...  
 [INFO]  
 [INFO] ------------------------------------------------------------------------  
 [INFO] Building Apache CloudStack Client UI 4.2.0-SNAPSHOT  
 [INFO] ------------------------------------------------------------------------  
 [INFO]  
 [INFO] --- maven-dependency-plugin:2.5.1:tree (default-cli) @ cloud-client-ui ---  
 [INFO] org.apache.cloudstack:cloud-client-ui:war:4.2.0-SNAPSHOT  
 ...  
 [INFO] +- commons-io:commons-io:jar:1.4:provided  
 ...  

Solution:

Inspect the effective POM to see how POM inheritance has modified the dependency specification.

The command mvn help:effective-pom will return the pom.xml as modified by properties inherited from it's parent and other pom.xml files in its inheritance hierarchy   Run the command in the same folder as the pom.xml you are interested.

E.g. for mvn -pl :cloud-client-ui jetty:run, first find the folder for the cloud-client-ui project.

 root@mgmtserver:~/github/cshv3# grep -R cloud-client-ui * --include=pom.xml  
 client/pom.xml: <artifactId>cloud-client-ui</artifactId>  

Then use mvn help:effective-pom to see how the dependency specification for commons-io differs from that of other files.

 root@mgmtserver:~/github/cshv3# cd client  
 root@mgmtserver:~/github/cshv3/client# mvn help:effective-pom  
 ...  
    <dependency>  
     <groupId>commons-configuration</groupId>  
     <artifactId>commons-configuration</artifactId>  
     <version>1.8</version>  
    </dependency>  
    <dependency>  
     <groupId>commons-io</groupId>  
     <artifactId>commons-io</artifactId>  
     <version>1.4</version>  
     <scope>provided</scope>  
    </dependency>  
 ...  

Notice that commons-io has the <scope> element, which changes the dependency scope.  Dependency scope tells Maven when it is responsible for satisfying a dependency.  A scope of provided tells Maven not to bother adding commons-io to the runtime environment.  Specifically, provided "indicates you expect the JDK or a container to provide the dependency at runtime."

Therefore, when you look at the list of libraries Maven makes available at runtime, commons-configuration is available, but commons-io is not.  E.g.

 root@mgmtserver:~/github/cshv3# ls -al client/target/cloud-client-ui-4.2.0-SNAPSHOT/WEB-INF/lib | grep commons  
 -rw-r--r-- 1 root root  168760 Jul 16 17:23 commons-beanutils-core-1.7.0.jar  
 -rw-r--r-- 1 root root  232771 Jul 16 17:23 commons-codec-1.6.jar  
 -rw-r--r-- 1 root root  571259 Jul 22 10:14 commons-collections-3.2.jar  
 -rw-r--r-- 1 root root  354491 Jul 16 17:23 commons-configuration-1.8.jar  
 -rw-r--r-- 1 root root  24242 Jul 16 20:06 commons-daemon-1.0.10.jar  
 -rw-r--r-- 1 root root  160519 Jul 16 17:23 commons-dbcp-1.4.jar  
 -rw-r--r-- 1 root root  53082 Jul 16 17:23 commons-fileupload-1.2.jar  
 -rw-r--r-- 1 root root  305001 Jul 16 17:24 commons-httpclient-3.1.jar  
 -rw-r--r-- 1 root root  284220 Jul 16 17:23 commons-lang-2.6.jar  
 -rw-r--r-- 1 root root  60686 Jul 16 17:23 commons-logging-1.1.1.jar  
 -rw-r--r-- 1 root root  111119 Jul 16 17:23 commons-pool-1.6.jar  
 -rw-r--r-- 1 root root  34407 Jul 16 20:08 ws-commons-util-1.0.2.jar  

Remove <scope>provided</scope> element from the inherited dependency, and the problem disappears.

Final Remarks:

If you run into a dependency problem, tell the Apache CloudStack mailing list.  Dependency problems are difficult to spot.  Pointing them out greatly helps the project keep the build in good shape.