Merge pull request #14 in DEVEX/fsdevtools from DEVEX-98 to master

Squashed commit of the following:

commit 5e9f591
Author: Nicolai Henczi <henczi@e-spirit.com>
Date:   Fri Jun 2 15:54:00 2017 +0200

    DEVEX-98 fixed fs-cli.sh

commit b692d1d
Author: Hannes Pernpeintner <pernpeintner@e-spirit.com>
Date:   Wed May 31 09:09:04 2017 +0200

    DEVEX-98 fixed review issues

commit 047dfe9
Author: Hannes Pernpeintner <pernpeintner@e-spirit.com>
Date:   Mon May 29 10:27:06 2017 +0200

    DEVEX-98 changed developer documentation for unrestricted classpath scanning and custom artifact support, changed linux command line file

commit fc193c2
Author: Hannes Pernpeintner <pernpeintner@e-spirit.com>
Date:   Wed May 24 13:18:22 2017 +0200

    DEVEX-98 added unrestricted classpath scanning and example custom command module

documentation/DEV_DOC.md

@@ -1,7 +1,7 @@
 # Developer Documentation
 
 ## Dependencies
-The FS DevTools compiles only with **Java 8** and **FirstSpirit 5.2R3** (version 5.2.311) or newer. Since the required FirstSpirit artifacts are not public available the steps in the next section must be done.
+The FS DevTools compiles only with **Java 8** and **FirstSpirit 5.2R8** (version 5.2.802) or newer. Since the required FirstSpirit artifacts are not public available the steps in the next section must be done.
 
 ### Use FirstSpirit Access API as maven dependency
 
@@ -62,12 +62,24 @@ mvn clean package -Dci.version=VERSION
 ## Extending
 
 This tool should be easily expandable with further commands, while the *execution framework* should rarely needed to be touched at all. 
+You can either place your code right into this repository and compile it all together (hence have your own distribution), or you can build your own little software module and use fsdevtools as a dependency. Take a look at the chapters below for further details
 
-For your convenience, you can add commands and groups. Our cli assumes, that you place your commands in the existing command package ([com.espirit.moddev.cli.commands](https://github.com/e-Spirit/FSDevTools/tree/master/fsdevtools-cli/src/main/java/com/espirit/moddev/cli/commands)) and your new groups in the existing group package ([com.espirit.moddev.cli.groups](https://github.com/e-Spirit/FSDevTools/tree/master/fsdevtools-cli/src/main/java/com/espirit/moddev/cli/groups)) in the cli module. Since those packages are configured to be scanned, there's no need to further register commands or anything. 
-Since our tool relies on the [airline library](https://github.com/airlift/airline) here on github, you have to annotate your class with a `@Command` annotation and implement our [`Command`](https://github.com/e-Spirit/FSDevTools/blob/master/fsdevtools-cli-api/src/main/java/com/espirit/moddev/cli/api/command/Command.java) interface. 
+You can add commands and groups. For command and group implementations, arbitrary packages can be used. There's no need to further register commands or use special package names.
+Since our tool relies on the [airline library](https://github.com/airlift/airline) here on github, you have to annotate your command class with a `@Command` annotation and implement our [`Command`](https://github.com/e-Spirit/FSDevTools/blob/master/fsdevtools-cli-api/src/main/java/com/espirit/moddev/cli/api/command/Command.java) interface (or extend other command classes, as the [`SimpleCommand`](https://github.com/e-Spirit/FSDevTools/blob/master/fsdevtools-cli/src/main/java/com/espirit/moddev/cli/commands/SimpleCommand.java).
+Take a look at the fsdevtools-customcommand-example submodule, to get a short impression what you have to do. Keep in mind, that you have to provide a default command, if you add a custom group class implementation.
 
-By default, our commands use a connection to a FirstSpirit server. A global configuration for commands, as well as a context, is made available through the [`Config`](https://github.com/e-Spirit/FSDevTools/blob/master/fsdevtools-cli-api/src/main/java/com/espirit/moddev/cli/api/configuration/Config.java) interface. A general implementation is provided by our [`GlobalConfig`](https://github.com/e-Spirit/FSDevTools/blob/master/fsdevtools-cli/src/main/java/com/espirit/moddev/cli/configuration/GlobalConfig.java) class. If you implement a configuration, our execution environment uses the command itself for the connection configuration and initializes the connection for you right before the command execution. 
+By default, our commands use a connection to a FirstSpirit server. A global configuration for commands, as well as a context, is made available through the [`Config`](https://github.com/e-Spirit/FSDevTools/blob/master/fsdevtools-cli-api/src/main/java/com/espirit/moddev/cli/api/configuration/Config.java) interface. A general implementation is provided by our [`GlobalConfig`](https://github.com/e-Spirit/FSDevTools/blob/master/fsdevtools-cli/src/main/java/com/espirit/moddev/cli/configuration/GlobalConfig.java) class. If you implement a configuration, our execution environment uses the command itself for the connection configuration and initializes the connection for you right before the command execution.
 
 For your convenience, we provided the [`SimpleCommand`](https://github.com/e-Spirit/FSDevTools/blob/master/fsdevtools-cli/src/main/java/com/espirit/moddev/cli/commands/SimpleCommand.java) class that can be extended to prevent you from specifying standard connection logic for each command. The pure logic you want to program can then be placed in the generic `call` method you know from java's `Callable` interface and you are all done.
 
 For help configurations, take a look at existing commands and their annotations. If you really need it, you can have dynamic descriptions via a `public static String getDescription()` method in your command class (have a look at our [ExportCommand class](https://github.com/e-Spirit/FSDevTools/blob/master/fsdevtools-cli/src/main/java/com/espirit/moddev/cli/commands/export/ExportCommand.java)).
+
+### Build your own distribution
+You can clone this repository, extend the codebase and give us a pull request or maintain it as your own. The compilation and unit test stages can be executed with the above mentioned dependencies installed.
+
+### Use fsdevtools as a dependency in your own project
+With mvn install, you install the modules' artifacts of this repository to your maven repository, which enables using them as dependencies. After creating your own (for example maven) project,
+you can define dependencies to the api package (fsdevtools-api) as in the fsdevtools-customcommand-example module, or you can have a dependency on our implementations (fsdevtools-cli) - for example if you want
+to subclass our SimpleCommand class (Keep in mind, that implementation classes are less stable than api interfaces). Those libraries are part of the cli runtime, so they are defined with a *provided* scope.
+After packaging your library as a jar file, you have to place it in the *libs* folder of the cli distribution. All libraries in this folder are automatically added to the classpath, and hence your command implementations can be found
+by our classpath scanner.

fsdevtools-cli/pom.xml

@@ -96,6 +96,12 @@
             <scope>test</scope>
         </dependency>
 
+        <dependency>
+            <groupId>com.espirit.moddev.fsdevtools</groupId>
+            <artifactId>fsdevtools-customcommand-example</artifactId>
+            <version>${ci.version}</version>
+            <scope>test</scope>
+        </dependency>
     </dependencies>
 
     <build>

fsdevtools-cli/src/main/archive/fs-cli.cmd

@@ -2,5 +2,4 @@
 set java_cmd="%JAVA_HOME%/bin/java"
 set script_path=%~dp0
 set CLI_DIR=%script_path:~0,-4%
-set jarfile="%CLI_DIR%\lib\${project.artifactId}-${project.version}.jar"
-%java_cmd% -Dlog4j.configuration=file:"%CLI_DIR%conf/log4j.properties" -jar %jarfile% %*
+%java_cmd% -Dlog4j.configuration=file:"%CLI_DIR%conf/log4j.properties" -cp %CLI_DIR%\lib\* com.espirit.moddev.cli.Cli %*

fsdevtools-cli/src/main/archive/fs-cli.sh

@@ -1,7 +1,7 @@
 #!/bin/sh
 JAVACMD="$JAVA_HOME/bin/java";
 FS_CLI_DIR="$(dirname $(readlink -f $0))/../";
-JARFILE="${FS_CLI_DIR}lib/${project.artifactId}-${project.version}.jar";
-$JAVACMD -Dlog4j.configuration=file:"${FS_CLI_DIR}conf/log4j.properties" -jar $JARFILE "$@";
+$JAVACMD -Dlog4j.configuration=file:"${FS_CLI_DIR}conf/log4j.properties" -cp "${FS_CLI_DIR}/lib/*" com.espirit.moddev.cli.Cli "$@";
 RETVAL=$?;
 exit ${RETVAL};
+

fsdevtools-cli/src/main/java/com/espirit/moddev/cli/Cli.java

@@ -65,19 +65,9 @@
  */
 public final class Cli {
 
-    /**
-     * Default package for classes that define cli command groups.
-     */
-    public static final String DEFAULT_GROUP_PACKAGE_NAME = "com.espirit.moddev.cli.groups";
-
-    /**
-     * Default package for classes that define cli commands.
-     */
-    public static final String DEFAULT_COMMAND_PACKAGE_NAME = "com.espirit.moddev.cli.commands";
-
     private static final Logger LOGGER = LoggerFactory.getLogger(Cli.class);
-    private static Set<Class<? extends Command>> commandClasses = CommandUtils.scanForCommandClasses(DEFAULT_COMMAND_PACKAGE_NAME);
-    private static Set<Class<?>> groupClasses = GroupUtils.scanForGroupClasses(DEFAULT_GROUP_PACKAGE_NAME);
+    private static final Set<Class<? extends Command>> commandClasses = CommandUtils.scanForCommandClasses();
+    private static final Set<Class<?>> groupClasses = GroupUtils.scanForGroupClasses();
 
     private final Properties buildProperties;
     private final Properties gitProperties;
@@ -124,6 +114,7 @@ public static void main(final String[] args) {
      *
      * @param args the input arguments
      */
+    @SuppressWarnings("squid:S1162")
     public void execute(final String[] args) throws Exception {
         setLoggingSystemProperties();
 
@@ -222,13 +213,16 @@ private static void setLoggingSystemProperties() {
      *
      * @param command the command instance to execute
      */
+    @SuppressWarnings("squid:S1162")
     public void executeCommand(Command<Result> command) throws Exception {
         LOGGER.info("Executing " + command.getClass().getSimpleName());
         CliContext context = null;
         try {
             context = getCliContextOrNull(command);
             Result result = command.call();
             logResult(result);
+        } catch (ClassCastException e) {
+            LOGGER.trace("Cannot perform a cast - most likely because the command's call method returns Object as a result, instead of Result.", e);
         } catch (Exception e) {
             LOGGER.trace("Exception occurred during context initialization or command execution", e);
             throw e;
@@ -247,6 +241,7 @@ static void closeContext(CliContext context) {
         }
     }
 
+    @SuppressWarnings("squid:S1162")
     private static void logResult(Result result) throws Exception{
         if (result != null) {
             if(result.isError()){
@@ -300,7 +295,7 @@ public static Command parseCommandLine(String[] args, CliBuilder<Command> builde
 
 
     /**
-     * A getter for command classes from the package specified by {@link #DEFAULT_COMMAND_PACKAGE_NAME} only. The classes are loaded at class-load
+     * A getter for command classes that can be found on the classpath. The classes are loaded at class-load
      * time only once.
      *
      * @return a reference to the actual list of loaded commands
@@ -310,9 +305,9 @@ public static Set<Class<? extends Command>> getCommandClasses() {
     }
 
     /**
-     * Look up all class that define command classes in the package specified by {@link #DEFAULT_GROUP_PACKAGE_NAME}.
+     * Look up all class that define command classes in the classpath.
      *
-     * @return {@link java.util.Set} of all class that define command classes in the package specified by {@link #DEFAULT_GROUP_PACKAGE_NAME}
+     * @return {@link java.util.Set} of all class that define command classes in the classpath
      */
     public static Set<Class<?>> getGroupClasses() {
         return Collections.unmodifiableSet(groupClasses);

fsdevtools-cli/src/main/java/com/espirit/moddev/cli/reflection/CommandUtils.java

@@ -23,11 +23,15 @@
 package com.espirit.moddev.cli.reflection;
 
 import com.espirit.moddev.cli.api.command.Command;
-
 import org.apache.log4j.Logger;
 import org.reflections.Reflections;
+import org.reflections.util.ClasspathHelper;
+import org.reflections.util.ConfigurationBuilder;
+import org.reflections.util.FilterBuilder;
 
 import java.lang.reflect.Modifier;
+import java.net.URL;
+import java.util.Collection;
 import java.util.Set;
 import java.util.stream.Collectors;
 
@@ -43,19 +47,40 @@ public final class CommandUtils {
     private static final Logger LOGGER = Logger.getLogger(CommandUtils.class);
 
     private CommandUtils() {
-        // Not used
     }
 
+    /**
+     * Scans the whole classpath for classes that implement the {@link Command} interface. Ignores abstract classes.
+     * Ignores furthermore everything that is not a .class file. Ignores every artifact from the Java runtime.
+     * Ingores the package com.github.rvesse.airline.annotations, because it contains malformed classes.
+     *
+     * @return a set of matching classes
+     */
+    public static Set<Class<? extends Command>> scanForCommandClasses() {
+        FilterBuilder filter = new FilterBuilder().add(input -> input.endsWith(".class")).excludePackage("com.github.rvesse.airline.annotations");
+        Collection<URL> classPathUrls = ClasspathHelper.forJavaClassPath();
+        Collection<URL> classPathUrlsExceptJre = classPathUrls.stream().filter(url -> !url.toString().contains("/jre/lib")).collect(Collectors.toList());
+        ConfigurationBuilder configuration = new ConfigurationBuilder().addUrls(classPathUrlsExceptJre).filterInputsBy(filter);
+        return scanForCommandClasses(new Reflections(configuration));
+    }
     /**
      * Scans the given package for classes that implement the {@link Command} interface. Ignores abstract classes.
      *
-     * @param packageToScan the package, that should be scanned recursively
+     * @param packageToScanForCommands the package, that should be scanned recursively
      * @return a set of matching classes
+     * @throws IllegalArgumentException if null or empty package string is passed
      */
-    public static Set<Class<? extends Command>> scanForCommandClasses(String packageToScan) {
-        String packageToScanForCommands = packageToScan;
-        LOGGER.debug("Scanning for command classes in package " + packageToScanForCommands);
-        Reflections reflections = new Reflections(packageToScanForCommands);
+    public static Set<Class<? extends Command>> scanForCommandClasses(String packageToScanForCommands) {
+        if(packageToScanForCommands == null || packageToScanForCommands.isEmpty()) {
+            throw new IllegalArgumentException("Don't pass a null or empty string! Use scanForCommandClasses() if you don't want to define a package");
+        }
+        FilterBuilder inputsFilter = new FilterBuilder().includePackage(packageToScanForCommands);
+        ConfigurationBuilder configuration = new ConfigurationBuilder().forPackages(packageToScanForCommands).filterInputsBy(inputsFilter);
+        Reflections reflections = new Reflections(configuration);
+        return scanForCommandClasses(reflections);
+    }
+
+    private static Set<Class<? extends Command>> scanForCommandClasses(Reflections reflections) {
         Set<Class<? extends Command>> commandClasses = reflections.getSubTypesOf(Command.class);
         commandClasses = commandClasses
             .stream()

fsdevtools-cli/src/main/java/com/espirit/moddev/cli/reflection/GroupUtils.java

@@ -23,10 +23,14 @@
 package com.espirit.moddev.cli.reflection;
 
 import com.github.rvesse.airline.annotations.Group;
-
 import org.apache.log4j.Logger;
 import org.reflections.Reflections;
+import org.reflections.util.ClasspathHelper;
+import org.reflections.util.ConfigurationBuilder;
+import org.reflections.util.FilterBuilder;
 
+import java.net.URL;
+import java.util.Collection;
 import java.util.Set;
 import java.util.stream.Collectors;
 
@@ -44,6 +48,24 @@ private GroupUtils() {
         // Not used
     }
 
+    /**
+     * Scans the classpath for classes that are annotated with airline's {@link Group} annotation.
+     * Excludes the package com.github.rvesse.airline.annotations, because it contains a malformed
+     * command class that causes an exception when loaded.
+     *
+     * @return a set of matching classes
+     */
+    public static Set<Class<?>> scanForGroupClasses() {
+        FilterBuilder filter = new FilterBuilder().add(input -> input.endsWith(".class")).excludePackage("com.github.rvesse.airline.annotations");
+        Collection<URL> classPathUrls = ClasspathHelper.forJavaClassPath();
+        Collection<URL> classPathUrlsExceptJre = classPathUrls.stream().filter(url -> !url.toString().contains("/jre/lib")).collect(Collectors.toList());
+        ConfigurationBuilder configuration = new ConfigurationBuilder()
+                .addUrls(classPathUrlsExceptJre)
+                .filterInputsBy(filter);
+
+        return scanForGroupClasses(new Reflections(configuration));
+    }
+
     /**
      * Scans the given package for classes that are annotated with airline's {@link Group} annotation.
      *
@@ -52,7 +74,10 @@ private GroupUtils() {
      */
     public static Set<Class<?>> scanForGroupClasses(String packageToScan) {
         LOGGER.debug("Scanning for group classes in package " + packageToScan);
-        Reflections reflections = new Reflections(packageToScan);
+        return scanForGroupClasses(new Reflections(packageToScan));
+    }
+
+    private static Set<Class<?>> scanForGroupClasses(Reflections reflections) {
         Set<Class<?>> groupClasses = reflections.getTypesAnnotatedWith(Group.class);
 
         String commaSeparatedGroups = groupClasses.stream()

fsdevtools-cli/src/test/java/com/espirit/moddev/cli/testcommands/reflectiontest/CommandUtilsTest.java

@@ -23,31 +23,43 @@
 package com.espirit.moddev.cli.testcommands.reflectiontest;
 
 import com.espirit.moddev.cli.api.command.Command;
+import com.espirit.moddev.cli.commands.example.ExampleCustomCommand;
 import com.espirit.moddev.cli.commands.export.ExportCommand;
 import com.espirit.moddev.cli.reflection.CommandUtils;
 import com.espirit.moddev.cli.reflection.ReflectionUtils;
-
 import org.junit.Assert;
 import org.junit.Test;
 
 import java.util.Set;
 
-/**
- * @author e-Spirit AG
- */
 public class CommandUtilsTest {
 
-    public static final String DEFAULT_COMMAND_TEST_PACKAGE_NAME = "com.espirit.moddev.cli.testcommands.reflectiontest";
+    private static final String DEFAULT_COMMAND_TEST_PACKAGE_NAME = "com.espirit.moddev.cli.testcommands.reflectiontest";
 
+    private static final int EXPECTED_COMMANDCLASSES_IN_TESTCOMMANDSPACKAGE = 4;
+
+    /**
+     * If there are command class implementations added or removed from the test package, this test may fail. In
+     * order to make it green, you should expect the correct count of command classes in the test package. Since
+     * we test runtime behaviour (reflection) influenced by compile time changes (class removed/added) here,
+     * this test fails at test runtime, even though the project compiles fine.
+     */
     @Test
     public void packageScanRetrievesCorrectCommandClassCount() {
         Set<Class<? extends Command>> commandClassesInPackage = CommandUtils.scanForCommandClasses(DEFAULT_COMMAND_TEST_PACKAGE_NAME);
-        Assert.assertEquals(4, commandClassesInPackage.size());
+        Assert.assertEquals(EXPECTED_COMMANDCLASSES_IN_TESTCOMMANDSPACKAGE, commandClassesInPackage.size());
     }
 
     /**
-     * @author e-Spirit AG
+     * This test ensures that the whole classpath is scanned for commands without specifying a package name.
+     * Asserting the presence of an explicit class from an external dependency is probably the best Test for this.
      */
+    @Test
+    public void classpathScanRetrievesAllCommandClasses() {
+        Set<Class<? extends Command>> commandClassesInClasspath = CommandUtils.scanForCommandClasses();
+        Assert.assertTrue("Classpath scan should retrieve custom command class", commandClassesInClasspath.contains(ExampleCustomCommand.class));
+    }
+
     public static class ReflectionTest {
         @Test
         public void readsCommandDescriptionFromAnnotatedMethodTest() {

fsdevtools-cli/src/test/java/com/espirit/moddev/cli/testgroups/reflectiontest/GroupUtilsTest.java

@@ -22,6 +22,7 @@
 
 package com.espirit.moddev.cli.testgroups.reflectiontest;
 
+import com.espirit.moddev.cli.groups.example.ExampleCustomGroup;
 import com.espirit.moddev.cli.reflection.GroupUtils;
 
 import org.junit.Assert;
@@ -41,5 +42,10 @@ public void packageScanRetrievesCorrectCommandClassCount() {
         final Set<Class<?>> groupClassesInPackage = GroupUtils.scanForGroupClasses(DEFAULT_GROUP_TEST_PACKAGE_NAME);
         Assert.assertEquals(5, groupClassesInPackage.size());
     }
+    @Test
+    public void classpathScanRetrievesExampleGroup() {
+        final Set<Class<?>> groupClassesInPackage = GroupUtils.scanForGroupClasses();
+        Assert.assertTrue("Expected example group class to be found", groupClassesInPackage.contains(ExampleCustomGroup.class));
+    }
 
 }