Plugin Compatibility Testing

    A plugin can be executed inside of a Spinnaker version that is different than the version it was compiled against. For example, a plugin compiled against Spinnaker might be compatible with Spinnaker versions up to 1.23.

    This range is important for plugin developers so that they can support users across Spinnaker versions; it’s important for plugin consumers so that they can safely install and upgrade plugins. Fortunately, testing can resolve this compatibility range automatically.

    To start, you should write Spring Boot-style integration tests using service test fixtures .

    These integration tests have good properties. They execute a service’s main and demonstrate that a plugin can be loaded into the service at runtime. You can inject Spring’s ApplicationContext into your test class and attempt to retrieve the beans defined by your plugin.

    You can also use these tests to verify the end-to-end behavior of your plugin. For example, if you’re writing a new stage as a plugin, you can ask Orca’s /orchestrate endpoint to execute a pipeline that includes your new stage type.

    Automated compatibility testing

    These integration tests demonstrate that a compiled plugin can execute inside of a service at runtime. By executing a test multiple times but changing the underlying service version, we can resolve which versions of Spinnaker a plugin is compatible with.

    Before using this Gradle plugin, check that your integration tests depend on Spinnaker’s exported Gradle platforms, which take the form <service>-bom. For example, if your test relies on orca-api-tck, your plugin’s subproject build file should look something like this:

    To use the test runner, configure your plugin’s build files:

    1. // Inside root project
    3. }
    5. spinnakerBundle {
    6.   // ...
    7.   compatibility {
    8.   }
    11. // Inside service extension subproject
    12. apply plugin: "io.spinnaker.plugin.compatibility-test-runner"

    The plugin will create a set of tasks: a set of compatibilityTest-<subproject-name>-<spinnaker-version> tasks, and a top-level compatibilityTest task that will run all of the compatibility test subtasks.