This chapter provides a quick overview of some of the important considerations that you need to think about when you move from a simple build automation to a continuous integration environment.
If you have been following the examples in this book, you would have seen how to use Maven to manage the build process for projects which are targeted for deployment on Oracle Fusion Middleware environments.
The next logical step is to move towards a continuous integration approach, so that the builds of all of your projects can be triggered, managed and monitored in one central place.
The advantage of continuous integration comes from componentization of an application and constant integration of those components as they are independently modified, often by different groups of developers. Maven projects are used to represent these components and their relationships to each other. Since Hudson understands Maven's project relationship model, it can automatically rebuild and retest affected components in the order that they should be built in. When Hudson detects the changes to the code-base, the affected components are built and reintegrated in correct order to ensure proper function of the entire application.
This chapter includes some of the important things to consider while moving to a continuous integration environment with Hudson. This chapter includes the following sections:
Dependency management is a key feature of Maven and something that distinguishes it from other build automation technologies, like ANT, which Fusion Middleware has supported for some time. This section explores some important dependency management topics.
It includes the following topics:
Snapshot versioning is covered more extensively in Chapter 8. Using snapshots for components that are under development is required for the automated continuous integration system to work properly. Note that a fixed version, non-snapshot versioned artifact should not be modified and replaced. The best practice is that you should not update artifacts after they are released. This is a core assumption of the Maven approach. However, it is worth noting that often this assumption is not correct in enterprise software development, where vendors and end users do sometimes update "finished" artifacts without changing the version number, for example through patching them in place. Even though it is possible to violate this rule, every attempt should be made to comply to ensure integration stability.
Most projects have dependencies on other artifacts. At build time, Maven obtains these artifacts from the configured artifact repositories and use them to resolve compilation, runtime and test dependencies.
Dependencies explicitly listed in the POM may also have dependencies of their own. These are commonly referred to as transitive dependencies. Based on dependency attributes such as scope and version, Maven uses rules to determine which dependencies the build should utilize. An important part of this resolution process has to do with version conflicts. It is possible that a project may have transitive dependencies on multiple versions of the same artifact (identical groupId and artifactId). In such a case, Maven uses the nearest definition which means that it uses the version of the closest dependency to your project in the tree of dependencies. You can always guarantee a particular version by declaring it explicitly in your project's POM.
Note:
If two dependency versions are at the same depth in the dependency tree, until Maven 2.0.8 it was not defined which one would win, but since Maven 2.0.9 it is the order in the declaration that counts. Hence, the first declaration wins.Dependencies may optionally specify a scope. In addition to determining whether or not a dependency is made available to the classpath during a particular build phase, scope affects how transitive dependency is propagated to the classpath.
There are six scopes available.
Compile: This is the default scope, used if no scope is specified. Compile dependencies are available in all classpaths of a project. Furthermore, these dependencies are propagated to dependent projects.
Provided: This is much like compile, but indicates you expect the JDK or a container to provide the dependency at runtime. For example, when building a web application for the Java Enterprise Edition, you can set the dependency on the servlet API and related Java EE APIs to scope provided because the web container provides those classes. This scope is only available on the compilation and test classpath, and is not transitive.
Runtime: This scope indicates that the dependency is not required for compilation, but is for execution. It is in the runtime and test classpaths, but not the compile classpath.
Test: This scope indicates that the dependency is not required for normal use of the application, and is only available for the test compilation and execution phases.
System: This scope is similar to Provided except that you have to provide the JAR which contains it explicitly. The artifact is always available and is not looked up in a repository.
Import: (only available in Maven 2.0.9 or higher) This scope is only used on a dependency of type POM. It indicates that the specified POM should be replaced with the dependencies in that POMs. Since they are replaced, dependencies with a scope of import do not actually participate in limiting the transitivity of a dependency.
A series of interdependent projects, such as an application, can be aggregated by a multi-module POM. This should not be confused with a parent POM which provides inherited configuration. A multi-module POM may also be an inheritance parent to sub-module projects. When a Maven build is executed upon a multi module POM, Maven examines the tree of sub-projects and calculates the correct order of dependency to build the modules.
Multiple module POMs can be useful for organizing multiple component builds in Hudson.
This section describes some aspects of Maven that you should consider while moving to a continuous integration environment.
This section contains the following topics:
Every project that is part of continuous integration must specify a distributionManagement section in its POM. This section tells Maven where the artifacts are going to be deployed at the end of the build process, that is, which repository (local or remote). The examples used in this book use the Archiva repository. Deploying artifacts to a repository makes them available for other projects to use as dependencies.
You must define a distributionManagement section that describes which repository to deploy snapshots and releases to. It is recommended that the distributionManagement configuration be placed at a common inherited POM that is shared among all projects such as the oracle-common POM, as described in Chapter 9.
The following shows an example of a distributionManagement configuration:
<distributionManagement>
<repository>
<uniqueVersion>false</uniqueVersion>
<id>releases</id>
<name>Releases</name>
<url>http://server:port/archiva/repository/releases/</url>
<layout>default</layout>
</repository>
<snapshotRepository>
<uniqueVersion>true</uniqueVersion>
<id>snapshots</id>
<name>Snapshots</name>
<url>http://server:port/archiva/repository/snapshots</url>
<layout>default</layout>
</snapshotRepository>
</distributionManagement>
There are some important settings that govern how and when Maven accesses repositories:
Update Policy: This controls how often Maven checks with a remote repository for updates to an artifact that it already has in its local repository. Configure your snapshot repository in your settings.xml in Hudson to use updatePolicy as always. The effect of updatePolicy is on your development systems. The default value is daily. If you want to integrate the changes as they occur in Hudson, you should change their updatePolicy accordingly. Dependencies may change suddenly and without warning. While the continuous integration system should have sufficient tests in place to reduce the occurrence of regressions, you can still run into issues depending on up-to-the-minute snapshots while developing. One such example is the API changes.
You should get all project snapshot dependencies up-to-date so that their local build reflects the current state of the deployed code-base prior to check-in.
Server credentials: This tells Maven the credentials that are needed to access a remote repository; typically Maven repositories require you to authenticate before you are allowed to make changes to the repository, for example, publishing a new artifact). Unless you have given the Archiva guest user global upload privileges, which is not recommended, you must specify correct credentials for the snapshot repository in the servers section. You should have a unique Hudson user with snapshot repository upload permissions. See Chapter 4 for details about user and role configuration.
Use Maven's password encryption for the password value. The Maven guide to password encryption can be found here:
http://maven.apache.org/guides/mini/guide-encryption.html.
The following shows a sample settings.xml configuration for the Hudson user:
<settings>
...
<servers>
...
<server>
<id>snapshots</id>
<username>hudson</username>
<password>{COQLCE6DU6GtcS5P=}</password>
</server>
...
</servers>
...
</settings>
This section discusses how to set up your build jobs in Hudson. There are various options available to build Maven projects. This section describes the approach recommended by Oracle.
Before proceeding, ensure that you have configured Hudson, as described in Chapter 7.
This section contains the following topics:
To create a basic Maven Hudson job:
Open the Hudson web interface and log in. if necessary.
Create a new job:
Select New Job from the right-hand menu.
Provide a unique name and select Build a free-style software project.
Click OK.
Configure the source code management.
Ensure that you complete configuring the Subversion server, including the SSH public and private key configuration.
Under Source Code Management, select Subversion.
Provide the repository URL for your project directory. For example, svn+ssh://subversion-server/ciroot/subversion/repository/trunk/projects/my-project
Note:
In this example we are using asvn+ssh URL, which accesses Subversion using SSH. If you are using a different protocol, the steps that are necessary to configure it may vary slightly.
Hudson attempts to verify the URL and may respond with an error message like the following:
Unable to access svn+ssh://hostname/ciroot/subversion/repository/trunk : svn: E200015: authentication cancelled(show details) (Maybe you need to enter credential?)
If you get this error message, do the following:
From the message, click enter credentials.
Select SSH public key authentication (svn+ssh).
Enter the user name.
Enter the SSH private-key passphrase if required.
Select the private-key file from the Hudson file system. It should be in a ~/.ssh/id_rsa format.
Add a Maven build step:
Under the Build section, select Invoke Maven 3 from the Add Build Step drop-down menu.
Select Maven 3 home. Add necessary goals and properties in the appropriate text fields.
If you have a SNAPSHOT continuous integration build environment, then configure the goals to perform a clean deploy.
If necessary, open the Advanced settings and ensure that the Settings entry points to the Maven settings that you created in the Hudson web interface, while configuring Hudson.
Save the configuration
Click Save at the bottom of the page.
Hudson provides number of ways to manage a continuous integration build's triggers in Hudson. These include manual and automated triggers. The option to manually start a build is always available for any job. When choosing an automated trigger, you may consider factors like the structure of the project, the location of the source code, the presence of any branches, and so on.
This section contains the following topics:
Regardless of how the build is triggered, the job is added to the pending job queue and completed when the resources become available.
All jobs can be started from the user interface with the Build Now link.
This type of build trigger is vital to establishing a healthy continuous integration build. As changes are committed to project source, Hudson triggers builds of the associated Hudson jobs. The trigger does this by periodically checking the associated Subversion URL for changes.
To enable this trigger, select the Poll SCM option. You must then provide a cron expression to determine the schedule Hudson uses to poll the repository.
For some job types, you can trigger them on a schedule. Long running system integration tests are an example of a build that you might want to run periodically as opposed to every time the test source is modified.
Schedule based triggers are configured with cron expressions exactly like the Poll SCM trigger.
Trivial projects may contain multiple builds that produce unique artifacts that have dependencies on each other. If Hudson rebuilds an artifact as the result of any trigger type, it must also build and test dependent artifacts to ensure integration is still valid. When dependencies that are also built on this Hudson server are successfully completed, Hudson recognizes these relationships automatically and triggers the build. In order for this trigger to work, the dependencies must also enable the post-build action Notify that Maven dependencies have been updated by Maven 3 integration.
If there are dependencies that are undergoing concurrent development and being managed as snapshots in your common Maven repository, then they should be managed by your Hudson instance. If this is not practical, then you can use the SNAPSHOT dependency trigger to monitor the Maven repository for changes in such dependencies. When an updated SNAPSHOT dependency has been detected, the build triggers and downloads the new dependency for integration.
This trigger also uses a cron expression to configure the polling interval.
To manage your build dependencies correctly, you may add each project as a separate Hudson build and configure the dependency triggers manually, or you can configure a multi-module Maven POM as a parent Hudson job. The multi-module solution reduces the possibility of making mistakes wiring the dependencies manually. It also automatically stays up-to-date as the dependencies are changed in the Maven configuration.
Configuration of a multi-module build is identical to configuration for regular projects. To examine the results of component project builds, Hudson provides a tab in the build results page in the Maven 3 Build Information link. At the top of the page, the Modules tab summarizes the component build results.
To examine the log for any of the sub-project builds, use the Console Log link. All project and sub-project builds are logged in their sequence of execution.
Hudson should be configured to send notifications to the correct parties when the build breaks. The continuous integration system must ensure that their changes do not break the build and test process. If this does happen, they need to be notified of the breakage and must address the issue as soon as possible. Hudson should have each user registered as a unique user. The Hudson user name must match the Subversion user name that they ordinarily commit under. Hudson relies on this name to look up the proper contact email to send notification to.
General email notification configuration can be found under Manage Hudson -> Configure System -> E-mail Notification.
You must also make sure some form of user management is enabled. You can do this by selecting Enable Security from the Configure System panel. There are a number of choices for user management and additional third-party plug-ins to support most other popular solutions, such as LDAP. The best option is to use Hudson's own user database. Select this choice from the Access Control section. There are additional options for limiting permissions to particular users and groups.
To add a new user, the user simply needs to follow the sign up link at the top of the Hudson home page and fill out the necessary information. The user name must match the corresponding Subversion user name.
Normally, automated notification is sufficient to ensure the continuous build system is kept healthy and produces effective results. However, there are conditions that can require additional monitoring and coordination to get the system back to an operational state. It is a good policy to designate a build coordinator to track down such problems, coordinate solutions for broad problems and perform troubleshooting when the system itself is suspect.