This chapter gives an overview of issues related with dependency management and presents how Gradle can be used to overcome them.
Gradle offers a very good support for dependency management. If you are familiar with Maven or Ivy approach you will be delighted to learn that:
All the concepts that you already know and like are still there and are fully supported by Gradle. The current dependency management solutions all require to work with XML descriptor files and are usually based on remote repositories for downloading the dependencies. Gradle fully supports this approach.
Gradle works perfectly with your existent dependency management infrastructure, be it Maven or Ivy. All the repositories you have set up with your custom POM or ivy files can be used as they are. No changes necessary.
Additionally, Gradle offers a simpler approach, which might be better suited for some projects.
We think dependency management is very important for almost any project. Yet the kind of dependency management you need depends on the complexity and the environment of your project. Is your project a distribution or a library? Is it part of an enterprise environment, where it is integrated into other projects builds or not? But all types of projects share the following requirements:
The version of the jar must be easy to recognize. Sometimes the version is in the Manifest file of the jar, often not. And even if, it is rather painful to always look into the Manifest file to learn about the version. Therefore we think that you should only use jars which have their version as part of their file name.
It hopes to be clear what are the first level dependencies and what are the transitive ones. There are different ways to achieve this. We will look at this later.
Conflicting versions of the same jar should be detected and either resolved or cause an exception.
Why do we think this is necessary? Without a dependency management as described above, your are likely to burn your fingers sooner or later. If it is unclear which version of a jar your are using, this can introduce subtle bugs which are very hard to find. For example there might be a project which uses Hibernate 3.0.4. There are some problems with Hibernate so a developer installs version 3.0.5 of Hibernate on her machine. This did not solve the problem but she forgot to roll back Hibernate to 3.0.4. Weeks later there is an exception on the integration machine which can't be reproduced on the developer machine. Without a version in the jar name this problem might take a long time to debug. Version in the jar names increases the expressiveness of your project and makes it easier to maintain.
Why is transitive dependency management so important? If you don't know which dependencies are first level dependencies and which ones are transitive you will soon lose control over your build. Even Gradle has already 20+ jars. An enterprise project using Spring, Hibernate, etc. easily ends up with 100+ jars. There is no way to memorize where all these jars come from. If you want to get rid of a first level dependency you can't be sure which other jars you should remove. Because a dependency of a first level dependency might also be a first level dependency itself. Or it might be a transitive dependency of another of your first level dependencies. Many first level dependencies are runtime dependencies and the transitive dependencies are of course all runtime dependencies. So the compiler won't help you much here. The end of the story is, as we have seen very often, no one dares to remove any jar any longer. The project classpath is a complete mess and if a classpath problem arises, hell on earth invites you for a ride. In one of my former projects, I found some ldap related jar in the classpath, whose sheer presence, as I found out after much research, accelerated LDAP access. So removing this jar would not have led to any errors at compile or runtime.
Gradle offers you different ways to express what are first level and what are transitive dependencies. Gradle allows you for example to store your jars in CVS or SVN without XML descriptor files and still use transitive dependency management. Gradle also validates your dependency hierarchy against the reality of your code by using only the first level dependencies for compiling.
In your dependency description you tell Gradle which version of a dependency is needed by another dependency. This frequently leads to conflicts. Different dependencies rely on different versions of another dependency. The JVM unfortunately does not offer yet any easy way, to have different versions of the same jar in the classpath (see Section 43.2.5, “Dependency management and Java”).
Gradle offers following conflict resolution strategies:
1.0-milestone-6
. See ResolutionStrategy
for reference on managing the conflict resolution strategies.
Gradle provides means to resolve version conflicts:
DependencyHandler
1.0-milestone-7
.
This approach is useful if the dependency incurring conflict is a transitive dependency.
It also can be used to force versions of first level dependencies.
See examples in ResolutionStrategy
To deal with problems due to version conflicts, reports with dependency graphs are also very helpful. Such reports are another feature of dependency management.
Sometimes, you always want to use the latest version of a particular dependency, or the latest in a range of versions.
You can easily do this using a dynamic version. A dynamic version can be either a version range (eg. 2.+
)
or it can be a placeholder for the latest version available (eg. latest.integration
).
Alternatively, sometimes the module you request can change over time, even for the same version.
An example of this type of changing module is a maven SNAPSHOT
module,
which always points at the latest artifacts published.
The main difference between a dynamic version and a changing module is that when you resolve a dynamic version, you'll get the real, static version as the module name. When you resolve a changing module, the artifacts are named using the version you requested, but the underlying artifacts may change over time.
By default, Gradle caches dynamic versions and changing modules for 24 hours.
You can override the default cache modes using command line options.
You can change the cache expiry times in your build using the resolution strategy
(see Section 43.8.3, “Fine-tuned control over dependency caching”).
Traditionally, Java has offered no support at all for dealing with libraries and versions. There are
no standard ways to say that
foo-1.0.jar
depends on a bar-2.0.jar
. This has led to proprietary solutions. The most popular ones
are Maven and Ivy. Maven is a complete build system whereas Ivy focuses solely on dependency management.
Both approaches rely on descriptor XML files, which contains information about the dependencies of a particular jar. Both also use repositories where the actual jars are placed together with their descriptor files. And both offer resolution for conflicting jar versions in one form or the other. Yet we think the differences of both approaches are significant in terms of flexibility and maintainability. Beside this, Ivy fully supports the Maven dependency handling. So with Ivy you have access to both worlds. We like Ivy very much. Gradle uses it under the hood for its dependency management. Ivy is most often used via Ant and XML descriptors. But it also has an API. We integrate deeply with Ivy via its API. This enables us to build new concepts on top of Ivy which Ivy does not offer itself.
Right now there is a lot of movement in the field of dependency handling. There is OSGi and there is JSR-294, Improved Modularity Support in the JavaTM Programming Language. OSGi is available already, JSR-294 is supposed to be shipped with Java 7. These technologies deal, amongst many other things, also with a painful problem which is neither solved by Maven nor by Ivy. This is enabling different versions of the same jar to be used at runtime.
In Gradle dependencies are grouped into configurations. Configurations have a name, a number of other properties, and they can extend each other. Many Gradle plugin add pre-defined configurations to your project. The Java plugin, for example, adds some configurations to represent the various classpaths it needs. see Section 23.5, “Dependency management” for details. Of course you can add your add custom configurations on top of that. There are many use cases for custom configurations. This is very handy for example for adding dependencies not needed for building or testing your software (e.g. additional JDBC drivers to be shipped with your distribution).
A project's configurations are managed by a configurations
object. The closure you pass to
the configurations object is applied against its API. To learn more about this API have a look at
ConfigurationContainer
.
To define a configuration:
To access a configuration:
Example 43.2. Accessing a configuration
build.gradle
println configurations.compile.name
println configurations['compile'].name
To configure a configuration:
Example 43.3. Configuration of a configuration
build.gradle
configurations { compile { description = 'compile classpath' transitive = true } runtime { extendsFrom compile } } configurations.compile { description = 'compile classpath' }
There are several different types of dependencies that you can declare:
Table 43.1. Dependency types
Type | Description |
External module dependency | A dependency on an external module in some repository. |
Project dependency | A dependency on another project in the same build. |
File dependency | A dependency on a set of files on the local filesystem. |
Client module dependency | A dependency on an external module, where the artifacts are located in some repository but the module meta-data is specified by the local build. You use this kind of dependency when you want to override the meta-data for the module. |
Gradle API dependency | A dependency on the API of the current Gradle version. You use this kind of dependency when you are developing custom Gradle plugins and task types. |
Local Groovy dependency | A dependency on the Groovy version used by the current Gradle version. You use this kind of dependency when you are developing custom Gradle plugins and task types. |
External module dependencies are the most common dependencies. They refer to a module in an external repository.
Example 43.4. Module dependencies
build.gradle
dependencies { runtime group: 'org.springframework', name: 'spring-core', version: '2.5' runtime 'org.springframework:spring-core:2.5', 'org.springframework:spring-aop:2.5' runtime( [group: 'org.springframework', name: 'spring-core', version: '2.5'], [group: 'org.springframework', name: 'spring-aop', version: '2.5'] ) runtime('org.hibernate:hibernate:3.0.5') { transitive = true } runtime group: 'org.hibernate', name: 'hibernate', version: '3.0.5', transitive: true runtime(group: 'org.hibernate', name: 'hibernate', version: '3.0.5') { transitive = true } }
Please see the DependencyHandler
for more examples and complete reference. Please read on to get thorough understanding of the Gradle's dependency management.
Gradle provides different notations for module dependencies. There is a string notation and
a map notation. A module dependency has an API which allows for further configuration. Have a look at
ExternalModuleDependency
to learn all about the API.
This API provides properties and configuration methods. Via the string notation you can define a subset
the properties. With the map notation you can define all properties. To have access to the complete API,
either with the map or with the string notation, you can assign a single dependency to a configuration
together with a closure.
If you declare a module dependency, Gradle looks for a corresponding module descriptor file (pom.xml
or
ivy.xml
) in the repositories. If such a module descriptor file exists, it is parsed and the artifacts of
this module (e.g. hibernate-3.0.5.jar
) as well as its dependencies (e.g. cglib) are downloaded. If no such
module descriptor file exists, Gradle looks for a file called hibernate-3.0.5.jar
to retrieve. In Maven
a module can only have one and only one artifact. In Gradle and Ivy a module can have multiple artifacts.
Each artifact can have a different set of dependencies.
ivy.xml
) can declare multiple artifacts.
For more information, see Ivy reference for ivy.xml
.
In Gradle, when you declare a dependency on an ivy module you actually declare dependency on the 'default'
configuration of that module.
So the actual list of artifacts (typically jars) your project depends on, are all artifacts that are attached to the default
configuration of that module.
This is very important in following exemplary use cases:
default
configuration of some module contains some artifacts
you don't want on the classpath. You might need to configure a dependency on specific artifact(s) of given module,
rather than pulling all artifacts of the default
dependency
default
one. This means this artifact will not be pulled in by Gradle.
Unless you explicitly declare what configuration of the module you depend on.
DependencyHandler
for examples and complete reference on declaring dependencies.
As said above, if no module descriptor file can be found, Gradle by default
downloads a jar with the name of the module. But sometimes, even if the repository contains module descriptors, you want to download only the artifact jar, without
the dependencies.
[12]
And sometimes you want to download a zip from a repository, that does not have module descriptors.
Gradle provides an artifact only notation for those use cases - simply prefix the extension that you want to be downloaded with '@'
sign:
Example 43.5. Artifact only notation
build.gradle
dependencies { runtime "org.groovy:groovy:1.8.6@jar" runtime group: 'org.groovy', name: 'groovy', version: '1.8.6', ext: 'jar' }
An artifact only notation creates a module dependency which downloads only the artifact file with
the specified extension. Existing module descriptors are ignored.
The Maven dependency management has the notion of classifiers. [13] Gradle supports this. To retrieve classified dependencies from a maven repository you can write:
Example 43.6. Dependency with classifier
build.gradle
compile "org.gradle.test.classifiers:service:1.0:jdk15@jar" otherConf group: 'org.gradle.test.classifiers', name: 'service', version: '1.0', classifier: 'jdk14'
As you can see in the example, classifiers can be used together with setting an explicit extension (artifact only notation).
To use the external dependencies of a configuration:
Example 43.7. Usage of external dependency of a configuration
build.gradle
task listJars << { configurations.compile.each { File file -> println file.name } }
Output of gradle -q listJars
> gradle -q listJars hibernate-core-3.6.7.Final.jar antlr-2.7.6.jar commons-collections-3.1.jar dom4j-1.6.1.jar slf4j-api-1.6.1.jar hibernate-commons-annotations-3.2.0.Final.jar hibernate-jpa-2.0-api-1.0.1.Final.jar jta-1.1.jar
Client module dependencies enable you to declare transitive dependencies directly in your build script. They are a replacement for a module descriptor XML file in an external repository.
Example 43.8. Client module dependencies - transitive dependencies
build.gradle
dependencies { runtime module("org.codehaus.groovy:groovy-all:1.8.6") { dependency("commons-cli:commons-cli:1.0") { transitive = false } module(group: 'org.apache.ant', name: 'ant', version: '1.8.4') { dependencies "org.apache.ant:ant-launcher:1.8.4@jar", "org.apache.ant:ant-junit:1.8.4" } } }
This declares a dependency of your project on Groovy. Groovy itself has dependencies. But Gradle does
not look for an XML descriptor to figure them out but gets the information from the build file. The
dependencies of a client module can be normal module dependencies or artifact dependencies or another
client module. Have also a look at the API documentation: ClientModule
In the current release client modules have one limitation. Let's say your project is a library and you want this library to be uploaded to your company's Maven or Ivy repository. Gradle uploads the jars of your project to the company repository together with the XML descriptor file of the dependencies. If you use client modules the dependency declaration in the XML descriptor file is not correct. We will improve this in a future release of Gradle.
Gradle distinguishes between external dependencies and dependencies on projects which are part of the same multi-project build. For the latter you can declare Project Dependencies.
For more information see the API documentation for ProjectDependency
Multi-project builds are discussed in Chapter 49, Multi-project Builds.
File dependencies allow you to directly add a set of files to a configuration, without first adding them to a repository. This can be useful if you cannot, or do not want to, place certain files in a repository. Or if you do not want to use any repositories at all for storing your dependencies.
To add some files as a dependency for a configuration, you simply pass a file collection as a dependency:
Example 43.10. File dependencies
build.gradle
dependencies { runtime files('libs/a.jar', 'libs/b.jar') runtime fileTree(dir: 'libs', include: '*.jar') }
File dependencies are not included in the published dependency descriptor for your project. However, file dependencies are included in transitive project dependencies within the same build. This means they cannot be used outside the current build, but they can be used with the same build.
You can declare which tasks produce the files for a file dependency. You might do this when, for example, the files are generated by the build.
Example 43.11. Generated file dependencies
build.gradle
dependencies { compile files("$buildDir/classes") { builtBy 'compile' } } task compile << { println 'compiling classes' } task list(dependsOn: configurations.compile) << { println "classpath = ${configurations.compile.collect {File file -> file.name}}" }
Output of gradle -q list
> gradle -q list compiling classes classpath = [classes]
You can declare a dependency on the API of the current version of Gradle by using the
DependencyHandler.gradleApi()
method. This is
useful when you are developing custom Gradle tasks or plugins.
You can declare a dependency on the Groovy that is distributed with Gradle by using the
DependencyHandler.localGroovy()
method. This is
useful when you are developing custom Gradle tasks or plugins in Groovy.
You can exclude a transitive dependency either by configuration or by dependency:
Example 43.14. Excluding transitive dependencies
build.gradle
configurations { compile.exclude module: 'commons' all*.exclude group: 'org.gradle.test.excludes', module: 'reports' } dependencies { compile("org.gradle.test.excludes:api:1.0") { exclude module: 'shared' } }
If you define
an exclude for a particular configuration, the excluded transitive dependency will be filtered for all
dependencies when resolving this configuration or any inheriting configuration.
If you want to exclude a transitive dependency from all your
configurations you can use the Groovy spread-dot operator to express this in a concise way, as shown in the example.
When defining an exclude, you can
specify either only the organization or only the module name or both.
Have also a look at the API documentation of Dependency
and
Configuration
.
Not every transitive dependency can be excluded - some transitive dependencies might be essential for correct runtime behavior of the application. Generally, one can exclude transitive dependencies that are either not required by runtime or that are guaranteed to be available on the target environment/platform.
Should you exclude per-dependency or per-configuration? It turns out that in majority of cases you want to use the per-configuration exclusion. Here are the some exemplary reasons why one might want to exclude a transitive dependency. Bear in mind that for some of those use cases there are better solutions than exclusions!
ResolutionStrategy
for a potentially better solution to the problem.
Basically, in most of the cases excluding the transitive dependency should be done per configuration. This way the dependency declaration is more explicit. It is also more accurate because a per-dependency exclude rule does not guarantee the given transitive dependency does not show up in the configuration. For example, some other dependency, which does not have any exclude rules, might pull in that unwanted transitive dependency.
Other examples of the dependency exclusions can be found in the reference for
ModuleDependency
or
DependencyHandler
.
All attributes for a dependency are optional, except the name. It depends on the repository type, which information is need for actually finding the dependencies in the repository. See Section 43.6, “Repositories”. If you work for example with Maven repositories, you need to define the group, name and version. If you work with filesystem repositories you might only need the name or the name and the version.
Example 43.15. Optional attributes of dependencies
build.gradle
dependencies { runtime ":junit:4.10", ":testng" runtime name: 'testng' }
You can also assign collections or arrays of dependency notations to a configuration:
Example 43.16. Collections and arrays of dependencies
build.gradle
List groovy = ["org.codehaus.groovy:groovy-all:1.8.6@jar", "commons-cli:commons-cli:1.0@jar", "org.apache.ant:ant:1.8.4@jar"] List hibernate = ['org.hibernate:hibernate:3.0.5@jar', 'somegroup:someorg:1.0@jar'] dependencies { runtime groovy, hibernate }
In Gradle a dependency can have different configurations (as your project can have different configurations). If you don't specify anything explicitly, Gradle uses the default configuration of the dependency. For dependencies from a Maven repository, the default configuration is the only available one anyway. If you work with Ivy repositories and want to declare a non-default configuration for your dependency you have to use the map notation and declare:
Example 43.17. Dependency configurations
build.gradle
dependencies { runtime group: 'org.somegroup', name: 'somedependency', version: '1.0', configuration: 'someConfiguration' }
To do the same for project dependencies you need to declare:
Example 43.18. Dependency configurations for project
build.gradle
dependencies { compile project(path: ':api', configuration: 'spi') }
You can generate dependency reports from the command line (see Section 11.5.3, “Listing project dependencies”). With the help of the Project report plugin (see Chapter 39, The Project Report Plugin) such a report can be created by your build.
For the examples below we have the following dependencies setup:
Example 43.19. Configuration.copy
build.gradle
configurations { sealife alllife } dependencies { sealife "sea.mammals:orca:1.0", "sea.fish:shark:1.0", "sea.fish:tuna:1.0" alllife configurations.sealife alllife "air.birds:albatros:1.0" }
The dependencies have the following transitive dependencies:
shark-1.0 -> seal-2.0, tuna-1.0
orca-1.0 -> seal-1.0
tuna-1.0 -> herring-1.0
You can use the configuration to access the declared dependencies or a subset of those:
Example 43.20. Accessing declared dependencies
build.gradle
task dependencies << {
configurations.alllife.dependencies.each { dep -> println dep.name }
println()
configurations.alllife.allDependencies.each { dep -> println dep.name }
println()
configurations.alllife.allDependencies.findAll { dep -> dep.name != 'orca' }.each { dep -> println dep.name }
}
Output of gradle -q dependencies
> gradle -q dependencies albatros albatros orca shark tuna albatros shark tuna
dependencies
returns only the dependencies belonging explicitly to the configuration.
allDependencies
includes the dependencies from extended
configurations.
To get the library files of the configuration dependencies you can do:
Example 43.21. Configuration.files
build.gradle
task allFiles << { configurations.sealife.files.each { file -> println file.name } }
Output of gradle -q allFiles
> gradle -q allFiles orca-1.0.jar shark-1.0.jar tuna-1.0.jar seal-2.0.jar herring-1.0.jar
Sometimes you want the library files of a subset of the configuration dependencies (e.g. of a single dependency).
Example 43.22. Configuration.files with spec
build.gradle
task files << {
configurations.sealife.files { dep -> dep.name == 'orca' }.each { file ->
println file.name
}
}
Output of gradle -q files
> gradle -q files orca-1.0.jar seal-2.0.jar
The Configuration.files
method always retrieves all artifacts of the whole configuration. It
then filters the retrieved files by specified dependencies. As you can see in the example, transitive dependencies are included.
You can also copy a configuration. You can optionally specify that only a subset of dependencies from the original configuration
should be copied. The copying methods come in two flavors. The copy
method copies only the dependencies belonging
explicitly to the configuration. The copyRecursive
method copies all the dependencies, including the dependencies from extended
configurations.
Example 43.23. Configuration.copy
build.gradle
task copy << {
configurations.alllife.copyRecursive { dep -> dep.name != 'orca' }.allDependencies.each { dep ->
println dep.name
}
println()
configurations.alllife.copy().allDependencies.each { dep ->
println dep.name
}
}
Output of gradle -q copy
> gradle -q copy albatros shark tuna albatros
It is important to note that the returned files of the copied configuration are often but not always the same than the returned files of the dependency subset of the original configuration. In case of version conflicts between dependencies of the subset and dependencies not belonging to the subset the resolve result might be different.
Example 43.24. Configuration.copy vs. Configuration.files
build.gradle
task copyVsFiles << { configurations.sealife.copyRecursive { dep -> dep.name == 'orca' }.each { file -> println file.name } println() configurations.sealife.files { dep -> dep.name == 'orca' }.each { file -> println file.name } }
Output of gradle -q copyVsFiles
> gradle -q copyVsFiles orca-1.0.jar seal-1.0.jar orca-1.0.jar seal-2.0.jar
In the example above, orca
has a dependency on seal-1.0
whereas
shark
has a dependency on seal-2.0
. The original configuration has therefore a version
conflict which is resolved to the newer seal-2.0
version. The files
method therefore
returns seal-2.0
as a transitive dependency of orca
. The copied configuration only has orca
as a dependency and therefore there is no version conflict and seal-1.0
is returned as a transitive
dependency.
Once a configuration is resolved it is immutable. Changing its state or the state of one of its dependencies will cause an exception. You can always copy a resolved configuration. The copied configuration is in the unresolved state and can be freshly resolved.
To learn more about the API of the configuration class see the API documentation:
Configuration
.
Gradle repository management, based on Apache Ivy, gives you a lot of freedom regarding repository layout and retrieval policies. Additionally Gradle provides various convenience method to add pre-configured repositories.
You may configure any number of repositories, each of which is treated independently by Gradle. If Gradle finds a module descriptor in a particular repository, it will attempt to download all of the artifacts for that module from the same repository. Although module meta-data and module artifacts must be located in the same repository, it is possible to compose a single repository of multiple URLs, giving multiple locations to search for meta-data files and jar files.
There are several different types of repositories you can declare:
Table 43.2. Repository types
Type | Description |
Maven central repository | A pre-configured repository that looks for dependencies in Maven Central. |
Maven local repository | A pre-configured repository that looks for dependencies in the local Maven repository. |
Maven repository | A Maven repository. Can be located on the local filesystem or at some remote location. |
Ivy repository | An Ivy repository. Can be located on the local filesystem or at some remote location. |
Flat directory repository | A simple repository on the local filesystem. Does not support any meta-data formats. |
To add the central Maven 2 repository (http://repo1.maven.org/maven2) simply add this to your build script:
Now Gradle will look for your dependencies in this repository.
To use the local Maven cache as a repository you can do:
Example 43.26. Adding the local Maven cache as a repository
build.gradle
repositories { mavenLocal() }
Gradle uses the same logic as maven to identify the location of your local maven cache. If a local repository location is defined in a settings.xml
, this location will be used.
The settings.xml
in
takes precedence over the USER_HOME
/.m2settings.xml
in
.
If no M2_HOME
/confsettings.xml
is available, Gradle uses the default location
.USER_HOME
/.m2/repository
For adding a custom Maven repository you can do:
Example 43.27. Adding custom Maven repository
build.gradle
repositories {
maven {
url "http://repo.mycompany.com/maven2"
}
}
Sometimes a repository will have the POMs published to one location, and the JARs and other artifacts published at another location. To define such a repository, you can do:
Example 43.28. Adding additional Maven repositories for JAR files
build.gradle
repositories { maven { // Look for POMs and artifacts, such as JARs, here url "http://repo2.mycompany.com/maven2" // Look for artifacts here if not found at the above location artifactUrls "http://repo.mycompany.com/jars" artifactUrls "http://repo.mycompany.com/jars2" } }
Gradle will look at the first URL for the POM and the JAR. If the JAR can't be found there, the artifact URLs are used to look for JARs.
To access a Maven repository which uses basic authentication, you specify the username and password to use when you define the repository:
Example 43.29. Accessing password protected Maven repository
build.gradle
repositories { maven { credentials { username 'user' password 'password' } url "http://repo.mycompany.com/maven2" } }
It is advisable to keep your username and password in gradle.properties
rather than directly in the build file.
If you want to use a (flat) filesystem directory as a repository, simply type:
Example 43.30. Flat repository resolver
build.gradle
repositories { flatDir { dirs 'lib' } flatDir { dirs 'lib1', 'lib2' } }
This adds repositories which look into one or more directories for finding dependencies. If you only work with flat directory resolvers you don't need to set all attributes of a dependency. See Section 43.4.8, “Optional attributes”
To use an Ivy repository with a standard layout:
Example 43.31. Ivy repository
build.gradle
repositories { ivy { url "http://repo.mycompany.com/repo" layout "maven" } }
See IvyArtifactRepository
for details.
To define an Ivy repository with a non-standard layout, you can define a pattern layout for the repository:
Example 43.32. Ivy repository with pattern layout
build.gradle
repositories { ivy { url "http://repo.mycompany.com/repo" layout 'pattern', { artifact "[module]/[revision]/[artifact].[ext]" ivy "[module]/[revision]/ivy.xml" } } }
To define an Ivy repository which fetches ivy files and artifacts from different locations, you can explicitly define complete URL patterns for artifacts and ivy files:
Example 43.33. Ivy repository with custom patterns
build.gradle
repositories { ivy { artifactPattern "http://repo.mycompany.com/3rd-party-artifacts/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" artifactPattern "http://repo.mycompany.com/company-artifacts/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" ivyPattern "http://repo.mycompany.com/ivy-files/[organisation]/[module]/[revision]/ivy.xml" } }
To access an Ivy repository which uses basic authentication, you specify the username and password to use when you define the repository:
Example 43.34. Ivy repository
build.gradle
repositories { ivy { credentials { username 'user' password 'password' } artifactPattern "http://repo.mycompany.com/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" } }
To access a repository:
Example 43.35. Accessing a repository
build.gradle
println repositories.localRepository.name
println repositories['localRepository'].name
To configure a repository:
Example 43.36. Configuration of a repository
build.gradle
repositories { flatDir { name 'localRepository' } } repositories { localRepository { dirs 'lib' } } repositories.localRepository { dirs 'lib' }
Gradle, thanks to Ivy under its hood, is extremely flexible regarding repositories:
There are many options for the protocol to communicate with the repository (e.g. filesystem, http, ssh, ...)
Each repository can have its own layout.
Let's say, you declare a dependency on the
junit:junit:3.8.2
library.
Now how does Gradle find it in the repositories? Somehow the dependency information has to be mapped to a
path. In contrast to Maven, where this path is fixed, with Gradle you can define a pattern that defines
what the path will look like. Here are some examples:
[14]
// Maven2 layout (if a repository is marked as Maven2 compatible, the organization (group) is split into subfolders according to the dots.) someroot/[organisation]/[module]/[revision]/[module]-[revision].[ext] // Typical layout for an ivy repository (the organization is not split into subfolder) someroot/[organisation]/[module]/[revision]/[type]s/[artifact].[ext] // Simple layout (the organization is not used, no nested folders.) someroot/[artifact]-[revision].[ext]
To add any kind of repository (you can pretty easy write your own ones) you can do:
Example 43.37. Definition of a custom repository
build.gradle
repositories { ivy { ivyPattern "$projectDir/repo/[organisation]/[module]-ivy-[revision].xml" artifactPattern "$projectDir/repo/[organisation]/[module]-[revision](-[classifier]).[ext]" } }
An overview of which Resolvers are offered by Ivy and thus also by Gradle can be found here. With Gradle you just don't configure them via XML but directly via their API.
Gradle takes your dependency declarations and repository definitions and attempts to download all of your dependencies by a process called dependency resolution. Below is a brief outline of how this process works.
Given a required dependency, Gradle first attempts to resolve the module for that dependency. Each repository is inspected in order, searching first for a module descriptor file (pom or ivy file) that indicates the presence of that module. If no module descriptor is found, Gradle will search for the presence of the primary module artifact file indicating that the module exists in the repository.
If the dependency is declared as a dynamic version (like 1.+
), Gradle will resolve this to the newest available static version (like 1.2
)
in the repository. For maven repositories, this is done using the maven-metadata.xml
file, while for ivy repositories this is done by directory listing.
If the module descriptor is a pom
file that has a parent pom declared, Gradle will recursively attempt to resolve each of the parent modules for the pom.
Once each repository has been inspected for the module, Gradle will choose the 'best' one to use. This is done using the following criteria:
When the dependency is declared by a static version and a module descriptor file is found in a repository, there is no need to continue searching later repositories and the remainder of the process is short-circuited.
All of the artifacts for the module are then requested from the same repository that was chosen in the process above.
Gradle contains a highly sophisticated dependency caching mechanism, which seeks to minimise the number of remote requests made in dependency resolution, while striving to guarantee that the results of dependency resolution are correct and reproducible.
The Gradle dependency cache consists of 2 key types of storage:
A file-based store of downloaded artifacts, including binaries like jars as well as raw downloaded meta-data like pom files and ivy files. The storage path for a downloaded artifact includes the SHA1 checksum, meaning that 2 artifacts with the same name but different content can easily be cached.
A binary store of resolved module meta-data, including the results of resolving dynamic versions, module descriptors, and artifacts.
Separating the storage of downloaded artifacts from the cache metadata permits us to do some very powerful things with our cache that would be difficult with a transparent, file-only cache layout.
The Gradle cache does not allow the local cache to hide problems and creating mysterious and difficult to debug behavior that has been a challenge with many build tools. This new behavior is implemented in a bandwidth and storage efficient way. In doing so, Gradle enables reliable and reproducible enterprise builds.
Gradle keeps a record of various aspects of dependency resolution in binary format in the metadata cache. The information stored in the metadata cache includes:
Every entry in the metadata cache includes a record of the repository that provided the information as well as a timestamp that can be used for cache expiry.
As described above, for each repository there is a separate metadata cache. A repository is identified by its URL, type and layout. If a module or artifact has not been previously resolved from this repository, Gradle will attempt to resolve the module against the repository. This will always involve a remote lookup on the repository, however in many cases no download will be required (see Section 43.8.1.3, “Artifact reuse”, below).
Dependency resolution will fail if the required artifacts are not available in any repository specified by the build, regardless whether the local cache has retrieved this artifact from a different repository. Repository independence allows builds to be isolated from each other in an advanced way that no build tool has done before. This is a key feature to create builds that are reliable and reproducible in any environment.
Before downloading an artifact, Gradle tries to determine the checksum of the required artifact by downloading the sha file associated with that artifact. If the checksum can be retrieved, an artifact is not downloaded if an artifact already exists with the same id and checksum. If the checksum cannot be retrieved from the remote server, the artifact will be downloaded (and ignored if it matches an existing artifact).
As well as considering artifacts downloaded from a different repository, Gradle will also attempt to reuse artifacts found in the local Maven Repository. If a candidate artifact has been downloaded by Maven, Gradle will use this artifact if it can be verified to match the checksum declared by the remote server.
It is possible for different repositories to provide a different binary artifact in response to the same artifact identifier. This is often the case with Maven SNAPSHOT artifacts, but can also be true for any artifact which is republished without changing it's identifier. By caching artifacts based on their SHA1 checksum, Gradle is able to maintain multiple versions of the same artifact. This means that when resolving against one repository Gradle will never overwrite the cached artifact file from a different repository. This is done without requiring a separate artifact file store per repository.
The Gradle dependency cache uses file-based locking to ensure that it can safely be used by multiple Gradle processes concurrently. The lock is held whenever the binary meta-data store is being read or written, but is released for slow operations such as downloading remote artifacts.
The --offline
command line switch tells Gradle to always use dependency modules from the cache, regardless if they are due to be checked again.
When running with offline, Gradle will never attempt to access the network to perform dependency resolution.
If required modules are not present in the dependency cache, build execution will fail.
At times, the Gradle Dependency Cache can be out of sync with the actual state of the configured repositories. Perhaps a repository was initially misconfigured,
or perhaps a "non-changing" module was published incorrectly. To refresh all dependencies in the dependency cache, use the --refresh-dependencies
option on the command line.
The --refresh-dependencies
option tells Gradle to ignore all cached entries for resolved modules and artifacts.
A fresh resolve will be performed against all configured repositories, with dynamic versions recalculated, modules refreshed, and artifacts downloaded.
However, where possible Gradle will attempt to if the previously downloaded artifacts are valid before downloading again.
This is done by comparing published SHA1 values in the repository with the SHA1 values for existing downloaded artifacts.
You can fine-tune certain aspects of caching using the ResolutionStrategy
for a configuration.
By default, Gradle caches dynamic versions for 24 hours. To change how long Gradle will cache the resolved version for a dynamic version, use:
Example 43.38. Dynamic version cache control
build.gradle
configurations.all { resolutionStrategy.cacheDynamicVersionsFor 10, 'minutes' }
By default, Gradle caches changing modules for 24 hours. To change how long Gradle will cache the meta-data and artifacts for a changing module, use:
Example 43.39. Changing module cache control
build.gradle
configurations.all { resolutionStrategy.cacheChangingModulesFor 30, 'days' }
For more details, take a look at the API documentation for ResolutionStrategy
.
Many projects rely on the Maven Central repository. This is not without problems.
The Maven Central repository can be down or has a very long response time.
The pom.xml
's of many projects have wrong information (as one example, the POM of
commons-httpclient-3.0
declares JUnit as a runtime dependency).
For many projects there is not one right set of dependencies (as more or less imposed by the
pom
format).
If your project relies on the Maven Central repository you are likely to need an additional custom repository, because:
You might need dependencies that are not uploaded to Maven Central yet.
You want to deal properly with wrong metadata in a Maven Central pom.xml
.
You don't want to expose people who want to build your project, to the downtimes or sometimes very long response times of Maven Central.
It is not a big deal to set-up a custom repository. [15] But it can be tedious, to keep it up to date. For a new version, you have always to create the new XML descriptor and the directories. And your custom repository is another infrastructure element which might have downtimes and needs to be updated. To enable historical builds, you need to keep all the past libraries and you need a backup. It is another layer of indirection. Another source of information you have to lookup. All this is not really a big deal but in its sum it has an impact. Repository Manager like Artifactory or Nexus make this easier. But for example open source projects don't usually have a host for those products.
This is a reason why some projects prefer to store their libraries in their version control system. This approach is fully supported by Gradle. The libraries can be stored in a flat directory without any XML module descriptor files. Yet Gradle offers complete transitive dependency management. You can use either client module dependencies to express the dependency relations, or artifact dependencies in case a first level dependency has no transitive dependencies. People can check out such a project from svn and have everything necessary to build it.
If you are working with a distributed version control system like Git you probably don't want to use the version control system to store libraries as people check out the whole history. But even here the flexibility of Gradle can make your life easier. For example you can use a shared flat directory without XML descriptors and yet you can have full transitive dependency management as described above.
You could also have a mixed strategy. If your main concern is bad metadata in the pom.xml
and maintaining
custom XML descriptors,
Client Modules
offer an alternative. But you can of course still use Maven2 repo and your custom repository as a
repository for
jars only
and still enjoy
transitive
dependency management. Or you can only provide client modules for POMs with bad metadata. For the
jars and the correct POMs you still use the remote repository.
There is another way to deal with transitive dependencies without XML descriptor files. You can do this with Gradle, but we don't recommend it. We mention it for the sake of completeness and comparison with other build tools.
The trick is to use only artifact dependencies and group them in lists. That way you have somehow
expressed, what are your first level dependencies and what are transitive dependencies (see
Section 43.4.8, “Optional attributes”).
But the draw-back is, that for the Gradle dependency management all dependencies are considered first level dependencies. The
dependency reports don't show your real dependency graph and the
compile
task uses all dependencies, not just the first level dependencies. All in all, your build is less
maintainable and reliable than it could be when using client modules. And you don't gain anything.
[12] Gradle supports partial multiproject builds (seeChapter 49, Multi-project Builds).
[13] http://www.sonatype.com/books/maven-book/reference/pom-relationships-sect-project-relationships.html
[14] At http://ant.apache.org/ivy/history/latest-milestone/concept.html you can learn more about ivy patterns.
[15] If you want to shield your project from the downtimes of Maven Central things get more complicated. You probably want to set-up a repository proxy for this. In an enterprise environment this is rather common. For an open source project it looks like overkill.