Understanding variant selection
Gradle’s dependency management engine is known as variant aware.
In a traditional dependency management engine like Apache Maven™, dependencies are bound to components published at GAV coordinates.
This means that the set of transitive dependencies for a component is solely determined by the GAV coordinates of this component.
It doesn’t matter what artifact is actually resolved, the set of dependencies is always the same.
In addition, selecting a different artifact for a component (for example, using the jdk7 artifact) is cumbersome as it requires the use of classifiers.
One issue with this model is that it cannot guarantee global graph consistency because there are no common semantics associated with classifiers.
What this means is that there’s nothing which prevents from having both the jdk7 and jdk8 versions of a single module on classpath, because the engine has no idea what semantics are associated with the classifier name.
Gradle, in addition to the concept of a module published at GAV coordinates, introduces the concept of variants of this module. Variants correspond to the different "views" of a component that is published at the same GAV coordinates. In the Gradle model, artifacts are attached to variants, not modules. This means, in practice, that different artifacts can have a different set of dependencies:
This intermediate level, which associates artifacts and dependencies to variants instead of directly to the component, allows Gradle to model properly what each artifact is used for.
However, this raises the question about how variants are selected: how does Gradle know which variant to choose when there’s more than one? In practice, variants are selected thanks to the use of attributes, which provide semantics to the variants and help the engine in achieving a consistent resolution result.
For historical reasons, Gradle differentiates between two kind of components:
-
local components, built from sources, for which variants are mapped to outgoing configurations
-
external components, published on repositories, in which case either the module was published with Gradle Module Metadata and variants are natively supported, or the module is using Ivy/Maven metadata and variants are derived from metadata.
In both cases, Gradle performs variant aware selection.
Configuration and variant attributes
Local components expose variants as outgoing configurations, which are consumable configurations. When dependency resolution happens, the engine will select one variant of an outgoing component by selecting one of its consumable configurations.
|
There are 2 noticeable exception to this rule:
In this case, variant aware resolution is bypassed. |
Attributes are used on both resolvable configurations (also known as a consumer) and consumable configurations (on the producer). Adding attributes to other kinds of configurations simply has no effect, as attributes are not inherited between configurations.
The role of the dependency resolution engine is to find a suitable variant of a producer given the constraints expressed by a consumer.
This is where attributes come into play: their role is to perform the selection of the right variant of a component.
|
Variants vs configurations
For external components, the terminology is to use the word variants, not configurations. Configurations are a super-set of variants. This means that an external component provides variants, which also have attributes. However, sometimes the term configuration may leak into the DSL for historical reasons, or because you use Ivy which also has this concept of configuration. |
Visualizing variant information
Gradle offers a report task called outgoingVariants that displays the variants of a project, with their capabilities, attributes and artifacts.
It is conceptually similar to the dependencyInsight reporting task.
By default, outgoingVariants prints information about all variants.
It offers the optional parameter --variant <variantName> to select a single variant to display.
It also accepts the --all flag to include information about legacy and deprecated configurations.
Here is the output of the outgoingVariants task on a freshly generated java-library project:
> Task :outgoingVariants
--------------------------------------------------
Variant apiElements
--------------------------------------------------
Description = API elements for main.
Capabilities
- [default capability]
Attributes
- org.gradle.category = library
- org.gradle.dependency.bundling = external
- org.gradle.jvm.version = 8
- org.gradle.libraryelements = jar
- org.gradle.usage = java-api
Artifacts
- build/libs/variant-report.jar (artifactType = jar)
Secondary variants (*)
- Variant : classes
- Attributes
- org.gradle.category = library
- org.gradle.dependency.bundling = external
- org.gradle.jvm.version = 8
- org.gradle.libraryelements = classes
- org.gradle.usage = java-api
- Artifacts
- build/classes/java/main (artifactType = java-classes-directory)
--------------------------------------------------
Variant runtimeElements
--------------------------------------------------
Description = Elements of runtime for main.
Capabilities
- [default capability]
Attributes
- org.gradle.category = library
- org.gradle.dependency.bundling = external
- org.gradle.jvm.version = 8
- org.gradle.libraryelements = jar
- org.gradle.usage = java-runtime
Artifacts
- build/libs/variant-report.jar (artifactType = jar)
Secondary variants (*)
- Variant : classes
- Attributes
- org.gradle.category = library
- org.gradle.dependency.bundling = external
- org.gradle.jvm.version = 8
- org.gradle.libraryelements = classes
- org.gradle.usage = java-runtime
- Artifacts
- build/classes/java/main (artifactType = java-classes-directory)
- Variant : resources
- Attributes
- org.gradle.category = library
- org.gradle.dependency.bundling = external
- org.gradle.jvm.version = 8
- org.gradle.libraryelements = resources
- org.gradle.usage = java-runtime
- Artifacts
- build/resources/main (artifactType = java-resources-directory)
(*) Secondary variants are variants created via the Configuration#getOutgoing(): ConfigurationPublications API which also participate in selection, in addition to the configuration itself.
From this you can see the two main variants that are exposed by a java library, apiElements and runtimeElements.
Notice that the main difference is on the org.gradle.usage attribute, with values java-api and java-runtime.
As they indicate, this is where the difference is made between what needs to be on the compile classpath of consumers, versus what’s needed on the runtime classpath.
It also shows secondary variants, which are exclusive to Gradle projects and not published.
For example, the secondary variant classes from apiElements is what allows Gradle to skip the JAR creation when compiling against a java-library project.
Variant aware matching
Let’s take the example of a lib library which exposes 2 variants: its API (via a variant named exposedApi) and its runtime (via a variant named exposedRuntime).
|
About producer variants
The variant name is there mostly for debugging purposes and to get a nicer display in error messages. The name, in particular, doesn’t participate in the id of a variant: only its attributes do. That is to say that to search for a particular variant, one must rely on its attributes, not its name. There are no restrictions on the number of variants a component can expose. Traditionally, a component would expose an API and an implementation, but we may, for example, want to expose the test fixtures of a component too. It is also possible to expose different APIs for different consumers (think about different environments, like Linux vs Windows). |
A consumer needs to explain what variant it needs and this is done by setting attributes on the consumer.
Attributes consist of a name and a value pair.
For example, Gradle comes with a standard attribute named org.gradle.usage specifically to deal with the concept of selecting the right variant of a component based on the usage of the consumer (compile, runtime …).
It is however possible to define an arbitrary number of attributes.
As a producer, we can express that a consumable configuration represents the API of a component by attaching the (org.gradle.usage,JAVA_API) attribute to the variant.
As a consumer, we can express that we need the API of the dependencies of a resolvable configuration by attaching the (org.gradle.usage,JAVA_API) attribute to it.
Doing this, Gradle has a way to automatically select the appropriate variant by looking at the configuration attributes:
-
the consumer wants
org.gradle.usage=JAVA_API -
the producer,
libexposes 2 different variants. One withorg.gradle.usage=JAVA_API, the other withorg.gradle.usage=JAVA_RUNTIME. -
Gradle chooses the
org.gradle.usage=JAVA_APIvariant of the producer because it matches the consumer attributes
In other words: attributes are used to perform the selection based on the values of the attributes.
A more elaborate example involves more than one attribute. Typically, a Java Library project in Gradle will involve 4 different attributes, found both on the producer and consumer sides:
-
org.gradle.usage, explaining if the variant is the API of a component, or its implementation -
org.gradle.dependency.bundling, which declares how the dependencies of the component are bundled (for example, if the artifact is a fat jar, then the bundling isEMBEDDED) -
org.gradle.libraryelements, which is used to explain what parts of the library the variant contains (classes, resources or everything) -
org.gradle.jvm.version, which is used to explain what minimal version of Java this variant is targeted at
Now imagine that our library comes in two different flavors:
-
one for JDK 8
-
one for JDK 9+
This is typically achieved, in Maven, by producing 2 different artifacts, a "main" artifact and a "classified" one. However, in Maven a consumer cannot express the fact it needs the most appropriate version of the library based on the runtime.
With Gradle, this is elegantly solved by having the producer declare 2 variants:
-
one with
org.gradle.jvm.version=8, for consumers at least running on JDK 8 -
one with
org.gradle.jvm.version=9, for consumers starting from JDK 9
Note that the artifacts for both variants will be different, but their dependencies may be different too. Typically, the JDK 8 variant may need a "backport" library of JDK 9+ to work, that only consumers running on JDK 8 should get.
On the consumer side, the resolvable configuration will set all four attributes above, and, depending on the runtime, will set its org.gradle.jvm.version to 8 or more.
|
A note about compatibility of variants
What if the consumer sets Then resolution would fail with an error message explaining that there’s no matching variant of the producer. This is because Gradle recognizes that the consumer wants a Java 7 compatible library, but the minimal version of Java available on the producer is 8. If, on the other hand, the consumer needs 11, then Gradle knows both the 8 and 9 variant would work, but it will select 9 because it’s the highest compatible version. |
Variant selection errors
In the process of identifying the right variant of a component, two situations will result in a resolution error:
-
More than one variant from the producer match the consumer attributes, there is variant ambiguity
-
No variant from the producer match the consumer attributes
Dealing with ambiguous variant selection errors
An ambiguous variant selection looks somewhat like the following:
> Could not resolve all files for configuration ':compileClasspath'.
> Could not resolve project :lib.
Required by:
project :ui
> Cannot choose between the following variants of project :lib:
- feature1ApiElements
- feature2ApiElements
All of them match the consumer attributes:
- Variant 'feature1ApiElements' capability org.test:test-capability:1.0:
- Unmatched attribute:
- Found org.gradle.category 'library' but wasn't required.
- Compatible attributes:
- Provides org.gradle.dependency.bundling 'external'
- Provides org.gradle.jvm.version '11'
- Required org.gradle.libraryelements 'classes' and found value 'jar'.
- Provides org.gradle.usage 'java-api'
- Variant 'feature2ApiElements' capability org.test:test-capability:1.0:
- Unmatched attribute:
- Found org.gradle.category 'library' but wasn't required.
- Compatible attributes:
- Provides org.gradle.dependency.bundling 'external'
- Provides org.gradle.jvm.version '11'
- Required org.gradle.libraryelements 'classes' and found value 'jar'.
- Provides org.gradle.usage 'java-api'
As can be seen, all compatible candidate variants are displayed, with their attributes. These are then grouped into two sections:
-
Unmatched attributes are presented first, as they might be the missing piece in selecting the proper variant.
-
Compatible attributes are presented second as they indicate what the consumer wanted and how these variants do match that request.
There cannot be any mismatched attributes as the variant would not be a candidate then. Similarly, the set of displayed variants also excludes the ones that have been disambiguated.
In the example above, the fix does not lie in attribute matching but in capability matching, which are shown next to the variant name.
Because these two variants effectively provide the same attributes and capabilities, they cannot be disambiguated.
So in this case, the fix is most likely to provide different capabilities on the producer side (project :lib) and express a capability choice on the consumer side (project :ui).
Dealing with no matching variant errors
A no matching variant error looks somewhat like the following:
> No variants of project :lib match the consumer attributes:
- Configuration ':lib:compile':
- Incompatible attribute:
- Required artifactType 'dll' and found incompatible value 'jar'.
- Other compatible attribute:
- Provides usage 'api'
- Configuration ':lib:compile' variant debug:
- Incompatible attribute:
- Required artifactType 'dll' and found incompatible value 'jar'.
- Other compatible attributes:
- Found buildType 'debug' but wasn't required.
- Provides usage 'api'
- Configuration ':lib:compile' variant release:
- Incompatible attribute:
- Required artifactType 'dll' and found incompatible value 'jar'.
- Other compatible attributes:
- Found buildType 'release' but wasn't required.
- Provides usage 'api'
As can be seen, all candidate variants are displayed, with their attributes. These are then grouped into two sections:
-
Incompatible attributes are presented first, as they usually are the key in understanding why a variant could not be selected.
-
Other attributes are presented second, this includes required and compatible ones as well as all extra producer attributes that are not requested by the consumer.
Similarly with the ambiguous variant error, the goal is then to understand which variant is to be selected and see which attribute or capability can be tweaked on the consumer for this to happen.
Mapping from Maven/Ivy to variants
Neither Maven nor Ivy have the concept of variants, which are only natively supported by Gradle Module Metadata. However, it doesn’t prevent Gradle from working with them thanks to different strategies.
|
Relationship with Gradle Module Metadata
Gradle Module Metadata is a metadata format for modules published on Maven, Ivy or other kind of repositories.
It is similar to See the Gradle Module Metadata specification for more information. |
Mapping of POM files to variants
Modules published on a Maven repository are converted into variant-aware modules. A particularity of Maven modules is that there is no way to know what kind of component is published. In particular, there’s no way to make the difference between a BOM representing a platform, and a BOM used as a super-POM. Sometimes, it is even possible for a POM file to act both as a platform and a library.
As a consequence, Maven modules are derived into 6 distinct variants, which allows Gradle users to explain precisely what they depend on:
-
2 "library" variants (attribute
org.gradle.category=library)-
the
compilevariant maps the<scope>compile</scope>dependencies. This variant is equivalent to theapiElementsvariant of the Java Library plugin. All dependencies of this scope are considered API dependencies. -
the
runtimevariant maps both the<scope>compile</scope>and<scope>runtime</scope>dependencies. This variant is equivalent to theruntimeElementsvariant of the Java Library plugin. All dependencies of those scopes are considered runtime dependencies.-
in both cases, the
<dependencyManagement>dependencies are not converted to constraints
-
-
-
4 "platform" variants derived from the
<dependencyManagement>block (attributeorg.gradle.category=platform):-
the
platform-compilevariant maps the<scope>compile</scope>dependency management dependencies as dependency constraints. -
the
platform-runtimevariant maps both the<scope>compile</scope>and<scope>runtime</scope>dependency management dependencies as dependency constraints. -
the
enforced-platform-compileis similar toplatform-compilebut all the constraints are forced -
the
enforced-platform-runtimeis similar toplatform-runtimebut all the constraints are forced
-
You can understand more about the use of platform and enforced platforms variants by looking at the importing BOMs section of the manual.
By default, whenever you declare a dependency on a Maven module, Gradle is going to look for the library variants.
However, using the platform or enforcedPlatform keyword, Gradle is now looking for one of the "platform" variants, which allows you to import the constraints from the POM files, instead of the dependencies.
Mapping of Ivy files to variants
Contrary to Maven, there is no derivation strategy implemented for Ivy files by default. The reason fo this is that, contrary to pom, Ivy is a flexible format that allows you to publish arbitrary many and customized configurations. So there is no notion of compile/runtime scope or compile/runtime variants in Ivy in general. Only if you use the ivy-publish plugin to publish ivy files with Gradle, you get a structure that follows a similar pattern as pom files. But since there is not guarantee that all ivy metadata files consumed by a build follow this pattern, Gradle cannot enforce a derivation strategy based on it.
However, if you want to implement a derivation strategy for compile and runtime variants for Ivy, you can do so with component metadata rule.
The component metadata rules API allows you to access ivy configurations and create variants based on them.
If you know that all the ivy modules your are consuming have been published with Gradle without further customizations of the ivy.xml file, you can add the following rule to your build:
class IvyVariantDerivationRule implements ComponentMetadataRule {
@Inject ObjectFactory getObjects() { }
void execute(ComponentMetadataContext context) {
// This filters out any non Ivy module
if(context.getDescriptor(IvyModuleDescriptor) == null) {
return
}
context.details.addVariant("runtimeElements", "default") {
attributes {
attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, getObjects().named(LibraryElements, LibraryElements.JAR))
attribute(Category.CATEGORY_ATTRIBUTE, getObjects().named(Category, Category.LIBRARY))
attribute(Usage.USAGE_ATTRIBUTE, getObjects().named(Usage, Usage.JAVA_RUNTIME))
}
}
context.details.addVariant("apiElements", "compile") {
attributes {
attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, getObjects().named(LibraryElements, LibraryElements.JAR))
attribute(Category.CATEGORY_ATTRIBUTE, getObjects().named(Category, Category.LIBRARY))
attribute(Usage.USAGE_ATTRIBUTE, getObjects().named(Usage, Usage.JAVA_API))
}
}
}
}
dependencies {
components { all(IvyVariantDerivationRule) }
}open class IvyVariantDerivationRule : ComponentMetadataRule {
@Inject open fun getObjects(): ObjectFactory = throw UnsupportedOperationException()
override fun execute(context: ComponentMetadataContext) {
// This filters out any non Ivy module
if(context.getDescriptor(IvyModuleDescriptor::class) == null) {
return
}
context.details.addVariant("runtimeElements", "default") {
attributes {
attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, getObjects().named(LibraryElements.JAR))
attribute(Category.CATEGORY_ATTRIBUTE, getObjects().named(Category.LIBRARY))
attribute(Usage.USAGE_ATTRIBUTE, getObjects().named(Usage.JAVA_RUNTIME))
}
}
context.details.addVariant("apiElements", "compile") {
attributes {
attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, getObjects().named(LibraryElements.JAR))
attribute(Category.CATEGORY_ATTRIBUTE, getObjects().named(Category.LIBRARY))
attribute(Usage.USAGE_ATTRIBUTE, getObjects().named(Usage.JAVA_API))
}
}
}
}
dependencies {
components { all<IvyVariantDerivationRule>() }
}The rule creates an apiElements variant based on the compile configuration and a runtimeElements variant based on the default configuration of each ivy module.
For each variant, it sets the corresponding Java ecosystem attributes.
Dependencies and artifacts of the variants are taken from the underlying configurations.
If not all consumed ivy modules follow this pattern, the rule can be adjusted or only applied to a selected set of modules.
For all ivy modules without variants, Gradle falls back to legacy configuration selection (i.e. Gradle does not perform variant aware resolution for these modules).
This means either the default configuration or the configuration explicitly defined in the dependency to the corresponding module is selected.
(Note that explicit configuration selection is only possible from build scripts or ivy metadata, and should be avoided in favor of variant selection.)