Introduction
In this installment I'll try to explain concepts behind SNAPSHOT artifact handling in Maven and Aether. How this is done in pax-url-aether will be described in next part of the series.
SNAPSHOT concept seems simple, it may however be very confusing at the same time - especially if multiple repositories are involved or Aether is used as part of wider solution - like pax-url-aether or Apache Karaf.
Ideally we may start with these definitions:
- non-SNAPSHOT version (like e.g.,
1.2.0.RELEASE) never changes - each copy of an artifact with non-SNAPSHOT version should be exactly the same, no matter what repository does it come from - SNAPSHOT version (like e.g.,
1.2.0.BUILD-SNAPSHOT) may always change, so there's a need to verify if we may get newer build of the artifact.
This is usually obvious when dealing with 3rd party libraries and general rule is:
Do not use SNAPSHOT dependencies for 3rd party libraries when releasing a product containing such library
This general rule doesn't apply during development - before releasing a product, some 3rd party (or internal) dependency libraries may use SNAPSHOT versions. I this case it's helpful to know when exactly SNAPSHOT versions are updated, when Maven/Aether checks for new SNAPSHOT build and when it's actually downloaded and written as current SNAPSHOT version inside Maven's local repository.
I'd like to provide detailed information on the subject in this part of Maven in OSGi series.
Metadata
The appealing idea behind SNAPSHOTs is to make easier to just depend on latest/current version of dependency without specifying the exact version number.
To be able to achieve such goal, Maven uses the concept of metadata. Instead of referring to external documentation, I'll present the metadata concept by example.
Let's start with simplest Maven project, without any dependencies, using SNAPSHOT version.
<project
xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>grgr</groupId>
<artifactId>universalis-api</artifactId>
<version>0.1.0.BUILD-SNAPSHOT</version>
</project>
This POM declares exactly three things:
- groupId =
grgr - artifactId =
universalis-api - version =
0.1.0.BUILD-SNAPSHOT
There's no need to have any code inside such project. Let's see what happens when we simply install such project in
local repository by invoking mvn clean install.
$ mvn clean install
[INFO] Scanning for projects...
...
[INFO] --- maven-install-plugin:2.4:install (default-install) @ universalis-api ---
[INFO] Installing /data/ggrzybek/sources/_testing/grgr-universalis-api/target/universalis-api-0.1.0.BUILD-SNAPSHOT.jar ↵
to /home/ggrzybek/.m2/repository/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/universalis-api-0.1.0.BUILD-SNAPSHOT.jar
[INFO] Installing /data/ggrzybek/sources/_testing/grgr-universalis-api/pom.xml ↵
to /home/ggrzybek/.m2/repository/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/universalis-api-0.1.0.BUILD-SNAPSHOT.pom
...
I've extracted the information about maven-install-plugin execution, but in fact, the two above artifacts are not the only files
that are stored in
~/.m2/repository/grgr/universalis-api directory. Remaining artifacts are metadata files. Before describing them, we'll
mvn clean install again to see what has changed (git init; git add .; git commit trick inside
~/.m2/repository/grgr/universalis-api).
~/.m2/repository/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/_remote.repositories
This is Aether-specific file (we've described it shortly in first part of the series). It contains:
#NOTE: This is an Aether internal implementation file, its format can be changed without prior notice. #Tue Oct 18 09:29:14 CEST 2016 universalis-api-0.1.0.BUILD-SNAPSHOT.pom>= universalis-api-0.1.0.BUILD-SNAPSHOT.jar>=
This is
*.properties file with two properties. Only the keys are relevant, values are empty. By looking at this file, we can't say much, but
after looking at other _remote.repositories files in our ~/.m2/repository, we can say:
- It's written in
org.eclipse.aether.internal.impl.TrackingFileManager#update() - Keys are constructed with: file name +
>+ repository ID - Most
_remote.repositoriesfiles in~/.m2/repositorycontainxyz.jar>central=property, becausecentralrepository is used by default in Maven. - This file is one of the most important sources of confusion when we operate in multi repository environment. I'm pretty sure we've all seen the following:
[ERROR] Failed to execute goal on project universalis-api: Could not resolve dependencies for project grgr:universalis-api:jar:0.1.0.BUILD-SNAPSHOT: ↵ Failure to find commons-pool:commons-pool:jar:1.6.0.redhat-9 in https://repo.maven.apache.org/maven2 was cached in the local repository, ↵ resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]The version shown is arbitrary, but the problem is that Aether, when usingorg.eclipse.aether.internal.impl.EnhancedLocalRepositoryManager, not only checks if artifact is available locally - it also checks if it's downloaded from one of the repositories that are currently configured (in current Maven build or Aether session).
The above error occurs, because we don't use the same repository (that we've downloaded the artifact from originally) in current build. Maven, by default usesEnhancedLocalRepositoryManager, pax-url-aether usesSimpleLocalRepositoryManagerFactoryand doesn't have this problem. - Due to the above, it's important to carefully identify remote repositories used in parent POMs. To not end up with this:
$ find ~/.m2/repository -name _remote.repositories|xargs grep '>..'|cut -d '>' -f 2|sort|uniq ... apache-snapshots= apache.snapshots= ... ea= EA= ... jboss.public= jboss-public-repository= jboss-public-repository-group= ... ops4j.snapshot= ops4j-snapshots= ... sonatype= sonatype-nexus-snapshots= sonatype-snapshots= ...
~/.m2/repository/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/maven-metadata-local.xml
This file describes information about particular groupId:artifactId:version combination. For non-SNAPSHOT versions, we don't have this file created - there are simply no additional builds within single GAV where version is non-SNAPSHOT.
For SNAPSHOT versions however this file is used to list all different SNAPSHOT builds.
Here's the file after initial mvn clean install:
<?xml version="1.0" encoding="UTF-8"?>
<metadata modelVersion="1.1.0">
<groupId>grgr</groupId>
<artifactId>universalis-api</artifactId>
<version>0.1.0.BUILD-SNAPSHOT</version>
<versioning>
<snapshot>
<localCopy>true</localCopy>
</snapshot>
<lastUpdated>20161018072914</lastUpdated>
<snapshotVersions>
<snapshotVersion>
<extension>jar</extension>
<value>0.1.0.BUILD-SNAPSHOT</value>
<updated>20161018072914</updated>
</snapshotVersion>
<snapshotVersion>
<extension>pom</extension>
<value>0.1.0.BUILD-SNAPSHOT</value>
<updated>20161018072914</updated>
</snapshotVersion>
</snapshotVersions>
</versioning>
</metadata>
And here's the diff after 2nd mvn clean install:
--- a/0.1.0.BUILD-SNAPSHOT/maven-metadata-local.xml
+++ b/0.1.0.BUILD-SNAPSHOT/maven-metadata-local.xml
@@ -7,17 +7,17 @@
<snapshot>
<localCopy>true</localCopy>
</snapshot>
- <lastUpdated>20161018072914</lastUpdated>
+ <lastUpdated>20161018075358</lastUpdated>
<snapshotVersions>
<snapshotVersion>
<extension>jar</extension>
<value>0.1.0.BUILD-SNAPSHOT</value>
- <updated>20161018072914</updated>
+ <updated>20161018075358</updated>
</snapshotVersion>
<snapshotVersion>
<extension>pom</extension>
<value>0.1.0.BUILD-SNAPSHOT</value>
- <updated>20161018072914</updated>
+ <updated>20161018075358</updated>
</snapshotVersion>
</snapshotVersions>
</versioning>
Here's summary of important info about this file:
- There's only one build tracked. Each installed artifact (like
*.pomand*.jar) has its<snapshotVersion>. versioning/snapshot/localCopy = trueis an indication of local origin of the artifact.- There are timestamps for each artifact (and the metadata itself)
~/.m2/repository/grgr/universalis-api/maven-metadata-local.xml
This file describes information about particular groupId:artifactId across different versions. After several
mvn clean install invocations
for single SNAPSHOT versions, this file has only timestamp updated and it's neither particularly interesting nor complex:
<?xml version="1.0" encoding="UTF-8"?>
<metadata>
<groupId>grgr</groupId>
<artifactId>universalis-api</artifactId>
<versioning>
<versions>
<version>0.1.0.BUILD-SNAPSHOT</version>
</versions>
<lastUpdated>20161018075358</lastUpdated>
</versioning>
</metadata>
It becomes however more interesting after our simple project evolves and other versions are installed. After we
mvn clean install versions 0.1.0, 0.1.1, 0.2.0, 0.1.2
(in that order), we can see:
<?xml version="1.0" encoding="UTF-8"?>
<metadata>
<groupId>grgr</groupId>
<artifactId>universalis-api</artifactId>
<versioning>
<release>0.1.2</release>
<versions>
<version>0.1.0.BUILD-SNAPSHOT</version>
<version>0.1.0</version>
<version>0.1.1</version>
<version>0.2.0</version>
<version>0.1.2</version>
</versions>
<lastUpdated>20161018085259</lastUpdated>
</versioning>
</metadata>
- It's a kind of index of versions for particular combination of groupId:artifactId
versioning/releasemay not exactly point to latest version (more aboutLATEST… later).
Low level, canonical example
All right. Having the metadata background in mind, let's actually try to resolve this
grgr:universalis-api:0.1.0.BUILD-SNAPSHOT
artifact using Aether code.
(I've reverted back to the state where we had only 0.1.0.BUILD-SNAPSHOT available in
~/.m2/repository/grgr/universalis-api).
DefaultServiceLocator locator = MavenRepositorySystemUtils.newServiceLocator();
locator.setService(LocalRepositoryManagerFactory.class, SimpleLocalRepositoryManagerFactory.class);
locator.setService(org.eclipse.aether.spi.log.LoggerFactory.class, Slf4jLoggerFactory.class);
RepositorySystem system = locator.getService(RepositorySystem.class);
RepositorySystemSession session = MavenRepositorySystemUtils.newSession();
LocalRepositoryManager lrm = system.newLocalRepositoryManager(session, new LocalRepository("/home/ggrzybek/.m2/repository"));
((DefaultRepositorySystemSession)session).setLocalRepositoryManager(lrm);
ArtifactRequest req = new ArtifactRequest();
req.setArtifact(new DefaultArtifact("grgr", "universalis-api", "jar", "0.1.0.BUILD-SNAPSHOT"));
ArtifactResult res = system.resolveArtifact(session, req);
Nothing strange here. We're resolving an artifact without any reference to remote repositories - we expect to have SNAPSHOT artifact in local repository.
What Aether does internally?
org.eclipse.aether.resolution.VersionRequest is executed in version resolver. This request concerns
grgr:universalis-api:jar:0.1.0.BUILD-SNAPSHOT artifact. The result of version resolution may be
a change of version in original ArtifactRequest - usually after checking that remote SNAPSHOT metadata contains newer build than we already have.
If aether.versionResolver.noCache config option is not
true, Aether's session cache is consulted (this is more important in such cases as full Maven build).
If version is not a release version (i.e., it is equal to RELEASE,
LATEST or ends with SNAPSHOT), then
Aether executes another request (or collection of such requests) -
org.eclipse.aether.resolution.MetadataRequest.
MetadataRequest is prepared for local repository and for each remote repository added to
ArtifactRequest in our code (we've added none).
Metadata is resolved by org.eclipse.aether.impl.MetadataResolver
Only one
MetadataRequest - for local repository - is executed. This is handled by
org.eclipse.aether.internal.impl.SimpleLocalRepositoryManager#find() method.
For local repository,
grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/maven-metadata-local.xml file is examined. If we had remote repositories
added to initial ArtifactRequest,
grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/maven-metadata-central.xml (for repository with ID=central)
would have been checked as well.
For each metadata found (for local and remote repositories. In our case - only for local one), versions are read. Versions correspond to
/metadata/versioning XPath element from maven-metadata-<ID>.xml file.
Versions from each examined metadata file are merged to find
latest version (based on timestamp from metadata). Latest version
found from all MetadataResults is used as version in VersionResult
If aether.versionResolver.noCache config option is not
true, Aether's session cache is updated.
Version from VersionResult is set as the version of
Artifact inside our ArtifactRequest. In our case
nothing has changed - 0.1.0.BUILD-SNAPSHOT was left untouched. We'll see how it
may change when dealing with LATEST version
and with remote metadata later.
org.eclipse.aether.internal.impl.SimpleLocalRepositoryManager#find() is used again - this time to find locally available
grgr:universalis-api:jar:0.1.0.BUILD-SNAPSHOT artifact. Simple
java.io.File#isFile() call is enough to determine whether the artifact is
locally available. In case of EnhancedLocalRepositoryManager,
_remote.repositories file would be examined as well.
ArtifactResult is returned and its
artifact.file points to locally available artifact.
Here's important information about the above process:
- We didn't mention about policies, but it's important to be aware that in case of local repositories, no policies are enforced. Metadata about SNAPSHOTs is read unconditionally.
- No version translation happens here. It'd happen if we had remote repositories. (more about this later).
RELEASE and LATEST versions
To explain these concepts, let's mvn clean install some more versions of grgr:universalis-api. In order: 0.1.0, 0.1.1 and 0.2.0.BUILD-SNAPSHOT
Here's current grgr/universalis-api/maven-metadata-local.xml:
<?xml version="1.0" encoding="UTF-8"?>
<metadata>
<groupId>grgr</groupId>
<artifactId>universalis-api</artifactId>
<versioning>
<release>0.1.1</release>
<versions>
<version>0.1.0.BUILD-SNAPSHOT</version>
<version>0.1.0</version>
<version>0.1.1</version>
<version>0.2.0.BUILD-SNAPSHOT</version>
</versions>
<lastUpdated>20161018102447</lastUpdated>
</versioning>
</metadata>
We can try resolving grgr:universalis-api:RELEASE artifact:
ArtifactRequest req = new ArtifactRequest();
req.setArtifact(new DefaultArtifact("grgr", "universalis-api", "jar", "RELEASE"));
ArtifactResult res = system.resolveArtifact(session, req);
org.eclipse.aether.internal.impl.SimpleLocalRepositoryManager#find()is used by metadata resolver to fetch~/.m2/repository/grgr/universalis-api/maven-metadata-local.xmlfile - which is metadata per groupId:artifactId - for all versions./metadata/versioning/releaseand/metadata/versioning/latestare taken into account (in that order)ArtifactResultrefers to0.1.1version after resolution
We can also try resolving grgr:universalis-api:LATEST artifact:
ArtifactRequest req = new ArtifactRequest();
req.setArtifact(new DefaultArtifact("grgr", "universalis-api", "jar", "LATEST"));
ArtifactResult res = system.resolveArtifact(session, req);
~/.m2/repository/grgr/universalis-api/maven-metadata-local.xmlfile is read as above/metadata/versioning/releaseand/metadata/versioning/latestare taken into account (in that order)ArtifactResultrefers to0.1.1version after resolution...
Hey - what's the difference then between RELEASE and LATEST? None - at least when only local repositories are involved.
What's more, if we mvn clean install versions 0.2.0 and 0.1.2 (in that order), resolving grgr:universalis-api:LATEST (or RELEASE)
would result in … version 0.1.2 instead of 0.2.0.
Remote repositories
Let's start fresh and this time use remote repository. We can use Sonatype Nexus as our remote repository manager
(even if installed locally). We'll use mvn clean deploy after correct configuration of server credentials.
Our POM contains now:
<distributionManagement>
<repository>
<id>local-nexus</id>
<url>http://localhost:8081/nexus/content/repositories/releases</url>
</repository>
<snapshotRepository>
<id>local-nexus</id>
<url>http://localhost:8081/nexus/content/repositories/snapshots</url>
</snapshotRepository>
</distributionManagement>
Deploying a project:
$ mvn clean deploy
[INFO] Scanning for projects...
...
[INFO] --- maven-install-plugin:2.4:install (default-install) @ universalis-api ---
[INFO] Installing /data/ggrzybek/sources/_testing/grgr-universalis-api/target/universalis-api-0.1.0.BUILD-SNAPSHOT.jar ↵
to /home/ggrzybek/.m2/repository/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/universalis-api-0.1.0.BUILD-SNAPSHOT.jar
[INFO] Installing /data/ggrzybek/sources/_testing/grgr-universalis-api/pom.xml ↵
to /home/ggrzybek/.m2/repository/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/universalis-api-0.1.0.BUILD-SNAPSHOT.pom
...
[INFO] --- maven-deploy-plugin:2.7:deploy (default-deploy) @ universalis-api ---
Downloading: http://localhost:8081/nexus/content/repositories/snapshots/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/maven-metadata.xml
Uploading: http://localhost:8081/nexus/content/repositories/snapshots/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/universalis-api-0.1.0.BUILD-20161018.111155-1.jar
Uploaded: http://localhost:8081/nexus/content/repositories/snapshots/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/universalis-api-0.1.0.BUILD-20161018.111155-1.jar (3 KB at 12.8 KB/sec)
Uploading: http://localhost:8081/nexus/content/repositories/snapshots/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/universalis-api-0.1.0.BUILD-20161018.111155-1.pom
Uploaded: http://localhost:8081/nexus/content/repositories/snapshots/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/universalis-api-0.1.0.BUILD-20161018.111155-1.pom (1020 B at 15.8 KB/sec)
Downloading: http://localhost:8081/nexus/content/repositories/snapshots/grgr/universalis-api/maven-metadata.xml
Uploading: http://localhost:8081/nexus/content/repositories/snapshots/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/maven-metadata.xml
Uploaded: http://localhost:8081/nexus/content/repositories/snapshots/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/maven-metadata.xml (787 B at 19.2 KB/sec)
Uploading: http://localhost:8081/nexus/content/repositories/snapshots/grgr/universalis-api/maven-metadata.xml
Uploaded: http://localhost:8081/nexus/content/repositories/snapshots/grgr/universalis-api/maven-metadata.xml (285 B at 9.0 KB/sec)
...
maven-deploy-plugin:
- Downloads
maven-metadata.xmlfor groupId:artifactId:version - if Nexus is empty, none is found - Uploads
*.jarand*.pomartifacts withSNAPSHOTversion changed to<timestamp>-<build number> - Downloads
maven-metadata.xmlfor groupId:artifactId - if Nexus is empty, none is found - Uploads both groupId:artifactId and groupId:artifactId:version metadata - after changing them. This is important information - repository managers
don't generate metadata - it's Maven that does it (after examining currently deployed metadata - by previous
mvn deploy) and uploads changed version back to Nexus.
Example with remote repository
Let's run our example again - after removing local copy of SNAPSHOT. We'll try to resolve SNAPSHOT from remote repository.
DefaultServiceLocator locator = MavenRepositorySystemUtils.newServiceLocator();
locator.setService(RepositoryConnectorFactory.class, BasicRepositoryConnectorFactory.class);
locator.addService(TransporterFactory.class, HttpTransporterFactory.class);
locator.setService(LocalRepositoryManagerFactory.class, SimpleLocalRepositoryManagerFactory.class);
locator.setService(org.eclipse.aether.spi.log.LoggerFactory.class, Slf4jLoggerFactory.class);
RepositorySystem system = locator.getService(RepositorySystem.class);
RepositorySystemSession session = MavenRepositorySystemUtils.newSession();
LocalRepositoryManager lrm = system.newLocalRepositoryManager(session, new LocalRepository("/home/ggrzybek/.m2/repository"));
((DefaultRepositorySystemSession)session).setLocalRepositoryManager(lrm);
RemoteRepository.Builder b = new RemoteRepository.Builder("local-nexus", "default", "http://localhost:8081/nexus/content/repositories/snapshots");
RepositoryPolicy enabledPolicy = new RepositoryPolicy(true, RepositoryPolicy.UPDATE_POLICY_NEVER, RepositoryPolicy.CHECKSUM_POLICY_IGNORE);
RepositoryPolicy disabledPolicy = new RepositoryPolicy(false, RepositoryPolicy.UPDATE_POLICY_NEVER, RepositoryPolicy.CHECKSUM_POLICY_IGNORE);
b.setReleasePolicy(disabledPolicy);
b.setSnapshotPolicy(enabledPolicy);
ArtifactRequest req = new ArtifactRequest();
req.addRepository(b.build());
req.setArtifact(new DefaultArtifact("grgr", "universalis-api", "jar", "0.1.0.BUILD-SNAPSHOT"));
ArtifactResult res = system.resolveArtifact(session, req);
LOG.info("Result: " + res);
We've added one RemoteRepository to ArtifactRequest. This repository has two policies set:
- disabled releases
- enabled snapshots, ignoring checksum verification errors and no-update policy (more about this soon).
Assuming that there's currently no ~/.m2/repository/grgr/univrsalis-api directory, what are the additional operations
performed by Aether? Let's trace the metadata resolution fragment:
MetadataRequest is prepared for local repository and for each remote repository - we have two such requests now.
MetadataRequest for local repository uses grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/maven-metadata-local.xml path
- there's no such file.
MetadataRequest for remote repository uses grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/maven-metadata-local-nexus.xml path
(local-nexus is ID of our remote repository in pom.xml) - there's no such file (yet). This request has favorLocalRepository set to true.
For each remote repository, org.eclipse.aether.impl.UpdateCheck is started.
org.eclipse.aether.impl.UpdateCheck#policy is set to never (as we've requested).
org.eclipse.aether.impl.UpdateCheckManager is invoked to verify if update is needed
touch file is checked (~/.m2/repository/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/resolver-status.properties) - there's no such file (yet)
Because we don't have any metadata file available locally, the check is marked as required. Even if we've specified "never" as update policy.
For each required check, org.eclipse.aether.internal.impl.DefaultMetadataResolver.ResolveTask is scheduled.
RepositoryConnector is used to download metadata. After downloading, we have first file: ~/.m2/repository/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/maven-metadata-local-nexus.xml:
<?xml version="1.0" encoding="UTF-8"?>
<metadata modelVersion="1.1.0">
<groupId>grgr</groupId>
<artifactId>universalis-api</artifactId>
<version>0.1.0.BUILD-SNAPSHOT</version>
<versioning>
<snapshot>
<timestamp>20161018.111155</timestamp>
<buildNumber>1</buildNumber>
</snapshot>
<lastUpdated>20161018111155</lastUpdated>
<snapshotVersions>
<snapshotVersion>
<extension>jar</extension>
<value>0.1.0.BUILD-20161018.111155-1</value>
<updated>20161018111155</updated>
</snapshotVersion>
<snapshotVersion>
<extension>pom</extension>
<value>0.1.0.BUILD-20161018.111155-1</value>
<updated>20161018111155</updated>
</snapshotVersion>
</snapshotVersions>
</versioning>
</metadata>
touch file is updated. Aether writes the following properties to ~/.m2/repository/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/resolver-status.properties:
#NOTE: This is an Aether internal implementation file, its format can be changed without prior notice. #Tue Oct 18 14:12:00 CEST 2016 maven-metadata-local-nexus.xml.lastUpdated=1476792720594
For each metadata found (for local and remote repositories), versions are read. Versions correspond to
/metadata/versioning XPath element from maven-metadata-<ID>.xml file.
Versions from each examined metadata file are merged to find
latest version (based on timestamp from metadata). Latest version
found from all MetadataResults is used as version in VersionResult
Our version result is 0.1.0.BUILD-20161018.111155-1 - from remote metadata.
This time such artifact is not available locally (yet).
Artifact is downloaded.
If aether.artifactResolver.snapshotNormalization is true (which is default), downloaded
universalis-api-0.1.0.BUILD-20161018.111155-1.jar artifact is copied to universalis-api-0.1.0.BUILD-SNAPSHOT.jar as well.
Update policy
As we've seen, the key information required to resolve SNAPSHOT artifacts is contained in maven-metadata.xml. There are different scenarios now:
- Someone else may have deployed newer version of SNAPSHOT artifact to Remote repository
- We may have installed (
mvn clean install) newer version of SNAPSHOT ourselves.
With SNAPSHOTs, the two questions are:
- When (if at all)
maven-metadata.xmlis downloaded from remote repository? - When (if at all) actual artifact is downloaded from remote repository?
The answer is "it depends".
Let's assume that Nexus contains two builds of an artifact: 0.1.0.BUILD-20161018.111155-1 and 0.1.0.BUILD-20161018.124558-2:
- We may have no local version of remote
maven-metadata-local-nexus.xml. - We may have local version of remote
maven-metadata-local-nexus.xmlwhere0.1.0.BUILD-20161018.111155-1is the latest - older than remote. - We may have local version of remote
maven-metadata-local-nexus.xmlwhere0.1.0.BUILD-20161018.124558-2is the latest - up to date with remote.
We also may have local maven-metadata-local.xml describing locally mvn clean installed SNAPSHOTs.
Here are the rules for determining when remote maven-metadata.xml is downloaded:
- Initial resolution request (
ArtifactRequest) must have one or more remote repositories added. - Assuming we're resolving with remote repository having
local-nexusID, Aether will look formaven-metadata-local-nexus.xmlfile. - If there's no
~/.m2/repository/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/maven-metadata-local-nexus.xmlfile, it'll be downloaded. - If there's such file, its timestamp (
java.io.File#lastModified()) is verified against effective update policy:never- remote metadata won't be downloadedinterval:N- remote metadata will be downloaded if the timestamp is older than N minutesdaily- remote metadata will be downloaded if the timestamp is not created today (even if it was created a minute before midnight and it's 00:00:01 now).always- remote metadata will be downloaded
That's all! These rules are quite straightforward and clean. There's however another resolution performed by Aether - to determine which version will actually be used.
org.eclipse.aether.impl.VersionResolver uses org.eclipse.aether.impl.MetadataResolver. Metadata is gathered using the above rules. In simplest case,
when artifact resolution is performed with one remote repository, we have at most two metadata files:
~/.m2/repository/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/maven-metadata-local.xml- frommvn clean install, may not exist~/.m2/repository/grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/maven-metadata-local-nexus.xml- from remote repository, must exist, but may be not up to date with remote version
Now, Aether checks /metadata/snapshotVersions/snapshotVersion elements (case when at 11:11:55 artifact was deployed to nexus and at 12:10:33 we've invoked mvn clean install):
- In
maven-metadata-local.xml:<snapshotVersions> <snapshotVersion> <extension>jar</extension> <value>0.1.0.BUILD-SNAPSHOT</value> <updated>20161018121033</updated> </snapshotVersion> <snapshotVersion> <extension>pom</extension> <value>0.1.0.BUILD-SNAPSHOT</value> <updated>20161018121033</updated> </snapshotVersion> </snapshotVersions> - In
maven-metadata-local-nexus.xml:<snapshotVersions> <snapshotVersion> <extension>jar</extension> <value>0.1.0.BUILD-20161018.111155-1</value> <updated>20161018111155</updated> </snapshotVersion> <snapshotVersion> <extension>pom</extension> <value>0.1.0.BUILD-20161018.111155-1</value> <updated>20161018111155</updated> </snapshotVersion> </snapshotVersions>
What Aether does is simply comparing snapshotVersion/updated timestamps, so org.eclipse.aether.impl.VersionResolver after
examining all MetadataResults, returns latest version, which is 0.1.0.BUILD-SNAPSHOT. VersionResult has:
- versionResult.version = 0.1.0.BUILD-SNAPSHOT
- versionResult.repository = "/home/ggrzybek/.m2/repository (simple)" (
LocalRepository)
Let's now force download of current remote metadata (using e.g., UPDATE_POLICY_ALWAYS), to get:
<snapshotVersions>
<snapshotVersion>
<extension>jar</extension>
<value>0.1.0.BUILD-20161018.124558-2</value>
<updated>20161018124558</updated>
</snapshotVersion>
<snapshotVersion>
<extension>pom</extension>
<value>0.1.0.BUILD-20161018.124558-2</value>
<updated>20161018124558</updated>
</snapshotVersion>
</snapshotVersions>
Now, after examining all MetadataResults, VersionResult has:
- versionResult.version = 0.1.0.BUILD-20161018.124558-2
- versionResult.repository = "local-nexus (http://localhost:8081/nexus/content/repositories/snapshots, default, snapshots)" (
RemoteRepository)
A quick summary now - so far, update policy was used to determine whether to download remote version of maven-metadata-local-nexus.xml and entire resolution
process updates requested version inside ArtifactResult.artifact.version.
After examining all available metadata files, resolution process changed our initial GAV from grgr:universalis-api:jar:0.1.0.BUILD-SNAPSHOT
to grgr:universalis-api:jar:0.1.0.BUILD-20161018.124558-2. Now usual resolution process continues:
- local repository is checked if it contains
grgr/universalis-api/0.1.0.BUILD-SNAPSHOT/universalis-api-0.1.0.BUILD-20161018.124558-2.jar- it doesn't (yet) - artifact is scheduled to be downloaded. It's written to
universalis-api-0.1.0.BUILD-20161018.124558-2.jar, and copied (becauseaether.artifactResolver.snapshotNormalizationistrue) touniversalis-api-0.1.0.BUILD-SNAPSHOT.jar.
Update policy is not used at all for actual artifacts! It's used only for metadata.
Summary
Whew. It was long material. Let's summarize SNAPSHOT handling with Aether:
- If resolution is performed without remote repository, local SNAPSHOT artifact must be available.
- If resolution is performed with remote repository, update policy is taken into account.
- update policy is used to determine whether metadata from remote repositories is downloaded.
- After possible download of metadata we have following
maven-metadata-<ID>.xmlfiles:- one for each remote repository used
- possibly one related to locally
mvn clean installed SNAPSHOT
- Using timestamps, newest SNAPSHOT version is passed for further artifact resolution
- if latest SNAPSHOT comes from one of remote metadata files,
SNAPSHOTis translated to<timestamp>-<build number>. - if latest SNAPSHOT comes from local metadata file, version is unchanged
- if latest SNAPSHOT comes from one of remote metadata files,
- The effect of version resolution and metadata resolution is possible change of version for pending artifact resolution.
- If translated artifact coordinates (groupId and artifactId don't change, version may be changed) describe locally available artifact, it's returned.
- If the artifact is not available, it's being downloaded from repository which was related to metadata file that declared latest SNAPSHOT.
- Update policy is not used at all for artifact download.
In next installment I'll describe how update policies may be configured with pax-url-aether.