Introduction
In
previous installment I've described the foundation block of Maven -
Eclipse Aether library. I've mentioned declarative usage of Maven using
mvn: URI scheme.
In Karaf 4:
karaf@root()> bundle:install mvn:commons-io/commons-io/2.5 Bundle ID: 52or in JBoss Fuse:
JBossFuse:karaf@root> osgi:install mvn:commons-io/commons-io/2.5 Bundle ID: 295
In this part I'll describe how Maven and Aether library are used in Karaf 4 and in JBoss Fuse 6.3.x (standalone mode). I'll leave JBoss Fuse fabric mode for next installment.
Let's recall some fundamental concepts:
- local repository - accessed by Aether with the help of
org.eclipse.aether.repository.LocalRepositoryManagerinterface andorg.eclipse.aether.repository.LocalRepositoryclass. Effectively local repository is a wrapper for locally accessible filesystem directory that follows specific structure (organization of Maven artifacts). - remote repository - accessed by Aether with the help of
org.eclipse.aether.repository.RemoteRepositoryinterface. Effectively remote repository is a wrapper forURI, a set of policies related to snapshot/release versions plus proxy, mirroring and authentication information.
pax-url-aether
The above commands rely on mvn: protocol implementation of
java.net.URLStreamHandler interface. This implementation is provided
by
OPS4J PAX URL Aether library and uses Eclipse Aether to handle Maven artifact resolution.
When pax-url-aether bundle is installed and active in OSGi framework, it provides OSGi service with
org.ops4j.pax.url.mvn.MavenResolver interface.
In Karaf 4:
karaf@root()> bundle:services -p 4 OPS4J Pax Url - aether: (4) provides: ------------------------------------- objectClass = [org.osgi.service.cm.ManagedService] service.bundleid = 4 service.id = 9 service.pid = org.ops4j.pax.url.mvn service.scope = singleton ---- objectClass = [org.osgi.service.url.URLStreamHandlerService] service.bundleid = 4 service.id = 10 service.scope = singleton url.handler.protocol = mvn ---- objectClass = [org.ops4j.pax.url.mvn.MavenResolver] service.bundleid = 4 service.id = 19 service.scope = singleton
In JBoss Fuse:
JBossFuse:karaf@root> ls 4 You are about to access system bundle 4. Do you wish to continue (yes/no): yes OPS4J Pax Url - aether: (4) provides: ------------------------------------- objectClass = [org.osgi.service.cm.ManagedService] service.id = 8 service.pid = org.ops4j.pax.url.mvn ---- objectClass = [org.osgi.service.url.URLStreamHandlerService] service.id = 9 url.handler.protocol = mvn ---- objectClass = [org.ops4j.pax.url.mvn.MavenResolver] service.id = 21
Configuration
This time I won't show any code examples. We don't need them. As with most of other OSGi services we can configure how pax-url-aether uses Aether with Configuration Admin service. Describing Configuration Admin is a good task for another blog post. There are some articles about Configuration Admin and if something's not clear, please refer to original documentation.
To configure org.ops4j.pax.url.mvn.MavenResolver service (the implementation is
org.ops4j.pax.url.mvn.internal.AetherBasedResolver) we
use
org.ops4j.pax.url.mvn PID (persistent identifier). We can configure this PID manually using Configuration Admin API or using
etc/org.ops4j.pax.url.mvn.cfg file.
Internally, pax-url-aether uses
org.ops4j.pax.url.mvn.internal.config.MavenConfigurationImpl object. It contains information used by
org.ops4j.pax.url.mvn.internal.AetherBasedResolver.
Let's see what's the default configuration (sightly formatted) provided by Karaf 4 (/work directory is a docker mount volume):
karaf@root()> property-list --pid org.ops4j.pax.url.mvn
felix.fileinstall.filename = file:/work/etc/org.ops4j.pax.url.mvn.cfg
org.ops4j.pax.url.mvn.defaultRepositories = \
file:/work/system@id=system.repository@snapshots, \
file:/work/data/kar@id=kar.repository@multi@snapshots, \
file:/work/system@id=child.system.repository@snapshots
org.ops4j.pax.url.mvn.repositories = \
http://repo1.maven.org/maven2@id=central, \
http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, \
http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, \
http://zodiac.springsource.com/maven/bundles/release@id=gemini, \
http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, \
https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, \
https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases, \
http://repository.springsource.com/maven/bundles/external@id=spring-ebr-repository@snapshots@noreleases
org.ops4j.pax.url.mvn.useFallbackRepositories = false
service.pid = org.ops4j.pax.url.mvn
And JBoss Fuse:
JBossFuse:karaf@root> config:proplist --pid org.ops4j.pax.url.mvn
felix.fileinstall.filename = file:/data/servers/jboss-fuse-6.3.0.redhat-145/etc/org.ops4j.pax.url.mvn.cfg
org.ops4j.pax.url.mvn.defaultRepositories = \
file:/data/servers/jboss-fuse-6.3.0.redhat-145/system@snapshots@id=karaf.system,\
file:/home/ggrzybek/.m2/repository@snapshots@id=local,\
file:/data/servers/jboss-fuse-6.3.0.redhat-145/local-repo@snapshots@id=karaf.local-repo,\
file:/data/servers/jboss-fuse-6.3.0.redhat-145/system@snapshots@id=child.karaf.system
org.ops4j.pax.url.mvn.globalChecksumPolicy = warn
org.ops4j.pax.url.mvn.globalUpdatePolicy = daily
org.ops4j.pax.url.mvn.localRepository = /data/servers/jboss-fuse-6.3.0.redhat-145/data/repository
org.ops4j.pax.url.mvn.repositories = \
http://repo1.maven.org/maven2@id=maven.central.repo, \
https://maven.repository.redhat.com/ga@id=redhat.ga.repo, \
https://maven.repository.redhat.com/earlyaccess/all@id=redhat.ea.repo, \
https://repository.jboss.org/nexus/content/groups/ea@id=fuseearlyaccess
org.ops4j.pax.url.mvn.settings = /data/servers/jboss-fuse-6.3.0.redhat-145/etc/maven-settings.xml
org.ops4j.pax.url.mvn.useFallbackRepositories = false
service.pid = org.ops4j.pax.url.mvn
The above configurations differ a bit. JBoss Fuse provides more explicit configuration. Let's describe each used (and assumed) properties
that can be used to configure pax-url-aether (org.ops4j.pax.url.mvn.internal.AetherBasedResolver).
- org.ops4j.pax.url.mvn.defaultRepositories
this is a list of local repositories searched for an artifact in the first phase of artifact resolution. This repository should not contain URIs other than
file://-based. Each repository from this list is treated as local repository. pax-url-aether iterates over this list and checks one repository at a time. If neither location contains the artifact being resolved, pax-url-aether switches to second phase that involves remote repositories.
Also these repositories do not require write access - Aether doesn't write any files there.Access to these repositories can be presented using the following code (did I promise not to show any code example? sorry...):
RepositorySystem system = locator.getService(RepositorySystem.class); RepositorySystemSession session = MavenRepositorySystemUtils.newSession(); String basedir = singleRepositoryFromListOfDefaultRepositories; ((DefaultRepositorySystemSession)session).setLocalRepositoryManager(system.newLocalRepositoryManager(session, new LocalRepository(basedir))); ArtifactRequest req = new ArtifactRequest(); req.setArtifact(new DefaultArtifact("commons-io", "commons-io", "jar", "2.5")); ArtifactResult res = system.resolveArtifact(session, req);We don't invoke any
req.addRepository(repositoryBuilder.build());, so Aether doesn't try to go to any external location.Line 4 shows that we're trying one of repositories from
org.ops4j.pax.url.mvn.defaultRepositoriesat a time. Each such local repository is checked independently.- org.ops4j.pax.url.mvn.repositories
this is a list of remote repositories searched for an artifact in the second phase of artifact resolution. This repository may contain URIs with
file:scheme, but it's better to add such repositories to org.ops4j.pax.url.mvn.defaultRepositories. Each repository is accessed using some configured connector (pax-url-aether uses a connector that underneath invokes httpclient 4.x library - that's why configuringorg.apache.http.headerslogger may be a good idea)- org.ops4j.pax.url.mvn.localRepository
this is a local repository that supports Aether in second phase of artifact resolution. Its role is a bit different than the role of org.ops4j.pax.url.mvn.defaultRepositories. When Aether actually resolves artifact in one of the remote repositories, it stores the downloaded artifact to org.ops4j.pax.url.mvn.localRepository. That's why write access is required for this location.
If not specified, this property defaults to:${user.home}/.m2/repository! Be aware of this if Aether seems to find artifacts that are not expected.Second phase of artifact resolution can be presented using the following code (I love clean code!):
RepositorySystem system = locator.getService(RepositorySystem.class); RepositorySystemSession session = MavenRepositorySystemUtils.newSession(); String basedir = localRepository; ((DefaultRepositorySystemSession)session).setLocalRepositoryManager(system.newLocalRepositoryManager(session, new LocalRepository(basedir))); ArtifactRequest req = new ArtifactRequest(); req.addRepository(new RemoteRepository.Builder("ID1", "default", "http://uri1").build()); req.addRepository(new RemoteRepository.Builder("ID2", "default", "http://uri2").build()); req.setArtifact(new DefaultArtifact("commons-io", "commons-io", "jar", "2.5")); ArtifactResult res = system.resolveArtifact(session, req);Here, for each remote repository from the list of URIs in org.ops4j.pax.url.mvn.repositories property we call
org.eclipse.aether.resolution.ArtifactRequest.addRepository().Also, in line 4 we use local repository from org.ops4j.pax.url.mvn.localRepository property - the same local repository is used for each remote repository being searched.
- org.ops4j.pax.url.mvn.useFallbackRepositories
If
true, then Aether will always use http://repo1.maven.org/maven2 repository in addition to any remote repositories specified. I prefer explicit declaration of Maven Central repository (if needed), so it's better to sayfalsehere.- org.ops4j.pax.url.mvn.settings
-
Ah, big topic here. We can specify an explicit location of an XML document following Maven Settings XML Schema.
If not specified, pax-url-aether searches for settings file in the following locations:${user.home}/.m2/settings.xml${maven.home}/conf/settings.xml$M2_HOME/conf/settings.xml
In Karaf 4, implicit location is used (most probably
${user.home}/.m2/settings.xml). In JBoss Fuse, explicit${karaf.etc}/maven-settings.xmlvalue is configured and default, commented template is shipped.Why specify custom
settings.xmlfile, when we have properties such asorg.ops4j.pax.url.mvn.repositories? There are few things that can be specified only there:- HTTP proxies
- custom HTTP headers added when accessing particular remote repositories
Here's the example of HTTP proxy configuration:
<!-- This is the place to configure http proxies used by Aether. If there's no proxy for "https" protocol, proxy for "http" will be used when accessing remote repository --> <proxies> <proxy> <id>proxy</id> <host>127.0.0.1</host> <port>3128</port> <protocol>http</protocol> <username></username> <password></password> <nonProxyHosts>127.0.0.*|*.repository.corp</nonProxyHosts> </proxy> </proxies>And here's the example of specifying custom HTTP headers:
<!-- pax-url-aether may use the below configuration to add custom HTTP headers when accessing remote repositories with a given identifier --> <servers> <server> <id>maven.central.repo</id> <configuration> <httpHeaders> <httpHeader> <name>User-Agent</name> <value>Karaf</value> </httpHeader> <httpHeader> <name>Secret-Header</name> <value>secret_value</value> </httpHeader> </httpHeaders> </configuration> </server> </servers>With custom headers specification, we can see these in logs when accessing repository with ID=
maven.central.repo(see below for repository URI specification):17:30:44,590 | DEBUG | ... | http-outgoing-0 >> GET /maven2/commons-io/commons-io/2.7/commons-io-2.7.jar HTTP/1.1 17:30:44,590 | DEBUG | ... | http-outgoing-0 >> Cache-control: no-cache 17:30:44,590 | DEBUG | ... | http-outgoing-0 >> Cache-store: no-store 17:30:44,590 | DEBUG | ... | http-outgoing-0 >> Pragma: no-cache 17:30:44,591 | DEBUG | ... | http-outgoing-0 >> Expires: 0 17:30:44,591 | DEBUG | ... | http-outgoing-0 >> Accept-Encoding: gzip 17:30:44,591 | DEBUG | ... | http-outgoing-0 >> User-Agent: Karaf 17:30:44,591 | DEBUG | ... | http-outgoing-0 >> Secret-Header: secret_value 17:30:44,591 | DEBUG | ... | http-outgoing-0 >> Host: repo1.maven.org
- org.ops4j.pax.url.mvn.repositories - again
After describing org.ops4j.pax.url.mvn.settings let's get back for a moment to org.ops4j.pax.url.mvn.repositories.
If a list of remote repositories in org.ops4j.pax.url.mvn.repositories is prefixed with
+sign, all repositories available in all active profiles defined insettings.xmlfile are appended to effective list of remote repositories searched.For example if we have this:
org.ops4j.pax.url.mvn.repositories= \ +http://repo1.maven.org/maven2@id=maven.central.repoAnd this in
settings.xml:<!-- If org.ops4j.pax.url.mvn.repositories property is _prepended_ with '+' sign, repositories from all active profiles will be _appended_ to the list of searched remote repositories --> <profiles> <profile> <id>default</id> <repositories> <repository> <id>private.repository</id> <url>http://localhost:8181/maven-repository</url> </repository> </repositories> </profile> </profiles> <activeProfiles> <activeProfile>default</activeProfile> </activeProfiles>We can see this in logs during sample resolution:
18:10:17,734 | DEBUG | ... | Using transporter WagonTransporter with priority -1.0 for http://repo1.maven.org/maven2/ 18:10:17,736 | DEBUG | ... | Using connector BasicRepositoryConnector with priority 0.0 for http://repo1.maven.org/maven2/ 18:10:17,800 | DEBUG | ... | http-outgoing-8 >> GET /maven2/commons-io/commons-io/2.7/commons-io-2.7.jar HTTP/1.1 18:10:17,802 | DEBUG | ... | http-outgoing-8 >> Host: repo1.maven.org ... 18:10:17,872 | DEBUG | ... | Using transporter WagonTransporter with priority -1.0 for http://localhost:8181/maven-repository/ 18:10:17,873 | DEBUG | ... | Using connector BasicRepositoryConnector with priority 0.0 for http://localhost:8181/maven-repository/ 18:10:17,875 | DEBUG | ... | http-outgoing-9 >> GET /maven-repository/commons-io/commons-io/2.7/commons-io-2.7.jar HTTP/1.1 18:10:17,876 | DEBUG | ... | http-outgoing-9 >> Host: localhost:8181 ...
- org.ops4j.pax.url.mvn.globalChecksumPolicy
When Aether fetches artifact from remote repository, it always tries to download SHA1/MD5 checksum for the artifact. It may fail to do so. If repository URI doesn't specify per-repository value, this global property's value is used. Actually if this global value is specified, per-repository values are ignored.
This property may have 3 values determining Aether's behavior:
fail- resolution failswarn- information is printed at WARN levelignore- nothing happens.
Note that there's no way to prevent fetching checksums.
- org.ops4j.pax.url.mvn.globalUpdatePolicy
When Aether fetches SNAPSHOT artifacts, it needs to fetch
maven-metadata.xmlfirst. Before hitting org.ops4j.pax.url.mvn.repositories, Aether checks the presence ofresolver-status.propertiesfile in org.ops4j.pax.url.mvn.localRepository location (this status file is specific to given groupId, artifactId and version, for example:<REPOSITORY>/commons-io/commons-io/2.5-SNAPSHOT/resolver-status.properties). We can control whether Aether actually should refresh metadata information:always- Aether always fetchesmaven-metadata.xmlwhen resolving SNAPSHOTsnever- opposite of the abovedaily- Aether fetchesmaven-metadata.xmlif a day passed since timestamp written inmaven-metadata-ID_OF_REPOSITORY.xml.lastUpdatedproperty insideresolver-status.propertiesfile.interval:<NUMBER_OF_MINUTES>- Aether fetchesmaven-metadata.xmlif given number of minutes passed.
Maven Repository URI
I've mentioned repository URI in few places. When specifying URI on a org.ops4j.pax.url.mvn.repositories list, we may use the following format:
http(s)://host:port/path@snapshots@noreleases@id=ID@other_options
Options that may be specified are:
id=ID- this option may (should) be specified to identify a repository. We may then refer to the repository for example when specifying custom headers.snapshots- whether the repository should be used when resolving SNAPSHOT artifactsnoreleases- whether the repository should not be used when resolving non-SNAPSHOT artifactsreleasesUpdate=daily|never|always|interval:MINUTES- see description of org.ops4j.pax.url.mvn.globalUpdatePolicy propertysnapshotsUpdate=daily|never|always|interval:MINUTES- see description of org.ops4j.pax.url.mvn.globalUpdatePolicy propertyupdate=daily|never|always|interval:MINUTES- see description of org.ops4j.pax.url.mvn.globalUpdatePolicy propertyreleasesChecksum=fail|warn|ignore- see description of org.ops4j.pax.url.mvn.globalChecksumPolicy propertysnapshotsChecksum=fail|warn|ignore- see description of org.ops4j.pax.url.mvn.globalChecksumPolicy propertychecksum=fail|warn|ignore- see description of org.ops4j.pax.url.mvn.globalChecksumPolicy property
Other options
There are other properties that can be configured in org.ops4j.pax.url.mvn PID:
- org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote
- Whether local repository specified in org.ops4j.pax.url.mvn.localRepository should be added as first remote repository inserted to the list configured with org.ops4j.pax.url.mvn.repositories property - it's a bad idea...
Caveats
Due to highly asynchronous nature of OSGi™ (and in particular - a slight race condition between pax-url-aether that configures org.ops4j.pax.url.mvn.MavenResolver service on one side
and felix.fileinstall and felix.configadmin bundles that create configuration for org.ops4j.pax.url.mvn PID on other side), there's short period of time where other configuration may
be used in org.ops4j.pax.url.mvn.MavenResolver service.
To prevent such interregnum, I suggest duplicating Maven/Aether properties from ${karaf.etc}/org.ops4j.pax.url.mvn.cfg in ${karaf.etc}/config.properties. If pax-url-aether can't find ConfigurationAdmin (yet), it defaults to bundle properties and these may be specified in etc/config.properties.
Summary
I hope the above information will clear all confusion related to pax-url-aether configuration in OSGi framework as JBoss Fuse or Karaf.