grpc-core and grpc-context jars exporting the same package doesnt go well with OSGI #2727

mksriram · 2017-02-15T15:29:04Z

Please answer these questions before submitting your issue.

What version of gRPC are you using?

GRPC version 1.1.2

What JVM are you using (`java -version`)?

1.8

What did you do?

I am trying to add a GRPC Server into Karaf OSGI Container as a feature. To this library, grpc-core and grpc-context libraries are dependencies.

Because these jars are not OSGI bundles, the osgi wrap is performed. While running the application, we get an error that io.grpc.Context could not be found in bundle grpc-core.jar (NoClassDefFoundError)

For information please refer to the OSGI bundle definitions.

karaf@root()> bundle:headers 83

wrap_file__home_apache-karaf-4.0.8_data_kar_mvm-core-grpcapi-17.6.0-rel.3-SNAPSHOT_io_grpc_grpc-core_1.1.2_grpc-core-1.1.2.jar (83)

Bnd-LastModified = 1486762343975
Built-By = root
Built-JDK = 1.8.0_45
Created-By = 1.8.0_121 (Oracle Corporation)
Generated-By-Ops4j-Pax-From = wrap:file:/home/apache-karaf-4.0.8/data/kar/mvm-core-grpcapi-17.6.0-rel.3-SNAPSHOT/io/grpc/grpc-core/1.1.2/grpc-core-1.1.2.jar
Implementation-Title = grpc-core
Implementation-Version = 1.1.2
Manifest-Version = 1.0
Source-Compatibility = 1.6
Target-Compatibility = 1.6
Tool = Bnd-2.3.0.201405100607

Bundle-ManifestVersion = 2
Bundle-Name = wrap_file__home_apache-karaf-4.0.8_data_kar_mvm-core-grpcapi-17.6.0-rel.3-SNAPSHOT_io_grpc_grpc-core_1.1.2_grpc-core-1.1.2.jar
Bundle-SymbolicName = wrap_file__home_apache-karaf-4.0.8_data_kar_mvm-core-grpcapi-17.6.0-rel.3-SNAPSHOT_io_grpc_grpc-core_1.1.2_grpc-core-1.1.2.jar
Bundle-Version = 0
Require-Capability =
osgi.ee;filter:=(&(osgi.ee=JavaSE)(version=1.6))

Export-Package =
io.grpc;uses:="com.google.common.base,com.google.errorprone.annotations,javax.annotation,javax.net.ssl",
io.grpc.inprocess;uses:="com.google.instrumentation.stats,io.grpc,io.grpc.internal",
io.grpc.internal;uses:="com.google.common.base,com.google.instrumentation.stats,
io.grpc,javax.annotation",
io.grpc.util;uses:="io.grpc,javax.annotation"
Import-Package =
com.google.common.base;resolution:=optional,
com.google.common.io;resolution:=optional,
com.google.common.util.concurrent;resolution:=optional,
com.google.errorprone.annotations;resolution:=optional,
com.google.instrumentation.stats;resolution:=optional,
javax.annotation;resolution:=optional,
javax.net.ssl;resolution:=optional

karaf@root()> bundle:headers 82

wrap_file__home_apache-karaf-4.0.8_data_kar_mvm-core-grpcapi-17.6.0-rel.3-SNAPSHOT_io_grpc_grpc-context_1.1.2_grpc-context-1.1.2.jar (82)

Bnd-LastModified = 1486762343849
Built-By = root
Built-JDK = 1.8.0_45
Created-By = 1.8.0_121 (Oracle Corporation)
Generated-By-Ops4j-Pax-From = wrap:file:/home/apache-karaf-4.0.8/data/kar/mvm-core-grpcapi-17.6.0-rel.3-SNAPSHOT/io/grpc/grpc-context/1.1.2/grpc-context-1.1.2.jar
Implementation-Title = grpc-context
Implementation-Version = 1.1.2
Manifest-Version = 1.0
Source-Compatibility = 1.6
Target-Compatibility = 1.6
Tool = Bnd-2.3.0.201405100607

Bundle-ManifestVersion = 2
Bundle-Name = wrap_file__home_apache-karaf-4.0.8_data_kar_mvm-core-grpcapi-17.6.0-rel.3-SNAPSHOT_io_grpc_grpc-context_1.1.2_grpc-context-1.1.2.jar
Bundle-SymbolicName = wrap_file__home_apache-karaf-4.0.8_data_kar_mvm-core-grpcapi-17.6.0-rel.3-SNAPSHOT_io_grpc_grpc-context_1.1.2_grpc-context-1.1.2.jar
Bundle-Version = 0

Require-Capability =
osgi.ee;filter:=(&(osgi.ee=JavaSE)(version=1.6))

Export-Package =
io.grpc

As you can notice, the problem seems to be because of the same export-packages defined in grpc-core and grpc-context which doesnt seem to go well with OSGI.

I reverted my GRPC version to 1.0.0 in which the jars were broken down to core and context and it worked well.

So the requests are as below:

a) A single jar which includes both core and context
b) If context is intended to be separate, i suggest different packages be used in context so that there wont be a collision on the export-packages.

Is there a plan to create a OSGI bundle for these jars?

The text was updated successfully, but these errors were encountered:

ejona86 · 2017-02-15T21:12:06Z

We can't easily move Context to a different package, and splitting the jar was necessary to avoid circular dependencies and allow others outside of gRPC to use context.

I guess we could try to make new API, say, io.grpc.context with somehow provide compatibility between the two, but that will be quite involved and would involve some pain for users.

We have zero OSGi knowledge, such that I don't really know what is to be gained by providing a bundle.

CC @zhangkun83

mindcrime · 2017-04-07T03:05:54Z

I am running into this same problem, trying to run a gRPC service in ServiceMix (which is basically just Karaf). It would be a huge help if grpc provided proper osgi bundle jars, or at least get rid of this "dual export" problem so the wrap process can work.

Maybe one good option would be to provide a merged core/context jar as a separate download, that puts all of those classes into one jar. Otherwise, end users are going to have to repackage things themselves, if they are using osgi.

tjian123 · 2017-07-14T01:33:03Z

Hi, @mksriram @ejona86 and @mindcrime, I am also facing this problem, do you have any suggestions to solve it? I am using karaf-4.1.1 and grpc-*-1.4.0. Thanks!

tjian123 · 2017-07-14T01:39:05Z

I also tried the wrap of grpc-all, but it just had no Export-package.

wrap_mvn_io.grpc_grpc-all_1.4.0 (60)
------------------------------------
Bnd-LastModified = 1499995367999
Built-By = root
Built-JDK = 1.8.0_45
Created-By = 1.8.0_111 (Oracle Corporation)
Generated-By-Ops4j-Pax-From = wrap:mvn:io.grpc/grpc-all/1.4.0
Implementation-Title = grpc-all
Implementation-Version = 1.4.0
Manifest-Version = 1.0
Source-Compatibility = 1.6
Target-Compatibility = 1.6
Tool = Bnd-2.3.0.201405100607

Bundle-ManifestVersion = 2
Bundle-Name = wrap_mvn_io.grpc_grpc-all_1.4.0
Bundle-SymbolicName = wrap_mvn_io.grpc_grpc-all_1.4.0
Bundle-Version = 0

pieterjanpintens · 2017-09-04T06:43:32Z

Merging context and core worked for me. Now i,'m stuck on the service loader problem.

mksriram · 2017-09-04T07:23:14Z

Hi We did solve this issue by merging the jars. I am in vacation and don't have access to computer. I will be back on 12th and in case you still have problems, I can share the jar. Thanks Sriram

…

Sent from my iPhone

On Jul 14, 2017, at 7:03 AM, tjian123 ***@***.***> wrote: Hi, @mksriram @ejona86 and @mindcrime, I am also facing this problem, do you have any suggestions to solve it? I am using karaf-4.1.1 and grpc-*-1.4.0. Thanks! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

pieterjanpintens · 2017-09-04T11:23:06Z

I already have a working 'fat' jar file but it would be better if #3273 was fixed this way my osgi server app only needs to ship the jars that are really required. Apart from the context and core split package problem the plain grpc jars can be 'converted' to osgi bundles relatively easy. You will need to have spi fly and weaving as described in the other issue. I don't really mind merging core and context jars is as you probably need both anyway. Sure it would be better if they could live as separate bundles but I understand that this is not easy because sushi a refactor would not be very compatible with previous releases.

jeethridge · 2017-11-29T20:43:57Z

Would anyone be able to provide a repackaged jar (or DIY instructions)? I'm trying to use the latest java etcd driver (com.coreos.jetcd-core) in an OSGi project and I'm having the exact same issue because jetcd depends on grpc-1.5.0. I'm somewhat new to OSGi and I think I'm probably doing something wrong when repackaging grpc or writing the manifest.

mksriram · 2017-11-29T21:23:36Z

Hi James Which version of GRPC are you using. We recent upgraded our GRPC library version to 1.7.0 and protobuf to 3.4.0 // build.gradle def grpcVersion = '1.7.0' def protobufVersion = '3.4.0' compile "io.grpc:grpc-all:${grpcVersion}" compile "com.google.protobuf:protobuf-java:${protobufVersion}" We did not seem to hit this issue with this version. If you don't want to use a later version, i can pass on the instructions for building the jar. Thanks Sriram

…

On Wed, Nov 29, 2017 at 3:44 PM, James Ethridge ***@***.***> wrote: Would anyone be able to provide a repackaged jar (or DIY instructions)? I'm trying to use the latest java etcd driver (com.coreos.jetcd-core) in an OSGi project and I'm having the exact same issue because jetcd depends on grpc-1.5.0. I'm somewhat new to OSGi and I think I'm probably doing something wrong when repackaging grpc or writing the manifest. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2727 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQoI7bxH7wcvygOqydtiOpDcPyOpUA7Kks5s7cIdgaJpZM4MB2qR> .

mksriram · 2017-11-29T21:26:21Z

My bad, i see you are using 1.5.0. Will send u the instructions

jeethridge · 2017-11-29T22:48:58Z

@mksriram Thanks!

mksriram · 2017-11-30T14:48:40Z

I did the following when we were using 1.1.2 version of GRPC and this worked. Not sure if it would still work with 1.5.0. But here is the procedure.

Download the jar for grpc-core (v1.5.0) from https://mvnrepository.com/artifact/io.grpc/grpc-core/1.5.0
Download the jar for grpc-context (v1.5.0) from https://mvnrepository.com/artifact/io.grpc/grpc-context/1.5.0

In the downloads folder, create a new directory called grpc and copy the above two jars into the grpc folder.

Extract both the jars.

create the new jar as below
jar cvf osgi-grpc-core-context.jar io/ META-INF/

The new jar should be included in the dependencies rather than using grpc-core or grpc-context.

Now that we have a single jar, the osgi manifest for this jar would export io.grpc as against grpc-core and grpc-context jars exporting io.grpc.

Hope this helps.

jeethridge · 2017-12-04T16:50:54Z

@mksriram This fixed the problem I was having with loading dependencies on grpc v1.5.0. I haven't verified whether or not this is still an issue with any later versions. Thanks again.

baymaxhuang · 2018-01-31T14:00:02Z

I also faced the same problem when deploying gRPC in karaf container. At present I have a solution as follows:

First I use the maven-bundle-plugin and its Export-package instruction to merge the packages in grpc-core and grpc context:

<plugin>
    <groupId>org.apache.felix</groupId>
    <artifactId>maven-bundle-plugin</artifactId>
    <version>3.0.1</version>
    <extensions>true</extensions>
    <configuration>
        <instructions>
            <Export-Package>
                io.grpc;version="${grpc.package.version}",
                io.grpc.inprocess;version="${grpc.package.version}",
                io.grpc.internal;version="${grpc.package.version}",
                io.grpc.util;version="${grpc.package.version}"
            </Export-Package>
            <Import-Package>!com.google.errorprone.annotations,*</Import-Package>
        </instructions>
    </configuration>
</plugin>

Then the wrap deployer can be used to "hot deploy" non-OSGi jar files, edit feature.xml as follows:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>

<features xmlns="http://karaf.apache.org/xmlns/features/v1.2.0" name="${project.artifactId}-${project.version}">
    <feature name="${project.artifactId}" version="${project.version}"
             description="${project.description}">
        <feature version="${project.version}">grpc</feature>
        <bundle>mvn:com.google.protobuf/protobuf-java/${protobuf.version}</bundle>
        <bundle>mvn:${project.groupId}/${project.artifactId}/${project.version}</bundle>
    </feature>

    <feature name="grpc-netty" version="${project.version}"
             description="gRPC Netty dependencies">
        <bundle>mvn:io.netty/netty-common/${grpc.netty.version}</bundle>
        <bundle>mvn:io.netty/netty-buffer/${grpc.netty.version}</bundle>
        <bundle>mvn:io.netty/netty-transport/${grpc.netty.version}</bundle>
        <bundle>mvn:io.netty/netty-handler/${grpc.netty.version}</bundle>
        <bundle>mvn:io.netty/netty-codec/${grpc.netty.version}</bundle>
        <bundle>mvn:io.netty/netty-codec-http/${grpc.netty.version}</bundle>
        <bundle>mvn:io.netty/netty-codec-http2/${grpc.netty.version}</bundle>
        <bundle>mvn:io.netty/netty-resolver/${grpc.netty.version}</bundle>
    </feature>

    <feature name="grpc" version="${project.version}" description="gRPC dependencies">
        <feature version="${project.version}">grpc-netty</feature>
        <bundle>wrap:mvn:io.grpc/grpc-protobuf-lite/${grpc.version}$Bundle-SymbolicName=io.grpc.grpc-protobuf-lite&amp;Bundle-Version=${grpc.package.version}</bundle>
        <bundle>wrap:mvn:io.grpc/grpc-protobuf/${grpc.version}$Bundle-SymbolicName=io.grpc.grpc-protobuf&amp;Bundle-Version=${grpc.package.version}</bundle>
        <bundle>wrap:mvn:io.grpc/grpc-stub/${grpc.version}$Bundle-SymbolicName=io.grpc.grpc-stub&amp;Bundle-Version=${grpc.package.version}</bundle>
        <bundle>wrap:mvn:io.grpc/grpc-netty/${grpc.version}$Bundle-SymbolicName=io.grpc.grpc-netty&amp;Bundle-Version=${grpc.package.version}</bundle>
        <!--<bundle>wrap:mvn:io.grpc/grpc-auth/${grpc.version}$Bundle-SymbolicName=io.grpc.grpc-auth&amp;Bundle-Version=${grpc.package.version}&amp;Import-Package=!com.google.auth,*</bundle>-->
        <bundle>wrap:mvn:io.grpc/grpc-auth/${grpc.version}$Bundle-SymbolicName=io.grpc.grpc-auth&amp;Bundle-Version=${grpc.package.version}</bundle>
        <bundle>wrap:mvn:com.google.instrumentation/instrumentation-api/0.3.0$Bundle-SymbolicName=com.google.instrumentation.instrumentation-api&amp;Bundle-Version=${google.instrumentation.version}</bundle>
    </feature>
</features>

Refer to: gRPC demo for full application.

ejona86 · 2018-08-13T23:58:06Z

Note: the generated code also can run afoul of Java 9 modules. If nothing else, the gRPC generated code is frequently in a separate JAR from the protobuf generated code, but the two exist in the same package.

* gRPC classes * PB converters WIP * PB converters WIP * PB converters WIP * PB converters WIP * PB converters WIP * Hash fixes * Fixes * Fixes * Fixes * Fixes * Fixes * Fixes

ejona86 · 2023-11-25T16:19:01Z

This has been resolved since 1.57.0. grpc-context was merged in with grpc-api (which had been split from grpc-core much earlier).

zhangkun83 mentioned this issue Mar 22, 2017

Context in its own package? #2847

Open

ejona86 added this to the Unscheduled milestone Jul 27, 2017

ejona86 mentioned this issue Sep 11, 2018

How to use grpc in osgi environment now? #4822

Closed

ejona86 mentioned this issue Oct 12, 2018

gRPC Java is not usable from Java 9 modules #3522

Closed

ejona86 mentioned this issue Nov 20, 2018

Remove Census dependency from grpc-core #5076

Closed

ejona86 mentioned this issue Apr 15, 2019

Consider supporting Guava 19 in grpc-api #5592

Open

suadzh referenced this issue in wavesplatform/WavesJ May 24, 2019

gRPC classes (#33)

44b6413

* gRPC classes * PB converters WIP * PB converters WIP * PB converters WIP * PB converters WIP * PB converters WIP * Hash fixes * Fixes * Fixes * Fixes * Fixes * Fixes * Fixes

Leejjon mentioned this issue Jul 6, 2019

Using google-cloud-datastore dependency in modular Java 11 project gives error googleapis/google-cloud-java#5609

Closed

codefromthecrypt mentioned this issue Sep 29, 2019

remove grpc-context dependency from the api package open-telemetry/opentelemetry-java#575

Closed

ejona86 modified the milestones: Unscheduled, 1.57 Nov 25, 2023

ejona86 closed this as completed Nov 25, 2023

github-actions bot locked as resolved and limited conversation to collaborators Feb 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

grpc-core and grpc-context jars exporting the same package doesnt go well with OSGI #2727

grpc-core and grpc-context jars exporting the same package doesnt go well with OSGI #2727

grpc-core and grpc-context jars exporting the same package doesnt go well with OSGI #2727

grpc-core and grpc-context jars exporting the same package doesnt go well with OSGI #2727

Comments

What version of gRPC are you using?

What JVM are you using (java -version)?

What did you do?

wrap_file__home_apache-karaf-4.0.8_data_kar_mvm-core-grpcapi-17.6.0-rel.3-SNAPSHOT_io_grpc_grpc-core_1.1.2_grpc-core-1.1.2.jar (83)

wrap_file__home_apache-karaf-4.0.8_data_kar_mvm-core-grpcapi-17.6.0-rel.3-SNAPSHOT_io_grpc_grpc-context_1.1.2_grpc-context-1.1.2.jar (82)

What JVM are you using (`java -version`)?