Skip to content

InProcess: client streaming memory leak when retry enabled  #8712

Description

@systemfreund

What version of gRPC-Java are you using?

1.42.0

What is your environment?

Linux 5.15.2
openjdk version "11.0.13" 2021-10-19

What did you expect to see?

No java heap space exception.

What did you see instead?

java.lang.OutOfMemoryError: Java heap space

Steps to reproduce the bug

We've recently updated our grpc dependency from 1.39.0 to 1.42.0 which now causes an out of memory error in one of our tests.

I've attached a test project that can reproduce the problem.
grpcheap.zip

How to run:

mvn clean test -Dgrpc.version=1.39.0   # test runs successfully
mvn clean test -Dgrpc.version=1.42.0   # test fails with OOM

Please be aware that the test is run with limited heap size (see pom.xml): -Xmx64M

The general idea of the test is this:
Client-side has an InputStream that produces ~200mb data. This data of that stream is (lazily) split into ~1mb chunks and these are then streamed to the server.

What happens when I run the test with grpc >1.40 is that I get an OOM when it has sent ~60mb worth of chunks, which obviously is near the -Xmx limit. However, what I would expect is that the heap usage would never rise to that level, as after each 1mb chunk has been sent that chunk can be gc'ed. I've also attached a screenshot of visualvm where you can see the heap graph of the test using grpc 1.39.0 vs 1.42.0:

grpc

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions