PubSub emulator exhibiting failures with Subscription JSON payload

[bilisie]

Hi,

I'm testing an emulator locally using a json payload as such:

curl -H "Content-Type: application/json" -X PUT  -d '{ "name": "projects\/sample-project\/subscriptions\/subscription-localhost-2017-06-28T12-25-43-55cd2ef3-edb2-4484-9d13-55004416057e", "topic": "projects\/sample-project\/topics\/sample-topic" }' http://localhost:8080/v1/projects/sample-project/subscriptions/subscription-localhost-2017-06-28T13-09-08-88285d03-fcef-48b8-b851-a27449a47812

On the emulator side the following stack trace is being produced:

Jun 28, 2017 1:15:54 PM io.gapi.emulators.grpc.HttpAdapter$UnaryMethodHandler handle
WARNING: Failed to convert request to message: Field google.pubsub.v1.Subscription.name has already been set.
com.google.protobuf.InvalidProtocolBufferException: Field google.pubsub.v1.Subscription.name has already been set.
	at io.gapi.emulators.grpc.JsonFormat$ParserImpl.mergeField(JsonFormat.java:1383)
	at io.gapi.emulators.grpc.JsonFormat$ParserImpl.mergeMessage(JsonFormat.java:1239)
	at io.gapi.emulators.grpc.JsonFormat$ParserImpl.merge(JsonFormat.java:1197)
	at io.gapi.emulators.grpc.JsonFormat$ParserImpl.merge(JsonFormat.java:1079)
	at io.gapi.emulators.grpc.JsonFormat$Parser.merge(JsonFormat.java:283)
	at io.gapi.emulators.grpc.HttpJsonAdapter.merge(HttpJsonAdapter.java:61)
	at io.gapi.emulators.grpc.HttpAdapter$UnaryMethodHandler.handle(HttpAdapter.java:466)
	at io.gapi.emulators.grpc.HttpAdapter.handleRequest(HttpAdapter.java:165)
	at io.gapi.emulators.netty.HttpHandler.channelRead(HttpHandler.java:52)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:39)
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:364)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
	at java.lang.Thread.run(Thread.java:748)

Jun 28, 2017 1:15:54 PM io.gapi.emulators.netty.HttpHandler$1 onError
INFO: Exception when handling request: INVALID_ARGUMENT: Payload isn't valid for request.

The pattern of usage of using json payloads does function properly against real endpoints.

[hbchai]

Hi @bilisie, one interesting feature of your request is that you are essentially specifying the name of the subscription to be created twice; that is first in the URL, and second in the request body as "name". Moreover, the two values differ in your example curl command (although I'm assuming that's a copy-paste error).

You are actually correct that it's not considered an error for the value to be specified twice (I just verified this by testing against the live service - the specification is a bit ambiguous about this, see https://github.com/googleapis/googleapis/blob/master/google/api/http.proto#L206), so the emulator behavior is indeed different, and we should fix that. However, note that the name specified in the request body is being ignored. You can try using a different value against the live system as a test; you should see that the name of the subscription that is created comes from the URL.

Thanks for reporting this, BTW.

[bilisie]

Hi @hbchai

Thanks for the update. I've added better reproduction of the issue here:
https://github.com/bilisie/pubsub-emulator-issue

Removing the name from the URL breaks the integration tests against real PubSub as I've tested earlier. I can also try removing it from the payload or leaving it the URL to see if that satisfied both the emulator and prod.

[hbchai]

The subscription name definitely needs to be specified in the URL. Please do try removing it from the payload.

[garrettjonesgoogle]

This isn't an issue in google-cloud-java, and it looks like there is a workaround, so I am going to close it out now.