[Nymphium]

This PR adds Metainfo module to each generated files.
That module has information to implement gRPC reflection API:

• file name: its source proto file name path
• file descriptor: protobuf binary format of the file
• service list: list of services of the file

[andersfugmann]

Thanks. LGTM. I've have one question on naming though.

Also before we merge, could you give a simple example of all the metadata created around service endpoints for gRPC? I'd like to understand if more fields could be moved into this structure to clean up and hopefully it would also be easier to add additional fields going forward.

[andersfugmann]

Does metainfo have a special definition in the gRPC standard? If not, then I think it would be better to call it something like Service_info

[Nymphium]

No it may not be a standard. I agree with Service_info 👍

[Nymphium]

@andersfugmann
Here is an example of the generated service info and how reflection is utilized:
Note that fd_of_service is used here to bundle the generated artifacts for the reflection service.

https://github.com/Nymphium/oresai/blob/main/server/protos/generated/grpc_reflection_v1_reflection.ml#L2025
https://github.com/Nymphium/oresai/blob/main/server/interface/grpc/services/grpc/reflection/v1/reflection.ml
https://github.com/Nymphium/oresai/blob/main/server/protos/gen_protos.ml

[andersfugmann]

Thanks for providing examples.
file_descriptor proto can be quite large, and is only needed for gRPC. I dont think that should be included by default. Would it make sense to add a flag to the generation to enable gRPC relevant fields when needed?

Would it be possible to make the list of endpoints be a list of modules instead to avoid string duplication?
So instead have

let package_service_names = 
[ (module Grpc.Reflection.V1.ServerReflection : Rpc); ... ]

Should hold the same (if not more) information and avoid string duplication.

Btw. Since this information is only needed for gRPC, maybe call the module Grpc_info.

[Nymphium]

@andersfugmann I've applied the changes in 4fb8d6e.

Would it make sense to add a flag to the generation to enable gRPC relevant fields when needed?

I addeed service_info flag and false by default 👍

Would it be possible to make the list of endpoints be a list of modules instead to avoid string duplication?

It's difficult to achieve that in some cases. protoc invokes plugins multiple times with different target protos, and these plugin processes run independently.
That said, it might be possible if we split the service info and other specs into separate files.

[andersfugmann]

Looks very good. Thanks for making the updates.

[andersfugmann]

It's difficult to achieve that in some cases. protoc invokes plugins multiple times with different target protos, and these plugin processes run independently. That said, it might be possible if we split the service info and other specs into separate files.

I'm not sure I follow. The service names are already collected as part of emitting the signature and implementation of all messages and services defined in the proto file.

I may take a stab at implementing that, but I would prefer to the two other comments + tests addressed first.

Again, Thanks for doing this work. Really appreciated!

[Nymphium]

@andersfugmann I've updated the code based on your suggestions and added tests 🎍

[andersfugmann]

Hi. I took a closer look at this a while ago.
My first approach was that it would be much simpler to just include the complete Service_info struct as an ocaml value (no need to hold the binary information and then deserialize later).

This should remove the need of a lot of fields currently used for gRPC. See https://github.com/andersfugmann/ocaml-protoc-plugin/tree/andersfugmann/gRPC

However, I think the overall approach is wrong. All this could easilly be done outside ocaml-protoc. Since the build system already have access to all the protobuf files, I think it would scale much better if gRPC just created its own files/information as needed. This would allow gRPC maintainers to change and add fields as needed and not be tightly coupled with ocaml-protocol-plugin.

wdyt?