Network.GRPC.Spec

Protobuf RPC

This exists only as a type-level marker

Wrapper around Protobuf messages and Protobuf enums

Protobuf messages and enums behave differently to normal Haskell datatypes. Fields in messages always have defaults, enums can have unknown values, etc. We therefore mark them at the type-level with this Proto wrapper. Most of the time you can work with Proto values as if the wrapper is not there, because Proto msg inherits Message and Data.ProtoLens.Field HasField instances from msg. For example, you can create a 'Proto Point' value as

p = defMessage
      & #latitude  .~ ..
      & #longitude .~ ..

and access fields from such a value using

p ^. #latitude

as per usual.

One advantage of the Proto wrapper is that we can give blanket instances for all Protobuf messages; we use this to provide GHC.Records HasField and GHC.Records.Compat HasField instances. This means that you can also use OverloadedRecordDot to access fields

p.latitude

or even OverloadedRecordUpdate to set fields

p{latitude = ..}

Field accessor for Proto

gRPC using JSON as the message encoding

"JSON over gRPC" is a bit of an ambiguous phrase. It can be a very general term, simply meaning using an otherwise-unspecified JSON encoding, or it can refer to "Protobuf over JSON" (see https://protobuf.dev/programming-guides/proto3/#json). In this module we deal with the former, and don't deal with anything Protobuf-specific at all, nor do we rely on any of the infrastructure generated by the Protobuf compiler (in other words, there is no need to use protoc). See https://grpc.io/blog/grpc-with-json/ for a Java example of using gRPC with JSON without Protobuf.

In the absence of the infrastructure provided by protoc, you will need to manually provide Input and Output instances for each RPC you use. For example:

type Create   = JsonRpc KeyValueService "Create"
type Delete   = JsonRpc KeyValueService "Delete"
..

type instance Input  Create   = ..
type instance Output Create   = ..
type instance Input  Retrieve = ..
type instance Output Retrieve = ..
..

On the client, you will need ToJSON instances for inputs and FromJSON instances for outputs; on the server the situation is dual. You may find it convenient to use JsonObject (but this is certainly not required).

TODO: https://github.com/well-typed/grapesy/issues/166 We don't currently offer explicit support for "Protobuf JSON".

Convenient way to construct JSON values

Example:

type instance Input Create =
  JsonObject '[ '("key"   , Required Key)
              , '("value" , Required Value)
              ]

Required field

Optional field

Maybe will be represented by the absence of the field in the object.

Auxiliary class used for the FromJSON instance for JsonObject

It is not possible (nor necessary) to define additional instances.

Auxiliary class used for the ToJSON instance for JsonObject

It is not possible (nor necessary) to define additional instances.

Custom gRPC format

Usually gRPC runs over Protobuf, but it does not have to. RawRpc provides an alternative format, which does not use serialization/deserialization at all, just using raw bytestrings for messages. This is a non-standard format (which the gRPC specification explicitly permits).

This RPC supports the given streaming type

This is a weaker condition than HasStreamingType: some (non-Protobuf) RPCs may support more than one streaming type.

The streaming type supported by this RPC

Is there a next element in a stream?

Send a value

Receive a value

Nothing indicates no more values. Calling this function again after receiving Nothing is a bug.

Positive use of a

Negative use of a

Handler role

Type of a handler

Wrapper around Handler Server to avoid ambiguous types

Alias for ServerHandler' with the streaming type determined by the rpc

Wrapper around Handler Client to avoid ambiguous types

Alias for ClientHandler' with the streaming type determined by the rpc

Hoist server handler from one monad to another

RPC parameters that can be chosen on a per-call basis

Timeout

Tell the server that if the request cannot be completed within the specified amount of time, it should be aborted. If the timeout gets exceeded, a Network.GRPC.Common.GrpcDeadlineExceeded exception will be raised.

Custom metadata to be included in the request

All pseudo-headers

Partial pseudo headers: identify the server, but not a specific resource

Request pseudo-methods

https://datatracker.ietf.org/doc/html/rfc7540#section-8.1.2.3

Path

The gRPC spec specifies:

Path → ":path" "/" Service-Name "/" {method name} # But see note below.

Moreover, it says:

Path is case-sensitive. Some gRPC implementations may allow the Path format
shown above to be overridden, but this functionality is strongly
discouraged. gRPC does not go out of its way to break users that are using
this kind of override, but we do not actively support it, and some
functionality (e.g., service config support) will not work when the path is
not of the form shown above.

We don't support these non-standard paths at all.

Address

The address of a server to connect to. This is not standard gRPC nomenclature, but follows convention such as adopted by grpcurl and grpc-client-cli, which distinguish between the address of a server to connect to (hostname and port), and the (optional) HTTP authority, which is an (optional) string to be included as the HTTP2 :authority pseudo-header.

Scheme

See https://datatracker.ietf.org/doc/html/rfc3986#section-3.1.

Method

The only method supported by gRPC is POST.

See also https://datatracker.ietf.org/doc/html/rfc7231#section-4.

Construct path

Timeout

Positive integer with ASCII representation of at most 8 digits

Timeout unit

Translate Timeout to microseconds

For Nanosecond timeout we round up.

Note: the choice of Integer for the result is important: timeouts can be quite long, and might easily exceed the range of a 32-bit int: 2^31 microseconds is roughly 35 minutes (on 64-bit architectures this is much less important; 2^63 microseconds is 292,277.2 years). We could use Int64 or Word64, but Integer works nicely with the unbounded-delays package.

Valid timeout values

Timeout values cannot exceed 8 digits. If you need a longer timeout, consider using a different TimeoutUnit instead.

Response headers

Response headers (without allowing for invalid headers)

See RequestHeaders for an explanation of Undecorated.

Response headers allowing for invalid headers

See RequestHeaders' for an explanation of Checked and the purpose of e.

Information sent by the peer after the final output

Response trailers are a HTTP2 concept: they are HTTP headers that are sent after the content body. For example, imagine the server is streaming a file that it's reading from disk; it could use trailers to give the client an MD5 checksum when streaming is complete.

Trailers sent after the response (without allowing for invalid trailers)

Trailers sent after the response, allowing for invalid trailers

We do not parameterize this over the type of synthesized errors: unlike response (or request) headers, we have no opportunity to check the trailers for synthesized errors ahead of time, so having a type to signal "trailers without synthesized errors" is not particularly useful.

Trailers sent in the gRPC Trailers-Only case

We deal with the HTTP status elsewhere.

Trailers for the Trailers-Only case (without allowing for invalid trailers)

Trailers for the Trailers-Only case, allowing for invalid headers

Pushback

The server adds this header to push back against client retries. We do not yet support automatic retries (https://github.com/well-typed/grapesy/issues/104), but do we parse this header so that if the server includes it, we do not throw a parser error.

See also https://github.com/grpc/proposal/blob/master/A6-client-retries.md

Default constructor for ProperTrailers

Server indicated normal termination

This is only an exception if the client tries to send any further messages.

Translate gRPC exception to response trailers

Check if trailers correspond to an exceptional response

The gRPC spec states that

Trailers-Only is permitted for calls that produce an immediate error

However, in practice gRPC servers can also respond with Trailers-Only in non-error cases, simply indicating that the server considers the conversation over. To distinguish, we look at properTrailersGrpcStatus.

ProperTrailers is a subset of TrailersOnly

TrailersOnly is a superset of ProperTrailers

Translate GrpcStatus to numerical status code

See https://grpc.github.io/grpc/core/md_doc_statuscodes.html

Translate GrpcError to numerical status code

See also fromGrpcStatus

Inverse to fromGrpcStatus

Inverse to fromGrpcError

Server indicated a gRPC error

For the common case where you just want to set grpcError, you can use throwGrpcError.

Convenience function to throw an GrpcException with the specified error

Fields :

• Proto.Status_Fields.code :: Lens' Status Data.Int.Int32

• Proto.Status_Fields.message :: Lens' Status Data.Text.Text

• Proto.Status_Fields.details :: Lens' Status [Proto.Google.Protobuf.Any.Any]

• Proto.Status_Fields.vec'details :: Lens' Status (Data.Vector.Vector Proto.Google.Protobuf.Any.Any)

Map from header names to values

The gRPC spec mandates

Custom-Metadata header order is not guaranteed to be preserved except for
values with duplicate header names. Duplicate header names may have their
values joined with "," as the delimiter and be considered semantically
equivalent.

Internally we don't allow for these duplicates, but instead join the headers as mandated by the spec.

Construct CustomMetadataMap, joining duplicates

Flatten CustomMetadataMap to a list

Precondition: the map must be valid.

Insert value into CustomMetadataMap

If a header with the same name already exists, the value is appended to (the end of) the existing value.

Metadata included in the request

Often you can give a blanket metadata definition for all methods in a service. For example:

type instance RequestMetadata          (Protobuf RouteGuide meth) = NoMetadata
type instance ResponseInitialMetadata  (Protobuf RouteGuide meth) = NoMetadata
type instance ResponseTrailingMetadata (Protobuf RouteGuide meth) = NoMetadata

If you want to give specific types of metadata for specific methods but not for others, it can sometimes be useful to introduce an auxiliary closed type, so that you can give a catch-all case. For example:

type instance ResponseInitialMetadata (Protobuf Greeter meth) = GreeterResponseInitialMetadata meth

type family GreeterResponseInitialMetadata (meth :: Symbol) where
  GreeterResponseInitialMetadata "sayHelloStreamReply" = SayHelloMetadata
  GreeterResponseInitialMetadata meth                  = NoMetadata

Metadata included in the initial response

See RequestMetadata for discussion.

Metadata included in the response trailers

See RequestMetadata for discussion.

Response metadata

It occassionally happens that we do not know if we should expect the initial metadata from the server or the trailing metadata (when the server uses Trailers-Only); for example, see Network.GRPC.Client.recvResponseInitialMetadata.

Serialize metadata to custom metadata headers

Parse metadata from custom metadata headers

Some guidelines for defining instances:

• You can assume that the list of headers will not contain duplicates. The gRPC spec does allow for duplicate headers and specifies how to process them, but this will be taken care of before parseMetadata is called.

• However, you should assume no particular order.

• If there are unexpected headers present, you have a choice whether you want to consider this a error and throw an exception, or regard the additional headers as merely additional information and simply ignore them. There is no single right answer here: ignoring additional metadata runs the risk of not realizing that the peer is trying to tell you something important, but throwing an error runs the risk of unnecessarily aborting an RPC.

Metadata with statically known fields

This is required for the response trailing metadata. When the server sends the initial set of headers to the client, it must tell the client which trailers to expect (by means of the HTTP Trailer header; see https://datatracker.ietf.org/doc/html/rfc7230#section-4.4).

Any headers constructed in buildMetadata must be listed here; not doing so is a bug. However, the converse is not true: it is acceptable for a header to be listed in metadataHeaderNames but not in buildMetadata. Put another way: the list of "trailers to expect" included in the initial request headers is allowed to be an overapproximation, but not an underapproximation.

Wrapper around buildMetadata that catches any pure exceptions

These pure exceptions can arise when invalid headers are generated (for example, ASCII headers with non-ASCII values).

Convenience constructor around InvalidHeader

Convenience constructor around MissingHeader

Convenience constructor around UnexpectedHeader

Convenience constructor around InvalidHeaderSynthesize

Convenience function for throwing an invalidHeader exception.

Indicate that all synthesized errors have been handled

For some headers the gRPC spec mandates a specific gRPC error that should be synthesized when the header is invalid. We use HandledSynthesized in types to indicate that all errors that should have been synthesized have already been thrown.

For example, RequestHeaders' HandledSynthesized indicates that these request headers may still contain errors for some headers, but no errors for which the spec mandates that we synthesize a specific gRPC exception.

Evidence that HandledSynthesized is an empty type

Drop all synthesized errors, leaving just the original

Map over the errors

Pure version of mapSynthesizedM

Throw all synthesized errors

After this we are guaranteed that the synthesized errors have been handlded.

Extract all invalid headers

Render InvalidHeaders

HTTP status to report

If there are multiple headers, each of which with a mandated status, we just use the first; the spec is essentially ambiguous in this case.