Routing and Protocol Conversion
This task will show that, howe l7mp handle Routing and Protocol Conversion.
Before you begin
- You have to make sure that you have an installed minikube and you are able to install the l7mp with helm. here
- Make sure you are understand the basic conceps:
link to basic concepts.
About this task
This example demonstrates a simple UDP API gateway for video-game networking or
IoT. The worker service is exposed to the outside world though
the gateway, with the added twist that inbound packets received from the
Internet are processed through a transcoder service. This service, however,
is reachable only via UNIX domain socket (UDS) that does not allow remote access,
therefore the transcoder service will be exposed to the rest of the cluster on
a remote access protocol
WebSocket:8888, with the l7mp sidecar proxy doing proper
protocol-conversion for the app (WS <-> UDS). (NB: Currently l7mp supports only
byte-stream UDS so we will lose the original message framing at this point;
proper datagram-stream UDS will be added later.)
transcoder deployment, identified by the label
app:transcoder but with
no backing Kubernetes service, which will implement the transcoding functionality.
Each pod will contain two containers: a container for the transcoder process
itself that accepts connections via UDS (can be an UDS echo server for testing)
l7mp container that implements the sidecar.
To achieve this, you must first execute the following command, which will create a Deployment
with two Pods containing 2-2 containers. One container will be the l7mp as a sidecar
with a simple Controller configuration through which the l7mp operator will be able to configure
the sidecar. The second container will be a Unix Domain Socket echo server with a simple
command that reads from the
/tmp/uds-echo.sock socket and writes back the transcoded messages
there as follows:
Transcoded on <pod-name>: <message>.
kubectl apply -f https://l7mp.io/tasks/task1/transcoder-deployment.yaml
Now that this is the case, you should somehow configure the l7mp sidecar to be able to convert
incoming WebSocket packets to a Unix Domain Socket and write it to the
The first step is to create a cluster on the transcoder objects. Therefore, you must first
select the pods that have such a label and write a cluster in the sidecar that listens to and
directs traffic to /tmp/uds-echo.sock.
cat <<EOF | kubectl apply -f - apiVersion: l7mp.io/v1 kind: Target metadata: name: uds-cluster spec: selector: matchExpressions: - key: app operator: In values: - transcoder cluster: spec: UnixDomainSocket: filename: "/tmp/uds-echo.sock" endpoints: - spec: filename: "/tmp/uds-echo.sock" EOF
Secondly, the protocol conversion itself would need to be implemented by creating a listener object in the transcoder sidecar, which will listen WebSocket packets on port 8888 and redirect to the cluster you just created. Thus, the transcoder can only receive and transmit WebSocket traffic, but only UDS traffic within it.
cat <<EOF | kubectl apply -f - apiVersion: l7mp.io/v1 kind: VirtualService metadata: name: transcoder-vsvc namespace: default spec: selector: matchExpressions: - key: app operator: In values: - transcoder listener: name: websocket spec: WebSocket: port: 8888 rules: - action: route: destinationRef: /apis/l7mp.io/v1/namespaces/default/targets/uds-cluster EOF
Fantastic! Now you can send and receive WebSocket messages from transcoder. But you cannot specify a VirutalService/Listener as destinatination only Targets/Clusters. So you have to create a cluster inside the l7mp-ingress which can be addressed by the Gateway VirtualService at the end of this task.
cat <<EOF | kubectl apply -f - apiVersion: l7mp.io/v1 kind: Target metadata: name: ws-svc namespace: default spec: selector: matchLabels: app: l7mp-ingress linkedVirtualService: /apis/l7mp.io/v1/namespaces/default/virtualservices/transcoder-vsvc EOF
With the configuration above you can’t use a specific loadbalancer it always use trivial, which will always route traffic to the first endpoint in the cluster. So if you want to specify for example a ConsistentHash loadbalanacer you should use the configuration below instead the previously described.
cat <<EOF | kubectl apply -f - apiVersion: l7mp.io/v1 kind: Target metadata: name: websocket-cluster spec: selector: matchExpressions: - key: app operator: In values: - l7mp-ingress cluster: spec: WebSocket: port: 8888 loadbalancer: policy: ConsistentHash endpoints: - selector: matchLabels: app: transcoder EOF
The loadbalancer will allow pods to be accessed randomly each time you connect. The randomness stems from not specifying a key for the ConsistentHash loadbalancer.
The next command will create a very similar deployment to the transcoder, the only
difference being that there will be a UDP echo server here and not a UDS. This echo
server will echo back packets from any address on port 9999 in the following format:
Echo on <pod-name>: <message>.
kubectl apply -f https://l7mp.io/tasks/task1/worker-deployment.yaml
Now you need to create a new cluster, but this time it will not be written to sidecar
for the worker, but to the ingress gateway, for which you simply need to filter with this
key value pair when selecting: app:
cat <<EOF | kubectl apply -f - apiVersion: l7mp.io/v1 kind: Target metadata: name: udp-cluster spec: selector: matchExpressions: - key: app operator: In values: - l7mp-ingress cluster: spec: UDP: port: 9999 loadbalancer: policy: ConsistentHash endpoints: - selector: matchLabels: app: worker EOF
Please notice the endpoints section. There is no concrete address or configuration of an
endpoint only a selector. So if a pod has the
app: worker label it will be an endpoint of
Finally, you only have to connect these services together with an other Virtual Service. This will be the gateway-vsvc, which receive traffic from l7mp-ingress and first send to the transcoder and the result from transcoder to the worker and send back the traffic to the client.
cat <<EOF | kubectl apply -f - apiVersion: l7mp.io/v1 kind: VirtualService metadata: name: gateway-vsvc namespace: default spec: selector: matchLabels: app: l7mp-ingress listener: spec: UDP: port: 9001 rules: - action: route: destinationRef: /apis/l7mp.io/v1/namespaces/default/targets/udp-cluster ingress: - clusterRef: /apis/l7mp.io/v1/namespaces/default/targets/websocket-cluster EOF
If everything is working you have to test it. (You can use other tool for testing.)
socat - udp4:$(minikube ip):9001
You have to see something like this if you send something.
$ socat - udp4:$(minikube ip):9001 This is a test! Echo on worker-deployment-6bfdf5584c-97dgn: Transcoded on transcoder-deployment-6784fc494c-6t7l5: This is a test! Hurray it's working! Echo on worker-deployment-6bfdf5584c-97dgn: Transcoded on transcoder-deployment-6784fc494c-6t7l5: Hurray it's working!
To remove this setup use the following command:
curl -LO https://l7mp.io/tasks/task1/cleanup.sh chmod u+x cleanup.sh ./cleanup.sh