Cilium is now installed after these steps, but it doesn’t mean much as of now. To see the tip of the benefits iceberg, we need to explore Hubble.
Hubble is the observability layer of Cilium. It allows you to monitor the network traffic flowing through your nodes and pods.
1. Install Hubble
Install it with:
helm upgrade cilium cilium/cilium \
--namespace kube-system \
--reuse-values \
--set hubble.relay.enabled=true \
--set hubble.ui.enabled=true
And wait for Cilium to be ready with cilium status --wait:
cilium status –wait after some minutes while Hubble is being installed
2. Explore Hubble UI
Hubble is the observability layer of Cilium. Hubble UI is the web interface to explore the data collected by Hubble.
Enable it with:
cilium hubble ui
Open the URL in your browser and select the namespace you want to explore.
Hubble UI service graph, in a namespace of choice
3. Explore Hubble CLI
The main potential of Hubble can be glimpsed by using its CLI.
First of all let’s start the port-forwarding to the Hubble Relay:
kubectl port-forward -n kube-system deploy/hubble-relay 4245:grpc
Then, we can use the hubble observe command to see the traffic flowing through the network (-f follows the traffic):
hubble observe --namespace traefik -f
4. Explore L7 visibility
Cilium and Hubble can provide Layer 7 visibility, meaning that you can see the HTTP or DNS requests flowing through your network in real-time, and even filter them.
To see L7 flows, we need to enable L7 visibility for a specific namespace or pod. However, before enabling L7 visibility, we need to have a CiliumNetworkPolicy in place.
This is a simple tailored policy for one of my application pods.
1apiVersion: cilium.io/v2
2kind: CiliumNetworkPolicy
3metadata:
4 name: immich-l7-visibility
5 namespace: immich
6spec:
7 endpointSelector:
8 matchLabels:
9 k8s:io.kubernetes.pod.namespace: immich
10 app.kubernetes.io/name: server
11 ingress:
12 # Allow all traffic coming from otel namespace to metrics ports
13 - fromEndpoints:
14 - matchLabels:
15 io.kubernetes.pod.namespace: otel
16 toPorts:
17 - ports:
18 - port: "8081"
19 protocol: TCP
20 rules:
21 http: [{}]
22 - ports:
23 - port: "8082"
24 protocol: TCP
25 rules:
26 http: [{}]
27 # Allow all traffic coming from the internet to http port
28 - fromEndpoints:
29 - matchLabels:
30 app.kubernetes.io/name: traefik
31 io.kubernetes.pod.namespace: traefik
32 toPorts:
33 - ports:
34 - port: "2283"
35 protocol: TCP
36 rules:
37 http: [{}]
38 egress:
39 # Allow all traffic going to the DNS server
40 - toEndpoints:
41 - matchLabels:
42 io.kubernetes.pod.namespace: kube-system
43 toPorts:
44 - ports:
45 - port: "53"
46 protocol: UDP
47 rules:
48 dns:
49 - matchPattern: "*"
50 # Allow all traffic going to Redis
51 - toEndpoints:
52 - matchLabels:
53 app.kubernetes.io/name: redis
54 io.kubernetes.pod.namespace: immich
55 toPorts:
56 - ports:
57 - port: "6379"
58 protocol: TCP
59 # Allow all traffic going to PostgreSQL
60 - toEndpoints:
61 - matchLabels:
62 app.kubernetes.io/name: postgresql
63 io.kubernetes.pod.namespace: immich
64 toPorts:
65 - ports:
66 - port: "5432"
67 protocol: TCP
68 # Allow all traffic going to ML microservices
69 - toEndpoints:
70 - matchLabels:
71 app.kubernetes.io/name: machine-learning
72 io.kubernetes.pod.namespace: immich
73 toPorts:
74 - ports:
75 - port: "3003"
76 protocol: TCP
Note: the highlighted lines are the ones that enable L7 visibility for the specified ports.
In this case I enabled L7 visibility for 4 ports: 8081, 8082, 2283, and 53.
After that, we can simply observe L7 traffic with:
hubble observe --namespace immich -f -t l7
and we can see all the flows with their details, including HTTP URLs, status codes and DNS queries!
Hubble CLI showing L7 visibility for a specific namespace