Gloo Edge Workshop
Gloo Edge Enterprise

Gloo Edge Workshop

Introduction

Monolith to microservices
Companies are modernizing their applications by splitting traditional monoliths into micro services.
Adopting Kubernetes
These micro services are generally deployed on Kubernetes and one of the first requirements is to expose some of those services to the outside world.
Kubernetes Ingress
The natural way to expose them is through a standard Kubernetes Ingress Controller (NGINX, HAproxy, …).
But it quickly becomes obvious that this approach is limited. The Ingress Kubernetes object doesn’t provide any way to define advanced routing policies, for example.
Authentication at app level
More and more micro services are deployed and some functionalities (like external authentication) are being implemented by different application teams.
Those teams should focus on the business logic instead of reinventing the wheel.
A workaround implemented by many companies is to provide standard libraries to be used when developing new micro services, but it forces all the teams to use the same language, which is generating new challenges (slow down innovation, no global visibility for the security team, …).
Other capabilities
Other functionalities (rate limiting, Web Application Firewall, …) can’t easily be implemented in the micro services and generally require an API Gateway or Firewall running outside of Kubernetes.
It means that the configuration can’t be driven by yaml, so it’s not compatible with a modern Gitops deployment model.
Kubernetes native API gateway
Implementing a modern Kubernetes-native API Gateway becomes a natural choice.
We call it Kubernetes-native because it can be 100% configured by yaml (declarative API), so is compatible with a Gitops deployment model.
Authentication/Authorization can be implemented at the API Gateway level and relevant information (like the email address of the authenticated user) can be passed to the Upstream micro service (through a header, for example).
It also provides functionalities like rate limiting and Web Application Firewall, which can be configured through yaml.
Finally, it provides more visibility to the security team.
Other workloads
Even if it runs on Kubernetes, a modern Kubernetes-native API Gateway can be used to route traffic to legacy applications (running in Virtual Machines, for example).
And it can also sometimes provide interesting capabilities, like discovering functions running on AWS Lambda and route traffic to them.
Gloo Edge architecture
At Solo.io we have built a modern Kubernetes-native API Gateway built on Envoy.
It provides all the capabilities described above (and much more !).
Why would you choose an API Gateway based on Envoy ?
There are many good reasons why.
  • First of all, it’s a high performance software written in C++.
  • It’s driven by a neutral foundation (the CNCF, like Kubernetes), so its roadmap isn’t driven by a single vendor.
  • And probably more importantly, you have already adopted or you’re probably going to adopt a Service Mesh in the future. Chances are high that this Service Mesh will be Istio and if it’s not the case it will most probably be a Service Mesh based on Envoy.
  • So choosing an API Gateway based on Envoy will allow you to get the metrics for your API Gateway and your Service Mesh in the same format, will allow you troubleshoot issues in a common way, … It will make your life much easier to summarize.
Why would you choose Gloo Edge ?
  • It has been developed from the beginning with the idea to be configured 100% through yaml.
  • It provides all the functionalities you expect from a modern API Gateway:
  • External authentication based on OAuth2, JWT, api keys, …
  • Authorization based on OPA
  • Advanced rate limiting
  • Web Application Firewall based on ModSecurity
  • Advanced transformation
  • Customization through Web Assembly
  • A Kubernetes-native developer portal called Gloo Portal
  • And much more

Lab 1 - Deploy KinD cluster

From the terminal go to the /home/solo/workshops/gloo-edge directory:
1
cd /home/solo/workshops/gloo-edge
Copied!
Deploy a local Kubernetes cluster using this command:
1
./scripts/deploy.sh 1 gloo-edge
Copied!
Then verify that your Kubernetes cluster is ready:
1
./scripts/check.sh gloo-edge
Copied!
The check.sh script will return immediately with no output if the cluster is ready. Otherwise, it will output a series of periodic "waiting" messages until the cluster is up.

Lab 2 - Deploy Gloo Edge

Run the commands below to deploy Gloo Edge Enterprise:
1
export GLOO_VERSION=v1.8.17
2
curl -sL https://run.solo.io/gloo/install | sh
3
export PATH=$HOME/.gloo/bin:$PATH
4
glooctl upgrade --release=$GLOO_VERSION
5
6
helm repo add glooe https://storage.googleapis.com/gloo-ee-helm
7
helm upgrade --install gloo glooe/gloo-ee --namespace gloo-system \
8
--create-namespace --version 1.8.13 --set-string license_key=$LICENSE_KEY
Copied!
Gloo Edge can also be deployed using a Helm chart.
Use the following commands to wait for the Gloo Edge components to be deployed:
1
until kubectl get ns gloo-system
2
do
3
sleep 1
4
done
5
6
printf "\nWaiting for all the gloo-system pods to become ready"
7
until [ $(kubectl -n gloo-system get pods -o jsonpath='{range .items[*].status.containerStatuses[*]}{.ready}{"\n"}{end}' | grep false -c) -eq 0 ]; do
8
printf "%s" "."
9
sleep 1
10
done
11
printf "\n"
Copied!
Set the environment variable corresponding to the endpoint of the Gateway:
1
export ENDPOINT=$(kubectl -n gloo-system get svc gateway-proxy -o jsonpath='{.status.loadBalancer.ingress[0].*}')
Copied!

Lab 3 - Traffic management

Deploy Services

In this step you will expose two services to the outside world using Gloo Edge.
First let's deploy an application called bookinfo in bookinfo namespace:
1
kubectl create ns bookinfo
2
kubectl -n bookinfo apply -f https://raw.githubusercontent.com/istio/istio/1.7.3/samples/bookinfo/platform/kube/bookinfo.yaml
3
kubectl delete deployment reviews-v1 reviews-v3 -n bookinfo
Copied!
Gloo Edge with Bookinfo
The bookinfo app has 3 versions of a microservice called reviews. You will keep only the version 2 of the reviews microservice for this step and will add the other versions later. An easy way to distinguish among the different versions in the web interface is to look at the stars: v1 displays no stars in the reviews, v2 displays black stars, and v3 displays red stars.
Gloo Edge uses a discovery mechanism to create Upstreams automatically, but Upstreams can be also created manually using Kubernetes CRDs.
After a few seconds, Gloo Edge will discover the newly created service and create an Upstream called bookinfo-productpage-9080 (Gloo Edge uses the convention namespace-service-port for the discovered Upstreams).
To verify that the Upstream was created properly, run the following command:
1
until glooctl get upstream bookinfo-productpage-9080 2> /dev/null
2
do
3
echo waiting for upstream bookinfo-productpage-9080 to be discovered
4
sleep 3
5
done
Copied!
It should return the discovered upstream with an Accepted status:
1
+---------------------------+------------+----------+----------------------------+
2
| UPSTREAM | TYPE | STATUS | DETAILS |
3
+---------------------------+------------+----------+----------------------------+
4
| bookinfo-productpage-9080 | Kubernetes | Accepted | svc name: productpage |
5
| | | | svc namespace: bookinfo |
6
| | | | port: 9080 |
7
| | | | |
8
+---------------------------+------------+----------+----------------------------+
Copied!
Now, let's deploy the second application, called httpbin in the namespace team1.
This application is very useful when you have to debug routing, headers in requests, responses, status codes, etc. The public online version of it can be found here.
1
kubectl create ns team1
2
3
kubectl apply -f - <<EOF
4
apiVersion: v1
5
kind: ServiceAccount
6
metadata:
7
name: httpbin
8
namespace: team1
9
---
10
apiVersion: v1
11
kind: Service
12
metadata:
13
name: httpbin
14
namespace: team1
15
labels:
16
app: httpbin
17
spec:
18
ports:
19
- name: http
20
port: 8000
21
targetPort: 80
22
selector:
23
app: httpbin
24
---
25
apiVersion: apps/v1
26
kind: Deployment
27
metadata:
28
name: httpbin
29
namespace: team1
30
spec:
31
replicas: 1
32
selector:
33
matchLabels:
34
app: httpbin
35
version: v1
36
template:
37
metadata:
38
labels:
39
app: httpbin
40
version: v1
41
spec:
42
serviceAccountName: httpbin
43
containers:
44
- image: docker.io/kennethreitz/httpbin
45
imagePullPolicy: IfNotPresent
46
name: httpbin
47
ports:
48
- containerPort: 80
49
EOF
Copied!
To verify that the Upstream was created properly, run the following command:
1
until glooctl get upstream team1-httpbin-8000 2> /dev/null
2
do
3
echo waiting for upstream team1-httpbin-8000 to be discovered
4
sleep 3
5
done
Copied!
It should return the discovered upstream with an Accepted status:
1
+--------------------+------------+----------+------------------------+
2
| UPSTREAM | TYPE | STATUS | DETAILS |
3
+--------------------+------------+----------+------------------------+
4
| team1-httpbin-8000 | Kubernetes | Accepted | svc name: httpbin |
5
| | | | svc namespace: team1 |
6
| | | | port: 8000 |
7
| | | | |
8
+--------------------+------------+----------+------------------------+
Copied!

Routing to a Kubernetes Service

Now that your two Upstream CR have been created, you need to create the resources to route the traffic to them.
First, you create a Virtual Service. For bookinfo under / and for httpbin under /not-secured.
Please, notice that the order matters.
1
kubectl apply -f - <<EOF
2
apiVersion: gateway.solo.io/v1
3
kind: VirtualService
4
metadata:
5
name: demo
6
namespace: gloo-system
7
spec:
8
virtualHost:
9
domains:
10
- '*'
11
routes:
12
- matchers:
13
- prefix: /
14
routeAction:
15
single:
16
upstream:
17
name: bookinfo-productpage-9080
18
namespace: gloo-system
19
EOF
Copied!
The creation of the Virtual Service exposes the Kubernetes service through the gateway.
You can access the application using the web browser by running the following command:
1
/opt/google/chrome/chrome http://${ENDPOINT}/productpage
Copied!
It should return the bookinfo application webpage. Note that the review stars are black (v2).
Bookinfo Web Interface
Now, let's enable the httpbin route to be matched before bookinfo:
1
kubectl apply -f - <<EOF
2
apiVersion: gateway.solo.io/v1
3
kind: VirtualService
4
metadata:
5
name: demo
6
namespace: gloo-system
7
spec:
8
virtualHost:
9
domains:
10
- '*'
11
routes:
12
# ------------- Another route which is parsed before ------------------
13
- matchers:
14
- prefix: /not-secured
15
routeAction:
16
single:
17
upstream:
18
name: team1-httpbin-8000
19
namespace: gloo-system
20
options:
21
prefixRewrite: '/'
22
# ----------------------------------------------------------------------
23
- matchers:
24
- prefix: /
25
routeAction:
26
single:
27
upstream:
28
name: bookinfo-productpage-9080
29
namespace: gloo-system
30
EOF
Copied!
And to access the httpbin application:
1
/opt/google/chrome/chrome http://${ENDPOINT}/not-secured/get
Copied!
It should return a json object with details regarding your own request. Something similar to:
1
{
2
"args": {},
3
"headers": {
4
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
5
"Accept-Encoding": "gzip, deflate",
6
"Accept-Language": "en-GB,en;q=0.9,es-ES;q=0.8,es;q=0.7,sk-SK;q=0.6,sk;q=0.5,en-US;q=0.4",
7
"Cache-Control": "max-age=0",
8
"Host": "x.x.x.x",
9
"Upgrade-Insecure-Requests": "1",
10
"User-Agent": "xxxx",
11
"X-Envoy-Expected-Rq-Timeout-Ms": "15000",
12
"X-Envoy-Original-Path": "/not-secured/get"
13
},
14
"origin": "x.x.x.x",
15
"url": "http://x.x.x.x/get"
16
}
Copied!

Delegation

Gloo Edge provides a feature referred to as delegation. Delegation allows a complete routing configuration to be assembled from separate config objects. The root config object delegates responsibility to other objects, forming a tree of config objects. The tree always has a Virtual Service as its root, which delegates to any number of Route Tables. Route Tables can further delegate to other Route Tables.
Use cases for delegation include:
  • Allowing multiple tenants to own add, remove, and update routes without requiring shared access to the root-level Virtual Service
  • Sharing route configuration between Virtual Services
  • Simplifying blue-green routing configurations by swapping the target Route Table for a delegated route.
  • Simplifying very large routing configurations for a single Virtual Service
  • Restricting ownership of routing configuration for a tenant to a subset of the whole Virtual Service.
Let's rewrite your Virtual Service to delegate the routing to a Route Table.
First resource is the delegated Route Table. Second resource is a Virtual Service. Notice that there is a new delegateAction referencing the just created Route Table.
1
kubectl apply -f - <<EOF
2
# ------------- Delegation resource -----------------------
3
apiVersion: gateway.solo.io/v1
4
kind: RouteTable
5
metadata:
6
name: httpbin-routetable
7
namespace: team1
8
spec:
9
routes:
10
- matchers:
11
- prefix: /not-secured
12
options:
13
prefixRewrite: '/'
14
routeAction:
15
single:
16
upstream:
17
name: team1-httpbin-8000
18
namespace: gloo-system
19
# ---------------------------------------------------------
20
apiVersion: gateway.solo.io/v1
21
kind: VirtualService
22
metadata:
23
name: demo
24
namespace: gloo-system
25
spec:
26
virtualHost:
27
domains:
28
- '*'
29
routes:
30
- matchers:
31
- prefix: /not-secured
32
# ------------- Delegation action by reference ------------
33
delegateAction:
34
ref:
35
name: 'httpbin-routetable'
36
namespace: 'team1'
37
# ---------------------------------------------------------
38
- matchers:
39
- prefix: /
40
routeAction:
41
single:
42
upstream:
43
name: bookinfo-productpage-9080
44
namespace: gloo-system
45
EOF
Copied!

Delegation By Label Selector

Another way to delegate is to use labels. This approach helps you to create dynamic references. Let's update your Route Table to add a label. Then, the label will be used in the Virtual Service resource.
Let's make both changes:
1
kubectl apply -f - <<EOF
2
apiVersion: gateway.solo.io/v1
3
kind: RouteTable
4
metadata:
5
name: httpbin-routetable
6
namespace: team1
7
# ------------- Label to use as dynamic reference ---------
8
labels:
9
application-owner: team1
10
# ---------------------------------------------------------
11
spec:
12
routes:
13
- matchers:
14
- prefix: /not-secured
15
options:
16
prefixRewrite: '/'
17
routeAction:
18
single:
19
upstream:
20
name: team1-httpbin-8000
21
namespace: gloo-system
22
---
23
apiVersion: gateway.solo.io/v1
24
kind: VirtualService
25
metadata:
26
name: demo
27
namespace: gloo-system
28
spec:
29
virtualHost:
30
domains:
31
- '*'
32
routes:
33
- matchers:
34
- prefix: /not-secured
35
# ------------- Delegation by label selector --------------
36
delegateAction:
37
selector:
38
namespaces:
39
- team1
40
labels:
41
application-owner: team1
42
# ---------------------------------------------------------
43
- matchers:
44
- prefix: /
45
routeAction:
46
single:
47
upstream:
48
name: bookinfo-productpage-9080
49
namespace: gloo-system
50
EOF
Copied!

Routing to Multiple Upstreams

In many cases, you need to route traffic to two different versions of an application to test a new feature. In this step, you are going to update the Virtual Service to route traffic to two different Upstreams:
The first step is to create a new deployment of the bookinfo application, this time with the version 3 of the reviews microservice:
1
kubectl create ns bookinfo-beta
2
kubectl -n bookinfo-beta apply -f https://raw.githubusercontent.com/istio/istio/1.7.3/samples/bookinfo/platform/kube/bookinfo.yaml
3
kubectl delete deployment reviews-v1 reviews-v2 -n bookinfo-beta
Copied!
Weighted Routing Diagram
Verify that the Upstream for the beta application was created, using the following command:
1
until glooctl get upstream bookinfo-beta-productpage-9080 2> /dev/null
2
do
3
echo waiting for upstream bookinfo-beta-productpage-9080 to be discovered
4
sleep 3
5
done
Copied!
Now you can route to multiple Upstreams by updating the Virtual Service as follow:
1
kubectl apply -f - <<EOF
2
apiVersion: gateway.solo.io/v1
3
kind: VirtualService
4
metadata:
5
name: demo
6
namespace: gloo-system
7
spec:
8
virtualHost:
9
domains:
10
- '*'
11
routes:
12
- matchers:
13
- prefix: /not-secured
14
delegateAction:
15
selector:
16
namespaces:
17
- team1
18
labels:
19
application-owner: team1
20
- matchers:
21
- prefix: /
22
routeAction:
23
# ----------------------- Multi Destination ------------------
24
multi:
25
destinations:
26
- weight: 5
27
destination:
28
upstream:
29
name: bookinfo-productpage-9080
30
namespace: gloo-system
31
- weight: 5
32
destination:
33
upstream:
34
name: bookinfo-beta-productpage-9080
35
namespace: gloo-system
36
# ------------------------------------------------------------
37
EOF
Copied!
You should see either the black star reviews (v2) or the new red star reviews (v3) when refreshing the page.
Bookinfo Web Interface with v3

Lab 4 - Security and authentication

In this step, you will explore some Gloo Edge features related to security and Authentication.

Network Encryption - Server TLS

In this step you are going to secure your application using TLS.
Let's first create a private key and a self-signed certificate to use in your Virtual Service:
1
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
2
-keyout tls.key -out tls.crt -subj "/CN=*"
Copied!
Then, you have to store them in a Kubernetes secret running the following command:
1
kubectl create secret tls upstream-tls --key tls.key \
2
--cert tls.crt --namespace gloo-system
Copied!
To setup TLS you have to add the SSL config to the Virtual Service:
1
kubectl apply -f - <<EOF
2
apiVersion: gateway.solo.io/v1
3
kind: VirtualService
4
metadata:
5
name: demo
6
namespace: gloo-system
7
spec:
8
# ---------------- SSL config ---------------------------
9
sslConfig:
10
secretRef:
11
name: upstream-tls
12
namespace: gloo-system
13
# -------------------------------------------------------
14
virtualHost:
15
domains:
16
- '*'
17
routes:
18
- matchers:
19
- prefix: /not-secured
20
delegateAction:
21
selector:
22
namespaces:
23
- team1
24
labels:
25
application-owner: team1
26
- matchers:
27
- prefix: /
28
routeAction:
29
single:
30
upstream:
31
name: bookinfo-productpage-9080
32
namespace: gloo-system
33
EOF
Copied!
Now the application is securely exposed through TLS. To test the TLS configuration, run the following command to open the browser (note that now the traffic is served using https):
1
export APP_URL=https://${ENDPOINT}
Copied!
Let's check bookinfo:
1
/opt/google/chrome/chrome $APP_URL/productpage
Copied!
The browser will warn you that your connection is not private due to the self-signed certificate. Click through the Advanced button and the subsequent link to proceed to the "unsafe" destination. Then the bookinfo interface should display as expected, although the address bar may continue to warn you that the destination is "Not Secure." Don't worry; this is expected behavior given the self-signed certificate.
Self-Signed Certificate Warning
And httpbin:
Since you are using self-signed certificates, you need to allow insecure server connection when using SSL with -k
1
curl -k $APP_URL/not-secured/get
Copied!

Authentication with OIDC (OpenID Connect)

In many use cases, you need to restrict the access to your applications to authenticated users.
OIDC (OpenID Connect) is an identity layer on top of the OAuth 2.0 protocol. In OAuth 2.0 flows, authentication is performed by an external Identity Provider (IdP) which, in case of success, returns an Access Token representing the user identity. The protocol does not define the contents and structure of the Access Token, which greatly reduces the portability of OAuth 2.0 implementations.
The goal of OIDC is to address this ambiguity by additionally requiring Identity Providers to return a well-defined ID Token. OIDC ID tokens follow the JSON Web Token standard and contain specific fields that your applications can expect and handle. This standardization allows you to switch between Identity Providers – or support multiple ones at the same time – with minimal, if any, changes to your downstream services; it also allows you to consistently apply additional security measures like Role-based Access Control (RBAC) based on the identity of your users, i.e. the contents of their ID token.
In this step, you will secure the bookinfo application using an OIDC Identity Provider.
Let's start by installing Keycloak:
1
kubectl create -f https://raw.githubusercontent.com/keycloak/keycloak-quickstarts/12.0.4/kubernetes-examples/keycloak.yaml
2
kubectl rollout status deploy/keycloak
Copied!
Then, you need to configure it and create two users:
Let's create the users:
1
# Get Keycloak URL and token
2
KEYCLOAK_URL=http://$(kubectl get service keycloak -o jsonpath='{.status.loadBalancer.ingress[0].ip}'):8080/auth
3
KEYCLOAK_TOKEN=$(curl -d "client_id=admin-cli" -d "username=admin" -d "password=admin" -d "grant_type=password" "$KEYCLOAK_URL/realms/master/protocol/openid-connect/token" | jq -r .access_token)
4
5
# Create initial token to register the client
6
read -r client token <<<$(curl -H "Authorization: Bearer ${KEYCLOAK_TOKEN}" -X POST -H "Content-Type: application/json" -d '{"expiration": 0, "count": 1}' $KEYCLOAK_URL/admin/realms/master/clients-initial-access | jq -r '[.id, .token] | @tsv')
7
8
# Register the client
9
read -r id secret <<<$(curl -X POST -d "{ \"clientId\": \"${client}\" }" -H "Content-Type:application/json" -H "Authorization: bearer ${token}" ${KEYCLOAK_URL}/realms/master/clients-registrations/default| jq -r '[.id, .secret] | @tsv')
10
11
# Add allowed redirect URIs
12
curl -H "Authorization: Bearer ${KEYCLOAK_TOKEN}" -X PUT -H "Content-Type: application/json" -d '{"serviceAccountsEnabled": true, "directAccessGrantsEnabled": true, "authorizationServicesEnabled": true, "redirectUris": ["'${APP_URL}'/callback", "http://portal.example.com/callback"]}' $KEYCLOAK_URL/admin/realms/master/clients/${id}
13
14
# Add the group attribute in the JWT token returned by Keycloak
15
curl -H "Authorization: Bearer ${KEYCLOAK_TOKEN}" -X POST -H "Content-Type: application/json" -d '{"name": "group", "protocol": "openid-connect", "protocolMapper": "oidc-usermodel-attribute-mapper", "config": {"claim.name": "group", "jsonType.label": "String", "user.attribute": "group", "id.token.claim": "true", "access.token.claim": "true"}}' $KEYCLOAK_URL/admin/realms/master/clients/${id}/protocol-mappers/models
16
17
# Create first user
18
curl -H "Authorization: Bearer ${KEYCLOAK_TOKEN}" -X POST -H "Content-Type: application/json" -d '{"username": "user1", "email": "[email protected]", "enabled": true, "attributes": {"group": "users"}, "credentials": [{"type": "password", "value": "password", "temporary": false}]}' $KEYCLOAK_URL/admin/realms/master/users
19
20
# Create second user
21
curl -H "Authorization: Bearer ${KEYCLOAK_TOKEN}" -X POST -H "Content-Type: application/json" -d '{"username": "user2", "email": "[email protected]", "enabled": true, "attributes": {"group": "users"}, "credentials": [{"type": "password", "value": "password", "temporary": false}]}' $KEYCLOAK_URL/admin/realms/master/users
Copied!
Note: If you get a Not Authorized error, please, re-run this command and continue from the command started to fail:
1
KEYCLOAK_TOKEN=$(curl -d "client_id=admin-cli" -d "username=admin" -d "password=admin" -d "grant_type=password" "$KEYCLOAK_URL/realms/master/protocol/openid-connect/token" | jq -r .access_token)
Copied!
The architecture looks like this now:
Bookinfo with OIDC
The next step is to configure the authentication in the Virtual Service. For this, you will have to create a Kubernetes Secret that contains the OIDC secret:
1
glooctl create secret oauth --namespace gloo-system --name oauth --client-secret ${secret}
Copied!
Then you will create an AuthConfig, which is a Gloo Edge CRD that contains authentication information:
1
kubectl apply -f - <<EOF
2
apiVersion: enterprise.gloo.solo.io/v1
3
kind: AuthConfig
4
metadata:
5
name: oauth
6
namespace: gloo-system
7
spec:
8
configs:
9
- oauth2:
10
oidcAuthorizationCode:
11
appUrl: ${APP_URL}
12
callbackPath: /callback
13
clientId: ${client}
14
clientSecretRef:
15
name: oauth
16
namespace: gloo-system
17
issuerUrl: "${KEYCLOAK_URL}/realms/master/"
18
scopes:
19
- email
20
headers:
21
idTokenHeader: jwt
22
EOF
Copied!
Finally you activate the authentication on the Virtual Service by referencing the AuthConfig:
Notice that this only applies to a specific route, because it is configured within a matcher
1
kubectl apply -f - <<EOF
2
apiVersion: gateway.solo.io/v1
3
kind: VirtualService
4
metadata:
5
name: demo
6
namespace: gloo-system
7
spec:
8
sslConfig:
9
secretRef:
10
name: upstream-tls
11
namespace: gloo-system
12
virtualHost:
13
domains:
14
- '*'
15
routes:
16
- matchers:
17
- prefix: /not-secured
18
delegateAction:
19
selector:
20
namespaces:
21
- team1
22
labels:
23
application-owner: team1
24
- matchers:
25
- prefix: /
26
# ------------------- OIDC - Only applied to this matcher -------------------
27
options:
28
extauth:
29
configRef:
30
name: oauth
31
namespace: gloo-system
32
# ---------------------------------------------------------------------------
33
routeAction:
34
single:
35
upstream:
36
name: bookinfo-productpage-9080
37
namespace: gloo-system
38
EOF
Copied!
To test the Authentication, try first to access the httpbin application. You will see that Authentication is NOT required.
1
curl -k $APP_URL/not-secured/get
Copied!
On the other hand, if you refresh the web browser for the bookinfo application, you will be redirected to the authentication challenge.
1
/opt/google/chrome/chrome $APP_URL/productpage
Copied!
If you use username: user1 and password: password Gloo should redirect you back to the bookinfo application.
Keycloak Authentication Dialog

Lab 5 - Rate limiting

In this step, you are going to use rate limiting to protect only bookinfo applications.
To enable rate limiting on your Virtual Service, you will first create a RateLimitConfig resource:
1
kubectl apply -f - << EOF
2
apiVersion: ratelimit.solo.io/v1alpha1
3
kind: RateLimitConfig
4
metadata:
5
name: global-limit
6
namespace: gloo-system
7
spec:
8
raw:
9
descriptors:
10
- key: generic_key
11
value: count
12
rateLimit:
13
requestsPerUnit: 10
14
unit: MINUTE
15
rateLimits:
16
- actions:
17
- genericKey:
18
descriptorValue: count
19
EOF
Copied!
Now let's update your Virtual Service to use the bookinfo application with the new rate limit enforced. The changes affect only one matcher (the one for bookinfo):
1
kubectl apply -f - <<EOF
2
apiVersion: gateway.solo.io/v1
3
kind: VirtualService
4
metadata:
5
name: demo
6
namespace: gloo-system
7
spec:
8
sslConfig:
9
secretRef:
10
name: upstream-tls
11
namespace: gloo-system
12
virtualHost:
13
domains:
14
- '*'
15
routes:
16
- matchers:
17
- prefix: /not-secured
18
delegateAction:
19
selector:
20
namespaces:
21
- team1
22
labels:
23
application-owner: team1
24
- matchers:
25
- prefix: /
26
options:
27
extauth:
28
configRef:
29
name: oauth
30
namespace: gloo-system
31
# ---------------- Rate limit config ------------------
32
rateLimitConfigs:
33
refs:
34
- name: global-limit
35
namespace: gloo-system
36
#------------------------------------------------------
37
routeAction:
38
single:
39
upstream:
40
name: bookinfo-productpage-9080
41
namespace: gloo-system
42
EOF
Copied!
To test rate limiting, refresh the browser until you see a 429 message.
Browser 429 Too Many Requests Interface
1
/opt/google/chrome/chrome $APP_URL/productpage
Copied!

Different Rate Limiting For Authenticated And Anonymous Users

Following example of Rate Limit shows how you can specify different rate limiting for Authenticated users and for Anonymous users.
First, let's delete the Rate Limit Configuration you have created before:
1
kubectl delete ratelimitconfig global-limit -n gloo-system
Copied!
And now, let's update your Virtual Service to add the Rate Limit Configuration directly in the Virtual Service resource:
1
kubectl apply -f - <<EOF
2
apiVersion: gateway.solo.io/v1
3
kind: VirtualService
4
metadata:
5
name: demo
6
namespace: gloo-system
7
spec:
8
sslConfig:
9
secretRef:
10
name: upstream-tls
11
namespace: gloo-system
12
virtualHost:
13
domains:
14
- '*'
15
options:
16
# -------- Rate limit config by authenticated and anonymous -------
17
ratelimitBasic:
18
anonymousLimits:
19
requestsPerUnit: 5
20
unit: MINUTE
21
authorizedLimits:
22
requestsPerUnit: 20
23
unit: MINUTE
24
# -----------------------------------------------------------------
25
routes:
26
- matchers:
27
- prefix: /not-secured
28
delegateAction:
29
selector:
30
namespaces:
31
- team1
32
labels:
33
application-owner: team1
34
- matchers:
35
- prefix: /
36
options:
37
extauth:
38
configRef:
39
name: oauth
40
namespace: gloo-system
41
routeAction:
42
single:
43
upstream:
44
name: bookinfo-productpage-9080
45
namespace: gloo-system
46
EOF
Copied!
Access the httpbin application 10 times. After reaching the Rate Limit, you will retrieve a 429 error. This is because you are an anonymous user in that application.
1
for i in {1..10}
2
do
3
curl -k -s -o /dev/null -w "%{http_code}" $(glooctl proxy url --port https)/not-secured/get; echo
4
done
Copied!
Now, try it with the bookinfo application, and the number of requests will be limited to 20 as you are an Authenticated user.
Notice that the requests for static content also count since they share the route /. /productpage /static/
1
/opt/google/chrome/chrome $(glooctl proxy url --port https)/productpage
Copied!

Lab 6 - Web application firewall

A web application firewall (WAF) protects web applications by monitoring, filtering and blocking potentially harmful traffic and attacks that can overtake or exploit them.
Gloo Edge Enterprise includes the ability to enable the ModSecurity Web Application Firewall for any incoming and outgoing HTTP connections.
Let's update the Virtual Service to restrict requests with huge payloads to avoid Large Payload Post DDoS Attacks. The intention of this DDoS attack is abuse of the amount of memory used to decode the payload bringing the server down.
1
kubectl apply -f - <<EOF
2
apiVersion: gateway.solo.io/v1
3
kind: VirtualService
4
metadata:
5
name: demo
6
namespace: gloo-system
7
spec:
8
sslConfig:
9
secretRef:
10
name: upstream-tls
11
namespace: gloo-system
12
virtualHost:
13
domains:
14
- '*'
15
options:
16
# ---------------- Web Application Firewall -----------
17
waf:
18
customInterventionMessage: "Payload sizes above 1KB not allowed"
19
ruleSets:
20
- ruleStr: |
21
SecRuleEngine On
22
SecRequestBodyLimit 1
23
SecRequestBodyLimitAction Reject
24
#------------------------------------------------------
25
routes:
26
- matchers:
27
- prefix: /not-secured
28
delegateAction:
29
selector:
30
namespaces:
31
- team1
32
labels:
33
application-owner: team1
34
- matchers:
35
- prefix: /
36
options:
37
extauth:
38
configRef:
39
name: oauth
40
namespace: gloo-system
41
routeAction:
42
single:
43
upstream:
44
name: bookinfo-productpage-9080
45
namespace: gloo-system
46
EOF
Copied!
The rule means that the request body size cannot be greater than 1KB.
Run following command to test it:
1
curl -k $(glooctl proxy url --port https)/not-secured/post -d "<note><heading>Reminder</heading><body>are you sure this is a huge payload???</body></note>" -H "Content-Type: application/xml"
Copied!
You should get the following error message:
1
Payload sizes above 1KB not allowed
Copied!
WAF Blocks well-known harmful User-Agents
Another kind of attack is done by some recognized User-Agents.
In this step, you will block scammer:
1
kubectl apply -f - <<EOF
2
apiVersion: gateway.solo.io/v1
3
kind: VirtualService
4
metadata:
5
name: demo
6
namespace: gloo-system
7
spec:
8
sslConfig:
9
secretRef:
10
name: upstream-tls
11
namespace: gloo-system
12
virtualHost:
13
domains:
14
- '*'
15
options:
16
# -------- Web Application Firewall - Check User-Agent -----------
17
waf:
18
customInterventionMessage: "Blocked Scammer"
19
ruleSets:
20
- ruleStr: |
21
SecRuleEngine On
22
SecRule REQUEST_HEADERS:User-Agent "scammer" "deny,status:403,id:107,phase:1,msg:'blocked scammer'"
23
#-------------------------------------------------------------------
24
routes:
25
- matchers:
26
- prefix: /not-secured
27
delegateAction:
28
selector:
29
namespaces:
30
- team1
31
labels:
32
application-owner: team1
33
- matchers:
34
- prefix: /
35
options:
36
extauth:
37
configRef:
38
name: oauth
39
namespace: gloo-system
40
routeAction:
41
single:
42
upstream:
43
name: bookinfo-productpage-9080
44
namespace: gloo-system
45
EOF
Copied!
The rule means a well known scammer User-Agent will be blocked.
Run following command to test it:
1
curl -k $(glooctl proxy url --port https)/not-secured/get -H "Content-Type: application/xml" -H 'User-Agent: scammer'
Copied!
You should get the following error message:
1
Blocked Scammer
Copied!

Lab 7 - Data transformation

In this section you will explore how to transform requests and responses using Gloo Edge.

Response Transformation

Before, it was mentioned that httpbin is a good tool to debug and test things out. In this section, you will use that application to easily spot the results.
Following example demonstrates how to modify a response using Gloo Edge. You are going to return a basic html page when the response code is 429 (rate limited).
1
kubectl apply -f - <<EOF
2
apiVersion: gateway.solo.io/v1
3
kind: RouteTable
4
metadata:
5
name: httpbin-routetable
6
namespace: team1
7
labels:
8
application-owner: team1
9
spec:
10
routes:
11
# -------- Rate limit at route level requires to give a name -------
12
- name: "not-secured"
13
# ------------------------------------------------------------------
14
matchers:
15
- prefix: /not-secured
16
options:
17
prefixRewrite: '/'
18
# -------- Rate limit as you saw before ------------
19
ratelimitBasic:
20
anonymousLimits:
21
requestsPerUnit: 5
22
unit: MINUTE
23
# --------------------------------------------------
24
# ---------------- Transformation ------------------
25
transformations:
26
responseTransformation:
27
transformationTemplate:
28
parseBodyBehavior: DontParse
29
body:
30
text: '
31
32
<div data-gb-custom-block data-tag="if" data-0=':status' data-1=') == ' data-2='429' data-3='429'></div><html><body style="background-color:powderblue;"><h1>Too many Requests!</h1><p>Try again after 1 minute</p></body></html><div data-gb-custom-block data-tag="else">{{ body() }}</div>'
33
#---------------------------------------------------
34
routeAction:
35
single:
36
upstream:
37
name: team1-httpbin-8000
38
namespace: gloo-system
39
EOF
Copied!
Refreshing your browser more than 5 times, you should be able to see a styled HTML page indicating that you have reached the limit you had configured before.
1
/opt/google/chrome/chrome $(glooctl proxy url --port https)/not-secured/get
Copied!

Manipulate the response when a 401 Not Authorized is returned

The following example shows how to create a transformation which modifies the body when the error code is 401 Not Authorized.
This transformation applies to a scenario where you use ExtAuth. Now, you will only simulate the 401.
In Gloo, you can define transformations to happen:
  • In an early stage. Before all filters.
  • In a regular stage. After all filters.
Notice that the response transformation is flagged as early. The reason is that an Auth Server returning 401 will cancel any filter or transformation designed to happen after that (regular stage). Therefore, you need to prepare the response transformation before the Auth Server filter happens in an early stage.
1
kubectl apply -f - <<EOF
2
apiVersion: gateway.solo.io/v1
3
kind: RouteTable
4
metadata:
5
name: httpbin-routetable
6
namespace: team1
7
labels:
8
application-owner: team1
9
spec:
10
routes:
11
- matchers:
12
- prefix: /not-secured
13
options:
14
prefixRewrite: '/'
15
# ---------------- Transformation ------------------
16
stagedTransformations:
17
early:
18
responseTransforms:
19
- responseTransformation:
20
transformationTemplate:
21
parseBodyBehavior: DontParse
22
body:
23
text: |
24
<div data-gb-custom-block data-tag="if" data-0=':status' data-1=') == ' data-2='401' data-3='401'></div>
25
Hold the horses! You are not authenticated
26
<div data-gb-custom-block data-tag="else">{{ body() }}</div>
27
#---------------------------------------------------
28
routeAction:
29
single:
30
upstream:
31
name: team1-httpbin-8000
32
namespace: gloo-system
33
EOF
Copied!
With the httpbin application, you can simulate that the server returns a 401 status code. Let's run it and see that calling /status/401, the returned body is what you expected
1
curl -v -k $(glooctl proxy url --port https)/not-secured/status/401
Copied!
And any other status code, returns the expected body:
1
curl -v -k $(glooctl proxy url --port https)/not-secured/status/200
Copied!

Early response transformation

You can also extract information from one header with regular expressions and inject them into another header:
1
kubectl apply -f - <<EOF
2
apiVersion: gateway.solo.io/v1
3
kind: RouteTable
4
metadata:
5
name: httpbin-routetable
6
namespace: team1
7
labels:
8
application-owner: team1
9
spec:
10
routes:
11
- matchers:
12
- prefix: /not-secured
13
options:
14
prefixRewrite: '/'
15
stagedTransformations:
16
early:
17
# ---------------- Transformation with regex ------------------
18
requestTransforms:
19
- requestTransformation:
20
transformationTemplate:
21
extractors:
22
apiKey:
23
header: 'x-my-initial-header'
24
regex: '^Bearer (.*)#x27;
25
subgroup: 1
26
headers:
27
x-my-final-header:
28
text: '{{ apiKey }}'
29
#--------------------------------------------------------------
30
routeAction:
31
single:
32
upstream:
33
name: team1-httpbin-8000
34
namespace: gloo-system
35
EOF
Copied!
With the httpbin application, calling /get, you can debug the request headers. since you are transforming it to add a new header, after running:
1
curl -k $(glooctl proxy url --port https)/not-secured/get -H "x-my-initial-header: Bearer this_is_my_token_for_test"
Copied!
You can see the new X-My-Final-Header header only with the token:
1
{
2
"headers": {
3
[...]
4
"X-My-Final-Header": "this_is_my_token_for_test",
5
"X-My-Initial-Header": "Bearer this_is_my_token_for_test"
6
},
7
[...]
8
}
Copied!

Extract information from the JWT token

The JWT token contains a number of claims that you can use to drive RBAC decisions and potentially to provide other input to your upstream systems.
In this case, let's take a closer look at the claims that the Keycloak-generated JWT provides us. Paste the text of the Jwt header into the Encoded block at jwt.io, and it will show you the full set of claims available.
JWT Claims
Let's obtain those claims.
First, you need to assign the value to a variable of the keycloak master realm.
1
KEYCLOAK_MASTER_REALM_URL=http://$(kubectl get svc keycloak -ojsonpath='{.status.loadBalancer.ingress[0].ip}'):8080/auth/realms/master
Copied!
Now, you can update the resources to validate the token, extract claims from the token and create new headers based on these claims.
Notice that you will move the extauth block to a global location in the resource so that it affects all routes. You want to test this with httpbin.
1
kubectl apply -f - <<EOF
2
apiVersion: gateway.solo.io/v1
3
kind: VirtualService
4
metadata:
5
name: demo
6
namespace: gloo-system
7
spec:
8
sslConfig:
9
secretRef:
10
name: upstream-tls
11
namespace: gloo-system
12
virtualHost:
13
domains:
14
- '*'
15
options:
16
#-------------- Moved from specific route to a global place -----------------
17
extauth:
18
configRef:
19
name: oauth
20
namespace: gloo-system
21
#----------------------------------------------------------------------------
22
#--------------Extract claims-----------------
23
jwt:
24
providers:
25
keycloak:
26
issuer: ${KEYCLOAK_MASTER_REALM_URL}
27
keepToken: true
28
tokenSource:
29
headers:
30
- header: jwt
31
claimsToHeaders:
32
- claim: email
33
header: x-solo-claim-email
34
- claim: email_verified
35
header: x-solo-claim-email-verified
36
jwks:
37
remote:
38
url: http://keycloak.default.svc:8080/auth/realms/master/protocol/openid-connect/certs
39
upstreamRef:
40
name: default-keycloak-8080
41
namespace: gloo-system
42
#---------------------------------------------
43
routes:
44
- matchers:
45
- prefix: /not-secured
46
delegateAction:
47
selector:
48
namespaces:
49
- team1
50
labels:
51
application-owner: team1
52
- matchers:
53
- prefix: /
54
routeAction:
55
single:
56
upstream:
57
name: bookinfo-productpage-9080
58
namespace: gloo-system
59
EOF
Copied!
Once again, you need to refresh your bookinfo application to refresh the token in case it expires.
1
/opt/google/chrome/chrome $(glooctl proxy url --port https)/productpage
Copied!
And now, let's see the results with the fresh token:
1
/opt/google/chrome/chrome $(glooctl proxy url --port https)/not-secured/get
Copied!
Here is the output you should get if you refresh the web page:
1
{
2
"args": {},
3
"headers": {
4
[...]
5
"X-Solo-Claim-Email": "[email protected]",
6
"X-Solo-Claim-Email-Verified": "false",
7
[...]
8
},
9
[...]
10
}
Copied!
As you can see, Gloo Edge has added the x-solo-claim-email and x-solo-claim-email-verified headers using the information it has extracted from the JWT token.
It will allow the application to know the user's email identity and if that email has been verified.

Lab 8 - OPA authorization

Gloo Edge can also be used to set RBAC rules based on OPA (Open Policy Agent) and its rego rules.
This model allows you to get fine-grained control over the Authorization in your applications. As well, this model is well adopted by the kubernetes community.
Let's add another configuration to the AuthConfig you created in previous Labs:
1
kubectl apply -f - <<EOF
2
apiVersion: enterprise.gloo.solo.io/v1
3
kind: AuthConfig
4
metadata:
5
name: oauth
6
namespace: gloo-system
7
spec:
8
configs:
9
- oauth2:
10
oidcAuthorizationCode:
11
appUrl: ${APP_URL}
12
callbackPath: /callback
13
clientId: ${client}
14
clientSecretRef:
15
name: oauth
16
namespace: gloo-system
17
issuerUrl: "${KEYCLOAK_URL}/realms/master/"
18
scopes:
19
- email
20
headers:
21
idTokenHeader: jwt
22
- opaAuth:
23
modules:
24
- name: allow-solo-email-users
25
namespace: gloo-system
26
query: "data.test.allow == true"
27
EOF
Copied!
As you can see, you keep the existing keycloak configuration. And you create another config for OPA and its rego rule.
Now, let's create the rego rule in a ConfigMap, as follows:
1
kubectl apply -f - <<EOF
2
apiVersion: v1
3
data:
4
policy.rego: |-
5
package test
6
7
default allow = false
8
9
allow {
10
[header, payload, signature] = io.jwt.decode(input.state.jwt)
11
endswith(payload["email"], "@solo.io")
12
}
13
14
kind: ConfigMap
15
metadata:
16
name: allow-solo-email-users
17
namespace: gloo-system
18
EOF
Copied!
Notice that this rule will allow only users with email ending with @solo.io
Finally, let's update the resources. You will disable RBAC to not interfere with this authorization mechanism:
1
kubectl apply -f - <<EOF
2
apiVersion: gateway.solo.io/v1
3
kind: VirtualService
4
metadata:
5
name: demo
6
namespace: gloo-system
7
spec:
8
sslConfig:
9
secretRef:
10
name: upstream-tls
11
namespace: gloo-system
12
virtualHost:
13
domains:
14
- '*'
15
options:
16
jwt:
17
providers:
18
keycloak:
19
issuer: ${KEYCLOAK_MASTER_REALM_UR