Configuring
Configuring
Zowe provides sample configurations that make it easy for you to run Zowe in Kubernetes. You can use them directly or as a reference.
You can customize the configuration or make your own. If you do so, note the following objects that are expected by the container deployments:
Kind | Name | Note |
---|---|---|
Namespace | zowe | |
ServiceAccount | zowe-sa | |
ConfigMap | zowe-certificates-cm | Contains zowe-certificates.env with the same format as seen on z/OS keystore |
Secret | zowe-certificates-secret | Contains the base64 PEM and P12 data for keystore and truststore |
Ingress | discovery-ingress | Used for external access to the Discovery service |
Ingress | gateway-ingress | Used for external access to the Gateway service |
Route | discovery | Used for external access to the Discovery service |
Route | gateway | Used for external access to the Gateway service |
Service | discovery-service | Used for internal or external access to the Discovery service |
Service | gateway-service | Used for external access to the Gateway service |
Service | catalog-service | Used for access to the Catalog service |
PersistentVolumeClaim | zowe-workspace-pvc | |
HorizontalPodAutoscaler | * | Autoscalers exist for the various pods |
PodDisruptionBudget | * | Disruption budgets exist for the various pods |
To configure the Zowe container environment, complete the following procedure.
#
1. Create namespace and service accountRun the following commands to create Zowe's Namespace zowe
with Service Account zowe-sa
.
kubectl apply -f common/zowe-ns.yamlkubectl apply -f common/zowe-sa.yaml
Note that by default, zowe-sa
service account has automountServiceAccountToken
disabled for security purposes.
To verify, check the following configurations.
kubectl get namespaces
should show a Namespacezowe
.This displays the default Namespace zowe, if not set.
kubectl get serviceaccounts --namespace zowe
should show a ServiceAccountzowe-sa
.This displays the default ServiceAccount zowe-sa, if not set.
#
2. Create Persistent Volume Claim (PVC)Zowe's PVC has a default StorageClass value that may not apply to all Kubernetes clusters. Check and customize the storageClassName
value of samples/workspace-pvc.yaml
as needed. You can use kubectl get sc
to confirm which StorageClass you can use.
After you customize the storageClassName
value, apply the result by issuing the following commands:
kubectl apply -f samples/workspace-pvc.yaml
To verify, run the following commands and check if the STATUS
of line item zowe-workspace-pvc
shows as Bound
.
kubectl get pvc --namespace zowe
Note that zowe-workspace-pvc
PersistentVolumeClaim
must be declared in access mode ReadWriteMany
.
In some Kubernetes environment, you may need to define PeristentVolume
and define volumeName
in PersistentVolumeClaim
instead of defining storageClassName
. Please consult your Kubernetes administrator to confirm the appropriate way for your environment. This is an example to configure PersistentVolumeClaim
with pre-configured zowe-workspace-pv
PeristentVolume
.
apiVersion: v1kind: PersistentVolumeClaimmetadata: name: zowe-workspace-pvc namespace: zowe labels: app.kubernetes.io/name: zowe app.kubernetes.io/instance: zowe app.kubernetes.io/managed-by: manualspec: storageClassName: "" volumeName: zowe-workspace-pv accessModes: - ReadWriteMany resources: requests: storage: 50Mi
#
3. Create and modify ConfigMaps and SecretsSimilarly, to running Zowe services on z/OS, you can use either instance.env
or zowe.yaml
to customize Zowe.
You can modify samples/config-cm.yaml
, , and samples/certificates-secret.yaml
directly. Or more conveniently, if you have Zowe ZSS/ZIS running on z/OS, the Kubernetes environment can reuse instance and keystore configuration from that installation (supported in v1.25 and later).
If you want to manually create, or later customize the ConfigMaps and Secrets, see Customizing or manually creating ConfigMaps and Secrets for details.
To create and modify ConfigMaps and Secrets by using the migrate configuration script, complete the following steps:
a. On z/OS, run the following command:
cd <instance-dir> ./bin/utils/convert-for-k8s.sh -x "my-k8s-cluster.company.com,9.10.11.12"
This migration script supports these parameters:
-x
: is a comma-separated list of domains you will use to visit the Zowe Kubernetes cluster. These domains and IP addresses will be added to your new certificate if needed. This is optional. The default value islocalhost
.-n
: is the Zowe Kubernetes cluster namespace. This is optional. The default value iszowe
.-u
: is the Kubernetes cluster name. This is optional. The default value iscluster.local
.-p
: is the password of the local certificate authority PKCS#12 file. This is optional. The default value islocal_ca_password
.-a
: is the certificate alias of the local certificate authority. This is optional. The default value islocalca
.-v
: is a switch to enable verbose mode which will display more debugging information.
As a result, it displays ConfigMaps (zowe-config
, zowe-certificates-cm
) and Secrets (zowe-certificates-secret
) Kubernetes objects which are based on the Zowe instance and keystore used. The content looks similar to samples/config-cm.yaml
, samples/certificates-cm.yaml
and samples/certificates-secret.yaml
but with real values.
b. Follow the instructions in the script output to copy the output and save it as a YAML file configs.yaml
on your computer where you manage Kubernetes.
c. Apply the file into Kubernetes:
`kubectl apply -f /path/to/your/configs.yaml`
d. Remove the previously saved configs.yaml
file from all systems for security.
To verify:
kubectl get configmaps --namespace zowe
This command must display the two ConfigMaps
zowe-config
andzowe-certificates-cm
.kubectl get secrets --namespace zowe
This command must display a Secret
zowe-certificates-secret
.
#
4. Expose API Mediation Layer componentsThis step makes Zowe's Gateway, Discovery, and API Catalog servers available over a network.
The Gateway is always required to be externally accessible, and depending upon your environment the Discovery service may also need to be externally accessible.
The actions you need to take in this step vary depending upon your Kubernetes cluster configuration. If you are uncertain about this section, please contact your Kubernetes administrator or the Zowe community.
#
4a. Create serviceYou can set up either a LoadBalancer
or NodePort
type Service.
Note: Because NodePort
cannot be used together with NetworkPolicies
, LoadBalancer
and Ingress
is preferred configuration option.
Review the following table for steps you may take depending on the Kubernetes provider you use. If you don't need additional setups, you can skip steps 4b, 4c and jump directly to the Apply zowe section.
Kubernetes provider | Service | Additional setups required |
---|---|---|
minikube | LoadBalancer or NodePort | Port Forward (on next section Starting, stopping, and monitoring) |
docker-desktop | LoadBalancer | none |
bare-metal | LoadBalancer or NodePort | Create Ingress |
cloud-vendors | LoadBalancer | none |
OpenShift | LoadBalancer or NodePort | Create Route |
#
Defining api-catalog serviceapi-catalog-service
is required by Zowe, but not necessarily exposed to external users. Therefore, api-catalog-service
is defined as type ClusterIP
.
To define this service, run the command:
kubectl apply -f samples/api-catalog-service.yaml
Upon success, you should see the following output:
service/api-catalog-service created
Then, you can proceed with creating the Gateway and Discovery services according to your environment.
#
Applying Gateway ServiceIf using LoadBalancer
, run the command:
kubectl apply -f samples/gateway-service-lb.yaml
Or if using NodePort
instead, first check spec.ports[0].nodePort
as this will be the port to be exposed to external. In this case, the default gateway port is not 7554 but 32554. You will need to use https://<your-k8s-node>:32554/
to access APIML Gateway. To apply NodePort
type gateway-service
, run the following command:
kubectl apply -f samples/gateway-service-np.yaml
To verify either case, run the following command and check that the command displays the service gateway-service
.
kubectl get services --namespace zowe
#
Applying Discovery serviceExposing the Discovery service is only required when there is a Zowe service or extension which needs to be registered to the API Mediation Layer but is running outside of Kubernetes, such as on z/OS. Otherwise, the discovery service can remain accessible only within the Kubernetes environment.
Optional: To setup the discovery service without exposing it externally, edit samples/discovery-service-lb.yaml
if using LoadBalancer
type services, or samples/discovery-service-np.yaml
if using NodePort
type services. In either file, specify ClusterIP
as the type, replacing the NodePort
or LoadBalancer
value.
To enable the service externally when using LoadBalancer
services, run the command:
kubectl apply -f samples/discovery-service-lb.yaml
Or if using NodePort
instead, first check spec.ports[0].nodePort
as this will be the port to be exposed to external. In this case, the default discovery port is not 7553 but 32553. And you will need to use https://<your-k8s-node>:32553/
to access APIML Discovery. To apply NodePort
type discovery-service
, run the following command:
kubectl apply -f samples/discovery-service-np.yaml
To verify either case, run the following command and check that this command displays the service discovery-service
:
kubectl get services --namespace zowe
Upon completion of all the preceding steps in this a. Create service section, you may need to run additional setups. Refer to "Additional setups required" in the table. If you don't need additional setups, you can skip 4b, 4c, 4d, and jump directly to Apply Zowe section.
#
4b. Create Ingress (Bare-metal)An Ingress gives Services externally-reachable URLs and may provide other abilities such as traffic load balancing.
To create Ingress, perform the following steps:
a. Edit samples/gateway-ingress.yaml
and samples/discovery-ingress.yaml
before applying them, by uncommenting the lines (19 and 20) for defining spec.rules[0].host
and http:
, and then commenting out the line below, - http:
b. Run the following commands:
kubectl apply -f samples/bare-metal/gateway-ingress.yamlkubectl apply -f samples/bare-metal/discovery-ingress.yaml
To verify, run the following commands:
kubectl get ingresses --namespace zowe
This command must display two Ingresses gateway-ingress
and discovery-ingress
.
Upon completion, you can finish the setup by applying zowe and starting it.
#
4c. Create Route (OpenShift)If you are using OpenShift and choose to use LoadBalancer
services, you may already have an external IP for the service. You can use that external IP to access Zowe APIML Gateway. To verify your service external IP, run:
oc get svc -n zowe
If you see an IP in the EXTERNAL-IP
column, that means your OpenShift is properly configured and can provision external IP for you. If you see <pending>
and it does not change after waiting for a while, that means you may not be able to use LoadBalancer
services with your current configuration. Try ClusterIP
services and define Route
. A Route is a way to expose a service by giving it an externally-reachable hostname.
To create a route, perform the following steps:
a. Check and set the value of spec.port.targetPort
in samples/gateway-route.yaml
and samples/discovery-route.yaml
before applying the changes.
b. Run the following commands:
oc apply -f samples/openshift/gateway-route.yamloc apply -f samples/openshift/discovery-route.yaml
To verify, run the following commands:
oc get routes --namespace zowe
This command must display the two Services gateway
and discovery
.
Upon completion, you can finish the setup by applying zowe and starting it.
#
Customizing or manually creating ConfigMaps and SecretsThe z/OS to k8s convert tool can automatically create a config map and secret. However, if you want to customize or create your own, review the instructions in this section.
To manually create the ConfigMaps and Secrets used by Zowe containers, you must create the following objects:
A ConfigMap, with values based upon a Zowe instance's
instance.env
and similar to the examplesamples/config-cm.yaml
with the following differences to the values seen on a z/OS install:ZOWE_EXPLORER_HOST
,ZOWE_IP_ADDRESS
,ZWE_LAUNCH_COMPONENTS
,ZWE_DISCOVERY_SERVICES_LIST
andSKIP_NODE
are not needed for Zowe running in Kubernetes and will be ignored. You can remove them.JAVA_HOME
andNODE_HOME
are not usually needed if you are using Zowe base images.ROOT_DIR
must be set to/home/zowe/runtime
.KEYSTORE_DIRECTORY
must be set to/home/zowe/keystore
.ZWE_EXTERNAL_HOSTS
is suggested to define as a list of domains you are using to access your Kubernetes cluster.ZOWE_EXTERNAL_HOST=$(echo "${ZWE_EXTERNAL_HOSTS}" | awk -F, '{print $1}' | tr -d '[[:space:]]')
is needed to define afterZWE_EXTERNAL_HOSTS
. It's the primary external domain.ZWE_EXTERNAL_PORT
(orzowe.externalPort
if you are usingzowe.yaml
) must be the port you expose to end-user. This value is optional if it's same as defaultGATEWAY_PORT
7554
. With default settings,- if you choose
LoadBalancer
gateway-service
, this value is optional, or set to7554
, - if you choose
NodePort
gateway-service
and access the service directly, this value should be same asspec.ports[0].nodePort
with default value32554
, - if you choose
NodePort
gateway-service
and access the service through port forwarding, the value should be the forwarded port you set.
- if you choose
ZOWE_ZOS_HOST
is recommended to be set to where the z/OS system where your Zowe ZSS/ZIS is running.ZWE_DISCOVERY_SERVICES_REPLICAS
should be set to same value ofspec.replicas
defined inworkloads/discovery-statefulset.yaml
.- All components running in Kubernetes should use default ports:
CATALOG_PORT
is7552
,DISCOVERY_PORT
is7553
,GATEWAY_PORT
is7554
,ZWE_CACHING_SERVICE_PORT
is7555
,JOBS_API_PORT
is8545
,FILES_API_PORT
is8547
,JES_EXPLORER_UI_PORT
is8546
,MVS_EXPLORER_UI_PORT
is8548
,USS_EXPLORER_UI_PORT
is8550
,ZOWE_ZLUX_SERVER_HTTPS_PORT
is8544
.
ZOWE_ZSS_SERVER_PORT
should be set to the port where your Zowe ZSS is running onZOWE_ZOS_HOST
.APIML_GATEWAY_EXTERNAL_MAPPER
should be set tohttps://${GATEWAY_HOST}:${GATEWAY_PORT}/zss/api/v1/certificate/x509/map
.APIML_SECURITY_AUTHORIZATION_ENDPOINT_URL
should be set tohttps://${GATEWAY_HOST}:${GATEWAY_PORT}/zss/api/v1/saf-auth
.ZOWE_EXPLORER_FRAME_ANCESTORS
should be set to${ZOWE_EXTERNAL_HOST}:*
ZWE_CACHING_SERVICE_PERSISTENT
should NOT be set toVSAM
.redis
is suggested. Follow Redis configuration documentation to customize other Redis related variables. Leave the value to empty for debugging purposes.- Must append and customize these 2 values:
ZWED_agent_host=${ZOWE_ZOS_HOST}
ZWED_agent_https_port=${ZOWE_ZSS_SERVER_PORT}
If you are using
zowe.yaml
, the above configuration items are still valid but should use the matchingzowe.yaml
configuration entries. Check Updating the zowe.yaml configuration file for more details.A ConfigMap, with values based upon a Zowe keystore's
zowe-certificates.env
and similar to the examplesamples/certificates-cm.yaml
.A Secret, with values based upon a Zowe keystore's files, and similar to the example
samples/certificates-secret.yaml
.
PodDisruptionBudget
#
Zowe provides optional PodDisruptionBudget
which can provide high availability during upgrade. By default, Zowe defines minAvailable
to be 1
for all deployments. This configuration is optional but recommended. To apply PodDisruptionBudget
, run this command:
kubectl apply -f samples/pod-disruption-budget/
To verify this step, run:
kubectl get pdb --namespace zowe
This should show you a list of PodDisruptionBudget
like this:
NAME MIN AVAILABLE MAX UNAVAILABLE ALLOWED DISRUPTIONS AGEapi-catalog-pdb 1 N/A 0 1dapp-server-pdb 1 N/A 0 1dcaching-pdb 1 N/A 0 1ddiscovery-pdb 1 N/A 0 1dexplorer-jes-pdb 1 N/A 0 1dexplorer-mvs-pdb 1 N/A 0 1dexplorer-uss-pdb 1 N/A 0 1dfiles-api-pdb 1 N/A 0 1dgateway-pdb 1 N/A 0 1djobs-api-pdb 1 N/A 0 1d
HorizontalPodAutoscaler
#
Zowe provides optional HorizontalPodAutoscaler
which can automatically scale Zowe components based on resource usage. By default, each workload has a minimum of 1 replica and a maximum of 3 to 5 replicas based on CPU usage. This configuration is optional but recommended. HorizontalPodAutoscaler
relies on Kubernetes Metrics server monitoring to provide metrics through the Metrics API. To learn how to deploy the metrics-server, see the metrics-server documentation. Please adjust the HorizontalPodAutoscaler
definitions based on your cluster resources, then run this command to apply them to your cluster:
kubectl apply -f samples/horizontal-pod-autoscaler/
To verify this step, run:
kubectl get hpa --namespace zowe
This should show you a list of HorizontalPodAutoscaler
like this:
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGEapi-catalog-hpa Deployment/api-catalog 60%/70% 1 3 1 20mapp-server-hpa Deployment/app-server 2%/70% 1 5 1 9m59scaching-hpa Deployment/caching 7%/70% 1 3 1 9m59sdiscovery-hpa StatefulSet/discovery 34%/70% 1 3 1 8m15sexplorer-jes-hpa Deployment/explorer-jes 10%/70% 1 3 1 9m59sexplorer-mvs-hpa Deployment/explorer-mvs 10%/70% 1 3 1 9m59sexplorer-uss-hpa Deployment/explorer-uss 10%/70% 1 3 1 9m59sfiles-api-hpa Deployment/files-api 8%/70% 1 3 1 9m59sgateway-hpa Deployment/gateway 20%/60% 1 5 1 9m59sjobs-api-hpa Deployment/jobs-api 8%/70% 1 3 1 9m59s
#
Kubernetes v1.21+If you have Kubernetes v1.21+, several optional changes are recommended based on Deprecated API Migration Guide.
- Kind
CronJob
: changeapiVersion: batch/v1beta1
toapiVersion: batch/v1
onworkloads/zowe-yaml/cleanup-static-definitions-cronjob.yaml
andworkloads/instance-env/cleanup-static-definitions-cronjob.yaml
.apiVersion: batch/v1beta1
will stop working on Kubernetes v1.25. - Kind
PodDisruptionBudget
: changeapiVersion: policy/v1beta1
toapiVersion: policy/v1
on all files insamples/pod-disruption-budget/
.apiVersion: policy/v1beta1
will stop working on Kubernetes v1.25.