Helm install of stable/nginx-ingress fails to deploy the Ingress Controller

Article ID:360020511351
3 minute readKnowledge base

Issue

  • I installed the Nginx Ingress controller using Helm with a controller scope controller.scope.namespace=cloudbees-core but the ingress controllers fails to deploy. The logs of the failing pod shows:

F1127 06:13:04.673757       6 main.go:105] No namespace with name cloudbees-core found: namespaces "cloudbees-core" is forbidden: User "system:serviceaccount:ingress-nginx:nginx-ingress" cannot get namespaces in the namespace "cloudbees-core"

Environment

Related Issues

  • helm/charts #8914: Skipping clusterrole and clusterrolebinding when scope is restricted does not make sense

Explanation

This is related to a bug introduced in version 0.22.1 of the chart. This issue is tracked in helm/charts #8914.

Since version 0.22.1, the chart skips the creation of ClusterRole and ClusterRoleBinding when the controller scope is enabled, even though RBAC is enabled. So when installing ingress nginx restricting it to watch ingresses of only a specific namespace - ie.e the cloudbees-core namespace:

helm install --namespace ingress-nginx --name nginx-ingress stable/nginx-ingress \
             --set rbac.create=true \
             --set defaultBackend=false \
             --set controller.service.externalTrafficPolicy=Local \
             --set controller.scope.enabled=true \
             --set controller.scope.namespace=cloudbees-core

It fails with permission issues because the nginx-ingress service account lacks permission to get the cloudbees-core namespace.

Resolution

Until a solution is provided in the Helm chart, there are different workarounds.

Workaround using Helm

A workaround could be to install using Helm without the controller scope and configure it to watch the cloudbees-core namespace separately.

1) Create a YAML file patch-nginx-ingress-clusterrole.yaml (you may change nginx-ingress-0.31.0 by the version of the chart that is installed):

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-0.31.0
    heritage: Tiller
    release: nginx-ingress
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - namespaces
    resourceNames:
      - cloudbees-core
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses/status
    verbs:
      - update

2) Install the stable/nginx-ingress chart with the following command:

helm install --namespace ingress-nginx --name nginx-ingress stable/nginx-ingress \
             --set rbac.create=true \
             --set controller.service.externalTrafficPolicy=Local

3) Apply the patched ClusterRole:

kubectl apply -f patch-nginx-ingress-clusterrole.yaml

4) Patch the Ingress Nginx controller deployment to watch the namespace:

kubectl patch deployment nginx-ingress-controller -p '{"spec":{"template":{"spec":{"containers":[{"args":["/nginx-ingress-controller","--default-backend-service=ingress-nginx/nginx-ingress-default-backend","--election-id=ingress-controller-leader","--ingress-class=nginx","--configmap=ingress-nginx/nginx-ingress-controller","--watch-namespace=cloudbees-core"],"name":"nginx-ingress-controller"}]}}}}' -n ingress-nginx

Workaround using manual installation

Another workaround is to install the Nginx controller manually and configuring it to watch specific namespace.

1) Follow the installation of Kubernetes Ingress Nginx for the corresponding platform

2) Create a YAML file patch-nginx-ingress-clusterrole.yaml:

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - namespaces
    resourceNames:
      - cloudbees-core
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses/status
    verbs:
      - update

2) Apply the patched ClusterRole:

kubectl apply -f patch-nginx-ingress-clusterrole.yaml

3) Patch the Ingress Nginx controller deployment to watch the namespace:

kubectl patch deployment nginx-ingress-controller -p '{"spec":{"template":{"spec":{"containers":[{"args":["/nginx-ingress-controller","--configmap=$(POD_NAMESPACE)/nginx-configuration","--tcp-services-configmap=$(POD_NAMESPACE)/tcp-services","--udp-services-configmap=$(POD_NAMESPACE)/udp-services","--publish-service=$(POD_NAMESPACE)/ingress-nginx","--annotations-prefix=nginx.ingress.kubernetes.io","--watch-namespace=cloudbees-core"],"name":"nginx-ingress-controller"}]}}}}' -n ingress-nginx