[stable/artifactory-ha] Support for configMap to bootstrap Artifactory (#6010)

* added support for ePlus

* no message

* no message

* added replicator port

* removed replicator config from nodes

* added http version

* condition to avoid duplication

* allowing replicator to run in all nodes

* bug fix for nginx

* updated InactiveServerCleaner plugin logic

* support for configmap to bootstrap artifactory

* added support for static pv

* removed duplicate

* added env to skipAutoConfig update in nginx

* fix

stable/artifactory-ha/Chart.yaml

@@ -1,7 +1,7 @@
 apiVersion: v1
 name: artifactory-ha
 home: https://www.jfrog.com/artifactory/
-version: 0.2.0
+version: 0.2.1
 appVersion: 6.0.0
 description: Universal Repository Manager supporting all major packaging formats,
   build tools and CI servers.

stable/artifactory-ha/README.md

@@ -178,6 +178,34 @@ $ helm install --name artifactory-ha --set artifactory.license.secret=artifactor
 ```
 **NOTE:** You have to keep passing the license secret parameters as `--set artifactory.license.secret=artifactory-cluster-license,artifactory.license.dataKey=art.lic` on all future calls to `helm install` and `helm upgrade`!
 
+## Bootstrapping Artifactory
+**IMPORTANT:** Bootstrapping Artifactory needs license. Pass license as shown in above section.
+
+* User guide to [bootstrap Artifactory Global Configuration](https://www.jfrog.com/confluence/display/RTF/Configuration+Files#ConfigurationFiles-BootstrappingtheGlobalConfiguration)
+* User guide to [bootstrap Artifactory Security Configuration](https://www.jfrog.com/confluence/display/RTF/Configuration+Files#ConfigurationFiles-BootstrappingtheSecurityConfiguration)
+
+Create `bootstrap-config.yaml` with artifactory.config.import.xml and security.import.xml as shown below:
+```
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: my-release-bootstrap-config
+data:
+  artifactory.config.import.xml: |
+    <config contents>
+  security.import.xml: |
+    <config contents>
+```
+
+Create configMap in Kubernetes:
+```bash
+$ kubectl apply -f bootstrap-config.yaml
+```
+# Pass the configMap to helm
+```bash
+$ helm install --name artifactory-ha --set artifactory.license.secret=artifactory-cluster-license,artifactory.license.dataKey=art.lic,artifactory.configMapName=my-release-bootstrap-config stable/artifactory-ha
+```
+
 #### Scaling your Artifactory cluster
 A key feature in Artifactory HA is the ability to set an initial cluster size with `--set artifactory.node.replicaCount=${CLUSTER_SIZE}` and if needed, resize it.
 
@@ -268,7 +296,7 @@ The following table lists the configurable parameters of the artifactory chart a
 | `artifactory.name`                   | Artifactory name                     | `artifactory`                              |
 | `artifactory.image.pullPolicy`       | Container pull policy                | `IfNotPresent`                             |
 | `artifactory.image.repository`       | Container image                      | `docker.bintray.io/jfrog/artifactory-pro`  |
-| `artifactory.image.version`          | Container image tag                  | `5.10.1`                                    |
+| `artifactory.image.version`          | Container image tag                  | `6.0.0`                                    |
 | `artifactory.masterKey`      | Artifactory Master Key. Can be generated with `openssl rand -hex 32` |`FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF`|
 | `artifactory.license.secret` | Artifactory license secret name              |                                            |
 | `artifactory.license.dataKey`| Artifactory license secret data key          |                                            |
@@ -314,7 +342,7 @@ The following table lists the configurable parameters of the artifactory chart a
 | `artifactory.persistence.awsS3.path`                | AWS S3 path in bucket               | `artifactory-ha/filestore`   |
 | `artifactory.javaOpts.other` | Artifactory extra java options (for all nodes) | `-Dartifactory.locking.provider.type=db` |
 | `artifactory.replicator.enabled`            | Enable Artifactory Replicator | `false`  |
-| `artifactory.distributionCerts`            | Name of ConfigMap for Artifactory Distribution Certificate  | ``  |
+| `artifactory.distributionCerts`            | Name of ConfigMap for Artifactory Distribution Certificate  |   |
 | `artifactory.replicator.publicUrl`            | Artifactory Replicator Public URL |      |
 | `artifactory.primary.resources.requests.memory` | Artifactory primary node initial memory request  |                     |
 | `artifactory.primary.resources.requests.cpu`    | Artifactory primary node initial cpu request     |                     |
@@ -339,7 +367,7 @@ The following table lists the configurable parameters of the artifactory chart a
 | `nginx.name`                | Nginx name                        | `nginx`                                                |
 | `nginx.replicaCount`        | Nginx replica count               | `1`                                                    |
 | `nginx.image.repository`    | Container image                   | `docker.bintray.io/jfrog/nginx-artifactory-pro`        |
-| `nginx.image.version`       | Container version                 | `5.10.1`                                                |
+| `nginx.image.version`       | Container version                 | `6.0.0`                                                |
 | `nginx.image.pullPolicy`    | Container pull policy             | `IfNotPresent`                                         |
 | `nginx.service.type`        | Nginx service type                | `LoadBalancer`                                         |
 | `nginx.service.loadBalancerSourceRanges`| Nginx service array of IP CIDR ranges to whitelist (only when service type is LoadBalancer) |  |
@@ -364,6 +392,7 @@ The following table lists the configurable parameters of the artifactory chart a
 | `nginx.readinessProbe.failureThreshold`    | Minimum consecutive failures for the probe to be considered failed after having succeeded.   | 10 |
 | `nginx.tlsSecretName` |  SSL secret that will be used by the Nginx pod |    |
 | `nginx.env.ssl`                   | Nginx Environment enable ssl               | `true`                                  |
+| `nginx.env.skipAutoConfigUpdate`  | Nginx Environment to disable auto configuration update | `false`                     |
 | `nginx.resources.requests.memory` | Nginx initial memory request               | `250Mi`                                 |
 | `nginx.resources.requests.cpu`    | Nginx initial cpu request                  | `100m`                                  |
 | `nginx.resources.limits.memory`   | Nginx memory limit                         | `250Mi`                                 |

stable/artifactory-ha/templates/_helpers.tpl

@@ -62,3 +62,10 @@ If release name contains chart name it will be used as a full name.
 {{- end -}}
 {{- end -}}
 {{- end -}}
+
+{{/*
+Create chart name and version as used by the chart label.
+*/}}
+{{- define "artifactory-ha.chart" -}}
+{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
+{{- end -}}

stable/artifactory-ha/templates/artifactory-binarystore.yaml

@@ -4,7 +4,7 @@ metadata:
   name: {{ template "artifactory-ha.fullname" . }}-bs
   labels:
     app: {{ template "artifactory-ha.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    chart: {{ template "artifactory-ha.chart" . }}
     heritage: {{ .Release.Service }}
     release: {{ .Release.Name }}
 data:

stable/artifactory-ha/templates/artifactory-nfs-pvc.yaml

@@ -6,7 +6,7 @@ metadata:
   name: {{ template "artifactory-ha.fullname" . }}-data-pv
   labels:
     app: {{ template "artifactory-ha.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    chart: {{ template "artifactory-ha.chart" . }}
     heritage: {{ .Release.Service }}
     release: {{ .Release.Name }}
     id: {{ template "artifactory-ha.name" . }}-data-pv
@@ -28,7 +28,7 @@ metadata:
   name: {{ template "artifactory-ha.fullname" . }}-data-pvc
   labels:
     app: {{ template "artifactory-ha.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    chart: {{ template "artifactory-ha.chart" . }}
     heritage: {{ .Release.Service }}
     release: {{ .Release.Name }}
     type: nfs-volume
@@ -52,7 +52,7 @@ metadata:
   name: {{ template "artifactory-ha.fullname" . }}-backup-pv
   labels:
     app: {{ template "artifactory-ha.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    chart: {{ template "artifactory-ha.chart" . }}
     heritage: {{ .Release.Service }}
     release: {{ .Release.Name }}
     id: {{ template "artifactory-ha.name" . }}-backup-pv
@@ -74,7 +74,7 @@ metadata:
   name: {{ template "artifactory-ha.fullname" . }}-backup-pvc
   labels:
     app: {{ template "artifactory-ha.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    chart: {{ template "artifactory-ha.chart" . }}
     heritage: {{ .Release.Service }}
     release: {{ .Release.Name }}
     type: nfs-volume

stable/artifactory-ha/templates/artifactory-node-statefulset.yaml

@@ -4,12 +4,12 @@ metadata:
   name: {{ template "artifactory-ha.node.name" . }}
   labels:
     app: {{ template "artifactory-ha.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    chart: {{ template "artifactory-ha.chart" . }}
     component: {{ .Values.artifactory.name }}
     heritage: {{ .Release.Service }}
     release: {{ .Release.Name }}
 spec:
-  serviceName: "{{ template "artifactory-ha.node.name" . }}"
+  serviceName: {{ template "artifactory-ha.node.name" . }}
   replicas: {{ .Values.artifactory.node.replicaCount }}
   updateStrategy:
     type: RollingUpdate
@@ -223,11 +223,19 @@ spec:
         persistentVolumeClaim:
           claimName: {{ template "artifactory-ha.fullname" . }}-backup-pvc
       {{- end }}
+      {{- if .Values.artifactory.persistence.local }}
+      - name: volume
+        emptyDir: {}
+      {{- else }}
   volumeClaimTemplates:
   - metadata:
       name: volume
     spec:
-      accessModes: [ "ReadWriteOnce" ]
+    {{- if .Values.artifactory.node.persistence.existingClaim }}
+      selector:
+        matchLabels:
+          app: artifactory-ha-member
+    {{- else }}
       {{- if .Values.artifactory.persistence.storageClass }}
       {{- if (eq "-" .Values.artifactory.persistence.storageClass) }}
       storageClassName: ""
@@ -235,6 +243,9 @@ spec:
       storageClassName: "{{ .Values.artifactory.persistence.storageClass }}"
       {{- end }}
       {{- end }}
+      accessModes: [ "{{ .Values.artifactory.persistence.accessMode }}" ]
       resources:
         requests:
           storage: {{ .Values.artifactory.persistence.size }}
+    {{- end }}
+  {{- end }}

stable/artifactory-ha/templates/artifactory-plugin-inactiveservercleaner.yaml

@@ -4,7 +4,7 @@ metadata:
   name: {{ template "artifactory-ha.fullname" . }}-isc
   labels:
     app: {{ template "artifactory-ha.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    chart: {{ template "artifactory-ha.chart" . }}
     heritage: {{ .Release.Service }}
     release: {{ .Release.Name }}
 data:
@@ -18,12 +18,22 @@ data:
 
     jobs {
         clean(interval: 90000, delay: 900000) {
-            def artifactoryServersCommonService = ctx.beanForType(ArtifactoryServersCommonService)
-            def artifactoryInactiveServerCleaner = new ArtifactoryInactiveServersCleaner(artifactoryServersCommonService, log)
-            artifactoryInactiveServerCleaner.cleanInactiveArtifactoryServers()
+            runCleanupHAInactiveServers()
         }
     }
 
+    executions {
+        cleanHAInactiveServers() { params ->
+            runCleanupHAInactiveServers()
+        }
+    }
+
+    def runCleanupHAInactiveServers() {
+        def artifactoryServersCommonService = ctx.beanForType(ArtifactoryServersCommonService)
+        def artifactoryInactiveServerCleaner = new ArtifactoryInactiveServersCleaner(artifactoryServersCommonService, log)
+        artifactoryInactiveServerCleaner.cleanInactiveArtifactoryServers()
+    }
+
     public class ArtifactoryInactiveServersCleaner {
 
         private ArtifactoryServersCommonService artifactoryServersCommonService
@@ -40,7 +50,7 @@ data:
             for (member in allMembers) {
                 def heartbeat = TimeUnit.MILLISECONDS.toSeconds(System.currentTimeMillis() - member.getLastHeartbeat())
                 def noHeartbeat = heartbeat > ConstantValues.haHeartbeatStaleIntervalSecs.getInt()
-                if (member.getServerState() == ArtifactoryServerState.UNAVAILABLE || noHeartbeat) {
+                if (member.getServerState() == ArtifactoryServerState.UNAVAILABLE || ( noHeartbeat && member.getServerState() != ArtifactoryServerState.CONVERTING && member.getServerState() != ArtifactoryServerState.STARTING )) {
                     try {
                         log.info "Inactive artifactory servers cleaning task found server ${member.serverId} to remove"
                         artifactoryServersCommonService.removeServer(member.serverId)

stable/artifactory-ha/templates/artifactory-primary-statefulset.yaml

@@ -4,12 +4,12 @@ metadata:
   name: {{ template "artifactory-ha.primary.name" . }}
   labels:
     app: {{ template "artifactory-ha.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    chart: {{ template "artifactory-ha.chart" . }}
     component: {{ .Values.artifactory.name }}
     heritage: {{ .Release.Service }}
     release: {{ .Release.Name }}
 spec:
-  serviceName: "{{ template "artifactory-ha.primary.name" . }}"
+  serviceName: {{ template "artifactory-ha.primary.name" . }}
   replicas: 1
   updateStrategy:
     type: RollingUpdate
@@ -67,6 +67,9 @@ spec:
               - '/bin/sh'
               - '-c'
               - >
+                {{- if .Values.artifactory.configMapName }}
+                cp -Lrfv /bootstrap/* /artifactory_extra_conf/
+                {{- end }}
                 {{- if .Values.artifactory.replicator.enabled }}
                 chown -R {{ .Values.artifactory.uid }}:{{ .Values.artifactory.uid }} {{ .Values.artifactory.persistence.mountPath }}/access;
                 mkdir -p {{ .Values.artifactory.persistence.mountPath }}/replicator/etc;
@@ -173,6 +176,10 @@ spec:
           mountPath: "/artifactory_extra_conf/binarystore.xml"
           subPath: binarystore.xml
       {{- end }}
+      {{- if .Values.artifactory.configMapName }}
+        - name: bootstrap-config
+          mountPath: "/bootstrap/"
+      {{- end }}
       {{- if .Values.artifactory.license.secret }}
         - name: artifactory-license
           mountPath: "/artifactory_extra_conf/artifactory.cluster.license"
@@ -224,6 +231,11 @@ spec:
         configMap:
           name: {{ template "artifactory-ha.fullname" . }}-replicator-config
       {{- end }}
+      {{- if .Values.artifactory.configMapName }}
+      - name: bootstrap-config
+        configMap:
+          name: {{ .Values.artifactory.configMapName }}
+      {{- end}}
       - name: artifactory-inactiveservercleaner
         configMap:
           name: {{ template "artifactory-ha.fullname" . }}-isc
@@ -240,11 +252,19 @@ spec:
         persistentVolumeClaim:
           claimName: {{ template "artifactory-ha.fullname" . }}-backup-pvc
       {{- end }}
+      {{- if .Values.artifactory.persistence.local }}
+      - name: volume
+        emptyDir: {}
+      {{- else }}
   volumeClaimTemplates:
   - metadata:
       name: volume
     spec:
-      accessModes: [ "ReadWriteOnce" ]
+    {{- if .Values.artifactory.primary.persistence.existingClaim }}
+      selector:
+        matchLabels:
+          app: artifactory-ha-primary
+    {{- else }}
       {{- if .Values.artifactory.persistence.storageClass }}
       {{- if (eq "-" .Values.artifactory.persistence.storageClass) }}
       storageClassName: ""
@@ -252,6 +272,9 @@ spec:
       storageClassName: "{{ .Values.artifactory.persistence.storageClass }}"
       {{- end }}
       {{- end }}
+      accessModes: [ "{{ .Values.artifactory.persistence.accessMode }}" ]
       resources:
         requests:
           storage: {{ .Values.artifactory.persistence.size }}
+    {{- end }}
+  {{- end }}

stable/artifactory-ha/templates/artifactory-secrets.yaml

@@ -4,7 +4,7 @@ metadata:
   name: {{ template "artifactory-ha.fullname" . }}
   labels:
     app: {{ template "artifactory-ha.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    chart: {{ template "artifactory-ha.chart" . }}
     heritage: {{ .Release.Service }}
     release: {{ .Release.Name }}
 type: Opaque

stable/artifactory-ha/templates/artifactory-service.yaml

@@ -5,8 +5,8 @@ metadata:
   name: {{ template "artifactory-ha.fullname" . }}
   labels:
     app: {{ template "artifactory-ha.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
-    component: "{{ .Values.artifactory.name }}"
+    chart: {{ template "artifactory-ha.chart" . }}
+    component: {{ .Values.artifactory.name }}
     heritage: {{ .Release.Service }}
     release: {{ .Release.Name }}
 spec:
@@ -32,8 +32,8 @@ metadata:
   name: {{ template "artifactory-ha.primary.name" . }}
   labels:
     app: {{ template "artifactory-ha.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
-    component: "{{ .Values.artifactory.name }}"
+    chart: {{ template "artifactory-ha.chart" . }}
+    component: {{ .Values.artifactory.name }}
     heritage: {{ .Release.Service }}
     release: {{ .Release.Name }}
 spec:

stable/artifactory-ha/templates/ingress.yaml

@@ -7,7 +7,7 @@ metadata:
   name: {{ template "artifactory-ha.fullname" . }}
   labels:
     app: {{ template "artifactory-ha.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    chart: {{ template "artifactory-ha.chart" . }}
     release: {{ .Release.Name }}
     heritage: {{ .Release.Service }}
   annotations:

stable/artifactory-ha/templates/nginx-deployment.yaml

@@ -7,7 +7,7 @@ metadata:
   name: {{ template "artifactory-ha.nginx.fullname" . }}
   labels:
     app: {{ template "artifactory-ha.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    chart: {{ template "artifactory-ha.chart" . }}
     heritage: {{ .Release.Service }}
     release: {{ .Release.Name }}
     component: {{ .Values.nginx.name }}
@@ -22,7 +22,7 @@ spec:
     metadata:
       labels:
         app: {{ template "artifactory-ha.name" . }}
-        component: "{{ .Values.nginx.name }}"
+        component: {{ .Values.nginx.name }}
         release: {{ .Release.Name }}
     spec:
     {{- if .Values.imagePullSecrets }}
@@ -76,7 +76,7 @@ spec:
                     /etc/nginx/conf.d/artifactory.conf;
                 fi;
                 if ! grep -q 'proxy_http_version' /etc/nginx/conf.d/artifactory.conf; then
-                sed -i 's,\(proxy_pass .*\),proxy_http_version  1.1;\n    \1,g' /etc/nginx/conf.d/artifactory.conf;
+                sed -i 's,\(proxy_next_upstream .*\),proxy_http_version  1.1;\n    \1,g' /etc/nginx/conf.d/artifactory.conf;
                 fi;
                 sleep 5; nginx -s reload; touch /var/log/nginx/conf.done
         env:
@@ -88,6 +88,8 @@ spec:
           {{- end }}
         - name: SSL
           value: "{{ .Values.nginx.env.ssl }}"
+        - name: SKIP_AUTO_UPDATE_CONFIG
+          value: "{{ .Values.nginx.env.skipAutoConfigUpdate }}"
         ports:
         - containerPort: {{ .Values.nginx.internalPortHttp }}
         - containerPort: {{ .Values.nginx.internalPortHttps }}

stable/artifactory-ha/templates/nginx-pvc.yaml

@@ -6,9 +6,9 @@ metadata:
   name: {{ template "artifactory-ha.nginx.fullname" . }}
   labels:
     app: {{ template "artifactory-ha.name" . }}
-    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
-    release: "{{ .Release.Name }}"
-    heritage: "{{ .Release.Service }}"
+    chart: {{ template "artifactory-ha.chart" . }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
 spec:
   accessModes:
     - {{ .Values.nginx.persistence.accessMode | quote }}

stable/artifactory-ha/templates/nginx-replicator-conf.yaml

@@ -5,7 +5,7 @@ metadata:
   name: {{ template "artifactory-ha.fullname" . }}-replicator-nginx-config
   labels:
     app: {{ template "artifactory-ha.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    chart: {{ template "artifactory-ha.chart" . }}
     heritage: {{ .Release.Service }}
     release: {{ .Release.Name }}
 data:

stable/artifactory-ha/templates/nginx-service.yaml

@@ -5,7 +5,7 @@ metadata:
   name: {{ template "artifactory-ha.nginx.fullname" . }}
   labels:
     app: {{ template "artifactory-ha.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    chart: {{ template "artifactory-ha.chart" . }}
     heritage: {{ .Release.Service }}
     release: {{ .Release.Name }}
     component: {{ .Values.nginx.name }}
@@ -38,6 +38,6 @@ spec:
     name: https
   selector:
     app: {{ template "artifactory-ha.name" . }}
-    component: "{{ .Values.nginx.name }}"
+    component: {{ .Values.nginx.name }}
     release: {{ .Release.Name }}
 {{- end }}

stable/artifactory-ha/templates/replicator-configmap.yaml

@@ -5,7 +5,7 @@ metadata:
   name: {{ template "artifactory-ha.fullname" . }}-replicator-config
   labels:
     app: {{ template "artifactory-ha.name" . }}
-    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    chart: {{ template "artifactory-ha.chart" . }}
     heritage: {{ .Release.Service }}
     release: {{ .Release.Name }}
 data:

stable/artifactory-ha/values.yaml

@@ -75,6 +75,8 @@ artifactory:
   license:
     secret:
     dataKey:
+  ## Create configMap with artifactory.config.import.xml and security.import.xml and pass name of configMap in following parameter
+  configMapName:
   membershipPort: 10017
   externalPort: 8081
   internalPort: 8081
@@ -98,9 +100,10 @@ artifactory:
     periodSeconds: 10
     successThreshold: 1
   persistence:
+    enabled: true
+    local: false
     redundancy: 3
     mountPath: "/var/opt/jfrog/artifactory"
-    enabled: true
     accessMode: ReadWriteOnce
     size: 200Gi
     ## artifactory data Persistent Volume Storage Class
@@ -172,6 +175,10 @@ artifactory:
   ## Customising their resources and java parameters is done here.
   primary:
     name: artifactory-ha-primary
+    persistence:
+      ## Set existingClaim to true or false
+      ## If true, you must prepare a PVC with the name
+      existingClaim: false
     ## Resources for the primary node
     resources: {}
     #  requests:
@@ -188,6 +195,10 @@ artifactory:
     #  other:
   node:
     name: artifactory-ha-member
+    persistence:
+      ## Set existingClaim to true or false
+      ## If true, you must prepare a PVC with the name
+      existingClaim: false
     replicaCount: 2
     ## Resources for the member nodes
     resources: {}
@@ -251,6 +262,7 @@ nginx:
   env:
     ssl: true
     # artUrl: "http://artifactory:8081/artifactory"
+    skipAutoConfigUpdate: false
   persistence:
     mountPath: "/var/opt/jfrog/nginx"
     enabled: false