Add support for linting of charts on every PR via CircleCI (#2404)

This change does the following:
- Adds a circleci config file
- Has 2 scripts. One for installing tools and a separate one for
  executing tests
- Removes lint testing from the changed.sh script since it is
  performed elsewhere
- For linting changes the git diff mechanism to look at the merge
  base rather than all differences from master. The intent is to
  look for changes in this request rather than including those
  in PRs that may have already been merged to master while this
  one lingered.

Ref #2337

.circleci/config.yml

@@ -0,0 +1,14 @@
+version: 2
+jobs:
+   build:
+     docker:
+       # TODO: Update to 3.6.3 when the image become available
+       - image: circleci/python:3.6.2
+     steps:
+       - checkout
+       - run:
+           name: install tools
+           command: test/circle/install.sh
+       - run: 
+           name: helm lint
+           command: test/circle/lint.sh

test/README.md

@@ -2,7 +2,24 @@
 
 ## Pull Request Testing
 
-### Procedure
+There are two forms of testing run on a pull request. Static analysis, run
+automatically, and operational testing, run when a maintainer flags a pull
+request as being ready. The operational testing is run on Kubernertes test
+infrastructure.
+
+### Static Analysis
+
+Static analysis is performed on every pull request and is run by CircleCI. The
+configuration is stored in the [`.circleci/config.yml`](../.circleci/config.yml)
+file.
+
+The static analysis currently:
+
+* Performs `helm lint` on any changed charts to provide quick feedback
+
+### Operational Testing
+
+#### Procedure
 
 Pull requests testing is run via the [Kuberentes Test Infrastructure](https://github.com/kubernetes/test-infra).
 
@@ -19,13 +36,12 @@ The logic is as follows:
 1. [Install and initialize Helm](https://github.com/kubernetes/charts/blob/master/test/changed.sh#L47)
 1. [For any charts that have changed](https://github.com/kubernetes/charts/blob/master/test/changed.sh#L62):
     - Download dependent charts, if any, with `helm dep update`
-    - Run `helm lint` on the chart
     - Run `helm install` in a new namespace for this PR+build
     - Use the [test/verify-release.sh](https://github.com/kubernetes/charts/blob/master/test/verify-release.sh) to ensure that if any pods were launched that they get to the `Running` state
     - Run `helm test` on the release
     - Delete the release
 
-### Triggering
+#### Triggering
 
 In order for the tests to be kicked off one of the 
 [Kubernetes member](https://github.com/orgs/kubernetes/people) must add the 

test/changed.sh

@@ -67,7 +67,6 @@ for directory in ${CHANGED_FOLDERS}; do
     RELEASE_NAME="${CHART_NAME:0:7}-${BUILD_NUMBER}"
     CURRENT_RELEASE=${RELEASE_NAME}
     helm dep update ${directory}
-    helm lint ${directory}
     helm install --timeout 600 --name ${RELEASE_NAME} --namespace ${NAMESPACE} ${directory} | tee install_output
     ./test/verify-release.sh ${NAMESPACE}
     kubectl get pods --namespace ${NAMESPACE}

test/circle/README.md

@@ -0,0 +1,4 @@
+# CircleCI Testing
+
+This directory contains scripts tailored to run on CircleCI. For more information
+see the test documentation.

test/circle/install.sh

@@ -0,0 +1,23 @@
+#!/bin/bash -xe
+# Copyright 2017 The Kubernetes Authors All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Install Helm
+HELM_LATEST_VERSION="v2.6.1"
+
+wget http://storage.googleapis.com/kubernetes-helm/helm-${HELM_LATEST_VERSION}-linux-amd64.tar.gz
+tar -xvf helm-${HELM_LATEST_VERSION}-linux-amd64.tar.gz
+sudo mv linux-amd64/helm /usr/local/bin
+rm -f helm-${HELM_LATEST_VERSION}-linux-amd64.tar.gz
+rm -rf linux-amd64

test/circle/lint.sh

@@ -0,0 +1,35 @@
+#!/bin/bash -xe
+# Copyright 2017 The Kubernetes Authors All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Compare to the merge-base rather than master to limit scanning to changes
+# this PR/changeset is introducing. Making sure the comparison is to the
+# upstream charts repo rather than a fork.
+git remote add k8s https://github.com/kubernetes/charts
+git fetch k8s master
+CHANGED_FOLDERS=`git diff --find-renames --name-only $(git merge-base k8s/master HEAD) stable/ incubator/ | awk -F/ '{print $1"/"$2}' | uniq`
+
+# Exit early if no charts have changed
+if [ -z "$CHANGED_FOLDERS" ]; then
+  echo "No changes to charts found"
+  exit 0
+fi
+
+for directory in ${CHANGED_FOLDERS}; do
+  if [ "$directory" == "incubator/common" ]; then
+    continue
+  elif [ -d $directory ]; then
+    helm lint ${directory}
+  fi
+done