github/deprecated-helm-charts
1
0
Fork 0
mirror of https://github.com/helm/charts.git synced 2026-08-23 22:37:45 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update to Spark 2.4.0 (#9135)

Browse Source 
Signed-off-by: Chaoran Yu <yuchaoran2011@gmail.com>
This commit is contained in:
Chaoran Yu
2018-11-09 05:45:27 -08:00
committed by k8s-ci-robot
parent b1036550c9
commit c9d5a723bb
3 changed files with 8 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
stable/spark-history-server/Chart.yaml
+2 -2
View File
@@ -1,6 +1,6 @@
name: spark-history-server
version: 0.1.0
appVersion: 2.3.1
version: 0.1.1
appVersion: 2.4.0
description: A Helm chart for Spark History Server
home: https://spark.apache.org
sources:
stable/spark-history-server/README.md
+5 -5
View File
@@ -80,8 +80,8 @@ The following tables lists the configurable parameters of the Spark History Seve
| hdfs.hdfsSiteConfigMap |The name of the ConfigMap for hdfs-site.xml|hdfs-site|
| hdfs.coreSiteConfigMap |The name of the ConfigMap for core-site.xml|core-site|
| hdfs.HADOOP_CONF_DIR |The directory containing core-site.xml and hdfs-site.xml in the image|/etc/hadoop|
| image.repository |The Docker image used to start the history server daemon|lightbend/spark|
| image.tag |The tag of the image|2.4.0|
| image.repository |The Docker image used to start the history server daemon|gcr.io/spark-operator/spark|
| image.tag |The tag of the image|v2.4.0|
| service.type |The type of history server service that exposes the UI|LoadBalancer|
| service.port |The port on which the service UI can be accessed.|18080|
| pvc.enablePVC |Whether to use PVC storage|true|
@@ -136,7 +136,7 @@ bin/spark-submit \
--conf spark.eventLog.enabled=true \
--conf spark.eventLog.dir=file:/mnt \
--conf spark.executor.instances=2 \
--conf spark.kubernetes.container.image=lightbend/spark:k8s-rc-2.4 \
--conf spark.kubernetes.container.image=gcr.io/spark-operator/spark:v2.4.0 \
--conf spark.kubernetes.container.image.pullPolicy=Always \
--conf spark.kubernetes.driver.volumes.persistentVolumeClaim.checkpointpvc.options.claimName=nfs-pvc \
--conf spark.kubernetes.driver.volumes.persistentVolumeClaim.checkpointpvc.mount.path=/mnt \
@@ -178,8 +178,8 @@ bin/spark-submit \
--conf spark.hadoop.google.cloud.auth.service.account.json.keyfile=/etc/secrets/sparkonk8s.json \
--conf spark.kubernetes.driver.secrets.history-secrets=/etc/secrets \
--conf spark.kubernetes.executor.secrets.history-secrets=/etc/secrets \
--conf spark.kubernetes.container.image=lightbend/spark:k8s-daily-gcs \
local:///opt/spark/examples/jars/spark-examples_2.11-2.5.0-SNAPSHOT.jar
--conf spark.kubernetes.container.image=lightbend/spark:2.4.0-gcs \
local:///opt/spark/examples/jars/spark-examples_2.11-2.4.0.jar
```
Note that the image for your Spark job (i.e. `spark.kubernetes.container.image`, `spark.kubernetes.driver.container.image` and `spark.kubernetes.executor.container.image`) needs to have the [GCS connector](https://cloud.google.com/dataproc/docs/concepts/connectors/cloud-storage) dependency, otherwise the `gs://` scheme won't be recognized.
stable/spark-history-server/values.yaml
+1 -1
View File
@@ -9,7 +9,7 @@ serviceAccount:
image:
repository: gcr.io/spark-operator/spark
tag: v2.3.1
tag: v2.4.0
pullPolicy: IfNotPresent
service: