Compare commits

...
261 Commits
Author SHA1 Message Date
Salah Aldeen Al Saleh 4c72573936 os minor should default to 0 (#513) 2021-12-10 13:17:36 -08:00
Salah Aldeen Al Saleh 347d462d7f goreleaser should wait for supportbundle binary (#512) 2021-12-10 12:27:16 -08:00
Salah Aldeen Al Saleh d1f341b8ed host system packages collector/analyzer (#506)
* host system packages collector/analyzer
2021-12-10 12:05:21 -08:00
Ethan Mosbaugh 186180612b Merge pull request #502 from replicatedhq/dependabot/go_modules/sigs.k8s.io/controller-runtime-0.10.3
Bump sigs.k8s.io/controller-runtime from 0.7.0 to 0.10.3
2021-12-01 12:27:30 -08:00
dependabot[bot] bed930fa7c Bump sigs.k8s.io/controller-runtime from 0.7.0 to 0.10.3
Bumps [sigs.k8s.io/controller-runtime](https://github.com/kubernetes-sigs/controller-runtime) from 0.7.0 to 0.10.3.
- [Release notes](https://github.com/kubernetes-sigs/controller-runtime/releases)
- [Commits](https://github.com/kubernetes-sigs/controller-runtime/compare/v0.7.0...v0.10.3)

---
updated-dependencies:
- dependency-name: sigs.k8s.io/controller-runtime
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-12-01 18:43:11 +00:00
Ethan Mosbaugh 08b746d021 Merge pull request #503 from replicatedhq/dependabot/go_modules/k8s.io/apiextensions-apiserver-0.22.4
Bump k8s.io/apiextensions-apiserver from 0.22.2 to 0.22.4
2021-12-01 10:26:50 -08:00
Ethan Mosbaugh f9b55e2bff Merge pull request #504 from replicatedhq/dependabot/go_modules/github.com/shirou/gopsutil-3.21.11incompatible
Bump github.com/shirou/gopsutil from 3.21.10+incompatible to 3.21.11+incompatible
2021-12-01 10:26:37 -08:00
Ethan Mosbaugh f4db86960b Merge pull request #505 from replicatedhq/dependabot/go_modules/github.com/google/gofuzz-1.2.0
Bump github.com/google/gofuzz from 1.1.0 to 1.2.0
2021-12-01 10:02:45 -08:00
dependabot[bot] 40eea877ad Bump github.com/google/gofuzz from 1.1.0 to 1.2.0
Bumps [github.com/google/gofuzz](https://github.com/google/gofuzz) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/google/gofuzz/releases)
- [Commits](https://github.com/google/gofuzz/compare/v1.1.0...v1.2.0)

---
updated-dependencies:
- dependency-name: github.com/google/gofuzz
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-12-01 17:58:15 +00:00
dependabot[bot] 9e2dac4d55 Bump github.com/shirou/gopsutil
Bumps [github.com/shirou/gopsutil](https://github.com/shirou/gopsutil) from 3.21.10+incompatible to 3.21.11+incompatible.
- [Release notes](https://github.com/shirou/gopsutil/releases)
- [Commits](https://github.com/shirou/gopsutil/compare/v3.21.10...v3.21.11)

---
updated-dependencies:
- dependency-name: github.com/shirou/gopsutil
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-12-01 17:58:10 +00:00
dependabot[bot] 5c4825f4d4 Bump k8s.io/apiextensions-apiserver from 0.22.2 to 0.22.4
Bumps [k8s.io/apiextensions-apiserver](https://github.com/kubernetes/apiextensions-apiserver) from 0.22.2 to 0.22.4.
- [Release notes](https://github.com/kubernetes/apiextensions-apiserver/releases)
- [Commits](https://github.com/kubernetes/apiextensions-apiserver/compare/v0.22.2...v0.22.4)

---
updated-dependencies:
- dependency-name: k8s.io/apiextensions-apiserver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-12-01 17:58:01 +00:00
Ethan Mosbaugh 2252e61cbf Merge pull request #498 from replicatedhq/dependabot/go_modules/github.com/go-redis/redis/v7-7.4.1
Bump github.com/go-redis/redis/v7 from 7.2.0 to 7.4.1
2021-12-01 09:41:11 -08:00
dependabot[bot] 7582e58220 Bump github.com/go-redis/redis/v7 from 7.2.0 to 7.4.1
Bumps [github.com/go-redis/redis/v7](https://github.com/go-redis/redis) from 7.2.0 to 7.4.1.
- [Release notes](https://github.com/go-redis/redis/releases)
- [Changelog](https://github.com/go-redis/redis/blob/master/CHANGELOG.md)
- [Commits](https://github.com/go-redis/redis/compare/v7.2.0...v7.4.1)

---
updated-dependencies:
- dependency-name: github.com/go-redis/redis/v7
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-12-01 15:23:26 +00:00
Ethan Mosbaugh c18453711e Merge pull request #497 from replicatedhq/dependabot/go_modules/k8s.io/cli-runtime-0.22.4
Bump k8s.io/cli-runtime from 0.21.5 to 0.22.4
2021-12-01 06:49:20 -08:00
Ethan Mosbaugh 9603ccdc56 Merge pull request #499 from replicatedhq/dependabot/go_modules/github.com/spf13/viper-1.9.0
Bump github.com/spf13/viper from 1.8.1 to 1.9.0
2021-12-01 06:49:01 -08:00
Ethan Mosbaugh 891ee93cb1 Merge pull request #500 from replicatedhq/dependabot/go_modules/github.com/lib/pq-1.10.4
Bump github.com/lib/pq from 1.3.0 to 1.10.4
2021-12-01 06:48:47 -08:00
Ethan Mosbaugh 2cfa690838 Merge pull request #501 from replicatedhq/dependabot/go_modules/github.com/go-sql-driver/mysql-1.6.0
Bump github.com/go-sql-driver/mysql from 1.5.0 to 1.6.0
2021-12-01 06:48:37 -08:00
dependabot[bot] 9b9824bef5 Bump github.com/go-sql-driver/mysql from 1.5.0 to 1.6.0
Bumps [github.com/go-sql-driver/mysql](https://github.com/go-sql-driver/mysql) from 1.5.0 to 1.6.0.
- [Release notes](https://github.com/go-sql-driver/mysql/releases)
- [Changelog](https://github.com/go-sql-driver/mysql/blob/master/CHANGELOG.md)
- [Commits](https://github.com/go-sql-driver/mysql/compare/v1.5.0...v1.6.0)

---
updated-dependencies:
- dependency-name: github.com/go-sql-driver/mysql
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-12-01 01:01:27 +00:00
dependabot[bot] 60ef14a751 Bump github.com/lib/pq from 1.3.0 to 1.10.4
Bumps [github.com/lib/pq](https://github.com/lib/pq) from 1.3.0 to 1.10.4.
- [Release notes](https://github.com/lib/pq/releases)
- [Commits](https://github.com/lib/pq/compare/v1.3.0...v1.10.4)

---
updated-dependencies:
- dependency-name: github.com/lib/pq
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-12-01 01:01:14 +00:00
dependabot[bot] 3980a7b903 Bump github.com/spf13/viper from 1.8.1 to 1.9.0
Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.8.1 to 1.9.0.
- [Release notes](https://github.com/spf13/viper/releases)
- [Commits](https://github.com/spf13/viper/compare/v1.8.1...v1.9.0)

---
updated-dependencies:
- dependency-name: github.com/spf13/viper
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-12-01 01:00:57 +00:00
dependabot[bot] 655fa9b4a0 Bump k8s.io/cli-runtime from 0.21.5 to 0.22.4
Bumps [k8s.io/cli-runtime](https://github.com/kubernetes/cli-runtime) from 0.21.5 to 0.22.4.
- [Release notes](https://github.com/kubernetes/cli-runtime/releases)
- [Commits](https://github.com/kubernetes/cli-runtime/compare/v0.21.5...v0.22.4)

---
updated-dependencies:
- dependency-name: k8s.io/cli-runtime
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-12-01 00:59:45 +00:00
Ethan Mosbaugh 27c5e51920 Merge pull request #496 from replicatedhq/emosbaugh/update-containers-image-v5
Update github.com/containers/image/v5
2021-11-30 16:28:58 -08:00
Ethan Mosbaugh 177f2da16d Update github.com/containers/image/v5 2021-11-30 23:37:25 +00:00
Ethan Mosbaugh d774ad1f5a Merge pull request #493 from replicatedhq/dependabot/go_modules/k8s.io/apiserver-0.22.4
Bump k8s.io/apiserver from 0.22.2 to 0.22.4
2021-11-30 15:10:50 -08:00
Ethan Mosbaugh 9c5ae1f64e Merge pull request #494 from replicatedhq/dependabot/go_modules/github.com/segmentio/ksuid-1.0.4
Bump github.com/segmentio/ksuid from 1.0.3 to 1.0.4
2021-11-30 15:10:24 -08:00
Ethan Mosbaugh a237dba463 Merge pull request #495 from replicatedhq/dependabot/go_modules/github.com/fatih/color-1.13.0
Bump github.com/fatih/color from 1.12.0 to 1.13.0
2021-11-30 15:10:03 -08:00
dependabot[bot] 60ec620ba9 Bump github.com/fatih/color from 1.12.0 to 1.13.0
Bumps [github.com/fatih/color](https://github.com/fatih/color) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/fatih/color/releases)
- [Commits](https://github.com/fatih/color/compare/v1.12.0...v1.13.0)

---
updated-dependencies:
- dependency-name: github.com/fatih/color
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-30 22:10:22 +00:00
dependabot[bot] 0c422aefbd Bump github.com/segmentio/ksuid from 1.0.3 to 1.0.4
Bumps [github.com/segmentio/ksuid](https://github.com/segmentio/ksuid) from 1.0.3 to 1.0.4.
- [Release notes](https://github.com/segmentio/ksuid/releases)
- [Commits](https://github.com/segmentio/ksuid/compare/v1.0.3...v1.0.4)

---
updated-dependencies:
- dependency-name: github.com/segmentio/ksuid
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-30 22:10:13 +00:00
dependabot[bot] c2ac654e08 Bump k8s.io/apiserver from 0.22.2 to 0.22.4
Bumps [k8s.io/apiserver](https://github.com/kubernetes/apiserver) from 0.22.2 to 0.22.4.
- [Release notes](https://github.com/kubernetes/apiserver/releases)
- [Commits](https://github.com/kubernetes/apiserver/compare/v0.22.2...v0.22.4)

---
updated-dependencies:
- dependency-name: k8s.io/apiserver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-30 22:10:05 +00:00
Ethan Mosbaugh ffc8a347c6 Merge pull request #490 from replicatedhq/emosbaugh-create-dependabot-yaml
Create dependabot.yml
2021-11-30 13:59:27 -08:00
Ethan Mosbaugh 0a5e397d44 Create dependabot.yml 2021-11-30 13:56:43 -08:00
Ethan Mosbaugh e03cf6b37f Merge pull request #489 from replicatedhq/emosbaugh-delete-dependabot
Delete dependabot.yml
2021-11-30 13:56:08 -08:00
Ethan Mosbaugh 2ec15b69f6 Delete dependabot.yml 2021-11-30 13:53:04 -08:00
Ethan Mosbaugh 39bd7778f5 Merge pull request #488 from replicatedhq/emosbaugh/sc-38420/fix-go-mod
Fix go mod
2021-11-30 13:47:44 -08:00
Ethan Mosbaugh 59d50e7679 Fix go mod 2021-11-30 21:26:24 +00:00
Ethan Mosbaugh cb8dccbe7b Merge pull request #487 from replicatedhq/emosbaugh/20211130/fixes
Various fixes
2021-11-30 13:06:45 -08:00
Ethan Mosbaugh fba0f97225 found not ound 2021-11-30 20:12:29 +00:00
Ethan Mosbaugh 4d0eaf471f crd not storageClass 2021-11-30 20:12:09 +00:00
Ethan Mosbaugh 213d518136 Time parse doesnt support day notation 2021-11-30 20:11:51 +00:00
Ethan Mosbaugh c4ba3fe70e Merge pull request #486 from replicatedhq/enable-dependabot
Enable dependabot
2021-11-30 10:01:03 -08:00
Ethan Mosbaugh b524ac8ac4 Add labels and change interval to weekly 2021-11-30 10:00:49 -08:00
Ethan Mosbaugh 7403de58bd Enable dependabot 2021-11-30 09:52:03 -08:00
Marc Campbell 9eadae6771 Merge pull request #484 from replicatedhq/arm
Adding darwin/arm64
2021-11-12 11:57:14 -08:00
Marc Campbell 75a56d09b2 Windows/Arm64 2021-11-12 19:38:24 +00:00
Marc Campbell 4a72e7d2a5 Bump go version 2021-11-12 19:03:35 +00:00
Marc Campbell 2c53fb4da6 Bump go-ole 2021-11-12 18:02:08 +00:00
Marc Campbell 6a360b635f Merge branch 'arm' of https://github.com/replicatedhq/troubleshoot into arm 2021-11-12 18:01:36 +00:00
Marc Campbell 913963a809 Bump go-ole dep 2021-11-12 18:00:11 +00:00
Marc Campbell 49c56f6cd7 Adding darwin/arm64 2021-11-12 17:51:02 +00:00
Marc Campbell baa943474d Merge pull request #485 from replicatedhq/remove-precache
Remove precache until we have it enabled
2021-11-12 09:49:31 -08:00
Marc Campbell fa2c775abd Remove precache until we have it enabled 2021-11-11 22:31:11 +00:00
Marc Campbell ac65e4dc97 Adding darwin/arm64 2021-11-11 22:27:21 +00:00
Marc Campbell dfb0991fb2 Merge pull request #483 from replicatedhq/codespaces
Add codespace env
2021-11-11 14:05:03 -08:00
Marc Campbell 101ab5d2fa Add github action for codespace 2021-11-11 21:32:21 +00:00
Marc Campbell 5211cb0fe7 Add codespace env 2021-11-11 20:57:54 +00:00
Salah Aldeen Al Saleh c7c21e88fb fix custom resources redaction file path (#480)
* fix custom resources redaction file path
2021-11-02 12:36:15 -07:00
divolgin ea7756f8f2 Merge pull request #479 from replicatedhq/divolgin/no-files
Allow text analyzer to not generate an error if no files match
2021-10-29 11:23:34 -07:00
divolgin 739ee666af Allow text analyzer to not generate an error if no files match 2021-10-29 17:52:59 +00:00
divolgin 181df9c85b Merge pull request #478 from replicatedhq/divolgin/outcomes
Ensure outcomes are optional in every case
2021-10-28 17:41:02 -07:00
divolgin 742ddc8c06 Ensure outcomes are optional in every case 2021-10-29 00:23:32 +00:00
divolgin b4c97e377e Merge pull request #477 from replicatedhq/divolgin/selectors
replicaset analyzer supports label selectors
2021-10-28 17:21:11 -07:00
divolgin 7cb6d90a39 replicaset analyzer supports label selectors 2021-10-28 22:06:15 +00:00
Sean Rester 5d9f14fde5 Merge pull request #474 from replicatedhq/add-node-status-check
38798: Adding node status check
2021-10-28 17:52:18 -04:00
Salah Aldeen Al Saleh 14463642b0 a function to get a pod details from the support bundle (#476)
* a function to get a pod details from the support bundle
2021-10-28 14:06:02 -07:00
Salah Aldeen Al Saleh 45dd980012 update cluster pod analyzers comment (#475) 2021-10-28 10:31:59 -07:00
e100e7c478 get container logs for unhealthy pods (#469)
* get container logs for unhealthy pods

Co-authored-by: divolgin <dmitriy@replicated.com>
Co-authored-by: divolgin <divolgin@users.noreply.github.com>
2021-10-28 09:21:14 -07:00
Sean Rester 1345b200aa 38798: Adding node status check 2021-10-28 11:16:26 -04:00
divolgin 1cb0dab3c2 Merge pull request #471 from replicatedhq/divolgin/windows
Fix windows build
2021-10-27 15:39:15 -07:00
divolgin db3d27d38f Fix windows build 2021-10-27 21:29:11 +00:00
divolgin e7daba9d0c Merge pull request #470 from replicatedhq/divolgin/analyzers
Replicaset collector and analyzer
2021-10-27 13:51:42 -07:00
divolgin ada35eb31c Replicaset collector and analyzer 2021-10-27 20:24:14 +00:00
Salah Aldeen Al Saleh f2374cf113 add involved object to clusterPodStatuses analyzer result (#459)
* cluster pod statuses analyzer involved object
2021-10-27 12:18:49 -07:00
Salah Aldeen Al Saleh e7f7e75df1 change goreleaser version to v0.166.1 (#468) 2021-10-27 10:31:35 -07:00
Salah Aldeen Al Saleh 5a8561a31f include logs for init containers as well (#467) 2021-10-27 10:20:55 -07:00
divolgin 0df678929a Merge pull request #465 from replicatedhq/divolgin/analyzers
Jobs status analyzer
2021-10-26 17:13:36 -07:00
divolgin 1cdfd96768 Jobs status analyzer 2021-10-26 23:41:02 +00:00
divolgin 9666679ab4 Merge pull request #464 from replicatedhq/divolgin/analyzers
Analyze all deployments in all namespaces
2021-10-26 14:54:15 -07:00
divolgin f108c3ca57 Analyze all deployments in all namespaces 2021-10-26 21:36:27 +00:00
divolgin 491376b772 Merge pull request #462 from replicatedhq/divolgin/analyzers
Ability to analyze all statefulsets
2021-10-26 14:08:32 -07:00
divolgin 34724e7932 Ability to analyze all statefulsets 2021-10-26 20:51:45 +00:00
Marc Campbell 7e11498509 Merge pull request #461 from replicatedhq/deepsource-fix-97a5e72b
Fix check for empty string
2021-10-26 12:58:17 -07:00
deepsource-autofix[bot] 3e60fcda8e Fix check for empty string 2021-10-26 19:24:08 +00:00
Salah Aldeen Al Saleh b32afc1d17 pin goreleaser version to v0.182.1 (#460) 2021-10-26 11:37:37 -07:00
divolgin f048b426a5 Merge pull request #440 from replicatedhq/divolgin/tests
Better preflight validation
2021-10-26 08:45:40 -07:00
Salah Aldeen Al Saleh 26402a7b04 cluster pod statuses analyzer improvements (#458)
* add pod status reason to cluster pod statuses analyzer
2021-10-26 08:42:40 -07:00
Salah Aldeen Al Salehanddivolgin 3d1d53ee9d ClusterPodStatuses analyzer (#456)
* ClusterPodStatuses analyzer

Co-authored-by: divolgin <dmitriy@replicated.com>
2021-10-25 17:44:59 -07:00
divolgin 9cbdb70d16 Merge pull request #457 from replicatedhq/divolgin/pod-logs
Include pod logs for pods that are failing
2021-10-25 17:18:32 -07:00
divolgin 20f1b60f11 Include pod logs for pods that are failing 2021-10-26 00:01:26 +00:00
divolgin fafac18d29 Merge pull request #455 from replicatedhq/divolgin/ceph
Fix ceph collector
2021-10-22 16:38:55 -07:00
divolgin 072d2d7a36 Fix ceph collector 2021-10-22 23:01:13 +00:00
Andrew Reed 7b36e6a1f8 Copy in longhorn client (#454) 2021-10-22 15:24:07 -05:00
cf03503216 feat: Collect custom resources (#447)
* feat: Collect custom resources
Co-authored-by: Martin Hrabovcin<mhrabovcin@users.noreply.github.com>

Co-authored-by: Andrew Reed <andrew@replicated.com>
2021-10-21 16:49:59 -05:00
Dimitri KoshkinandSalah Aldeen Al Saleh 111396eb39 fix: pass redact flag when running support-bundle (#406)
Co-authored-by: Salah Aldeen Al Saleh <sg.alsaleh@gmail.com>
2021-10-20 10:47:27 -07:00
Jalaja Ganapathy 372454651e collector/analyzer for host operating system (#443)
* collector/analyzer for host operating system

* address cr comments

* cleanup

* fix invoking the analyzer
code cleanup

* fix cr comments

* add corner case unit-test

* fix kernel version parsing

* address review comments

* add default case

* parse using regex

* added more testcases and fixed the bug found in cr

* few small things
2021-10-12 14:42:23 -07:00
divolgin 5dece3eb75 Merge pull request #451 from replicatedhq/divolgin/panic
Check nil pointers
2021-10-12 10:12:11 -07:00
divolgin e095a7838f Check nil pointers 2021-10-12 16:10:02 +00:00
Vera Harless 08953d46d1 fix: add collect to goreleaser (#450) 2021-10-08 15:44:55 -04:00
Andrew Lavery bc197761ea Merge pull request #434 from croomes/handle-api-deprecations
Handle k8s api deprecations
2021-10-08 06:52:31 -07:00
Simon Croome dc8b38d249 Handle k8s api deprecations 2021-10-07 18:55:51 +01:00
Vera Harless 73609c4fef feat: add more detail to the ceph analyzer output (#445) 2021-10-06 11:22:56 -04:00
Simon Croome 977fc438ea Remote host collectors (#392)
* Add collect command and remote host collectors

Adds the ability to run a host collector on a set of remote k8s nodes.
Target nodes can be filtered using the --selector flag, with the same
syntax as kubectl.  Existing flags for --collector-image,
--collector-pullpolicy and --request-timeout are used.  To run on a
specified node, --selector="kubernetes.io/hostname=kind-worker2" could
be used.

The collect command is used by the remote collector to output the
results using a "raw" format, which uses the filename as the key, and
the value the output as a escaped json string.  When run manually it
defaults to fully decoded json. The existing block devices,
ipv4interfaces and services host collectors don't decode properly - the
fix is to convert their slice output to a map (fix not included as
unsure what depends on the existing format).

The collect command is also useful for troubleshooting preflight issues.

Examples are included to show remote collector usage.

```
bin/collect --collector-image=croomes/troubleshoot:latest  examples/collect/remote/memory.yaml --namespace test
{
  "kind-control-plane": {
    "system/memory.json": {
      "total": 1304207360
    }
  },
  "kind-worker": {
    "system/memory.json": {
      "total": 1695780864
    }
  },
  "kind-worker2": {
    "system/memory.json": {
      "total": 1726353408
    }
  }
}
```

The preflight command has been updated to run remote collectors.  To run
a host collector remotely it must be specified in the spec as a
`remoteCollector`:

```
apiVersion: troubleshoot.sh/v1beta2
kind: HostPreflight
metadata:
  name: memory
spec:
  remoteCollectors:
    - memory:
        collectorName: memory
  analyzers:
    - memory:
        outcomes:
          - fail:
              when: "< 8Gi"
              message: At least 8Gi of memory is required
          - warn:
              when: "< 32Gi"
              message: At least 32Gi of memory is recommended
          - pass:
              message: The system has as sufficient memory
```

Results for each node are analyzed separately, with the node name
appended to the title:

```
bin/preflight --interactive=false --collector-image=croomes/troubleshoot:latest examples/preflight/remote/memory.yaml --format=json
{memory running 0 1}
{memory completed 1 1}
{
  "fail": [
    {
      "title": "Amount of Memory (kind-worker2)",
      "message": "At least 8Gi of memory is required"
    },
    {
      "title": "Amount of Memory (kind-worker)",
      "message": "At least 8Gi of memory is required"
    },
    {
      "title": "Amount of Memory (kind-control-plane)",
      "message": "At least 8Gi of memory is required"
    }
  ]
}
```

Also added a host collector to allow preflight checks of required kernel
modules, which is the main driver for this change.
2021-10-06 09:03:53 -05:00
Andrew Reed 4d52760d35 Collector and analyzer for sysctl parameters (#441)
Collector and analyzer for sysctl parameters
2021-10-01 13:43:26 -05:00
divolgin 6e34aa615e Merge pull request #442 from replicatedhq/divolgin/closer
Allow memory writers
2021-09-30 11:42:14 -07:00
divolgin ca51e92878 Allow memory writers 2021-09-30 18:25:52 +00:00
divolgin c546577449 Better preflight validation 2021-09-29 22:28:25 +00:00
divolgin 06750d478e Merge pull request #439 from replicatedhq/divolgin/nil
Don't panic when no data is collected
2021-09-29 15:03:50 -07:00
divolgin 6d0a57b16e Don't panic when no data is collected 2021-09-29 21:25:28 +00:00
Jalaja Ganapathy 8a29442a2a Remove ID from host preflight spec (#438) 2021-09-29 09:49:54 -07:00
divolgin 299497c0c0 Merge pull request #429 from danbudris/copyFromHostForCpNodes
add toleration to copy-from-host daemonset to allow collection from CP nodes
2021-09-29 09:01:14 -07:00
divolgin 050f5939c6 Merge pull request #437 from replicatedhq/divolgin/memory
Save collector data to disk directly
2021-09-29 08:12:05 -07:00
divolgin 0e8bedc281 Save collector data to disk directly 2021-09-29 00:15:02 +00:00
Dan Stough bb0515830d Merge pull request #436 from replicatedhq/dans-fix-sbom-perms
chore(ci): fix sbom assets for krew
2021-09-28 15:09:47 -04:00
Dan Stough b903f1f1c4 chore(ci): fix sbom asset perms 2021-09-28 16:37:53 +00:00
Jalaja Ganapathy f26c9b4136 fix README syntax (#433) 2021-09-24 17:35:36 -07:00
Jalaja Ganapathy eb795c98b6 fix serializer for unique id (#432) 2021-09-24 14:20:37 -07:00
Jalaja Ganapathy a0b3b3f7dc add an unique id to each host preflights (#431)
* add an unique id to each host preflights

* auto generated files

* updated schemas for the new field id

* keeping it consistent with the rest of the spec
2021-09-24 13:29:14 -07:00
danbudris 67987a4432 add toleration to allow copy-from-host daemonset to run on CP nodes 2021-09-23 17:53:57 -04:00
Salah Aldeen Al Saleh 1bdd3db8c5 update schemas (#428)
* update schemas

* update controller-gen
2021-09-23 11:03:19 -07:00
John Murphy a2b5edb551 added missing cosign.key (#427)
SBOM generation was failing because it missed a step to generate the private key needed for SBOM signing from Github secret.
2021-09-23 10:46:30 -05:00
Salah Aldeen Al Saleh 880c7dc3ea ability to specify a list of namespaces for the cluster resources collector (#424)
* ability to specify a list of namespaces for the cluster resources collector
2021-09-23 08:02:05 -07:00
divolgin 922f7c8b23 Merge pull request #425 from replicatedhq/divolgin/results
Analyzers should not return multiple results
2021-09-22 16:13:54 -07:00
divolgin afa08e5362 Analyzers should not return multiple results 2021-09-22 22:50:38 +00:00
Dan Stough 614aed52c9 Merge pull request #422 from replicatedhq/dans/fix-clean-noninteractive-output
fix(support-bundle): no client-go warnings or control chars if noninteractive.
2021-09-22 13:38:43 -04:00
Dan Stough 72a50ee3f2 fix(support-bundle): no client-go warnings or control chars if noninteractive 2021-09-22 15:59:35 +00:00
Salah Aldeen Al Saleh 0c7fede7b6 check for nil analyzers (#421) 2021-09-21 12:12:10 -07:00
John Murphy 639bf7a832 Add signed SBOM to troubleshoot (#414)
This change will generate a signed software bill of materials and add it to the repository release archives when the project is released.
2021-09-21 13:55:41 -05:00
John Murphy 48287097d8 added email alias to code of conduct (#420) 2021-09-21 13:52:00 -05:00
divolgin cb5ddf752f Merge pull request #419 from danbudris/machineReadableNonInteractiveOutput
make non-interactive `support-bundle` output more machine readable
2021-09-21 09:21:06 -07:00
danbudris 52e1a04f57 Merge branch 'machineReadableNonInteractiveOutput' of https://github.com/danbudris/troubleshoot into machineReadableNonInteractiveOutput 2021-09-17 11:21:34 -04:00
danbudris 5b4b548aa0 if interactive, only return the print archivePath to stdout; if non-interactive, print whole analysis as json 2021-09-17 11:20:39 -04:00
Daniel Budris f2a232d174 use analyzerResults not analysis for key 2021-09-17 11:05:34 -04:00
danbudris f4e675dae0 add json tags to output struct for easier unmarshalling 2021-09-17 10:57:52 -04:00
danbudris 867df407ea convert output bytearray to string before printing 2021-09-17 10:50:22 -04:00
danbudris e0fb748498 move non-interactive output to discreet struct with marshalling methods; dont show output for non-interactive; format everything in JSON 2021-09-17 10:38:38 -04:00
danbudris 463783d2fa resolve merge conflicts 2021-09-15 21:25:15 -04:00
danbudris 2ce78ac33a Merge branch 'master' of https://github.com/replicatedhq/troubleshoot into machineReadableNonInteractiveOutput 2021-09-15 21:19:01 -04:00
danbudris 4cf0f5881d make non-interactive support-bundle output more machine readable
when using the `interactive=false` flag of `support-bundle`, the spinner would still spin and the archive path and analysis output were kind of smooshed together with the logs.

now, if `interactive=false`, only print each recieved collector callback message once, and don't spin

also, add a key to the archivePath and analyzerOutput that are returned, for easier programatic parsing
2021-09-15 20:58:09 -04:00
Salah Aldeen Al Saleh 465a533640 store analysis in the support bundle (#417)
* store analysis in the support bundle
2021-09-10 11:58:16 -07:00
Andrew Reed 10785987c5 Merge pull request #415 from areed/areed/weave-analyzer
Weave report analyzers
2021-09-09 12:47:51 -05:00
Andrew Reed 91eb94baaa Weave report analyzers
The IPAM pool analyzer checks that utilization of the pod IP subnet is
less than 85%. For example, if using 10.32.0.0/12, this analyzer will
warn if 3,482 IPs are currently allocated to pods.

The pending allocation analyzer checks that the IPAM status in the
report has no items for the PendingAllocates field. This indicates the
IPAM service is not ready according to the code in the weave status
template
https://github.com/weaveworks/weave/blob/e3712152d2a0fe3bc998964c948e45bdf8ff6144/prog/weaver/http.go#L186.

The weave connections analyzer checks that all connections to remote
peers are in the established state. The state will be "pending" if UDP
is blocked between nodes and will be "failed" if the weave pod on the
remote node is in a crash loop. To force a pending state for testing,
run the commands `iptables -A INPUT -p udp --dport 6784 -j REJECT` and
`iptables -A INPUT -p udp --dport 6783 -j REJECT` on a peer.

The weave connections analyzer also checks that all connections are
using the fastdp protocol. A commopn issue seen in the field on
CentOS/RHEL 7 is that some sides of a connection are using fastdp and
other sides have fallen back to sleeve. Set the WEAVE_NO_FASTDP env var
on the weave daemonset to "true" to test this analyzer.
2021-09-08 21:29:38 +00:00
Andrew Lavery 1b65d1a544 Merge pull request #413 from replicatedhq/laverya/collect-jobs-and-cronjobs
collect jobs and cronjobs as part of cluster-resources
2021-09-03 17:25:41 -04:00
Dan Stough 6e09aa641d Merge pull request #412 from replicatedhq/dans-chore-goreleaser-175-updates
chore(ci): update gorelease.yaml to use v175 syntax
2021-09-03 17:23:14 -04:00
Andrew Lavery 7fcc951c9a collect jobs and cronjobs as part of cluster-resources 2021-09-03 15:46:03 -05:00
Dan Stough 123e2e1049 chore(ci): update gorelease.yaml to use 175 syntax 2021-09-03 20:45:18 +00:00
Dan Stough 558f92caef Merge pull request #411 from replicatedhq/dans-fix-missing-cluster-res-namespaces
fix: cluster-res collector fixed to one namespace
2021-09-03 15:41:50 -04:00
Dan Stough 0478a7a60f fix: cluster-res collector fixed to one namespace 2021-09-03 19:23:44 +00:00
divolgin 5b157381fe Merge pull request #410 from replicatedhq/divolgin/codeowners
Create CODEOWNERS
2021-09-01 14:30:13 -07:00
divolgin 1181bc21d0 Create CODEOWNERS 2021-09-01 21:14:48 +00:00
Salah Aldeen Al Saleh c7af0dc593 fix openshift cluster detection (#408) 2021-08-24 09:51:12 -07:00
Kyle Sorensenandmurphybytes f30e925580 Community building (#402)
Create CODE_OF_CONDUCT.md , CONTRIBUTING.md and bug report / feature request templates for issues

Co-authored-by: murphybytes <murphybytes@gmail.com>
2021-08-19 20:57:46 -06:00
Jalaja Ganapathy e23fb2ce59 run support-bundle and preflight checks even with restricted access (#404) 2021-08-13 07:52:49 -07:00
John Murphy fd3b32293c default result only when no other result exists (#398) 2021-07-28 11:19:41 -05:00
Kyle Sorensen 2977f8f0d3 Stop longhorn false positives on no results. (#397)
Longhorn analyzer no longer report positive results on no results
2021-07-28 09:37:54 -06:00
Kyle Sorensen bf7d658313 troubleshoot enables collecting all data from a configmap (#395)
Enabled collecting all data from a ConfigMap instead of by key
2021-07-26 13:00:06 -06:00
emosbaugh e2dbb623e3 Merge pull request #396 from replicatedhq/emosbaugh/20210726/cleanup
remove debug log
2021-07-26 11:38:29 -06:00
Joris 'Josh' De Winne 6349ae8aee Adding support for inverted regex (#370) 2021-07-26 13:06:30 -04:00
Ethan Mosbaugh 851c91b582 remove debug log 2021-07-26 16:28:11 +00:00
emosbaugh cf4d510413 Merge pull request #393 from replicatedhq/collect-copy-extractArchive
Copy collectors extractArchive property
2021-07-23 08:55:10 -06:00
John Murphy e0f6cab5b3 Fix removes control characters from non interactive preflight runs (#394) 2021-07-23 09:46:36 -05:00
Ethan Mosbaugh cf7864cd97 Copy collectors extractArchive property 2021-07-23 13:37:57 +00:00
emosbaugh 8dcfa9886d Copy from host collector (#391)
* Copy from host collector

* namespace improvements

* better support for multiple nodes
2021-07-22 12:25:59 -07:00
John Murphy 6007f15253 fixed issue where warnings are disseminated along with passes (#390) 2021-07-22 08:27:39 -05:00
Andrew Lavery ba65f92ca2 Merge pull request #389 from replicatedhq/laverya/simplify-text-analyze-code
simplify the text analyze code by combining with compareRegex code
2021-07-20 16:32:38 -07:00
Andrew Lavery 6a0fb2e19c greatly improve coverage by adding regex group tests 2021-07-20 19:15:09 -04:00
Andrew Lavery 6861660460 simplify the text analyze code by combining with compareRegex code 2021-07-20 18:43:09 -04:00
kwsorensen 1ed6100ac8 Feature/validate tcp load balancer address (#387)
Load Balancer Validation part of troubleshoot pre-flight checks
2021-07-14 14:30:47 -06:00
emosbaugh 39350b5722 ConfigMap collector and secrets can be collected by selectors (#384)
* ConfigMap collector and secrets can be collected by selectors

* follow docs

* Pass context and kubernetes client to collectors

* collect tests

* analyze tests

* fix tests

* improvements
2021-07-08 16:30:26 -07:00
emosbaugh d7b6aa2758 Log progress when interactive=false (#382)
* Log progress when interactive=false

* safe print statement
2021-07-08 13:57:35 -07:00
Andrew Reed 0d56371242 Merge pull request #385 from replicatedhq/longhorn-accumulate-analyzers
Accumulate all longhorn pass results
2021-07-08 14:16:06 -05:00
Andrew Reed c95dc489a2 Accumulate all longhorn pass results
If there are any error or warning results then return those. Otherwise
return a single healthy pass result.
2021-07-08 18:25:10 +00:00
John Murphy c119a16235 Fixed bugs introduced by handling multiple results in host preflights (#383)
Fixed bug caused by host preflights not handling empty when clauses, this cropped up because we now handle multiple host preflight results. Also expanded test coverage and added integration test script.
2021-07-08 11:08:53 -05:00
John Murphy d730e6cad6 fixed tests 2021-07-06 08:42:12 -05:00
John Murphy 7e32de464a implemented code review suggestion 2021-07-06 08:42:12 -05:00
John Murphy ae4c07027b host preflights can produce multiple results 2021-07-06 08:42:12 -05:00
John Murphy eef54d0021 force timezone to upper case 2021-07-06 08:42:12 -05:00
divolgin 972fe2c210 Merge pull request #380 from replicatedhq/divolgin/cursor
Make cursors visible on CTRL+C
2021-07-01 16:26:46 -07:00
divolgin 4047977b35 Make cursors visible on CTRL+C 2021-07-01 23:08:05 +00:00
divolgin 3382d29e96 Merge pull request #379 from replicatedhq/divolgin/apis
Update troubleshoot api schema
2021-07-01 11:32:24 -07:00
Andrew Reed 4b4eedf477 Merge pull request #378 from replicatedhq/longhorn-ns
Always use longhorn namespace
2021-07-01 13:12:19 -05:00
divolgin 7381d5086c Update troubleshoot api schema 2021-07-01 17:24:00 +00:00
Andrew Reed 1ed8532663 Speed up replica checksum 2021-07-01 16:52:59 +00:00
Andrew Reed 3833955a58 Always include longhorn namespace 2021-07-01 15:03:28 +00:00
divolgin 6aa97a9a51 Merge pull request #376 from replicatedhq/divolgin/insecretls
Don't skip TLS validation on http package's default client
2021-06-30 11:47:25 -07:00
divolgin 52bbc0f2bf Don't skip TLS validation on http package's default client 2021-06-30 18:22:15 +00:00
divolgin 8fbae89465 Merge pull request #375 from replicatedhq/divolgin/redact
Add a default redactor for postgres db connection string
2021-06-29 15:48:39 -07:00
divolgin 53e8ee5495 Add a default redactor for postgres db connection string 2021-06-29 22:38:32 +00:00
Andrew Reed c018452b8a Merge pull request #374 from areed/longhorn-corrupted-replica
Longhorn replica corruption analyzer
2021-06-22 17:38:19 -05:00
Andrew Reed cb3925a0af Longhorn replica corruption analyzer
This automates the procedure from
https://longhorn.io/docs/1.1.1/advanced-resources/data-recovery/corrupted-replica/
2021-06-22 21:55:12 +00:00
Andrew Reed 900dd9e417 Merge pull request #373 from areed/no-import-goprocinfo
Use updated longhorn-manager fork
2021-06-22 15:37:49 -05:00
Andrew Reed 776934af2b Use updated longhorn-manager fork 2021-06-22 20:20:41 +00:00
Vera Harless f92d6f9cfe fix: always print support-bundle filename after generation (#372) 2021-06-16 14:05:47 -04:00
Andrew Reed 904afc0b2a Merge pull request #369 from areed/longhorn-engines-analyzer
Analyze longhorn engine
2021-05-27 16:54:34 -05:00
Andrew Reed e1bccd74b5 Analyze longhorn engine 2021-05-27 21:37:39 +00:00
Andrew Reed 2f34561a39 Merge pull request #368 from areed/longhorn-pod-logs
Collect all longhorn pod logs
2021-05-27 15:29:45 -05:00
Andrew Reed a86f5cae7d Collect all longhorn pod logs 2021-05-27 20:14:05 +00:00
Andrew Reed ed6b3369b3 Merge pull request #367 from areed/longhorn-replica-analyzer
Analyze longhorn replica
2021-05-27 15:07:54 -05:00
Andrew Reed 0d5f17de3c Analyze longhorn replica 2021-05-27 19:44:52 +00:00
Andrew Reed 12fb05e5f9 Merge pull request #366 from areed/longhorn
Longhorn collector for all CRDs
2021-05-26 18:52:06 -05:00
Andrew Reed 646f7a6991 Longhorn collector for all CRDs
Also implement a single analyzer as a proof of concept. More analyzers
can be added using the collected CRDs.
2021-05-26 23:37:15 +00:00
divolgin 160fffe9ed Merge pull request #364 from replicatedhq/divolgin/insecure
Do insecure upload if user allows it
2021-05-14 11:33:47 -07:00
divolgin c17be3fb73 Do insecure upload if user allows it 2021-05-14 18:21:56 +00:00
Dan Stough 35718432fe Merge pull request #363 from replicatedhq/chore-refactor-support-bundle
fix: use temp directory for storing bundles
2021-05-05 10:17:21 -04:00
Dan Stough 0dd24c702c fix: use temp directory for storing bundles 2021-05-04 20:29:21 +00:00
Dan Stough 2c3909f585 Merge pull request #360 from replicatedhq/chore-refactor-support-bundle
chore(support-bundle): refactor out support bundle pkg
2021-05-04 10:53:29 -04:00
Dan Stough b8622a65ad chore(support-bundle): refactor out support bundle pkg 2021-05-04 14:41:54 +00:00
emosbaugh f1ca1a5345 Merge pull request #362 from replicatedhq/preflight-error-and-result
Include result if not nil regardless of error
2021-04-28 20:41:26 -07:00
Ethan Mosbaugh 9357d5ac96 Include result if not nil regardless of error 2021-04-28 02:58:59 +00:00
Andrew Lavery eef9ebe0e5 Merge pull request #361 from replicatedhq/laverya/rook-block-device-disk-stats
collect rook block device disk stats
2021-04-20 16:10:15 -05:00
Andrew Lavery 25a92dec56 collect rook block device disk stats
this contains both max size and currently used size for each PV
2021-04-20 15:41:47 -05:00
divolgin 39cf553a03 Merge pull request #359 from replicatedhq/divolgin/maxage
Honor maxAge for log collector if set in the spec
2021-04-19 13:26:29 -07:00
divolgin e5233dfcf5 Honor maxAge for log collector if set in the spec 2021-04-19 20:15:41 +00:00
Dan Stough 1562a645de Merge pull request #358 from replicatedhq/DanStough-patch-1
Fix linkage to KOTS
2021-04-15 12:01:33 -04:00
Dan Stough 3f43697b57 Fix linkage to KOTS 2021-04-15 11:46:59 -04:00
jgruica 67ec6f54bc Merge pull request #357 from replicatedhq/jelena-analyzer-checks
More analyzer types checks
2021-04-15 07:59:08 -07:00
Jelena c43da65afe More analyzer types checks 2021-04-15 14:30:20 +00:00
jgruica dd2c2f84e6 Merge pull request #352 from replicatedhq/jelena-analyze-supportbundle
Analyze kind support bundle
2021-04-14 14:15:24 -07:00
Andrew Reed 45decfcb7d Merge pull request #356 from areed/fs-perf-text
Remove html escaping in fs performance analyzer
2021-04-13 12:47:14 -07:00
Andrew Reed 7d7e3c2664 Remove html escaping in fs performance analyzer 2021-04-13 19:35:11 +00:00
Andrew Reed 822ada99a2 Merge pull request #355 from areed/fs-perf-fix-timeout
Fix background IOPS blocking until timeout
2021-04-13 12:05:54 -07:00
Andrew Reed 30f21ac71b Fix background IOPS blocking until timeout 2021-04-13 18:55:53 +00:00
Andrew Reed ce815844cd Merge pull request #354 from areed/fs-perf-timeout
Add timeout to filesystem performance collector
2021-04-13 11:43:19 -07:00
Andrew Reed 0a6c9836e0 Add timeout to filesystem performance collector 2021-04-13 18:30:18 +00:00
Andrew Lavery e21813535c Merge pull request #353 from replicatedhq/laverya/collect-rgw-status
collect RGW status/stats as part of ceph collector
2021-04-13 11:24:08 -05:00
Andrew Lavery 44993a5d0d collect RGW status as part of ceph collector 2021-04-12 23:14:00 -05:00
Jelena a2f4041a1b Analyze kind support bundle 2021-04-12 23:50:15 +00:00
Andrew Reed 744523cc09 Merge pull request #351 from areed/concurrent-iops
Benchmark write latency with background IOPS
2021-04-12 16:06:13 -07:00
Andrew Reed 477cde7228 Benchmark write latency with background IOPS
Add a background IOPS feature to the filesystem performance collector
that specifies separate read and write background IOPS to perform while
measuring latency. This allows for better assessment of whether etcd
will be stable when running alongside other workloads on the same
cluster.

Also add templating to the outcome message of the filesystem performance
analyzers to allow printing individual latency percentiles or the entire
table.

Remove the random IOPS benchmark since it was attempting to perform
unaligned direct I/O.
2021-04-12 22:56:00 +00:00
divolgin ae63ccba8d Merge pull request #350 from replicatedhq/divolgin/so
remove troubleshoot.so from builds
2021-04-12 11:53:18 -07:00
divolgin 5c5e30a9a5 remove troubleshoot.so from builds 2021-04-12 18:50:58 +00:00
Salah Aldeen Al Saleh ac2a0484e2 add build flags to ffi (#349) 2021-04-12 11:23:46 -07:00
divolgin bc91151883 Merge pull request #348 from replicatedhq/divolgin/goreleaser
fix ffi build
2021-04-12 10:57:34 -07:00
divolgin de8feb167e fix ffi build 2021-04-12 17:22:21 +00:00
divolgin 0a6ef1c2c3 Merge pull request #347 from replicatedhq/divolgin/goreleaser
fix ffi build
2021-04-12 10:10:08 -07:00
divolgin 4a89067912 fix ffi build 2021-04-12 17:07:09 +00:00
divolgin 1e17c5c4c2 Merge pull request #346 from replicatedhq/divolgin/goreleaser
fix go-releaser build
2021-04-12 09:40:23 -07:00
divolgin 32cca1215a fix go-releaser build 2021-04-12 16:34:53 +00:00
divolgin 052d10fec7 Merge pull request #345 from replicatedhq/divolgin/images
use containers package instead of go-containerregistry
2021-04-12 08:42:46 -07:00
divolgin 7a0c6e5383 use containers package instead of go-containerregistry 2021-04-11 21:39:44 +00:00
divolgin 1dc03db254 Merge pull request #343 from replicatedhq/divolgin/images
Docker registry collector/analyzer
2021-04-09 09:25:29 -07:00
divolgin fe414af556 Docker registry collector/analyzer 2021-04-09 16:17:15 +00:00
Andrew Lavery f8dca39899 Merge pull request #342 from replicatedhq/laverya/improve-systemctl-service-analyzer
expand systemctl service analyzer to also match service sub/load
2021-04-02 15:33:02 -04:00
Andrew Lavery 19aef8a02f expand systemctl service analyzer to also match service sub/load 2021-04-02 14:48:24 -04:00
Andrew Lavery 4b6606e323 Merge pull request #341 from replicatedhq/laverya/systemctl-services-collector
systemctl services collector
2021-03-30 16:41:30 -04:00
Andrew Lavery 559e18d996 lowercase errors 2021-03-30 16:32:19 -04:00
Andrew Lavery bf4d26acd2 add host_services analyzer 2021-03-30 16:15:18 -04:00
Andrew Lavery f3b599c19a collect host systemctl services 2021-03-30 16:15:17 -04:00
Salah Aldeen Al Saleh 810b3cbf78 support non-interactive mode in support bundle (#340)
* support non-interactive mode in support bundle
2021-03-23 18:01:39 -07:00
Andrew Lavery 81789181ed Merge pull request #339 from replicatedhq/laverya/host-unmounted-partitions
added two parameters to the eligible block device check
2021-03-18 19:36:56 -04:00
Andrew Lavery 256c68feca added two parameters to the eligible block device check
whether to accept unmounted partitions (default false) and minimum acceptable device size (default 0)
2021-03-18 19:03:39 -04:00
divolgin b01e6ef578 Merge pull request #338 from replicatedhq/divolgin/preogress2
Add progress percentage
2021-03-18 16:02:11 -07:00
divolgin 62afc87af8 Add progress percentage 2021-03-18 22:29:27 +00:00
divolgin 17bff4b2bd Merge pull request #337 from replicatedhq/divolgin/progress
Report back some basic progress
2021-03-18 12:11:54 -07:00
divolgin 5f2525b663 Report back some basic progress 2021-03-18 18:56:27 +00:00
Salah Aldeen Al Saleh afa0bc56d4 fix custom redactors file selectors in support bundle subdirectory (#336)
* fix custom redactors file selectors in support bundle subdirectory
2021-03-11 08:45:20 -08:00
Salah Aldeen Al Saleh c1f9b3906d support reading redactors from configmaps (#335) 2021-03-10 15:25:55 -08:00
DexandSalah Aldeen Al Saleh 0a19d35073 hide spinner if interactive false (#328)
* hide preflight spinner if interactive is false

Co-authored-by: Salah Aldeen Al Saleh <salahalsaleh1993@gmail.com>
2021-03-09 09:42:38 -08:00
emosbaugh 7c4135d563 Merge pull request #334 from replicatedhq/host-preflgihts-ux
Host preflight ux improvements
2021-03-02 12:03:17 -08:00
Ethan Mosbaugh 4b78c430ca Host preflight ux improvements 2021-03-02 17:27:01 +00:00
Ethan Mosbaugh 09d16ff185 Host preflights exclude 2021-03-01 22:45:16 +00:00
Andrew Lavery bdf843f84d Merge pull request #332 from replicatedhq/laverya/add-regex-case-insensitive-test
add a test that uses a case-insensitive regex analyzer
2021-03-01 13:13:35 -05:00
Andrew Lavery 47f7d98907 add a test that uses a case-insensitive regex analyzer 2021-03-01 13:02:30 -05:00
457 changed files with 68817 additions and 10530 deletions
+115
View File
@@ -0,0 +1,115 @@
#-------------------------------------------------------------------------------------------------------------
# Modified from Codespaces default container image: https://github.com/microsoft/vscode-dev-containers/blob/main/containers/codespaces-linux/history/1.6.3.md
# - Remove PHP, Ruby, Dotnet, Java, powershell, rust dependencies
# - Remove fish shell
# - Remove Oryx
# - Remove git-lfs
# - Change shell to zsh
#
# TODO (dans): find a better way to pull in library script dynamically from vscode repo
# TODO (dans): AWS CLI - make a common script in the dev-containers repo
# TODO (dans): Gcloud CLI - make a common script in the dev-containers repo
# TODO (dans): add gcommands alias
# TODO (dans): terraform
#-------------------------------------------------------------------------------------------------------------
FROM mcr.microsoft.com/oryx/build:vso-focal-20210902.1 as replicated
ARG USERNAME=codespace
ARG USER_UID=1000
ARG USER_GID=$USER_UID
ARG HOMEDIR=/home/$USERNAME
ARG GO_VERSION="latest"
# Default to bash shell (other shells available at /usr/bin/fish and /usr/bin/zsh)
ENV SHELL=/bin/bash \
ORYX_ENV_TYPE=vsonline-present \
NODE_ROOT="${HOMEDIR}/.nodejs" \
PYTHON_ROOT="${HOMEDIR}/.python" \
HUGO_ROOT="${HOMEDIR}/.hugo" \
NVM_SYMLINK_CURRENT=true \
NVM_DIR="/home/${USERNAME}/.nvm" \
NVS_HOME="/home/${USERNAME}/.nvs" \
NPM_GLOBAL="/home/${USERNAME}/.npm-global" \
KREW_HOME="/home/${USERNAME}/.krew/bin" \
PIPX_HOME="/usr/local/py-utils" \
PIPX_BIN_DIR="/usr/local/py-utils/bin" \
GOROOT="/usr/local/go" \
GOPATH="/go"
ENV PATH="${PATH}:${KREW_HOME}:${NVM_DIR}/current/bin:${NPM_GLOBAL}/bin:${ORIGINAL_PATH}:${GOROOT}/bin:${GOPATH}/bin:${PIPX_BIN_DIR}:/opt/conda/condabin:${NODE_ROOT}/current/bin:${PYTHON_ROOT}/current/bin:${HUGO_ROOT}/current/bin:${ORYX_PATHS}"
COPY library-scripts/* first-run-notice.txt /tmp/scripts/
COPY ./config/* /etc/replicated/
COPY ./lifecycle-scripts/* /var/lib/replicated/scripts/
# Install needed utilities and setup non-root user. Use a separate RUN statement to add your own dependencies.
RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
# Restore man command
&& yes | unminimize 2>&1 \
# Run common script and setup user
&& bash /tmp/scripts/common-debian.sh "true" "${USERNAME}" "${USER_UID}" "${USER_GID}" "true" "true" "true" \
&& bash /tmp/scripts/setup-user.sh "${USERNAME}" "${PATH}" \
# Change owner of opt contents since Oryx can dynamically install and will run as "codespace"
&& chown ${USERNAME} /opt/* \
&& chsh -s /bin/bash ${USERNAME} \
# Verify expected build and debug tools are present
&& apt-get -y install build-essential cmake python3-dev \
# Install tools and shells not in common script
&& apt-get install -yq vim vim-doc xtail software-properties-common libsecret-1-dev \
# Install additional tools (useful for 'puppeteer' project)
&& apt-get install -y --no-install-recommends libnss3 libnspr4 libatk-bridge2.0-0 libatk1.0-0 libx11-6 libpangocairo-1.0-0 \
libx11-xcb1 libcups2 libxcomposite1 libxdamage1 libxfixes3 libpango-1.0-0 libgbm1 libgtk-3-0 \
&& bash /tmp/scripts/sshd-debian.sh \
&& bash /tmp/scripts/github-debian.sh \
&& bash /tmp/scripts/azcli-debian.sh \
# Install Moby CLI and Engine
&& /bin/bash /tmp/scripts/docker-debian.sh "true" "/var/run/docker-host.sock" "/var/run/docker.sock" "${USERNAME}" "true" \
# && bash /tmp/scripts/docker-in-docker-debian.sh "true" "${USERNAME}" "true" \
&& bash /tmp/scripts/kubectl-helm-debian.sh \
# Build latest git from source
&& bash /tmp/scripts/git-from-src-debian.sh "latest" \
# Clean up
&& apt-get autoremove -y && apt-get clean -y \
# Move first run notice to right spot
&& mkdir -p /usr/local/etc/vscode-dev-containers/ \
&& mv -f /tmp/scripts/first-run-notice.txt /usr/local/etc/vscode-dev-containers/
# Install Python
RUN bash /tmp/scripts/python-debian.sh "none" "/opt/python/latest" "${PIPX_HOME}" "${USERNAME}" "true" \
&& apt-get clean -y
# Setup Node.js, install NVM and NVS
RUN bash /tmp/scripts/node-debian.sh "${NVM_DIR}" "none" "${USERNAME}" \
&& (cd ${NVM_DIR} && git remote get-url origin && echo $(git log -n 1 --pretty=format:%H -- .)) > ${NVM_DIR}/.git-remote-and-commit \
# Install nvs (alternate cross-platform Node.js version-management tool)
&& sudo -u ${USERNAME} git clone -c advice.detachedHead=false --depth 1 https://github.com/jasongin/nvs ${NVS_HOME} 2>&1 \
&& (cd ${NVS_HOME} && git remote get-url origin && echo $(git log -n 1 --pretty=format:%H -- .)) > ${NVS_HOME}/.git-remote-and-commit \
&& sudo -u ${USERNAME} bash ${NVS_HOME}/nvs.sh install \
&& rm ${NVS_HOME}/cache/* \
# Set npm global location
&& sudo -u ${USERNAME} npm config set prefix ${NPM_GLOBAL} \
&& npm config -g set prefix ${NPM_GLOBAL} \
# Clean up
&& rm -rf ${NVM_DIR}/.git ${NVS_HOME}/.git
# Install Go
RUN bash /tmp/scripts/go-debian.sh "${GO_VERSION}" "${GOROOT}" "${GOPATH}" "${USERNAME}"
# Install Replicated Tools
RUN bash /tmp/scripts/replicated-debian.sh \
&& rm -rf /tmp/scripts \
&& apt-get clean -y
# Userspace
ENV SHELL=/bin/zsh
USER ${USERNAME}
COPY --chown=${USERNAME}:root library-scripts/replicated-userspace.sh /tmp/scripts/
RUN bash /usr/local/share/docker-init.sh \
&& bash /tmp/scripts/replicated-userspace.sh \
&& rm -rf /tmp/scripts/scripts
# Fire Docker/Moby script if needed along with Oryx's benv
ENTRYPOINT [ "/usr/local/share/docker-init.sh", "/usr/local/share/ssh-init.sh", "benv" ]
CMD [ "sleep", "infinity" ]
+7
View File
@@ -0,0 +1,7 @@
# Replicated KOTS Codespace Container
Most of the code here is borrowed from this [Microsoft repo of base images](https://github.com/microsoft/vscode-dev-containers), except for replicated specific things.
## Notes
* k3d *DOES NOT* work with DinD. You have to use the docker with docker install instead.
* Might be faster to install kubectl plugins on the `$PATH` in the `Dockerfile` instead of downloading them `onCreate.sh`.
+10
View File
@@ -0,0 +1,10 @@
apiVersion: k3d.io/v1alpha3
kind: Simple
name: replicated
servers: 1
image: rancher/k3s:v1.21.4-k3s1 # v1.21.3-k3s1 default is broken
registries:
create:
name: k3d-replicated-registry.localhost
host: "0.0.0.0"
hostPort: "5000"
+63
View File
@@ -0,0 +1,63 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
// https://github.com/microsoft/vscode-dev-containers/tree/v0.162.0/containers/javascript-node
{
"name": "Replicated Codeserver",
"build": {
"dockerfile": "Dockerfile",
"args": {
"GO_VERSION": "1.17",
}
},
// Set *default* container specific settings.json values on container create.
"settings": {
"terminal.integrated.shell.linux": "/usr/bin/zsh",
"go.toolsManagement.checkForUpdates": "local",
"go.useLanguageServer": true,
"go.gopath": "/go",
"go.goroot": "/usr/local/go",
"python.pythonPath": "/opt/python/latest/bin/python",
"python.linting.enabled": true,
"python.linting.pylintEnabled": true,
"python.formatting.autopep8Path": "/usr/local/py-utils/bin/autopep8",
"python.formatting.blackPath": "/usr/local/py-utils/bin/black",
"python.formatting.yapfPath": "/usr/local/py-utils/bin/yapf",
"python.linting.banditPath": "/usr/local/py-utils/bin/bandit",
"python.linting.flake8Path": "/usr/local/py-utils/bin/flake8",
"python.linting.mypyPath": "/usr/local/py-utils/bin/mypy",
"python.linting.pycodestylePath": "/usr/local/py-utils/bin/pycodestyle",
"python.linting.pydocstylePath": "/usr/local/py-utils/bin/pydocstyle",
"python.linting.pylintPath": "/usr/local/py-utils/bin/pylint",
"lldb.executable": "/usr/bin/lldb",
"files.watcherExclude": {
"**/target/**": true
}
},
"remoteUser": "codespace",
"overrideCommand": false,
"runArgs": [
"--privileged",
"--init"
],
"mounts": [
"source=/var/run/docker.sock,target=/var/run/docker-host.sock,type=bind",
],
// Add the IDs of extensions you want installed when the container is created.
"extensions": [
"dbaeumer.vscode-eslint",
"GitHub.vscode-pull-request-github",
"golang.go",
"github.copilot",
"lizebang.bash-extension-pack",
"streetsidesoftware.code-spell-checker",
],
// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "bash /var/lib/replicated/scripts/onCreate.sh",
// Use 'postStartCommand' to run commands after the container is created like starting minikube.
"postStartCommand": "bash /var/lib/replicated/scripts/onStart.sh",
// Comment out connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
// "remoteUser": "node"
}
+9
View File
@@ -0,0 +1,9 @@
👋 Welcome to your Replicated Codespace!
There's a local Kubernetes cluster set up for you.
Drivers Manual:
* `k` alias is available for `kubectl` with auto-completion for your pleasure
* This is a `zsh` terminal with Oh My Zsh installed. Just thought you should know.
@@ -0,0 +1,67 @@
#!/usr/bin/env bash
#-------------------------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
#-------------------------------------------------------------------------------------------------------------
#
# Docs: https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/azcli.md
# Maintainer: The VS Code and Codespaces Teams
#
# Syntax: ./azcli-debian.sh
set -e
MICROSOFT_GPG_KEYS_URI="https://packages.microsoft.com/keys/microsoft.asc"
if [ "$(id -u)" -ne 0 ]; then
echo -e 'Script must be run as root. Use sudo, su, or add "USER root" to your Dockerfile before running this script.'
exit 1
fi
# Get central common setting
get_common_setting() {
if [ "${common_settings_file_loaded}" != "true" ]; then
curl -sfL "https://aka.ms/vscode-dev-containers/script-library/settings.env" 2>/dev/null -o /tmp/vsdc-settings.env || echo "Could not download settings file. Skipping."
common_settings_file_loaded=true
fi
if [ -f "/tmp/vsdc-settings.env" ]; then
local multi_line=""
if [ "$2" = "true" ]; then multi_line="-z"; fi
local result="$(grep ${multi_line} -oP "$1=\"?\K[^\"]+" /tmp/vsdc-settings.env | tr -d '\0')"
if [ ! -z "${result}" ]; then declare -g $1="${result}"; fi
fi
echo "$1=${!1}"
}
# Function to run apt-get if needed
apt_get_update_if_needed()
{
if [ ! -d "/var/lib/apt/lists" ] || [ "$(ls /var/lib/apt/lists/ | wc -l)" = "0" ]; then
echo "Running apt-get update..."
apt-get update
else
echo "Skipping apt-get update."
fi
}
# Checks if packages are installed and installs them if not
check_packages() {
if ! dpkg -s "$@" > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get -y install --no-install-recommends "$@"
fi
}
export DEBIAN_FRONTEND=noninteractive
# Install dependencies
check_packages apt-transport-https curl ca-certificates lsb-release gnupg2
# Import key safely (new 'signed-by' method rather than deprecated apt-key approach) and install
. /etc/os-release
get_common_setting MICROSOFT_GPG_KEYS_URI
curl -sSL ${MICROSOFT_GPG_KEYS_URI} | gpg --dearmor > /usr/share/keyrings/microsoft-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/microsoft-archive-keyring.gpg] https://packages.microsoft.com/repos/azure-cli/ ${VERSION_CODENAME} main" > /etc/apt/sources.list.d/azure-cli.list
apt-get update
apt-get install -y azure-cli
echo "Done!"
@@ -0,0 +1,478 @@
#!/usr/bin/env bash
#-------------------------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
#-------------------------------------------------------------------------------------------------------------
#
# Docs: https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/common.md
# Maintainer: The VS Code and Codespaces Teams
#
# Syntax: ./common-debian.sh [install zsh flag] [username] [user UID] [user GID] [upgrade packages flag] [install Oh My Zsh! flag] [Add non-free packages]
set -e
INSTALL_ZSH=${1:-"true"}
USERNAME=${2:-"automatic"}
USER_UID=${3:-"automatic"}
USER_GID=${4:-"automatic"}
UPGRADE_PACKAGES=${5:-"true"}
INSTALL_OH_MYS=${6:-"true"}
ADD_NON_FREE_PACKAGES=${7:-"false"}
SCRIPT_DIR="$(cd $(dirname "${BASH_SOURCE[0]}") && pwd)"
MARKER_FILE="/usr/local/etc/vscode-dev-containers/common"
if [ "$(id -u)" -ne 0 ]; then
echo -e 'Script must be run as root. Use sudo, su, or add "USER root" to your Dockerfile before running this script.'
exit 1
fi
# Ensure that login shells get the correct path if the user updated the PATH using ENV.
rm -f /etc/profile.d/00-restore-env.sh
echo "export PATH=${PATH//$(sh -lc 'echo $PATH')/\$PATH}" > /etc/profile.d/00-restore-env.sh
chmod +x /etc/profile.d/00-restore-env.sh
# If in automatic mode, determine if a user already exists, if not use vscode
if [ "${USERNAME}" = "auto" ] || [ "${USERNAME}" = "automatic" ]; then
USERNAME=""
POSSIBLE_USERS=("vscode" "node" "codespace" "$(awk -v val=1000 -F ":" '$3==val{print $1}' /etc/passwd)")
for CURRENT_USER in ${POSSIBLE_USERS[@]}; do
if id -u ${CURRENT_USER} > /dev/null 2>&1; then
USERNAME=${CURRENT_USER}
break
fi
done
if [ "${USERNAME}" = "" ]; then
USERNAME=vscode
fi
elif [ "${USERNAME}" = "none" ]; then
USERNAME=root
USER_UID=0
USER_GID=0
fi
# Load markers to see which steps have already run
if [ -f "${MARKER_FILE}" ]; then
echo "Marker file found:"
cat "${MARKER_FILE}"
source "${MARKER_FILE}"
fi
# Ensure apt is in non-interactive to avoid prompts
export DEBIAN_FRONTEND=noninteractive
# Function to call apt-get if needed
apt_get_update_if_needed()
{
if [ ! -d "/var/lib/apt/lists" ] || [ "$(ls /var/lib/apt/lists/ | wc -l)" = "0" ]; then
echo "Running apt-get update..."
apt-get update
else
echo "Skipping apt-get update."
fi
}
# Run install apt-utils to avoid debconf warning then verify presence of other common developer tools and dependencies
if [ "${PACKAGES_ALREADY_INSTALLED}" != "true" ]; then
package_list="apt-utils \
openssh-client \
gnupg2 \
iproute2 \
procps \
lsof \
htop \
net-tools \
psmisc \
curl \
wget \
rsync \
ca-certificates \
unzip \
zip \
nano \
vim-tiny \
less \
jq \
lsb-release \
apt-transport-https \
dialog \
libc6 \
libgcc1 \
libkrb5-3 \
libgssapi-krb5-2 \
libicu[0-9][0-9] \
liblttng-ust0 \
libstdc++6 \
zlib1g \
locales \
sudo \
ncdu \
man-db \
strace \
manpages \
manpages-dev \
init-system-helpers"
# Needed for adding manpages-posix and manpages-posix-dev which are non-free packages in Debian
if [ "${ADD_NON_FREE_PACKAGES}" = "true" ]; then
# Bring in variables from /etc/os-release like VERSION_CODENAME
. /etc/os-release
sed -i -E "s/deb http:\/\/(deb|httpredir)\.debian\.org\/debian ${VERSION_CODENAME} main/deb http:\/\/\1\.debian\.org\/debian ${VERSION_CODENAME} main contrib non-free/" /etc/apt/sources.list
sed -i -E "s/deb-src http:\/\/(deb|httredir)\.debian\.org\/debian ${VERSION_CODENAME} main/deb http:\/\/\1\.debian\.org\/debian ${VERSION_CODENAME} main contrib non-free/" /etc/apt/sources.list
sed -i -E "s/deb http:\/\/(deb|httpredir)\.debian\.org\/debian ${VERSION_CODENAME}-updates main/deb http:\/\/\1\.debian\.org\/debian ${VERSION_CODENAME}-updates main contrib non-free/" /etc/apt/sources.list
sed -i -E "s/deb-src http:\/\/(deb|httpredir)\.debian\.org\/debian ${VERSION_CODENAME}-updates main/deb http:\/\/\1\.debian\.org\/debian ${VERSION_CODENAME}-updates main contrib non-free/" /etc/apt/sources.list
sed -i "s/deb http:\/\/security\.debian\.org\/debian-security ${VERSION_CODENAME}\/updates main/deb http:\/\/security\.debian\.org\/debian-security ${VERSION_CODENAME}\/updates main contrib non-free/" /etc/apt/sources.list
sed -i "s/deb-src http:\/\/security\.debian\.org\/debian-security ${VERSION_CODENAME}\/updates main/deb http:\/\/security\.debian\.org\/debian-security ${VERSION_CODENAME}\/updates main contrib non-free/" /etc/apt/sources.list
sed -i "s/deb http:\/\/deb\.debian\.org\/debian ${VERSION_CODENAME}-backports main/deb http:\/\/deb\.debian\.org\/debian ${VERSION_CODENAME}-backports main contrib non-free/" /etc/apt/sources.list
sed -i "s/deb-src http:\/\/deb\.debian\.org\/debian ${VERSION_CODENAME}-backports main/deb http:\/\/deb\.debian\.org\/debian ${VERSION_CODENAME}-backports main contrib non-free/" /etc/apt/sources.list
echo "Running apt-get update..."
apt-get update
package_list="${package_list} manpages-posix manpages-posix-dev"
else
apt_get_update_if_needed
fi
# Install libssl1.1 if available
if [[ ! -z $(apt-cache --names-only search ^libssl1.1$) ]]; then
package_list="${package_list} libssl1.1"
fi
# Install appropriate version of libssl1.0.x if available
libssl_package=$(dpkg-query -f '${db:Status-Abbrev}\t${binary:Package}\n' -W 'libssl1\.0\.?' 2>&1 || echo '')
if [ "$(echo "$LIlibssl_packageBSSL" | grep -o 'libssl1\.0\.[0-9]:' | uniq | sort | wc -l)" -eq 0 ]; then
if [[ ! -z $(apt-cache --names-only search ^libssl1.0.2$) ]]; then
# Debian 9
package_list="${package_list} libssl1.0.2"
elif [[ ! -z $(apt-cache --names-only search ^libssl1.0.0$) ]]; then
# Ubuntu 18.04, 16.04, earlier
package_list="${package_list} libssl1.0.0"
fi
fi
echo "Packages to verify are installed: ${package_list}"
apt-get -y install --no-install-recommends ${package_list} 2> >( grep -v 'debconf: delaying package configuration, since apt-utils is not installed' >&2 )
# Install git if not already installed (may be more recent than distro version)
if ! type git > /dev/null 2>&1; then
apt-get -y install --no-install-recommends git
fi
PACKAGES_ALREADY_INSTALLED="true"
fi
# Get to latest versions of all packages
if [ "${UPGRADE_PACKAGES}" = "true" ]; then
apt_get_update_if_needed
apt-get -y upgrade --no-install-recommends
apt-get autoremove -y
fi
# Ensure at least the en_US.UTF-8 UTF-8 locale is available.
# Common need for both applications and things like the agnoster ZSH theme.
if [ "${LOCALE_ALREADY_SET}" != "true" ] && ! grep -o -E '^\s*en_US.UTF-8\s+UTF-8' /etc/locale.gen > /dev/null; then
echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen
locale-gen
LOCALE_ALREADY_SET="true"
fi
# Create or update a non-root user to match UID/GID.
if id -u ${USERNAME} > /dev/null 2>&1; then
# User exists, update if needed
if [ "${USER_GID}" != "automatic" ] && [ "$USER_GID" != "$(id -G $USERNAME)" ]; then
groupmod --gid $USER_GID $USERNAME
usermod --gid $USER_GID $USERNAME
fi
if [ "${USER_UID}" != "automatic" ] && [ "$USER_UID" != "$(id -u $USERNAME)" ]; then
usermod --uid $USER_UID $USERNAME
fi
else
# Create user
if [ "${USER_GID}" = "automatic" ]; then
groupadd $USERNAME
else
groupadd --gid $USER_GID $USERNAME
fi
if [ "${USER_UID}" = "automatic" ]; then
useradd -s /bin/bash --gid $USERNAME -m $USERNAME
else
useradd -s /bin/bash --uid $USER_UID --gid $USERNAME -m $USERNAME
fi
fi
# Add add sudo support for non-root user
if [ "${USERNAME}" != "root" ] && [ "${EXISTING_NON_ROOT_USER}" != "${USERNAME}" ]; then
echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME
chmod 0440 /etc/sudoers.d/$USERNAME
EXISTING_NON_ROOT_USER="${USERNAME}"
fi
# ** Shell customization section **
if [ "${USERNAME}" = "root" ]; then
user_rc_path="/root"
else
user_rc_path="/home/${USERNAME}"
fi
# Restore user .bashrc defaults from skeleton file if it doesn't exist or is empty
if [ ! -f "${user_rc_path}/.bashrc" ] || [ ! -s "${user_rc_path}/.bashrc" ] ; then
cp /etc/skel/.bashrc "${user_rc_path}/.bashrc"
fi
# Restore user .profile defaults from skeleton file if it doesn't exist or is empty
if [ ! -f "${user_rc_path}/.profile" ] || [ ! -s "${user_rc_path}/.profile" ] ; then
cp /etc/skel/.profile "${user_rc_path}/.profile"
fi
# .bashrc/.zshrc snippet
rc_snippet="$(cat << 'EOF'
if [ -z "${USER}" ]; then export USER=$(whoami); fi
if [[ "${PATH}" != *"$HOME/.local/bin"* ]]; then export PATH="${PATH}:$HOME/.local/bin"; fi
# Display optional first run image specific notice if configured and terminal is interactive
if [ -t 1 ] && [[ "${TERM_PROGRAM}" = "vscode" || "${TERM_PROGRAM}" = "codespaces" ]] && [ ! -f "$HOME/.config/vscode-dev-containers/first-run-notice-already-displayed" ]; then
if [ -f "/usr/local/etc/vscode-dev-containers/first-run-notice.txt" ]; then
cat "/usr/local/etc/vscode-dev-containers/first-run-notice.txt"
elif [ -f "/workspaces/.codespaces/shared/first-run-notice.txt" ]; then
cat "/workspaces/.codespaces/shared/first-run-notice.txt"
fi
mkdir -p "$HOME/.config/vscode-dev-containers"
# Mark first run notice as displayed after 10s to avoid problems with fast terminal refreshes hiding it
((sleep 10s; touch "$HOME/.config/vscode-dev-containers/first-run-notice-already-displayed") &)
fi
# Set the default git editor if not already set
if [ -z "$(git config --get core.editor)" ] && [ -z "${GIT_EDITOR}" ]; then
if [ "${TERM_PROGRAM}" = "vscode" ]; then
if [[ -n $(command -v code-insiders) && -z $(command -v code) ]]; then
export GIT_EDITOR="code-insiders --wait"
else
export GIT_EDITOR="code --wait"
fi
fi
fi
EOF
)"
# code shim, it fallbacks to code-insiders if code is not available
cat << 'EOF' > /usr/local/bin/code
#!/bin/sh
get_in_path_except_current() {
which -a "$1" | grep -A1 "$0" | grep -v "$0"
}
code="$(get_in_path_except_current code)"
if [ -n "$code" ]; then
exec "$code" "$@"
elif [ "$(command -v code-insiders)" ]; then
exec code-insiders "$@"
else
echo "code or code-insiders is not installed" >&2
exit 127
fi
EOF
chmod +x /usr/local/bin/code
# systemctl shim - tells people to use 'service' if systemd is not running
cat << 'EOF' > /usr/local/bin/systemctl
#!/bin/sh
set -e
if [ -d "/run/systemd/system" ]; then
exec /bin/systemctl/systemctl "$@"
else
echo '\n"systemd" is not running in this container due to its overhead.\nUse the "service" command to start services intead. e.g.: \n\nservice --status-all'
fi
EOF
chmod +x /usr/local/bin/systemctl
# Codespaces bash and OMZ themes - partly inspired by https://github.com/ohmyzsh/ohmyzsh/blob/master/themes/robbyrussell.zsh-theme
codespaces_bash="$(cat \
<<'EOF'
# Codespaces bash prompt theme
__bash_prompt() {
local userpart='`export XIT=$? \
&& [ ! -z "${GITHUB_USER}" ] && echo -n "\[\033[0;32m\]@${GITHUB_USER} " || echo -n "\[\033[0;32m\]\u " \
&& [ "$XIT" -ne "0" ] && echo -n "\[\033[1;31m\]➜" || echo -n "\[\033[0m\]➜"`'
local gitbranch='`\
export BRANCH=$(git symbolic-ref --short HEAD 2>/dev/null || git rev-parse --short HEAD 2>/dev/null); \
if [ "${BRANCH}" != "" ]; then \
echo -n "\[\033[0;36m\](\[\033[1;31m\]${BRANCH}" \
&& if git ls-files --error-unmatch -m --directory --no-empty-directory -o --exclude-standard ":/*" > /dev/null 2>&1; then \
echo -n " \[\033[1;33m\]✗"; \
fi \
&& echo -n "\[\033[0;36m\]) "; \
fi`'
local lightblue='\[\033[1;34m\]'
local removecolor='\[\033[0m\]'
PS1="${userpart} ${lightblue}\w ${gitbranch}${removecolor}\$ "
unset -f __bash_prompt
}
__bash_prompt
EOF
)"
codespaces_zsh="$(cat \
<<'EOF'
# Codespaces zsh prompt theme
__zsh_prompt() {
local prompt_username
if [ ! -z "${GITHUB_USER}" ]; then
prompt_username="@${GITHUB_USER}"
else
prompt_username="%n"
fi
PROMPT="%{$fg[green]%}${prompt_username} %(?:%{$reset_color%}➜ :%{$fg_bold[red]%}➜ )" # User/exit code arrow
PROMPT+='%{$fg_bold[blue]%}%(5~|%-1~/…/%3~|%4~)%{$reset_color%} ' # cwd
PROMPT+='$(git_prompt_info)%{$fg[white]%}$ %{$reset_color%}' # Git status
unset -f __zsh_prompt
}
ZSH_THEME_GIT_PROMPT_PREFIX="%{$fg_bold[cyan]%}(%{$fg_bold[red]%}"
ZSH_THEME_GIT_PROMPT_SUFFIX="%{$reset_color%} "
ZSH_THEME_GIT_PROMPT_DIRTY=" %{$fg_bold[yellow]%}✗%{$fg_bold[cyan]%})"
ZSH_THEME_GIT_PROMPT_CLEAN="%{$fg_bold[cyan]%})"
__zsh_prompt
EOF
)"
# Add notice that Oh My Bash! has been removed from images and how to provide information on how to install manually
omb_readme="$(cat \
<<'EOF'
"Oh My Bash!" has been removed from this image in favor of a simple shell prompt. If you
still wish to use it, remove "~/.oh-my-bash" and install it from: https://github.com/ohmybash/oh-my-bash
You may also want to consider "Bash-it" as an alternative: https://github.com/bash-it/bash-it
See here for infomation on adding it to your image or dotfiles: https://aka.ms/codespaces/omb-remove
EOF
)"
omb_stub="$(cat \
<<'EOF'
#!/usr/bin/env bash
if [ -t 1 ]; then
cat $HOME/.oh-my-bash/README.md
fi
EOF
)"
# Add RC snippet and custom bash prompt
if [ "${RC_SNIPPET_ALREADY_ADDED}" != "true" ]; then
echo "${rc_snippet}" >> /etc/bash.bashrc
echo "${codespaces_bash}" >> "${user_rc_path}/.bashrc"
echo 'export PROMPT_DIRTRIM=4' >> "${user_rc_path}/.bashrc"
if [ "${USERNAME}" != "root" ]; then
echo "${codespaces_bash}" >> "/root/.bashrc"
echo 'export PROMPT_DIRTRIM=4' >> "/root/.bashrc"
fi
chown ${USERNAME}:${USERNAME} "${user_rc_path}/.bashrc"
RC_SNIPPET_ALREADY_ADDED="true"
fi
# Add stub for Oh My Bash!
if [ ! -d "${user_rc_path}/.oh-my-bash}" ] && [ "${INSTALL_OH_MYS}" = "true" ]; then
mkdir -p "${user_rc_path}/.oh-my-bash" "/root/.oh-my-bash"
echo "${omb_readme}" >> "${user_rc_path}/.oh-my-bash/README.md"
echo "${omb_stub}" >> "${user_rc_path}/.oh-my-bash/oh-my-bash.sh"
chmod +x "${user_rc_path}/.oh-my-bash/oh-my-bash.sh"
if [ "${USERNAME}" != "root" ]; then
echo "${omb_readme}" >> "/root/.oh-my-bash/README.md"
echo "${omb_stub}" >> "/root/.oh-my-bash/oh-my-bash.sh"
chmod +x "/root/.oh-my-bash/oh-my-bash.sh"
fi
chown -R "${USERNAME}:${USERNAME}" "${user_rc_path}/.oh-my-bash"
fi
# Optionally install and configure zsh and Oh My Zsh!
if [ "${INSTALL_ZSH}" = "true" ]; then
if ! type zsh > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get install -y zsh
fi
if [ "${ZSH_ALREADY_INSTALLED}" != "true" ]; then
echo "${rc_snippet}" >> /etc/zsh/zshrc
ZSH_ALREADY_INSTALLED="true"
fi
# Adapted, simplified inline Oh My Zsh! install steps that adds, defaults to a codespaces theme.
# See https://github.com/ohmyzsh/ohmyzsh/blob/master/tools/install.sh for official script.
oh_my_install_dir="${user_rc_path}/.oh-my-zsh"
if [ ! -d "${oh_my_install_dir}" ] && [ "${INSTALL_OH_MYS}" = "true" ]; then
template_path="${oh_my_install_dir}/templates/zshrc.zsh-template"
user_rc_file="${user_rc_path}/.zshrc"
umask g-w,o-w
mkdir -p ${oh_my_install_dir}
git clone --depth=1 \
-c core.eol=lf \
-c core.autocrlf=false \
-c fsck.zeroPaddedFilemode=ignore \
-c fetch.fsck.zeroPaddedFilemode=ignore \
-c receive.fsck.zeroPaddedFilemode=ignore \
"https://github.com/ohmyzsh/ohmyzsh" "${oh_my_install_dir}" 2>&1
echo -e "$(cat "${template_path}")\nDISABLE_AUTO_UPDATE=true\nDISABLE_UPDATE_PROMPT=true" > ${user_rc_file}
sed -i -e 's/ZSH_THEME=.*/ZSH_THEME="codespaces"/g' ${user_rc_file}
mkdir -p ${oh_my_install_dir}/custom/themes
echo "${codespaces_zsh}" > "${oh_my_install_dir}/custom/themes/codespaces.zsh-theme"
# Shrink git while still enabling updates
cd "${oh_my_install_dir}"
git repack -a -d -f --depth=1 --window=1
# Copy to non-root user if one is specified
if [ "${USERNAME}" != "root" ]; then
cp -rf "${user_rc_file}" "${oh_my_install_dir}" /root
chown -R ${USERNAME}:${USERNAME} "${user_rc_path}"
fi
fi
fi
# Persist image metadata info, script if meta.env found in same directory
meta_info_script="$(cat << 'EOF'
#!/bin/sh
. /usr/local/etc/vscode-dev-containers/meta.env
# Minimal output
if [ "$1" = "version" ] || [ "$1" = "image-version" ]; then
echo "${VERSION}"
exit 0
elif [ "$1" = "release" ]; then
echo "${GIT_REPOSITORY_RELEASE}"
exit 0
elif [ "$1" = "content" ] || [ "$1" = "content-url" ] || [ "$1" = "contents" ] || [ "$1" = "contents-url" ]; then
echo "${CONTENTS_URL}"
exit 0
fi
#Full output
echo
echo "Development container image information"
echo
if [ ! -z "${VERSION}" ]; then echo "- Image version: ${VERSION}"; fi
if [ ! -z "${DEFINITION_ID}" ]; then echo "- Definition ID: ${DEFINITION_ID}"; fi
if [ ! -z "${VARIANT}" ]; then echo "- Variant: ${VARIANT}"; fi
if [ ! -z "${GIT_REPOSITORY}" ]; then echo "- Source code repository: ${GIT_REPOSITORY}"; fi
if [ ! -z "${GIT_REPOSITORY_RELEASE}" ]; then echo "- Source code release/branch: ${GIT_REPOSITORY_RELEASE}"; fi
if [ ! -z "${BUILD_TIMESTAMP}" ]; then echo "- Timestamp: ${BUILD_TIMESTAMP}"; fi
if [ ! -z "${CONTENTS_URL}" ]; then echo && echo "More info: ${CONTENTS_URL}"; fi
echo
EOF
)"
if [ -f "${SCRIPT_DIR}/meta.env" ]; then
mkdir -p /usr/local/etc/vscode-dev-containers/
cp -f "${SCRIPT_DIR}/meta.env" /usr/local/etc/vscode-dev-containers/meta.env
echo "${meta_info_script}" > /usr/local/bin/devcontainer-info
chmod +x /usr/local/bin/devcontainer-info
fi
# Write marker file
mkdir -p "$(dirname "${MARKER_FILE}")"
echo -e "\
PACKAGES_ALREADY_INSTALLED=${PACKAGES_ALREADY_INSTALLED}\n\
LOCALE_ALREADY_SET=${LOCALE_ALREADY_SET}\n\
EXISTING_NON_ROOT_USER=${EXISTING_NON_ROOT_USER}\n\
RC_SNIPPET_ALREADY_ADDED=${RC_SNIPPET_ALREADY_ADDED}\n\
ZSH_ALREADY_INSTALLED=${ZSH_ALREADY_INSTALLED}" > "${MARKER_FILE}"
echo "Done!"
@@ -0,0 +1,224 @@
#!/usr/bin/env bash
#-------------------------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
#-------------------------------------------------------------------------------------------------------------
#
# Docs: https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/docker.md
# Maintainer: The VS Code and Codespaces Teams
#
# Syntax: ./docker-debian.sh [enable non-root docker socket access flag] [source socket] [target socket] [non-root user] [use moby]
ENABLE_NONROOT_DOCKER=${1:-"true"}
SOURCE_SOCKET=${2:-"/var/run/docker-host.sock"}
TARGET_SOCKET=${3:-"/var/run/docker.sock"}
USERNAME=${4:-"automatic"}
USE_MOBY=${5:-"true"}
MICROSOFT_GPG_KEYS_URI="https://packages.microsoft.com/keys/microsoft.asc"
set -e
if [ "$(id -u)" -ne 0 ]; then
echo -e 'Script must be run as root. Use sudo, su, or add "USER root" to your Dockerfile before running this script.'
exit 1
fi
# Determine the appropriate non-root user
if [ "${USERNAME}" = "auto" ] || [ "${USERNAME}" = "automatic" ]; then
USERNAME=""
POSSIBLE_USERS=("vscode" "node" "codespace" "$(awk -v val=1000 -F ":" '$3==val{print $1}' /etc/passwd)")
for CURRENT_USER in ${POSSIBLE_USERS[@]}; do
if id -u ${CURRENT_USER} > /dev/null 2>&1; then
USERNAME=${CURRENT_USER}
break
fi
done
if [ "${USERNAME}" = "" ]; then
USERNAME=root
fi
elif [ "${USERNAME}" = "none" ] || ! id -u ${USERNAME} > /dev/null 2>&1; then
USERNAME=root
fi
# Get central common setting
get_common_setting() {
if [ "${common_settings_file_loaded}" != "true" ]; then
curl -sfL "https://aka.ms/vscode-dev-containers/script-library/settings.env" 2>/dev/null -o /tmp/vsdc-settings.env || echo "Could not download settings file. Skipping."
common_settings_file_loaded=true
fi
if [ -f "/tmp/vsdc-settings.env" ]; then
local multi_line=""
if [ "$2" = "true" ]; then multi_line="-z"; fi
local result="$(grep ${multi_line} -oP "$1=\"?\K[^\"]+" /tmp/vsdc-settings.env | tr -d '\0')"
if [ ! -z "${result}" ]; then declare -g $1="${result}"; fi
fi
echo "$1=${!1}"
}
# Function to run apt-get if needed
apt_get_update_if_needed()
{
if [ ! -d "/var/lib/apt/lists" ] || [ "$(ls /var/lib/apt/lists/ | wc -l)" = "0" ]; then
echo "Running apt-get update..."
apt-get update
else
echo "Skipping apt-get update."
fi
}
# Checks if packages are installed and installs them if not
check_packages() {
if ! dpkg -s "$@" > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get -y install --no-install-recommends "$@"
fi
}
# Ensure apt is in non-interactive to avoid prompts
export DEBIAN_FRONTEND=noninteractive
# Install dependencies
check_packages apt-transport-https curl ca-certificates gnupg2
# Install Docker / Moby CLI if not already installed
if type docker > /dev/null 2>&1; then
echo "Docker / Moby CLI already installed."
else
# Source /etc/os-release to get OS info
. /etc/os-release
if [ "${USE_MOBY}" = "true" ]; then
# Import key safely (new 'signed-by' method rather than deprecated apt-key approach) and install
get_common_setting MICROSOFT_GPG_KEYS_URI
curl -sSL ${MICROSOFT_GPG_KEYS_URI} | gpg --dearmor > /usr/share/keyrings/microsoft-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/microsoft-archive-keyring.gpg] https://packages.microsoft.com/repos/microsoft-${ID}-${VERSION_CODENAME}-prod ${VERSION_CODENAME} main" > /etc/apt/sources.list.d/microsoft.list
apt-get update
apt-get -y install --no-install-recommends moby-cli moby-buildx moby-compose
else
# Import key safely (new 'signed-by' method rather than deprecated apt-key approach) and install
curl -fsSL https://download.docker.com/linux/${ID}/gpg | gpg --dearmor > /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/${ID} ${VERSION_CODENAME} stable" > /etc/apt/sources.list.d/docker.list
apt-get update
apt-get -y install --no-install-recommends docker-ce-cli
fi
fi
# Install Docker Compose if not already installed and is on a supported architecture
if type docker-compose > /dev/null 2>&1; then
echo "Docker Compose already installed."
else
TARGET_COMPOSE_ARCH="$(uname -m)"
if [ "${TARGET_COMPOSE_ARCH}" = "amd64" ]; then
TARGET_COMPOSE_ARCH="x86_64"
fi
if [ "${TARGET_COMPOSE_ARCH}" != "x86_64" ]; then
# Use pip to get a version that runns on this architecture
if ! dpkg -s python3-minimal python3-pip libffi-dev python3-venv pipx > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get -y install python3-minimal python3-pip libffi-dev python3-venv pipx
fi
export PIPX_HOME=/usr/local/pipx
mkdir -p ${PIPX_HOME}
export PIPX_BIN_DIR=/usr/local/bin
export PIP_CACHE_DIR=/tmp/pip-tmp/cache
pipx install --system-site-packages --pip-args '--no-cache-dir --force-reinstall' docker-compose
rm -rf /tmp/pip-tmp
else
LATEST_COMPOSE_VERSION=$(basename "$(curl -fsSL -o /dev/null -w "%{url_effective}" https://github.com/docker/compose/releases/latest)")
curl -fsSL "https://github.com/docker/compose/releases/download/${LATEST_COMPOSE_VERSION}/docker-compose-$(uname -s)-${TARGET_COMPOSE_ARCH}" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
fi
fi
# If init file already exists, exit
if [ -f "/usr/local/share/docker-init.sh" ]; then
exit 0
fi
# By default, make the source and target sockets the same
if [ "${SOURCE_SOCKET}" != "${TARGET_SOCKET}" ]; then
touch "${SOURCE_SOCKET}"
ln -s "${SOURCE_SOCKET}" "${TARGET_SOCKET}"
fi
# Add a stub if not adding non-root user access, user is root
if [ "${ENABLE_NONROOT_DOCKER}" = "false" ] || [ "${USERNAME}" = "root" ]; then
echo '/usr/bin/env bash -c "\$@"' > /usr/local/share/docker-init.sh
chmod +x /usr/local/share/docker-init.sh
exit 0
fi
# If enabling non-root access and specified user is found, setup socat and add script
chown -h "${USERNAME}":root "${TARGET_SOCKET}"
if ! dpkg -s socat > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get -y install socat
fi
tee /usr/local/share/docker-init.sh > /dev/null \
<< EOF
#!/usr/bin/env bash
#-------------------------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
#-------------------------------------------------------------------------------------------------------------
set -e
SOCAT_PATH_BASE=/tmp/vscr-docker-from-docker
SOCAT_LOG=\${SOCAT_PATH_BASE}.log
SOCAT_PID=\${SOCAT_PATH_BASE}.pid
# Wrapper function to only use sudo if not already root
sudoIf()
{
if [ "\$(id -u)" -ne 0 ]; then
sudo "\$@"
else
"\$@"
fi
}
# Log messages
log()
{
echo -e "[\$(date)] \$@" | sudoIf tee -a \${SOCAT_LOG} > /dev/null
}
echo -e "\n** \$(date) **" | sudoIf tee -a \${SOCAT_LOG} > /dev/null
log "Ensuring ${USERNAME} has access to ${SOURCE_SOCKET} via ${TARGET_SOCKET}"
# If enabled, try to add a docker group with the right GID. If the group is root,
# fall back on using socat to forward the docker socket to another unix socket so
# that we can set permissions on it without affecting the host.
if [ "${ENABLE_NONROOT_DOCKER}" = "true" ] && [ "${SOURCE_SOCKET}" != "${TARGET_SOCKET}" ] && [ "${USERNAME}" != "root" ] && [ "${USERNAME}" != "0" ]; then
SOCKET_GID=\$(stat -c '%g' ${SOURCE_SOCKET})
if [ "\${SOCKET_GID}" != "0" ]; then
log "Adding user to group with GID \${SOCKET_GID}."
if [ "\$(cat /etc/group | grep :\${SOCKET_GID}:)" = "" ]; then
sudoIf groupadd --gid \${SOCKET_GID} docker-host
fi
# Add user to group if not already in it
if [ "\$(id ${USERNAME} | grep -E "groups.*(=|,)\${SOCKET_GID}\(")" = "" ]; then
sudoIf usermod -aG \${SOCKET_GID} ${USERNAME}
fi
else
# Enable proxy if not already running
if [ ! -f "\${SOCAT_PID}" ] || ! ps -p \$(cat \${SOCAT_PID}) > /dev/null; then
log "Enabling socket proxy."
log "Proxying ${SOURCE_SOCKET} to ${TARGET_SOCKET} for vscode"
sudoIf rm -rf ${TARGET_SOCKET}
(sudoIf socat UNIX-LISTEN:${TARGET_SOCKET},fork,mode=660,user=${USERNAME} UNIX-CONNECT:${SOURCE_SOCKET} 2>&1 | sudoIf tee -a \${SOCAT_LOG} > /dev/null & echo "\$!" | sudoIf tee \${SOCAT_PID} > /dev/null)
else
log "Socket proxy already running."
fi
fi
log "Success"
fi
# Execute whatever commands were passed in (if any). This allows us
# to set this script to ENTRYPOINT while still executing the default CMD.
set +e
exec "\$@"
EOF
chmod +x /usr/local/share/docker-init.sh
chown ${USERNAME}:root /usr/local/share/docker-init.sh
echo "Done!"
@@ -0,0 +1,237 @@
#!/usr/bin/env bash
#-------------------------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
#-------------------------------------------------------------------------------------------------------------
#
# Docs: https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/docker-in-docker.md
# Maintainer: The VS Code and Codespaces Teams
#
# Syntax: ./docker-in-docker-debian.sh [enable non-root docker access flag] [non-root user] [use moby]
ENABLE_NONROOT_DOCKER=${1:-"true"}
USERNAME=${2:-"automatic"}
USE_MOBY=${3:-"true"}
MICROSOFT_GPG_KEYS_URI="https://packages.microsoft.com/keys/microsoft.asc"
set -e
if [ "$(id -u)" -ne 0 ]; then
echo -e 'Script must be run as root. Use sudo, su, or add "USER root" to your Dockerfile before running this script.'
exit 1
fi
# Determine the appropriate non-root user
if [ "${USERNAME}" = "auto" ] || [ "${USERNAME}" = "automatic" ]; then
USERNAME=""
POSSIBLE_USERS=("vscode" "node" "codespace" "$(awk -v val=1000 -F ":" '$3==val{print $1}' /etc/passwd)")
for CURRENT_USER in ${POSSIBLE_USERS[@]}; do
if id -u ${CURRENT_USER} > /dev/null 2>&1; then
USERNAME=${CURRENT_USER}
break
fi
done
if [ "${USERNAME}" = "" ]; then
USERNAME=root
fi
elif [ "${USERNAME}" = "none" ] || ! id -u ${USERNAME} > /dev/null 2>&1; then
USERNAME=root
fi
# Get central common setting
get_common_setting() {
if [ "${common_settings_file_loaded}" != "true" ]; then
curl -sfL "https://aka.ms/vscode-dev-containers/script-library/settings.env" 2>/dev/null -o /tmp/vsdc-settings.env || echo "Could not download settings file. Skipping."
common_settings_file_loaded=true
fi
if [ -f "/tmp/vsdc-settings.env" ]; then
local multi_line=""
if [ "$2" = "true" ]; then multi_line="-z"; fi
local result="$(grep ${multi_line} -oP "$1=\"?\K[^\"]+" /tmp/vsdc-settings.env | tr -d '\0')"
if [ ! -z "${result}" ]; then declare -g $1="${result}"; fi
fi
echo "$1=${!1}"
}
# Function to run apt-get if needed
apt_get_update_if_needed()
{
if [ ! -d "/var/lib/apt/lists" ] || [ "$(ls /var/lib/apt/lists/ | wc -l)" = "0" ]; then
echo "Running apt-get update..."
apt-get update
else
echo "Skipping apt-get update."
fi
}
# Checks if packages are installed and installs them if not
check_packages() {
if ! dpkg -s "$@" > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get -y install --no-install-recommends "$@"
fi
}
# Ensure apt is in non-interactive to avoid prompts
export DEBIAN_FRONTEND=noninteractive
# Install dependencies
check_packages apt-transport-https curl ca-certificates lxc pigz iptables gnupg2
# Swap to legacy iptables for compatibility
if type iptables-legacy > /dev/null 2>&1; then
update-alternatives --set iptables /usr/sbin/iptables-legacy
update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy
fi
# Install Docker / Moby CLI if not already installed
if type docker > /dev/null 2>&1 && type dockerd > /dev/null 2>&1; then
echo "Docker / Moby CLI and Engine already installed."
else
# Source /etc/os-release to get OS info
. /etc/os-release
if [ "${USE_MOBY}" = "true" ]; then
# Import key safely (new 'signed-by' method rather than deprecated apt-key approach) and install
get_common_setting MICROSOFT_GPG_KEYS_URI
curl -sSL ${MICROSOFT_GPG_KEYS_URI} | gpg --dearmor > /usr/share/keyrings/microsoft-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/microsoft-archive-keyring.gpg] https://packages.microsoft.com/repos/microsoft-${ID}-${VERSION_CODENAME}-prod ${VERSION_CODENAME} main" > /etc/apt/sources.list.d/microsoft.list
apt-get update
apt-get -y install --no-install-recommends moby-cli moby-buildx moby-compose moby-engine
else
# Import key safely (new 'signed-by' method rather than deprecated apt-key approach) and install
curl -fsSL https://download.docker.com/linux/${ID}/gpg | gpg --dearmor > /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/${ID} ${VERSION_CODENAME} stable" > /etc/apt/sources.list.d/docker.list
apt-get update
apt-get -y install --no-install-recommends docker-ce-cli docker-ce
fi
fi
echo "Finished installing docker / moby"
# Install Docker Compose if not already installed and is on a supported architecture
if type docker-compose > /dev/null 2>&1; then
echo "Docker Compose already installed."
else
TARGET_COMPOSE_ARCH="$(uname -m)"
if [ "${TARGET_COMPOSE_ARCH}" = "amd64" ]; then
TARGET_COMPOSE_ARCH="x86_64"
fi
if [ "${TARGET_COMPOSE_ARCH}" != "x86_64" ]; then
# Use pip to get a version that runns on this architecture
if ! dpkg -s python3-minimal python3-pip libffi-dev python3-venv > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get -y install python3-minimal python3-pip libffi-dev python3-venv
fi
export PIPX_HOME=/usr/local/pipx
mkdir -p ${PIPX_HOME}
export PIPX_BIN_DIR=/usr/local/bin
export PYTHONUSERBASE=/tmp/pip-tmp
export PIP_CACHE_DIR=/tmp/pip-tmp/cache
pipx_bin=pipx
if ! type pipx > /dev/null 2>&1; then
pip3 install --disable-pip-version-check --no-warn-script-location --no-cache-dir --user pipx
pipx_bin=/tmp/pip-tmp/bin/pipx
fi
${pipx_bin} install --system-site-packages --pip-args '--no-cache-dir --force-reinstall' docker-compose
rm -rf /tmp/pip-tmp
else
LATEST_COMPOSE_VERSION=$(basename "$(curl -fsSL -o /dev/null -w "%{url_effective}" https://github.com/docker/compose/releases/latest)")
curl -fsSL "https://github.com/docker/compose/releases/download/${LATEST_COMPOSE_VERSION}/docker-compose-$(uname -s)-${TARGET_COMPOSE_ARCH}" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
fi
fi
# If init file already exists, exit
if [ -f "/usr/local/share/docker-init.sh" ]; then
echo "/usr/local/share/docker-init.sh already exists, so exiting."
exit 0
fi
echo "docker-init doesnt exist..."
# Add user to the docker group
if [ "${ENABLE_NONROOT_DOCKER}" = "true" ]; then
if ! getent group docker > /dev/null 2>&1; then
groupadd docker
fi
usermod -aG docker ${USERNAME}
fi
tee /usr/local/share/docker-init.sh > /dev/null \
<< 'EOF'
#!/usr/bin/env bash
#-------------------------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
#-------------------------------------------------------------------------------------------------------------
sudoIf()
{
if [ "$(id -u)" -ne 0 ]; then
sudo "$@"
else
"$@"
fi
}
# explicitly remove dockerd and containerd PID file to ensure that it can start properly if it was stopped uncleanly
# ie: docker kill <ID>
sudoIf find /run /var/run -iname 'docker*.pid' -delete || :
sudoIf find /run /var/run -iname 'container*.pid' -delete || :
set -e
## Dind wrapper script from docker team
# Maintained: https://github.com/moby/moby/blob/master/hack/dind
export container=docker
if [ -d /sys/kernel/security ] && ! sudoIf mountpoint -q /sys/kernel/security; then
sudoIf mount -t securityfs none /sys/kernel/security || {
echo >&2 'Could not mount /sys/kernel/security.'
echo >&2 'AppArmor detection and --privileged mode might break.'
}
fi
# Mount /tmp (conditionally)
if ! sudoIf mountpoint -q /tmp; then
sudoIf mount -t tmpfs none /tmp
fi
# cgroup v2: enable nesting
if [ -f /sys/fs/cgroup/cgroup.controllers ]; then
# move the init process (PID 1) from the root group to the /init group,
# otherwise writing subtree_control fails with EBUSY.
sudoIf mkdir -p /sys/fs/cgroup/init
sudoIf echo 1 > /sys/fs/cgroup/init/cgroup.procs
# enable controllers
sudoIf sed -e 's/ / +/g' -e 's/^/+/' < /sys/fs/cgroup/cgroup.controllers \
> /sys/fs/cgroup/cgroup.subtree_control
fi
## Dind wrapper over.
# Handle DNS
set +e
cat /etc/resolv.conf | grep -i 'internal.cloudapp.net'
if [ $? -eq 0 ]
then
echo "Setting dockerd Azure DNS."
CUSTOMDNS="--dns 168.63.129.16"
else
echo "Not setting dockerd DNS manually."
CUSTOMDNS=""
fi
set -e
# Start docker/moby engine
( sudoIf dockerd $CUSTOMDNS > /tmp/dockerd.log 2>&1 ) &
set +e
# Execute whatever commands were passed in (if any). This allows us
# to set this script to ENTRYPOINT while still executing the default CMD.
exec "$@"
EOF
chmod +x /usr/local/share/docker-init.sh
chown ${USERNAME}:root /usr/local/share/docker-init.sh
@@ -0,0 +1,140 @@
#!/usr/bin/env bash
#-------------------------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
#-------------------------------------------------------------------------------------------------------------
#
# Docs: https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/git-from-src.md
# Maintainer: The VS Code and Codespaces Teams
#
# Syntax: ./git-from-src-debian.sh [version] [use PPA if available]
GIT_VERSION=${1:-"latest"}
USE_PPA_IF_AVAILABLE=${2:-"false"}
GIT_CORE_PPA_ARCHIVE_GPG_KEY=E1DD270288B4E6030699E45FA1715D88E1DF1F24
GPG_KEY_SERVERS="keyserver hkp://keyserver.ubuntu.com:80
keyserver hkps://keys.openpgp.org
keyserver hkp://keyserver.pgp.com"
set -e
if [ "$(id -u)" -ne 0 ]; then
echo -e 'Script must be run as root. Use sudo, su, or add "USER root" to your Dockerfile before running this script.'
exit 1
fi
# Get central common setting
get_common_setting() {
if [ "${common_settings_file_loaded}" != "true" ]; then
curl -sfL "https://aka.ms/vscode-dev-containers/script-library/settings.env" 2>/dev/null -o /tmp/vsdc-settings.env || echo "Could not download settings file. Skipping."
common_settings_file_loaded=true
fi
if [ -f "/tmp/vsdc-settings.env" ]; then
local multi_line=""
if [ "$2" = "true" ]; then multi_line="-z"; fi
local result="$(grep ${multi_line} -oP "$1=\"?\K[^\"]+" /tmp/vsdc-settings.env | tr -d '\0')"
if [ ! -z "${result}" ]; then declare -g $1="${result}"; fi
fi
echo "$1=${!1}"
}
# Import the specified key in a variable name passed in as
receive_gpg_keys() {
get_common_setting $1
local keys=${!1}
get_common_setting GPG_KEY_SERVERS true
local keyring_args=""
if [ ! -z "$2" ]; then
mkdir -p "$(dirname \"$2\")"
keyring_args="--no-default-keyring --keyring $2"
fi
# Use a temporary locaiton for gpg keys to avoid polluting image
export GNUPGHOME="/tmp/tmp-gnupg"
mkdir -p ${GNUPGHOME}
chmod 700 ${GNUPGHOME}
echo -e "disable-ipv6\n${GPG_KEY_SERVERS}" > ${GNUPGHOME}/dirmngr.conf
# GPG key download sometimes fails for some reason and retrying fixes it.
local retry_count=0
local gpg_ok="false"
set +e
until [ "${gpg_ok}" = "true" ] || [ "${retry_count}" -eq "5" ];
do
echo "(*) Downloading GPG key..."
( echo "${keys}" | xargs -n 1 gpg -q ${keyring_args} --recv-keys) 2>&1 && gpg_ok="true"
if [ "${gpg_ok}" != "true" ]; then
echo "(*) Failed getting key, retring in 10s..."
(( retry_count++ ))
sleep 10s
fi
done
set -e
if [ "${gpg_ok}" = "false" ]; then
echo "(!) Failed to install rvm."
exit 1
fi
}
# Function to run apt-get if needed
apt_get_update_if_needed()
{
if [ ! -d "/var/lib/apt/lists" ] || [ "$(ls /var/lib/apt/lists/ | wc -l)" = "0" ]; then
echo "Running apt-get update..."
apt-get update
else
echo "Skipping apt-get update."
fi
}
# Checks if packages are installed and installs them if not
check_packages() {
if ! dpkg -s "$@" > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get -y install --no-install-recommends "$@"
fi
}
export DEBIAN_FRONTEND=noninteractive
# Source /etc/os-release to get OS info
. /etc/os-release
# If ubuntu, PPAs allowed, and latest - install from there
if ([ "${GIT_VERSION}" = "latest" ] || [ "${GIT_VERSION}" = "lts" ] || [ "${GIT_VERSION}" = "current" ]) && [ "${ID}" = "ubuntu" ] && [ "${USE_PPA_IF_AVAILABLE}" = "true" ]; then
echo "Using PPA to install latest git..."
check_packages apt-transport-https curl ca-certificates gnupg2
receive_gpg_keys GIT_CORE_PPA_ARCHIVE_GPG_KEY /usr/share/keyrings/gitcoreppa-archive-keyring.gpg
echo -e "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/gitcoreppa-archive-keyring.gpg] http://ppa.launchpad.net/git-core/ppa/ubuntu ${VERSION_CODENAME} main\ndeb-src [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/gitcoreppa-archive-keyring.gpg] http://ppa.launchpad.net/git-core/ppa/ubuntu ${VERSION_CODENAME} main" > /etc/apt/sources.list.d/git-core-ppa.list
apt-get update
apt-get -y install --no-install-recommends git
rm -rf "/tmp/tmp-gnupg"
exit 0
fi
# Install required packages to build if missing
check_packages build-essential curl ca-certificates tar gettext libssl-dev zlib1g-dev libcurl?-openssl-dev libexpat1-dev
# Partial version matching
if [ "$(echo "${GIT_VERSION}" | grep -o '\.' | wc -l)" != "2" ]; then
requested_version="${GIT_VERSION}"
version_list="$(curl -sSL -H "Accept: application/vnd.github.v3+json" "https://api.github.com/repos/git/git/tags" | grep -oP '"name":\s*"v\K[0-9]+\.[0-9]+\.[0-9]+"' | tr -d '"' | sort -rV )"
if [ "${requested_version}" = "latest" ] || [ "${requested_version}" = "lts" ] || [ "${requested_version}" = "current" ]; then
GIT_VERSION="$(echo "${version_list}" | head -n 1)"
else
set +e
GIT_VERSION="$(echo "${version_list}" | grep -E -m 1 "^${requested_version//./\\.}([\\.\\s]|$)")"
set -e
fi
if [ -z "${GIT_VERSION}" ] || ! echo "${version_list}" | grep "^${GIT_VERSION//./\\.}$" > /dev/null 2>&1; then
echo "Invalid git version: ${requested_version}" >&2
exit 1
fi
fi
echo "Downloading source for ${GIT_VERSION}..."
curl -sL https://github.com/git/git/archive/v${GIT_VERSION}.tar.gz | tar -xzC /tmp 2>&1
echo "Building..."
cd /tmp/git-${GIT_VERSION}
make -s prefix=/usr/local all && make -s prefix=/usr/local install 2>&1
rm -rf /tmp/git-${GIT_VERSION}
echo "Done!"
@@ -0,0 +1,188 @@
#!/usr/bin/env bash
#-------------------------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
#-------------------------------------------------------------------------------------------------------------
#
# Docs: https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/github.md
# Maintainer: The VS Code and Codespaces Teams
#
# Syntax: ./github-debian.sh [version]
CLI_VERSION=${1:-"latest"}
GITHUB_CLI_ARCHIVE_GPG_KEY=C99B11DEB97541F0
GPG_KEY_SERVERS="keyserver hkp://keyserver.ubuntu.com:80
keyserver hkps://keys.openpgp.org
keyserver hkp://keyserver.pgp.com"
set -e
if [ "$(id -u)" -ne 0 ]; then
echo -e 'Script must be run as root. Use sudo, su, or add "USER root" to your Dockerfile before running this script.'
exit 1
fi
# Get central common setting
get_common_setting() {
if [ "${common_settings_file_loaded}" != "true" ]; then
curl -sfL "https://aka.ms/vscode-dev-containers/script-library/settings.env" 2>/dev/null -o /tmp/vsdc-settings.env || echo "Could not download settings file. Skipping."
common_settings_file_loaded=true
fi
if [ -f "/tmp/vsdc-settings.env" ]; then
local multi_line=""
if [ "$2" = "true" ]; then multi_line="-z"; fi
local result="$(grep ${multi_line} -oP "$1=\"?\K[^\"]+" /tmp/vsdc-settings.env | tr -d '\0')"
if [ ! -z "${result}" ]; then declare -g $1="${result}"; fi
fi
echo "$1=${!1}"
}
# Import the specified key in a variable name passed in as
receive_gpg_keys() {
get_common_setting $1
local keys=${!1}
get_common_setting GPG_KEY_SERVERS true
# Use a temporary locaiton for gpg keys to avoid polluting image
export GNUPGHOME="/tmp/tmp-gnupg"
mkdir -p ${GNUPGHOME}
chmod 700 ${GNUPGHOME}
echo -e "disable-ipv6\n${GPG_KEY_SERVERS}" > ${GNUPGHOME}/dirmngr.conf
# GPG key download sometimes fails for some reason and retrying fixes it.
local retry_count=0
local gpg_ok="false"
set +e
until [ "${gpg_ok}" = "true" ] || [ "${retry_count}" -eq "5" ];
do
echo "(*) Downloading GPG key..."
( echo "${keys}" | xargs -n 1 gpg --recv-keys) 2>&1 && gpg_ok="true"
if [ "${gpg_ok}" != "true" ]; then
echo "(*) Failed getting key, retring in 10s..."
(( retry_count++ ))
sleep 10s
fi
done
set -e
if [ "${gpg_ok}" = "false" ]; then
echo "(!) Failed to install rvm."
exit 1
fi
}
# Figure out correct version of a three part version number is not passed
find_version_from_git_tags() {
local variable_name=$1
local requested_version=${!variable_name}
if [ "${requested_version}" = "none" ]; then return; fi
local repository=$2
local prefix=${3:-"tags/v"}
local separator=${4:-"."}
local last_part_optional=${5:-"false"}
if [ "$(echo "${requested_version}" | grep -o "." | wc -l)" != "2" ]; then
local escaped_separator=${separator//./\\.}
local last_part
if [ "${last_part_optional}" = "true" ]; then
last_part="(${escaped_separator}[0-9]+)?"
else
last_part="${escaped_separator}[0-9]+"
fi
local regex="${prefix}\\K[0-9]+${escaped_separator}[0-9]+${last_part}$"
local version_list="$(git ls-remote --tags ${repository} | grep -oP "${regex}" | tr -d ' ' | tr "${separator}" "." | sort -rV)"
if [ "${requested_version}" = "latest" ] || [ "${requested_version}" = "current" ] || [ "${requested_version}" = "lts" ]; then
declare -g ${variable_name}="$(echo "${version_list}" | head -n 1)"
else
set +e
declare -g ${variable_name}="$(echo "${version_list}" | grep -E -m 1 "^${requested_version//./\\.}([\\.\\s]|$)")"
set -e
fi
fi
if [ -z "${!variable_name}" ] || ! echo "${version_list}" | grep "^${!variable_name//./\\.}$" > /dev/null 2>&1; then
echo -e "Invalid ${variable_name} value: ${requested_version}\nValid values:\n${version_list}" >&2
exit 1
fi
echo "${variable_name}=${!variable_name}"
}
# Import the specified key in a variable name passed in as
receive_gpg_keys() {
get_common_setting $1
local keys=${!1}
get_common_setting GPG_KEY_SERVERS true
local keyring_args=""
if [ ! -z "$2" ]; then
keyring_args="--no-default-keyring --keyring $2"
fi
# Use a temporary locaiton for gpg keys to avoid polluting image
export GNUPGHOME="/tmp/tmp-gnupg"
mkdir -p ${GNUPGHOME}
chmod 700 ${GNUPGHOME}
echo -e "disable-ipv6\n${GPG_KEY_SERVERS}" > ${GNUPGHOME}/dirmngr.conf
# GPG key download sometimes fails for some reason and retrying fixes it.
local retry_count=0
local gpg_ok="false"
set +e
until [ "${gpg_ok}" = "true" ] || [ "${retry_count}" -eq "5" ];
do
echo "(*) Downloading GPG key..."
( echo "${keys}" | xargs -n 1 gpg -q ${keyring_args} --recv-keys) 2>&1 && gpg_ok="true"
if [ "${gpg_ok}" != "true" ]; then
echo "(*) Failed getting key, retring in 10s..."
(( retry_count++ ))
sleep 10s
fi
done
set -e
if [ "${gpg_ok}" = "false" ]; then
echo "(!) Failed to install rvm."
exit 1
fi
}
# Function to run apt-get if needed
apt_get_update_if_needed()
{
if [ ! -d "/var/lib/apt/lists" ] || [ "$(ls /var/lib/apt/lists/ | wc -l)" = "0" ]; then
echo "Running apt-get update..."
apt-get update
else
echo "Skipping apt-get update."
fi
}
# Checks if packages are installed and installs them if not
check_packages() {
if ! dpkg -s "$@" > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get -y install --no-install-recommends "$@"
fi
}
export DEBIAN_FRONTEND=noninteractive
# Install curl, apt-transport-https, curl, gpg, or dirmngr, git if missing
check_packages curl ca-certificates apt-transport-https dirmngr gnupg2
if ! type git > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get -y install --no-install-recommends git
fi
# Soft version matching
if [ "${CLI_VERSION}" != "latest" ] && [ "${CLI_VERSION}" != "lts" ] && [ "${CLI_VERSION}" != "stable" ]; then
find_version_from_git_tags CLI_VERSION "https://github.com/cli/cli"
version_suffix="=${CLI_VERSION}"
else
version_suffix=""
fi
# Install the GitHub CLI
echo "Downloading github CLI..."
# Import key safely (new method rather than deprecated apt-key approach) and install
. /etc/os-release
receive_gpg_keys GITHUB_CLI_ARCHIVE_GPG_KEY /usr/share/keyrings/githubcli-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages ${VERSION_CODENAME} main" > /etc/apt/sources.list.d/github-cli.list
apt-get update
apt-get -y install "gh${version_suffix}"
rm -rf "/tmp/gh/gnupg"
echo "Done!"
+201
View File
@@ -0,0 +1,201 @@
#!/usr/bin/env bash
#-------------------------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
#-------------------------------------------------------------------------------------------------------------
#
# Docs: https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/go.md
# Maintainer: The VS Code and Codespaces Teams
#
# Syntax: ./go-debian.sh [Go version] [GOROOT] [GOPATH] [non-root user] [Add GOPATH, GOROOT to rc files flag] [Install tools flag]
TARGET_GO_VERSION=${1:-"latest"}
TARGET_GOROOT=${2:-"/usr/local/go"}
TARGET_GOPATH=${3:-"/go"}
USERNAME=${4:-"automatic"}
UPDATE_RC=${5:-"true"}
INSTALL_GO_TOOLS=${6:-"true"}
set -e
if [ "$(id -u)" -ne 0 ]; then
echo -e 'Script must be run as root. Use sudo, su, or add "USER root" to your Dockerfile before running this script.'
exit 1
fi
# Ensure that login shells get the correct path if the user updated the PATH using ENV.
rm -f /etc/profile.d/00-restore-env.sh
echo "export PATH=${PATH//$(sh -lc 'echo $PATH')/\$PATH}" > /etc/profile.d/00-restore-env.sh
chmod +x /etc/profile.d/00-restore-env.sh
# Determine the appropriate non-root user
if [ "${USERNAME}" = "auto" ] || [ "${USERNAME}" = "automatic" ]; then
USERNAME=""
POSSIBLE_USERS=("vscode" "node" "codespace" "$(awk -v val=1000 -F ":" '$3==val{print $1}' /etc/passwd)")
for CURRENT_USER in ${POSSIBLE_USERS[@]}; do
if id -u ${CURRENT_USER} > /dev/null 2>&1; then
USERNAME=${CURRENT_USER}
break
fi
done
if [ "${USERNAME}" = "" ]; then
USERNAME=root
fi
elif [ "${USERNAME}" = "none" ] || ! id -u ${USERNAME} > /dev/null 2>&1; then
USERNAME=root
fi
updaterc() {
if [ "${UPDATE_RC}" = "true" ]; then
echo "Updating /etc/bash.bashrc and /etc/zsh/zshrc..."
echo -e "$1" >> /etc/bash.bashrc
if [ -f "/etc/zsh/zshrc" ]; then
echo -e "$1" >> /etc/zsh/zshrc
fi
fi
}
# Figure out correct version of a three part version number is not passed
find_version_from_git_tags() {
local variable_name=$1
local requested_version=${!variable_name}
if [ "${requested_version}" = "none" ]; then return; fi
local repository=$2
local prefix=${3:-"tags/v"}
local separator=${4:-"."}
local last_part_optional=${5:-"false"}
if [ "$(echo "${requested_version}" | grep -o "." | wc -l)" != "2" ]; then
local escaped_separator=${separator//./\\.}
local last_part
if [ "${last_part_optional}" = "true" ]; then
last_part="(${escaped_separator}[0-9]+)?"
else
last_part="${escaped_separator}[0-9]+"
fi
local regex="${prefix}\\K[0-9]+${escaped_separator}[0-9]+${last_part}$"
local version_list="$(git ls-remote --tags ${repository} | grep -oP "${regex}" | tr -d ' ' | tr "${separator}" "." | sort -rV)"
if [ "${requested_version}" = "latest" ] || [ "${requested_version}" = "current" ] || [ "${requested_version}" = "lts" ]; then
declare -g ${variable_name}="$(echo "${version_list}" | head -n 1)"
else
set +e
declare -g ${variable_name}="$(echo "${version_list}" | grep -E -m 1 "^${requested_version//./\\.}([\\.\\s]|$)")"
set -e
fi
fi
if [ -z "${!variable_name}" ] || ! echo "${version_list}" | grep "^${!variable_name//./\\.}$" > /dev/null 2>&1; then
echo -e "Invalid ${variable_name} value: ${requested_version}\nValid values:\n${version_list}" >&2
exit 1
fi
echo "${variable_name}=${!variable_name}"
}
# Function to run apt-get if needed
apt_get_update_if_needed()
{
if [ ! -d "/var/lib/apt/lists" ] || [ "$(ls /var/lib/apt/lists/ | wc -l)" = "0" ]; then
echo "Running apt-get update..."
apt-get update
else
echo "Skipping apt-get update."
fi
}
# Checks if packages are installed and installs them if not
check_packages() {
if ! dpkg -s "$@" > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get -y install --no-install-recommends "$@"
fi
}
export DEBIAN_FRONTEND=noninteractive
# Install curl, tar, git, other dependencies if missing
check_packages curl ca-certificates tar g++ gcc libc6-dev make pkg-config
if ! type git > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get -y install --no-install-recommends git
fi
# Get closest match for version number specified
find_version_from_git_tags TARGET_GO_VERSION "https://go.googlesource.com/go" "tags/go" "." "true"
architecture="$(uname -m)"
case $architecture in
x86_64) architecture="amd64";;
aarch64 | armv8*) architecture="arm64";;
aarch32 | armv7* | armvhf*) architecture="armv6l";;
i?86) architecture="386";;
*) echo "(!) Architecture $architecture unsupported"; exit 1 ;;
esac
# Install Go
GO_INSTALL_SCRIPT="$(cat <<EOF
set -e
echo "Downloading Go ${TARGET_GO_VERSION}..."
curl -sSL -o /tmp/go.tar.gz "https://golang.org/dl/go${TARGET_GO_VERSION}.linux-${architecture}.tar.gz"
echo "Extracting Go ${TARGET_GO_VERSION}..."
tar -xzf /tmp/go.tar.gz -C "${TARGET_GOROOT}" --strip-components=1
rm -f /tmp/go.tar.gz
EOF
)"
if [ "${TARGET_GO_VERSION}" != "none" ] && ! type go > /dev/null 2>&1; then
mkdir -p "${TARGET_GOROOT}" "${TARGET_GOPATH}"
chown -R ${USERNAME} "${TARGET_GOROOT}" "${TARGET_GOPATH}"
su ${USERNAME} -c "${GO_INSTALL_SCRIPT}"
else
echo "Go already installed. Skipping."
fi
# Install Go tools that are isImportant && !replacedByGopls based on
# https://github.com/golang/vscode-go/blob/0ff533d408e4eb8ea54ce84d6efa8b2524d62873/src/goToolsInformation.ts
# Exception `dlv-dap` is a copy of github.com/go-delve/delve/cmd/dlv built from the master.
GO_TOOLS="\
golang.org/x/tools/gopls@latest \
honnef.co/go/tools/cmd/staticcheck@latest \
golang.org/x/lint/golint@latest \
github.com/mgechev/revive@latest \
github.com/uudashr/gopkgs/v2/cmd/gopkgs@latest \
github.com/ramya-rao-a/go-outline@latest \
github.com/go-delve/delve/cmd/dlv@latest \
github.com/golangci/golangci-lint/cmd/golangci-lint@latest"
if [ "${INSTALL_GO_TOOLS}" = "true" ]; then
echo "Installing common Go tools..."
export PATH=${TARGET_GOROOT}/bin:${PATH}
mkdir -p /tmp/gotools /usr/local/etc/vscode-dev-containers ${TARGET_GOPATH}/bin
cd /tmp/gotools
export GOPATH=/tmp/gotools
export GOCACHE=/tmp/gotools/cache
# Use go get for versions of go under 1.17
go_install_command=install
if [[ "1.16" > "$(go version | grep -oP 'go\K[0-9]+\.[0-9]+(\.[0-9]+)?')" ]]; then
export GO111MODULE=on
go_install_command=get
echo "Go version < 1.17, using go get."
fi
(echo "${GO_TOOLS}" | xargs -n 1 go ${go_install_command} -v )2>&1 | tee -a /usr/local/etc/vscode-dev-containers/go.log
# Move Go tools into path and clean up
mv /tmp/gotools/bin/* ${TARGET_GOPATH}/bin/
# install dlv-dap (dlv@master)
go ${go_install_command} -v github.com/go-delve/delve/cmd/dlv@master 2>&1 | tee -a /usr/local/etc/vscode-dev-containers/go.log
mv /tmp/gotools/bin/dlv ${TARGET_GOPATH}/bin/dlv-dap
rm -rf /tmp/gotools
chown -R ${USERNAME} "${TARGET_GOPATH}"
fi
# Add GOPATH variable and bin directory into PATH in bashrc/zshrc files (unless disabled)
updaterc "$(cat << EOF
export GOPATH="${TARGET_GOPATH}"
if [[ "\${PATH}" != *"\${GOPATH}/bin"* ]]; then export PATH="\${PATH}:\${GOPATH}/bin"; fi
export GOROOT="${TARGET_GOROOT}"
if [[ "\${PATH}" != *"\${GOROOT}/bin"* ]]; then export PATH="\${PATH}:\${GOROOT}/bin"; fi
EOF
)"
echo "Done!"
+105
View File
@@ -0,0 +1,105 @@
#!/usr/bin/env bash
#
# This is a replicated script.
#
# Syntax: ./k3s-debian.sh [k3s version] [k3s SHA256]
set -e
K3S_VERSION="${1:-"latest"}" # latest is also valid
K3S_SHA256="${2:-"automatic"}"
GPG_KEY_SERVERS="keyserver hkp://keyserver.ubuntu.com:80
keyserver hkps://keys.openpgp.org
keyserver hkp://keyserver.pgp.com"
architecture="$(uname -m)"
case $architecture in
x86_64) architecture="amd64";;
aarch64 | armv8*) architecture="arm64";;
aarch32 | armv7* | armvhf*) architecture="armhf";;
*) echo "(!) Architecture $architecture unsupported"; exit 1 ;;
esac
# Figure out correct version of a three part version number is not passed
find_version_from_git_tags() {
local variable_name=$1
local requested_version=${!variable_name}
if [ "${requested_version}" = "none" ]; then return; fi
local repository=$2
local prefix=${3:-"tags/v"}
local separator=${4:-"."}
local last_part_optional=${5:-"false"}
if [ "$(echo "${requested_version}" | grep -o "." | wc -l)" != "2" ]; then
local escaped_separator=${separator//./\\.}
local last_part
if [ "${last_part_optional}" = "true" ]; then
last_part="(${escaped_separator}[0-9ks\+]+)?"
else
last_part="${escaped_separator}[0-9ks\+]+"
fi
local regex="${prefix}\\K[0-9]+${escaped_separator}[0-9]+${last_part}$"
local version_list="$(git ls-remote --tags ${repository} | grep -oP "${regex}" | tr -d ' ' | tr "${separator}" "." | sort -rV)"
echo $version_list
if [ "${requested_version}" = "latest" ] || [ "${requested_version}" = "current" ] || [ "${requested_version}" = "lts" ]; then
declare -g ${variable_name}="$(echo "${version_list}" | head -n 1)"
else
set +e
declare -g ${variable_name}="$(echo "${version_list}" | grep -E -m 1 "^${requested_version//./\\.}([\\.\\s+]|$)")"
set -e
fi
fi
if [ -z "${!variable_name}" ] || ! echo "${version_list}" | grep "^${!variable_name//./\\.}$" > /dev/null 2>&1; then
echo -e "Invalid ${variable_name} value: ${requested_version}\nValid values:\n${version_list}" >&2
exit 1
fi
echo "${variable_name}=${!variable_name}"
}
# Install K3s, verify checksum
if [ "${K3S_VERSION}" != "none" ]; then
echo "Downloading k3s..."
urlPrefix=
if [ "${K3S_VERSION}" = "latest" ] || [ "${K3S_VERSION}" = "lts" ] || [ "${K3S_VERSION}" = "current" ] || [ "${K3S_VERSION}" = "stable" ]; then
K3S_VERSION="latest"
urlPrefix="https://github.com/k3s-io/k3s/releases/latest/download"
else
find_version_from_git_tags K3S_VERSION https://github.com/k3s-io/k3s
if [ "${K3S_VERSION::1}" != "v" ]; then
K3S_VERSION="v${K3S_VERSION}"
fi
urlPrefix="https://github.com/k3s-io/k3s/releases/download/${K3S_VERSION}"
fi
# URL encode plus sign
K3S_VERSION="$(echo $K3S_VERSION | sed --expression='s/+/%2B/g')"
# latest is also valid in the download URLs
downloadUrl="${urlPrefix}/k3s${architecture}"
if [ "${architecture}" = "amd64" ]; then
downloadUrl="${urlPrefix}/k3s"
fi
curl -sSL -o /usr/local/bin/k3s "${downloadUrl}"
chmod 0755 /usr/local/bin/k3s
if [ "$K3S_SHA256" = "automatic" ]; then
shaUrl="${urlPrefix}/sha256sum-${architecture}.txt"
if [ "${architecture}" = "armhf" ]; then
shaUrl="${urlPrefix}/sha256sum-arm.txt"
fi
# Manifest contains image hashes, but we only need the binary
K3S_SHA256="$(curl -sSL $shaUrl | grep -P '(^|\s)\Kk3s(?=\s|$)' | cut -d ' ' -f1 )"
fi
echo $K3S_SHA256
([ "${K3S_SHA256}" = "dev-mode" ] || (echo "${K3S_SHA256} */usr/local/bin/k3s" | sha256sum -c -))
if ! type k3s > /dev/null 2>&1; then
echo '(!) k3s installation failed!'
exit 1
fi
fi
echo -e "\nDone!"
@@ -0,0 +1,218 @@
#!/usr/bin/env bash
#-------------------------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
#-------------------------------------------------------------------------------------------------------------
#
# Docs: https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/kubectl-helm.md
# Maintainer: The VS Code and Codespaces Teams
#
# Syntax: ./kubectl-helm-debian.sh [kubectl verison] [Helm version] [minikube version] [kubectl SHA256] [Helm SHA256] [minikube SHA256]
set -e
KUBECTL_VERSION="${1:-"latest"}"
HELM_VERSION="${2:-"latest"}"
MINIKUBE_VERSION="${3:-"none"}" # latest is also valid
KUBECTL_SHA256="${4:-"automatic"}"
HELM_SHA256="${5:-"automatic"}"
MINIKUBE_SHA256="${6:-"automatic"}"
HELM_GPG_KEYS_URI="https://raw.githubusercontent.com/helm/helm/main/KEYS"
GPG_KEY_SERVERS="keyserver hkp://keyserver.ubuntu.com:80
keyserver hkps://keys.openpgp.org
keyserver hkp://keyserver.pgp.com"
if [ "$(id -u)" -ne 0 ]; then
echo -e 'Script must be run as root. Use sudo, su, or add "USER root" to your Dockerfile before running this script.'
exit 1
fi
# Get central common setting
get_common_setting() {
if [ "${common_settings_file_loaded}" != "true" ]; then
curl -sfL "https://aka.ms/vscode-dev-containers/script-library/settings.env" 2>/dev/null -o /tmp/vsdc-settings.env || echo "Could not download settings file. Skipping."
common_settings_file_loaded=true
fi
if [ -f "/tmp/vsdc-settings.env" ]; then
local multi_line=""
if [ "$2" = "true" ]; then multi_line="-z"; fi
local result="$(grep ${multi_line} -oP "$1=\"?\K[^\"]+" /tmp/vsdc-settings.env | tr -d '\0')"
if [ ! -z "${result}" ]; then declare -g $1="${result}"; fi
fi
echo "$1=${!1}"
}
# Figure out correct version of a three part version number is not passed
find_version_from_git_tags() {
local variable_name=$1
local requested_version=${!variable_name}
if [ "${requested_version}" = "none" ]; then return; fi
local repository=$2
local prefix=${3:-"tags/v"}
local separator=${4:-"."}
local last_part_optional=${5:-"false"}
if [ "$(echo "${requested_version}" | grep -o "." | wc -l)" != "2" ]; then
local escaped_separator=${separator//./\\.}
local last_part
if [ "${last_part_optional}" = "true" ]; then
last_part="(${escaped_separator}[0-9]+)?"
else
last_part="${escaped_separator}[0-9]+"
fi
local regex="${prefix}\\K[0-9]+${escaped_separator}[0-9]+${last_part}$"
local version_list="$(git ls-remote --tags ${repository} | grep -oP "${regex}" | tr -d ' ' | tr "${separator}" "." | sort -rV)"
if [ "${requested_version}" = "latest" ] || [ "${requested_version}" = "current" ] || [ "${requested_version}" = "lts" ]; then
declare -g ${variable_name}="$(echo "${version_list}" | head -n 1)"
else
set +e
declare -g ${variable_name}="$(echo "${version_list}" | grep -E -m 1 "^${requested_version//./\\.}([\\.\\s]|$)")"
set -e
fi
fi
if [ -z "${!variable_name}" ] || ! echo "${version_list}" | grep "^${!variable_name//./\\.}$" > /dev/null 2>&1; then
echo -e "Invalid ${variable_name} value: ${requested_version}\nValid values:\n${version_list}" >&2
exit 1
fi
echo "${variable_name}=${!variable_name}"
}
# Function to run apt-get if needed
apt_get_update_if_needed()
{
if [ ! -d "/var/lib/apt/lists" ] || [ "$(ls /var/lib/apt/lists/ | wc -l)" = "0" ]; then
echo "Running apt-get update..."
apt-get update
else
echo "Skipping apt-get update."
fi
}
# Checks if packages are installed and installs them if not
check_packages() {
if ! dpkg -s "$@" > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get -y install --no-install-recommends "$@"
fi
}
# Ensure apt is in non-interactive to avoid prompts
export DEBIAN_FRONTEND=noninteractive
# Install dependencies
check_packages curl ca-certificates coreutils gnupg2 dirmngr bash-completion
if ! type git > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get -y install --no-install-recommends git
fi
architecture="$(uname -m)"
case $architecture in
x86_64) architecture="amd64";;
aarch64 | armv8*) architecture="arm64";;
aarch32 | armv7* | armvhf*) architecture="arm";;
i?86) architecture="386";;
*) echo "(!) Architecture $architecture unsupported"; exit 1 ;;
esac
# Install the kubectl, verify checksum
echo "Downloading kubectl..."
if [ "${KUBECTL_VERSION}" = "latest" ] || [ "${KUBECTL_VERSION}" = "lts" ] || [ "${KUBECTL_VERSION}" = "current" ] || [ "${KUBECTL_VERSION}" = "stable" ]; then
KUBECTL_VERSION="$(curl -sSL https://dl.k8s.io/release/stable.txt)"
else
find_version_from_git_tags KUBECTL_VERSION https://github.com/kubernetes/kubernetes
fi
if [ "${KUBECTL_VERSION::1}" != 'v' ]; then
KUBECTL_VERSION="v${KUBECTL_VERSION}"
fi
curl -sSL -o /usr/local/bin/kubectl "https://dl.k8s.io/release/${KUBECTL_VERSION}/bin/linux/${architecture}/kubectl"
chmod 0755 /usr/local/bin/kubectl
if [ "$KUBECTL_SHA256" = "automatic" ]; then
KUBECTL_SHA256="$(curl -sSL "https://dl.k8s.io/${KUBECTL_VERSION}/bin/linux/${architecture}/kubectl.sha256")"
fi
([ "${KUBECTL_SHA256}" = "dev-mode" ] || (echo "${KUBECTL_SHA256} */usr/local/bin/kubectl" | sha256sum -c -))
if ! type kubectl > /dev/null 2>&1; then
echo '(!) kubectl installation failed!'
exit 1
fi
# kubectl bash completion
kubectl completion bash > /etc/bash_completion.d/kubectl
# kubectl zsh completion
mkdir -p /home/${USERNAME}/.oh-my-zsh/completions
kubectl completion zsh > /home/${USERNAME}/.oh-my-zsh/completions/_kubectl
# Install Helm, verify signature and checksum
echo "Downloading Helm..."
find_version_from_git_tags HELM_VERSION "https://github.com/helm/helm"
if [ "${HELM_VERSION::1}" != 'v' ]; then
HELM_VERSION="v${HELM_VERSION}"
fi
mkdir -p /tmp/helm
helm_filename="helm-${HELM_VERSION}-linux-${architecture}.tar.gz"
tmp_helm_filename="/tmp/helm/${helm_filename}"
curl -sSL "https://get.helm.sh/${helm_filename}" -o "${tmp_helm_filename}"
curl -sSL "https://github.com/helm/helm/releases/download/${HELM_VERSION}/${helm_filename}.asc" -o "${tmp_helm_filename}.asc"
export GNUPGHOME="/tmp/helm/gnupg"
mkdir -p "${GNUPGHOME}"
chmod 700 ${GNUPGHOME}
get_common_setting HELM_GPG_KEYS_URI
get_common_setting GPG_KEY_SERVERS true
curl -sSL "${HELM_GPG_KEYS_URI}" -o /tmp/helm/KEYS
echo -e "disable-ipv6\n${GPG_KEY_SERVERS}" > ${GNUPGHOME}/dirmngr.conf
gpg -q --import "/tmp/helm/KEYS"
if ! gpg --verify "${tmp_helm_filename}.asc" > ${GNUPGHOME}/verify.log 2>&1; then
echo "Verification failed!"
cat /tmp/helm/gnupg/verify.log
exit 1
fi
if [ "${HELM_SHA256}" = "automatic" ]; then
curl -sSL "https://get.helm.sh/${helm_filename}.sha256" -o "${tmp_helm_filename}.sha256"
curl -sSL "https://github.com/helm/helm/releases/download/${HELM_VERSION}/${helm_filename}.sha256.asc" -o "${tmp_helm_filename}.sha256.asc"
if ! gpg --verify "${tmp_helm_filename}.sha256.asc" > /tmp/helm/gnupg/verify.log 2>&1; then
echo "Verification failed!"
cat /tmp/helm/gnupg/verify.log
exit 1
fi
HELM_SHA256="$(cat "${tmp_helm_filename}.sha256")"
fi
([ "${HELM_SHA256}" = "dev-mode" ] || (echo "${HELM_SHA256} *${tmp_helm_filename}" | sha256sum -c -))
tar xf "${tmp_helm_filename}" -C /tmp/helm
mv -f "/tmp/helm/linux-${architecture}/helm" /usr/local/bin/
chmod 0755 /usr/local/bin/helm
rm -rf /tmp/helm
if ! type helm > /dev/null 2>&1; then
echo '(!) Helm installation failed!'
exit 1
fi
# Install Minikube, verify checksum
if [ "${MINIKUBE_VERSION}" != "none" ]; then
echo "Downloading minikube..."
if [ "${MINIKUBE_VERSION}" = "latest" ] || [ "${MINIKUBE_VERSION}" = "lts" ] || [ "${MINIKUBE_VERSION}" = "current" ] || [ "${MINIKUBE_VERSION}" = "stable" ]; then
MINIKUBE_VERSION="latest"
else
find_version_from_git_tags MINIKUBE_VERSION https://github.com/kubernetes/minikube
if [ "${MINIKUBE_VERSION::1}" != "v" ]; then
MINIKUBE_VERSION="v${MINIKUBE_VERSION}"
fi
fi
# latest is also valid in the download URLs
curl -sSL -o /usr/local/bin/minikube "https://storage.googleapis.com/minikube/releases/${MINIKUBE_VERSION}/minikube-linux-${architecture}"
chmod 0755 /usr/local/bin/minikube
if [ "$MINIKUBE_SHA256" = "automatic" ]; then
MINIKUBE_SHA256="$(curl -sSL "https://storage.googleapis.com/minikube/releases/${MINIKUBE_VERSION}/minikube-linux-${architecture}.sha256")"
fi
([ "${MINIKUBE_SHA256}" = "dev-mode" ] || (echo "${MINIKUBE_SHA256} */usr/local/bin/minikube" | sha256sum -c -))
if ! type minikube > /dev/null 2>&1; then
echo '(!) minikube installation failed!'
exit 1
fi
fi
if ! type docker > /dev/null 2>&1; then
echo -e '\n(*) Warning: The docker command was not found.\n\nYou can use one of the following scripts to install it:\n\nhttps://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/docker-in-docker.md\n\nor\n\nhttps://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/docker.md'
fi
echo -e "\nDone!"
+1
View File
@@ -0,0 +1 @@
VERSION='dev'
@@ -0,0 +1,141 @@
#!/bin/bash
#-------------------------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
#-------------------------------------------------------------------------------------------------------------
#
# Docs: https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/node.md
# Maintainer: The VS Code and Codespaces Teams
#
# Syntax: ./node-debian.sh [directory to install nvm] [node version to install (use "none" to skip)] [non-root user] [Update rc files flag]
export NVM_DIR=${1:-"/usr/local/share/nvm"}
export NODE_VERSION=${2:-"lts"}
USERNAME=${3:-"automatic"}
UPDATE_RC=${4:-"true"}
export NVM_VERSION="0.38.0"
set -e
if [ "$(id -u)" -ne 0 ]; then
echo -e 'Script must be run as root. Use sudo, su, or add "USER root" to your Dockerfile before running this script.'
exit 1
fi
# Ensure that login shells get the correct path if the user updated the PATH using ENV.
rm -f /etc/profile.d/00-restore-env.sh
echo "export PATH=${PATH//$(sh -lc 'echo $PATH')/\$PATH}" > /etc/profile.d/00-restore-env.sh
chmod +x /etc/profile.d/00-restore-env.sh
# Determine the appropriate non-root user
if [ "${USERNAME}" = "auto" ] || [ "${USERNAME}" = "automatic" ]; then
USERNAME=""
POSSIBLE_USERS=("vscode" "node" "codespace" "$(awk -v val=1000 -F ":" '$3==val{print $1}' /etc/passwd)")
for CURRENT_USER in ${POSSIBLE_USERS[@]}; do
if id -u ${CURRENT_USER} > /dev/null 2>&1; then
USERNAME=${CURRENT_USER}
break
fi
done
if [ "${USERNAME}" = "" ]; then
USERNAME=root
fi
elif [ "${USERNAME}" = "none" ] || ! id -u ${USERNAME} > /dev/null 2>&1; then
USERNAME=root
fi
updaterc() {
if [ "${UPDATE_RC}" = "true" ]; then
echo "Updating /etc/bash.bashrc and /etc/zsh/zshrc..."
echo -e "$1" >> /etc/bash.bashrc
if [ -f "/etc/zsh/zshrc" ]; then
echo -e "$1" >> /etc/zsh/zshrc
fi
fi
}
# Function to run apt-get if needed
apt_get_update_if_needed()
{
if [ ! -d "/var/lib/apt/lists" ] || [ "$(ls /var/lib/apt/lists/ | wc -l)" = "0" ]; then
echo "Running apt-get update..."
apt-get update
else
echo "Skipping apt-get update."
fi
}
# Checks if packages are installed and installs them if not
check_packages() {
if ! dpkg -s "$@" > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get -y install --no-install-recommends "$@"
fi
}
# Ensure apt is in non-interactive to avoid prompts
export DEBIAN_FRONTEND=noninteractive
# Install dependencies
check_packages apt-transport-https curl ca-certificates tar gnupg2
# Install yarn
if type yarn > /dev/null 2>&1; then
echo "Yarn already installed."
else
# Import key safely (new method rather than deprecated apt-key approach) and install
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | gpg --dearmor > /usr/share/keyrings/yarn-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/yarn-archive-keyring.gpg] https://dl.yarnpkg.com/debian/ stable main" > /etc/apt/sources.list.d/yarn.list
apt-get update
apt-get -y install --no-install-recommends yarn
fi
# Adjust node version if required
if [ "${NODE_VERSION}" = "none" ]; then
export NODE_VERSION=
elif [ "${NODE_VERSION}" = "lts" ]; then
export NODE_VERSION="lts/*"
fi
# Install the specified node version if NVM directory already exists, then exit
if [ -d "${NVM_DIR}" ]; then
echo "NVM already installed."
if [ "${NODE_VERSION}" != "" ]; then
su ${USERNAME} -c ". $NVM_DIR/nvm.sh && nvm install ${NODE_VERSION} && nvm clear-cache"
fi
exit 0
fi
# Create nvm group, nvm dir, and set sticky bit
if ! cat /etc/group | grep -e "^nvm:" > /dev/null 2>&1; then
groupadd -r nvm
fi
umask 0002
usermod -a -G nvm ${USERNAME}
mkdir -p ${NVM_DIR}
chown :nvm ${NVM_DIR}
chmod g+s ${NVM_DIR}
su ${USERNAME} -c "$(cat << EOF
set -e
umask 0002
# Do not update profile - we'll do this manually
export PROFILE=/dev/null
curl -so- https://raw.githubusercontent.com/nvm-sh/nvm/v${NVM_VERSION}/install.sh | bash
source ${NVM_DIR}/nvm.sh
if [ "${NODE_VERSION}" != "" ]; then
nvm alias default ${NODE_VERSION}
fi
nvm clear-cache
EOF
)" 2>&1
# Update rc files
if [ "${UPDATE_RC}" = "true" ]; then
updaterc "$(cat <<EOF
export NVM_DIR="${NVM_DIR}"
[ -s "\$NVM_DIR/nvm.sh" ] && . "\$NVM_DIR/nvm.sh"
[ -s "\$NVM_DIR/bash_completion" ] && . "\$NVM_DIR/bash_completion"
EOF
)"
fi
echo "Done!"
@@ -0,0 +1,307 @@
#!/usr/bin/env bash
#-------------------------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
#-------------------------------------------------------------------------------------------------------------
#
# Docs: https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/python.md
# Maintainer: The VS Code and Codespaces Teams
#
# Syntax: ./python-debian.sh [Python Version] [Python intall path] [PIPX_HOME] [non-root user] [Update rc files flag] [install tools]
PYTHON_VERSION=${1:-"latest"}
PYTHON_INSTALL_PATH=${2:-"/usr/local/python"}
export PIPX_HOME=${3:-"/usr/local/py-utils"}
USERNAME=${4:-"automatic"}
UPDATE_RC=${5:-"true"}
INSTALL_PYTHON_TOOLS=${6:-"true"}
USE_PPA_IF_AVAILABLE=${7:-"true"}
DEADSNAKES_PPA_ARCHIVE_GPG_KEY="F23C5A6CF475977595C89F51BA6932366A755776"
PYTHON_SOURCE_GPG_KEYS="64E628F8D684696D B26995E310250568 2D347EA6AA65421D FB9921286F5E1540 3A5CA953F73C700D 04C367C218ADD4FF 0EDDC5F26A45C816 6AF053F07D9DC8D2 C9BE28DEE6DF025C 126EB563A74B06BF D9866941EA5BBD71 ED9D77D5"
GPG_KEY_SERVERS="keyserver hkp://keyserver.ubuntu.com:80
keyserver hkps://keys.openpgp.org
keyserver hkp://keyserver.pgp.com"
set -e
if [ "$(id -u)" -ne 0 ]; then
echo -e 'Script must be run as root. Use sudo, su, or add "USER root" to your Dockerfile before running this script.'
exit 1
fi
# Ensure that login shells get the correct path if the user updated the PATH using ENV.
rm -f /etc/profile.d/00-restore-env.sh
echo "export PATH=${PATH//$(sh -lc 'echo $PATH')/\$PATH}" > /etc/profile.d/00-restore-env.sh
chmod +x /etc/profile.d/00-restore-env.sh
# Determine the appropriate non-root user
if [ "${USERNAME}" = "auto" ] || [ "${USERNAME}" = "automatic" ]; then
USERNAME=""
POSSIBLE_USERS=("vscode" "node" "codespace" "$(awk -v val=1000 -F ":" '$3==val{print $1}' /etc/passwd)")
for CURRENT_USER in ${POSSIBLE_USERS[@]}; do
if id -u ${CURRENT_USER} > /dev/null 2>&1; then
USERNAME=${CURRENT_USER}
break
fi
done
if [ "${USERNAME}" = "" ]; then
USERNAME=root
fi
elif [ "${USERNAME}" = "none" ] || ! id -u ${USERNAME} > /dev/null 2>&1; then
USERNAME=root
fi
updaterc() {
if [ "${UPDATE_RC}" = "true" ]; then
echo "Updating /etc/bash.bashrc and /etc/zsh/zshrc..."
echo -e "$1" >> /etc/bash.bashrc
if [ -f "/etc/zsh/zshrc" ]; then
echo -e "$1" >> /etc/zsh/zshrc
fi
fi
}
# Get central common setting
get_common_setting() {
if [ "${common_settings_file_loaded}" != "true" ]; then
curl -sfL "https://aka.ms/vscode-dev-containers/script-library/settings.env" 2>/dev/null -o /tmp/vsdc-settings.env || echo "Could not download settings file. Skipping."
common_settings_file_loaded=true
fi
if [ -f "/tmp/vsdc-settings.env" ]; then
local multi_line=""
if [ "$2" = "true" ]; then multi_line="-z"; fi
local result="$(grep ${multi_line} -oP "$1=\"?\K[^\"]+" /tmp/vsdc-settings.env | tr -d '\0')"
if [ ! -z "${result}" ]; then declare -g $1="${result}"; fi
fi
echo "$1=${!1}"
}
# Import the specified key in a variable name passed in as
receive_gpg_keys() {
get_common_setting $1
local keys=${!1}
get_common_setting GPG_KEY_SERVERS true
local keyring_args=""
if [ ! -z "$2" ]; then
mkdir -p "$(dirname \"$2\")"
keyring_args="--no-default-keyring --keyring $2"
fi
# Use a temporary locaiton for gpg keys to avoid polluting image
export GNUPGHOME="/tmp/tmp-gnupg"
mkdir -p ${GNUPGHOME}
chmod 700 ${GNUPGHOME}
echo -e "disable-ipv6\n${GPG_KEY_SERVERS}" > ${GNUPGHOME}/dirmngr.conf
# GPG key download sometimes fails for some reason and retrying fixes it.
local retry_count=0
local gpg_ok="false"
set +e
until [ "${gpg_ok}" = "true" ] || [ "${retry_count}" -eq "5" ];
do
echo "(*) Downloading GPG key..."
( echo "${keys}" | xargs -n 1 gpg -q ${keyring_args} --recv-keys) 2>&1 && gpg_ok="true"
if [ "${gpg_ok}" != "true" ]; then
echo "(*) Failed getting key, retring in 10s..."
(( retry_count++ ))
sleep 10s
fi
done
set -e
if [ "${gpg_ok}" = "false" ]; then
echo "(!) Failed to install rvm."
exit 1
fi
}
# Figure out correct version of a three part version number is not passed
find_version_from_git_tags() {
local variable_name=$1
local requested_version=${!variable_name}
if [ "${requested_version}" = "none" ]; then return; fi
local repository=$2
local prefix=${3:-"tags/v"}
local separator=${4:-"."}
local last_part_optional=${5:-"false"}
if [ "$(echo "${requested_version}" | grep -o "." | wc -l)" != "2" ]; then
local escaped_separator=${separator//./\\.}
local last_part
if [ "${last_part_optional}" = "true" ]; then
last_part="(${escaped_separator}[0-9]+)?"
else
last_part="${escaped_separator}[0-9]+"
fi
local regex="${prefix}\\K[0-9]+${escaped_separator}[0-9]+${last_part}$"
local version_list="$(git ls-remote --tags ${repository} | grep -oP "${regex}" | tr -d ' ' | tr "${separator}" "." | sort -rV)"
if [ "${requested_version}" = "latest" ] || [ "${requested_version}" = "current" ] || [ "${requested_version}" = "lts" ]; then
declare -g ${variable_name}="$(echo "${version_list}" | head -n 1)"
else
set +e
declare -g ${variable_name}="$(echo "${version_list}" | grep -E -m 1 "^${requested_version//./\\.}([\\.\\s]|$)")"
set -e
fi
fi
if [ -z "${!variable_name}" ] || ! echo "${version_list}" | grep "^${!variable_name//./\\.}$" > /dev/null 2>&1; then
echo -e "Invalid ${variable_name} value: ${requested_version}\nValid values:\n${version_list}" >&2
exit 1
fi
echo "${variable_name}=${!variable_name}"
}
# Function to run apt-get if needed
apt_get_update_if_needed()
{
if [ ! -d "/var/lib/apt/lists" ] || [ "$(ls /var/lib/apt/lists/ | wc -l)" = "0" ]; then
echo "Running apt-get update..."
apt-get update
else
echo "Skipping apt-get update."
fi
}
# Checks if packages are installed and installs them if not
check_packages() {
if ! dpkg -s "$@" > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get -y install --no-install-recommends "$@"
fi
}
install_from_ppa() {
local requested_version="python${PYTHON_VERSION}"
echo "Using PPA to install Python..."
check_packages apt-transport-https curl ca-certificates gnupg2
receive_gpg_keys DEADSNAKES_PPA_ARCHIVE_GPG_KEY /usr/share/keyrings/deadsnakes-archive-keyring.gpg
echo -e "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/deadsnakes-archive-keyring.gpg] http://ppa.launchpad.net/deadsnakes/ppa/ubuntu ${VERSION_CODENAME} main\ndeb-src [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/deadsnakes-archive-keyring.gpg] http://ppa.launchpad.net/deadsnakes/ppa/ubuntu ${VERSION_CODENAME} main" > /etc/apt/sources.list.d/deadsnakes-ppa.list
apt-get update
if [ "${PYTHON_VERSION}" = "latest" ] || [ "${PYTHON_VERSION}" = "current" ] || [ "${PYTHON_VERSION}" = "lts" ]; then
requested_version="$(apt-cache search '^python3\.[0-9]$' | grep -oE '^python3\.[0-9]' | sort -rV | head -n 1)"
echo "Using ${requested_version} in place of ${PYTHON_VERSION}."
fi
apt-get -y install ${requested_version}
rm -rf /tmp/tmp-gnupg
exit 0
}
install_from_source() {
if [ -d "${PYTHON_INSTALL_PATH}" ]; then
echo "Path ${PYTHON_INSTALL_PATH} already exists. Remove this existing path or select a different one."
exit 1
else
echo "Building Python ${PYTHON_VERSION} from source..."
# Install prereqs if missing
check_packages curl ca-certificates tar make build-essential libssl-dev zlib1g-dev \
wget libbz2-dev libreadline-dev libxml2-dev xz-utils tk-dev gnupg2 \
libxmlsec1-dev libsqlite3-dev libffi-dev liblzma-dev llvm dirmngr
if ! type git > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get -y install --no-install-recommends git
fi
# Find version using soft match
find_version_from_git_tags PYTHON_VERSION "https://github.com/python/cpython"
# Download tgz of source
mkdir -p /tmp/python-src "${PYTHON_INSTALL_PATH}"
cd /tmp/python-src
TGZ_FILENAME="Python-${PYTHON_VERSION}.tgz"
TGZ_URL="https://www.python.org/ftp/python/${PYTHON_VERSION}/${TGZ_FILENAME}"
echo "Downloading ${TGZ_FILENAME}..."
curl -sSL -o "/tmp/python-src/${TGZ_FILENAME}" "${TGZ_URL}"
# Verify signature
if [ "${SKIP_SIGNATURE_CHECK}" != "true" ]; then
receive_gpg_keys PYTHON_SOURCE_GPG_KEYS
echo "Downloading ${TGZ_FILENAME}.asc..."
curl -sSL -o "/tmp/python-src/${TGZ_FILENAME}.asc" "${TGZ_URL}.asc"
gpg --verify "${TGZ_FILENAME}.asc"
fi
# Update min protocol for testing only - https://bugs.python.org/issue41561
cp /etc/ssl/openssl.cnf /tmp/python-src/
sed -i -E 's/MinProtocol[=\ ]+.*/MinProtocol = TLSv1.0/g' /tmp/python-src/openssl.cnf
export OPENSSL_CONF=/tmp/python-src/openssl.cnf
# Untar and build
tar -xzf "/tmp/python-src/${TGZ_FILENAME}" -C "/tmp/python-src" --strip-components=1
./configure --prefix="${PYTHON_INSTALL_PATH}" --enable-optimizations --with-ensurepip=install
make -j 8
make install
cd /tmp
rm -rf /tmp/python-src ${GNUPGHOME} /tmp/vscdc-settings.env
chown -R ${USERNAME} "${PYTHON_INSTALL_PATH}"
ln -s ${PYTHON_INSTALL_PATH}/bin/python3 ${PYTHON_INSTALL_PATH}/bin/python
ln -s ${PYTHON_INSTALL_PATH}/bin/pip3 ${PYTHON_INSTALL_PATH}/bin/pip
ln -s ${PYTHON_INSTALL_PATH}/bin/idle3 ${PYTHON_INSTALL_PATH}/bin/idle
ln -s ${PYTHON_INSTALL_PATH}/bin/pydoc3 ${PYTHON_INSTALL_PATH}/bin/pydoc
ln -s ${PYTHON_INSTALL_PATH}/bin/python3-config ${PYTHON_INSTALL_PATH}/bin/python-config
updaterc "export PATH=${PYTHON_INSTALL_PATH}/bin:\${PATH}"
fi
}
# Ensure apt is in non-interactive to avoid prompts
export DEBIAN_FRONTEND=noninteractive
# Install python from source if needed
if [ "${PYTHON_VERSION}" != "none" ]; then
# Source /etc/os-release to get OS info
. /etc/os-release
# If ubuntu, PPAs allowed - install from there
if [ "${ID}" = "ubuntu" ] && [ "${USE_PPA_IF_AVAILABLE}" = "true" ]; then
install_from_ppa
else
install_from_source
fi
fi
# If not installing python tools, exit
if [ "${INSTALL_PYTHON_TOOLS}" != "true" ]; then
echo "Done!"
exit 0;
fi
DEFAULT_UTILS="\
pylint \
flake8 \
autopep8 \
black \
yapf \
mypy \
pydocstyle \
pycodestyle \
bandit \
pipenv \
virtualenv"
export PIPX_BIN_DIR=${PIPX_HOME}/bin
export PATH=${PYTHON_INSTALL_PATH}/bin:${PIPX_BIN_DIR}:${PATH}
# Update pip
echo "Updating pip..."
python3 -m pip install --no-cache-dir --upgrade pip
# Create pipx group, dir, and set sticky bit
if ! cat /etc/group | grep -e "^pipx:" > /dev/null 2>&1; then
groupadd -r pipx
fi
usermod -a -G pipx ${USERNAME}
umask 0002
mkdir -p ${PIPX_BIN_DIR}
chown :pipx ${PIPX_HOME} ${PIPX_BIN_DIR}
chmod g+s ${PIPX_HOME} ${PIPX_BIN_DIR}
# Install tools
echo "Installing Python tools..."
export PYTHONUSERBASE=/tmp/pip-tmp
export PIP_CACHE_DIR=/tmp/pip-tmp/cache
pip3 install --disable-pip-version-check --no-warn-script-location --no-cache-dir --user pipx
/tmp/pip-tmp/bin/pipx install --pip-args=--no-cache-dir pipx
echo "${DEFAULT_UTILS}" | xargs -n 1 /tmp/pip-tmp/bin/pipx install --system-site-packages --pip-args '--no-cache-dir --force-reinstall'
rm -rf /tmp/pip-tmp
updaterc "$(cat << EOF
export PIPX_HOME="${PIPX_HOME}"
export PIPX_BIN_DIR="${PIPX_BIN_DIR}"
if [[ "\${PATH}" != *"\${PIPX_BIN_DIR}"* ]]; then export PATH="\${PATH}:\${PIPX_BIN_DIR}"; fi
EOF
)"
@@ -0,0 +1,13 @@
#!/usr/bin/env bash
# k3d
# v5 RC is needed to deterministically set the Registry port. Should be replaces with official release
curl -s https://raw.githubusercontent.com/rancher/k3d/main/install.sh | TAG=v5.0.0-rc.4 bash
# kustomize
pushd /tmp
curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash
popd
sudo mv /tmp/kustomize /usr/local/bin/
@@ -0,0 +1,46 @@
#!/usr/bin/env bash
# install Krew
# TODO (dans): ditch krew and just download the latest binaries on the path in Dockerfile
(
set -x; cd "$(mktemp -d)" &&
OS="$(uname | tr '[:upper:]' '[:lower:]')" &&
ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')" &&
curl -fsSLO "https://github.com/kubernetes-sigs/krew/releases/latest/download/krew.tar.gz" &&
tar zxvf krew.tar.gz &&
KREW=./krew-"${OS}_${ARCH}" &&
"$KREW" install krew
)
# install krew plugins
kubectl krew install schemahero
kubectl krew install support-bundle
kubectl krew install preflights
kubectl krew install view-secret
# Make the cache from master branch
pushd /tmp
git clone https://github.com/replicatedhq/troubleshoot.git
pushd troubleshoot
# TODO (dans): find a way to cache images on image build
go mod download
popd
rm -rf kots
popd
# Clone any extra repos here
# Autocomplete Kubernetes
cat >> ~/.zshrc << EOF
source <(kubectl completion zsh)
alias k=kubectl
complete -F __start_kubectl k
EOF
# Set Git Editor Preference
cat >> ~/.zshrc << EOF
export VISUAL=vim
export EDITOR="$VISUAL"
EOF
@@ -0,0 +1,16 @@
#!/bin/bash
# modified from https://github.com/microsoft/vscode-dev-containers/blob/main/containers/codespaces-linux/.devcontainer/setup-user.sh
# not part of the standard script library
USERNAME=${1:-codespace}
SECURE_PATH_BASE=${2:-$PATH}
echo "Defaults secure_path=\"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin:${SECURE_PATH_BASE}\"" >> /etc/sudoers.d/securepath
# Add user to a Docker group
sudo -u ${USERNAME} mkdir /home/${USERNAME}/.vsonline
groupadd -g 800 docker
usermod -a -G docker ${USERNAME}
# Create user's .local/bin
sudo -u ${USERNAME} mkdir -p /home/${USERNAME}/.local/bin
@@ -0,0 +1,165 @@
#!/usr/bin/env bash
#-------------------------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
#-------------------------------------------------------------------------------------------------------------
#
# Docs: https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/sshd.md
# Maintainer: The VS Code and Codespaces Teams
#
# Syntax: ./sshd-debian.sh [SSH Port (don't use 22)] [non-root user] [start sshd now flag] [new password for user] [fix environment flag]
#
# Note: You can change your user's password with "sudo passwd $(whoami)" (or just "passwd" if running as root).
SSHD_PORT=${1:-"2222"}
USERNAME=${2:-"automatic"}
START_SSHD=${3:-"false"}
NEW_PASSWORD=${4:-"skip"}
FIX_ENVIRONMENT=${5:-"true"}
set -e
if [ "$(id -u)" -ne 0 ]; then
echo -e 'Script must be run as root. Use sudo, su, or add "USER root" to your Dockerfile before running this script.'
exit 1
fi
# Determine the appropriate non-root user
if [ "${USERNAME}" = "auto" ] || [ "${USERNAME}" = "automatic" ]; then
USERNAME=""
POSSIBLE_USERS=("vscode" "node" "codespace" "$(awk -v val=1000 -F ":" '$3==val{print $1}' /etc/passwd)")
for CURRENT_USER in ${POSSIBLE_USERS[@]}; do
if id -u ${CURRENT_USER} > /dev/null 2>&1; then
USERNAME=${CURRENT_USER}
break
fi
done
if [ "${USERNAME}" = "" ]; then
USERNAME=root
fi
elif [ "${USERNAME}" = "none" ] || ! id -u ${USERNAME} > /dev/null 2>&1; then
USERNAME=root
fi
# Function to run apt-get if needed
apt_get_update_if_needed()
{
if [ ! -d "/var/lib/apt/lists" ] || [ "$(ls /var/lib/apt/lists/ | wc -l)" = "0" ]; then
echo "Running apt-get update..."
apt-get update
else
echo "Skipping apt-get update."
fi
}
# Checks if packages are installed and installs them if not
check_packages() {
if ! dpkg -s "$@" > /dev/null 2>&1; then
apt_get_update_if_needed
apt-get -y install --no-install-recommends "$@"
fi
}
# Ensure apt is in non-interactive to avoid prompts
export DEBIAN_FRONTEND=noninteractive
# Install openssh-server openssh-client
check_packages openssh-server openssh-client lsof
# Generate password if new password set to the word "random"
if [ "${NEW_PASSWORD}" = "random" ]; then
NEW_PASSWORD="$(openssl rand -hex 16)"
EMIT_PASSWORD="true"
elif [ "${NEW_PASSWORD}" != "skip" ]; then
# If new password not set to skip, set it for the specified user
echo "${USERNAME}:${NEW_PASSWORD}" | chpasswd
fi
# Add user to ssh group
if [ "${USERNAME}" != "root" ]; then
usermod -aG ssh ${USERNAME}
fi
# Setup sshd
mkdir -p /var/run/sshd
sed -i 's/session\s*required\s*pam_loginuid\.so/session optional pam_loginuid.so/g' /etc/pam.d/sshd
sed -i 's/#*PermitRootLogin prohibit-password/PermitRootLogin yes/g' /etc/ssh/sshd_config
sed -i -E "s/#*\s*Port\s+.+/Port ${SSHD_PORT}/g" /etc/ssh/sshd_config
# Need to UsePAM so /etc/environment is processed
sed -i -E "s/#?\s*UsePAM\s+.+/UsePAM yes/g" /etc/ssh/sshd_config
# Script to store variables that exist at the time the ENTRYPOINT is fired
store_env_script="$(cat << 'EOF'
# Wire in codespaces secret processing to zsh if present (since may have been added to image after script was run)
if [ -f /etc/zsh/zlogin ] && ! grep '/etc/profile.d/00-restore-secrets.sh' /etc/zsh/zlogin > /dev/null 2>&1; then
echo -e "if [ -f /etc/profile.d/00-restore-secrets.sh ]; then . /etc/profile.d/00-restore-secrets.sh; fi\n$(cat /etc/zsh/zlogin 2>/dev/null || echo '')" | sudoIf tee /etc/zsh/zlogin > /dev/null
fi
EOF
)"
# Script to ensure login shells get the latest Codespaces secrets
restore_secrets_script="$(cat << 'EOF'
#!/bin/sh
if [ "${CODESPACES}" != "true" ] || [ "${VSCDC_FIXED_SECRETS}" = "true" ] || [ ! -z "${GITHUB_CODESPACES_TOKEN}" ]; then
# Not codespaces, already run, or secrets already in environment, so return
return
fi
if [ -f /workspaces/.codespaces/shared/.env ]; then
set -o allexport
. /workspaces/.codespaces/shared/.env
set +o allexport
fi
export VSCDC_FIXED_SECRETS=true
EOF
)"
# Write out a scripts that can be referenced as an ENTRYPOINT to auto-start sshd and fix login environments
tee /usr/local/share/ssh-init.sh > /dev/null \
<< 'EOF'
#!/usr/bin/env bash
# This script is intended to be run as root with a container that runs as root (even if you connect with a different user)
# However, it supports running as a user other than root if passwordless sudo is configured for that same user.
set -e
sudoIf()
{
if [ "$(id -u)" -ne 0 ]; then
sudo "$@"
else
"$@"
fi
}
EOF
if [ "${FIX_ENVIRONMENT}" = "true" ]; then
echo "${store_env_script}" >> /usr/local/share/ssh-init.sh
echo "${restore_secrets_script}" > /etc/profile.d/00-restore-secrets.sh
chmod +x /etc/profile.d/00-restore-secrets.sh
# Wire in zsh if present
if type zsh > /dev/null 2>&1; then
echo -e "if [ -f /etc/profile.d/00-restore-secrets.sh ]; then . /etc/profile.d/00-restore-secrets.sh; fi\n$(cat /etc/zsh/zlogin 2>/dev/null || echo '')" > /etc/zsh/zlogin
fi
fi
tee -a /usr/local/share/ssh-init.sh > /dev/null \
<< 'EOF'
# ** Start SSH server **
sudoIf /etc/init.d/ssh start 2>&1 | sudoIf tee /tmp/sshd.log > /dev/null
set +e
exec "$@"
EOF
chmod +x /usr/local/share/ssh-init.sh
# If we should start sshd now, do so
if [ "${START_SSHD}" = "true" ]; then
/usr/local/share/ssh-init.sh
fi
# Output success details
echo -e "Done!\n\n- Port: ${SSHD_PORT}\n- User: ${USERNAME}"
if [ "${EMIT_PASSWORD}" = "true" ]; then
echo "- Password: ${NEW_PASSWORD}"
fi
echo -e "\nForward port ${SSHD_PORT} to your local machine and run:\n\n ssh -p ${SSHD_PORT} -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o GlobalKnownHostsFile=/dev/null ${USERNAME}@localhost\n"
@@ -0,0 +1,8 @@
#!/usr/bin/env bash
# Setup the cluster
k3d cluster create --config /etc/replicated/k3d-cluster.yaml --kubeconfig-update-default
# Clone any extra repos here
@@ -0,0 +1,5 @@
#!/usr/bin/env bash
# Start the cluster here
k3d cluster start replicated
+1
View File
@@ -0,0 +1 @@
* @replicatedhq/day2
+27
View File
@@ -0,0 +1,27 @@
---
name: Bug report
about: Report a problem with Troubleshoot
labels: 'bug'
---
**Bug Description**
Write a concise description of the problem here.
**Expected Behavior**
Include a description of the expected behaviour here.
**Steps To Reproduce**
Include the commands to reproduce the issue including any output. Any information that will help us to understand the problem is useful. Feel free to paste long output into a [Github gist](https://gist.github.com) and include the link here.
**Additional Context**
Include the following information.
- Troubleshoot version. If you built from source, note that including the version of Go you used to build with.
- Operating system
- Operating system version
- Other details that might be helpful in diagnosing the problem
+22
View File
@@ -0,0 +1,22 @@
---
name: Feature Request
about: Suggest a feature
labels: 'feature'
---
**Describe the rationale for the suggested feature.**
Insert a description of the rationale for the new feature here. For example, you could describe a problem that Troubleshoot doesn't address, a limitation, or an idea to improve Troubleshoot.
**Describe the feature**
Insert a description of the feature here. Be specific about how it addresses the any problems, limitations, or suggested improvements outlined in the rationale for the feature.
**Describe alternatives you've considered**
Describe alternative solutions here. Include any workarounds you've considered.
**Additional context**
Add additional context about the feature request. If the change is substantial, consider attaching files to the issue outlining architectural changes, data flows, file formats etc., anything that helps describe the requested change.
+15
View File
@@ -0,0 +1,15 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
version: 2
updates:
- package-ecosystem: "gomod" # See documentation for possible values
directory: "/" # Location of package manifests
labels:
- "dependencies"
- "go"
- "type::chore"
schedule:
interval: "weekly"
+71 -8
View File
@@ -1,10 +1,10 @@
on:
pull_request:
branches:
- master
- main
push:
branches:
- "master"
- "main"
tags:
- "v*.*.*"
@@ -14,7 +14,7 @@ jobs:
steps:
- uses: actions/setup-go@v1
with:
go-version: "1.14"
go-version: "1.17"
- name: setup env
run: |
@@ -32,7 +32,7 @@ jobs:
steps:
- uses: actions/setup-go@v1
with:
go-version: '1.14'
go-version: '1.17'
- name: setup env
run: |
echo "GOPATH=$(go env GOPATH)" >> $GITHUB_ENV
@@ -56,7 +56,32 @@ jobs:
path: bin/
- uses: engineerd/setup-kind@v0.5.0
- run: chmod +x bin/preflight
- run: ./bin/preflight --interactive=false --format=json https://preflight.replicated.com
- run: |
./bin/preflight --interactive=false --format=json https://preflight.replicated.com > result.json
cat result.json
EXIT_STATUS=0
if grep -q "was not collected" result.json; then
echo "Some files were not collected"
EXIT_STATUS=1
fi
if (( `jq '.pass | length' result.json` < 1 )); then
echo "No passing preflights found"
EXIT_STATUS=1
fi
if (( `jq '.warn | length' result.json` < 1 )); then
echo "No warnings found"
EXIT_STATUS=1
fi
if (( `jq '.fail | length' result.json` < 1 )); then
echo "No failed preflights found"
EXIT_STATUS=1
fi
exit $EXIT_STATUS
compile-supportbundle:
runs-on: ubuntu-latest
@@ -64,7 +89,7 @@ jobs:
steps:
- uses: actions/setup-go@v1
with:
go-version: '1.14'
go-version: '1.17'
- name: setup env
run: |
echo "GOPATH=$(go env GOPATH)" >> $GITHUB_ENV
@@ -93,10 +118,31 @@ jobs:
- run: ./bin/support-bundle ./examples/support-bundle/sample-supportbundle.yaml
- run: ./bin/support-bundle https://kots.io
goreleaser-test:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/v') != true
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Unshallow
run: git fetch --prune --unshallow
- uses: actions/setup-go@v1
with:
go-version: "1.17"
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v2
with:
version: "v0.183.0"
args: build --rm-dist --snapshot --config deploy/.goreleaser.yaml
goreleaser:
runs-on: ubuntu-latest
needs:
- validate-preflight
- validate-supportbundle
if: startsWith(github.ref, 'refs/tags/v')
steps:
- name: Checkout
@@ -112,12 +158,29 @@ jobs:
- uses: actions/setup-go@v1
with:
go-version: "1.14"
go-version: "1.17"
- uses: sigstore/cosign-installer@main
with:
cosign-release: "v1.2.1"
- name: Get Cosign Key
run: |
echo $COSIGN_KEY | base64 -d > ./cosign.key
env:
COSIGN_KEY: ${{secrets.COSIGN_KEY}}
- name: Generate SBOM
run: |
make sbom
env:
COSIGN_PASSWORD: ${{ secrets.COSIGN_PASSWORD }}
COSIGN_KEY: ${{ secrets.COSIGN_KEY }}
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v2
with:
version: latest
version: "v0.183.0"
args: release --rm-dist --config deploy/.goreleaser.yaml
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+7
View File
@@ -27,3 +27,10 @@ vendor
*~
dist
try.sh
.vscode/
workspace.*
cosign.key
sbom/
+128
View File
@@ -0,0 +1,128 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our
community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
oss@replicated.com.
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series
of actions.
**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within
the community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.0, available at
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
Community Impact Guidelines were inspired by [Mozilla's code of conduct
enforcement ladder](https://github.com/mozilla/diversity).
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see the FAQ at
https://www.contributor-covenant.org/faq. Translations are available at
https://www.contributor-covenant.org/translations.
+11
View File
@@ -0,0 +1,11 @@
# Contributing to Troubleshoot
Thank you for your interest in Troubleshoot, we welcome your participation. Please familiarize yourself with our [Code of Conduct](https://github.com/replicatedhq/troubleshoot/blob/master/CODE_OF_CONDUCT.md) prior to contributing. There are a number of ways to participate in Troubleshoot as outlined below:
## Issues
- [Request a New Feature](https://github.com/replicatedhq/troubleshoot/issues/new?assignees=&labels=feature&template=feature_enhancement.md) Create an issue to add functionality that addresses a problem or adds an enhancement.
- [Report a Bug](https://github.com/replicatedhq/troubleshoot/issues/new?assignees=&labels=bug&template=bug_report.md) Report a problem or unexpected behaviour with Troubleshoot.
## Pull Requests
If you are interested in contributing a change to the code or documentation please open a pull request with your set of changes. The pull request will be reviewed in a timely manner.
+49 -15
View File
@@ -1,8 +1,6 @@
# Image URL to use all building/pushing image targets
IMG ?= controller:latest
export GO111MODULE=on
export GOPROXY=https://proxy.golang.org
SHELL := /bin/bash -o pipefail
VERSION_PACKAGE = github.com/replicatedhq/troubleshoot/pkg/version
@@ -35,27 +33,33 @@ define LDFLAGS
"
endef
BUILDFLAGS = -tags "netgo containers_image_ostree_stub exclude_graphdriver_devicemapper exclude_graphdriver_btrfs containers_image_openpgp" -installsuffix netgo
all: test
.PHONY: ffi
ffi: fmt vet
go build ${LDFLAGS} -o bin/troubleshoot.so -buildmode=c-shared ffi/main.go
go build ${BUILDFLAGS} ${LDFLAGS} -o bin/troubleshoot.so -buildmode=c-shared ffi/main.go
# Run tests
test: generate fmt vet
go test ./pkg/... ./cmd/... -coverprofile cover.out
go test ${BUILDFLAGS} ./pkg/... ./cmd/... -coverprofile cover.out
.PHONY: support-bundle
support-bundle: generate fmt vet
go build -tags netgo ${LDFLAGS} -o bin/support-bundle github.com/replicatedhq/troubleshoot/cmd/troubleshoot
go build ${BUILDFLAGS} ${LDFLAGS} -o bin/support-bundle github.com/replicatedhq/troubleshoot/cmd/troubleshoot
.PHONY: preflight
preflight: generate fmt vet
go build -tags netgo ${LDFLAGS} -o bin/preflight github.com/replicatedhq/troubleshoot/cmd/preflight
go build ${BUILDFLAGS} ${LDFLAGS} -o bin/preflight github.com/replicatedhq/troubleshoot/cmd/preflight
.PHONY: analyze
analyze: generate fmt vet
go build -tags netgo ${LDFLAGS} -o bin/analyze github.com/replicatedhq/troubleshoot/cmd/analyze
go build ${BUILDFLAGS} ${LDFLAGS} -o bin/analyze github.com/replicatedhq/troubleshoot/cmd/analyze
.PHONY: collect
collect: generate fmt vet
go build ${BUILDFLAGS} ${LDFLAGS} -o bin/collect github.com/replicatedhq/troubleshoot/cmd/collect
.PHONY: fmt
fmt:
@@ -63,7 +67,7 @@ fmt:
.PHONY: vet
vet:
go vet ./pkg/... ./cmd/...
go vet ${BUILDFLAGS} ./pkg/... ./cmd/...
.PHONY: generate
generate: controller-gen client-gen
@@ -87,19 +91,14 @@ schemas: fmt vet openapischema
go build ${LDFLAGS} -o bin/schemagen github.com/replicatedhq/troubleshoot/cmd/schemagen
./bin/schemagen --output-dir ./schemas
.PHONY: contoller-gen
controller-gen:
ifeq (, $(shell which controller-gen))
go get sigs.k8s.io/controller-tools/cmd/controller-gen@v0.3.0
CONTROLLER_GEN=$(shell go env GOPATH)/bin/controller-gen
else
go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.7.0
CONTROLLER_GEN=$(shell which controller-gen)
endif
.PHONY: client-gen
client-gen:
ifeq (, $(shell which client-gen))
go get k8s.io/code-generator/cmd/client-gen@kubernetes-1.18.0
go install k8s.io/code-generator/cmd/client-gen@v0.22.2
CLIENT_GEN=$(shell go env GOPATH)/bin/client-gen
else
CLIENT_GEN=$(shell which client-gen)
@@ -135,3 +134,38 @@ run-troubleshoot: support-bundle
.PHONY: run-analyze
run-analyze: analyze
./bin/analyze --analyzers ./examples/support-bundle/sample-analyzers.yaml ./support-bundle.tar.gz
.PHONY: init-sbom
init-sbom:
mkdir -p sbom/spdx sbom/assets
.PHONY: install-spdx-sbom-generator
install-spdx-sbom-generator: init-sbom
./scripts/initialize-sbom-build.sh
SPDX_GENERATOR=./sbom/spdx-sbom-generator
.PHONY: generate-sbom
generate-sbom: install-spdx-sbom-generator
$(SPDX_GENERATOR) -o ./sbom/spdx
sbom/assets/troubleshoot-sbom.tgz: generate-sbom
tar -czf sbom/assets/troubleshoot-sbom.tgz sbom/spdx/*.spdx
sbom: sbom/assets/troubleshoot-sbom.tgz
cosign sign-blob -key cosign.key sbom/assets/troubleshoot-sbom.tgz > sbom/assets/troubleshoot-sbom.tgz.sig
cosign public-key -key cosign.key -outfile sbom/assets/key.pub
longhorn:
git clone https://github.com/longhorn/longhorn-manager.git
cd longhorn-manager && git checkout v1.2.2 && cd ..
rm -rf pkg/longhorn
mv longhorn-manager/k8s/pkg pkg/longhorn
mv longhorn-manager/types pkg/longhorn/types
mv longhorn-manager/util pkg/longhorn/util
rm -rf pkg/longhorn/util/daemon
rm -rf pkg/longhorn/util/server
find pkg/longhorn -type f | xargs sed -i "s/github.com\/longhorn\/longhorn-manager\/k8s\/pkg/github.com\/replicatedhq\/troubleshoot\/pkg\/longhorn/g"
find pkg/longhorn -type f | xargs sed -i "s/github.com\/longhorn\/longhorn-manager\/types/github.com\/replicatedhq\/troubleshoot\/pkg\/longhorn\/types/g"
find pkg/longhorn -type f | xargs sed -i "s/github.com\/longhorn\/longhorn-manager\/util/github.com\/replicatedhq\/troubleshoot\/pkg\/longhorn\/util/g"
rm -rf longhorn-manager
+16 -3
View File
@@ -18,11 +18,11 @@ curl https://krew.sh/preflight | bash
kubectl preflight https://preflight.replicated.com
```
For a details on creating the custom resource files that drive preflight checks, visit [creating preflight checks](https://troubleshoot.sh/docs/preflight/introduction/).
For more details on creating the custom resource files that drive preflight checks, visit [creating preflight checks](https://troubleshoot.sh/docs/preflight/introduction/).
## Support Bundle
A support bundle is an archive that's created in-cluster, by collecting logs and cluster information, and executing specified commands (including redaction of sensitive information). After creating a support bundle, the cluster operator will normally deliver it to the 3rd-party application vendor for analysis and disconnected debugging. Another Replicated project, [Kotsadm](https://github.com/replicatedhq/kotsadm), provides cluster operators with an in-cluster UI for processing support bundles and viewing analyzers (as well as support bundle collection).
A support bundle is an archive that's created in-cluster, by collecting logs and cluster information, and executing specified commands (including redaction of sensitive information). After creating a support bundle, the cluster operator will normally deliver it to the 3rd-party application vendor for analysis and disconnected debugging. Another Replicated project, [KOTS](https://github.com/replicatedhq/kots), provides k8s apps an in-cluster UI for processing support bundles and viewing analyzers (as well as support bundle collection).
To collect a sample support bundle, install the troubleshoot kubectl plugin:
@@ -34,8 +34,21 @@ curl https://krew.sh/support-bundle | bash
```shell
kubectl support-bundle https://support-bundle.replicated.com
```
For details on creating the custom resource files that drive support-bundle collection, visit [creating collectors](https://troubleshoot.sh/docs/collect/) and [creating analyzers](https://troubleshoot.sh/docs/analyze/).
For more details on creating the custom resource files that drive support-bundle collection, visit [creating collectors](https://troubleshoot.sh/docs/collect/) and [creating analyzers](https://troubleshoot.sh/docs/analyze/).
# Community
For questions about using Troubleshoot, there's a [Replicated Community](https://help.replicated.com/community) forum, and a [#app-troubleshoot channel in Kubernetes Slack](https://kubernetes.slack.com/channels/app-troubleshoot).
# Software Bill of Materials
A signed SBOM that includes Troubleshoot dependencies is included in each release.
- **troubleshoot-sbom.tgz** contains a software bill of materials for Troubleshoot.
- **troubleshoot-sbom.tgz.sig** is the digital signature for troubleshoot-sbom.tgz
- **key.pub** is the public key from the key pair used to sign troubleshoot-sbom.tgz
The following example illustrates using [cosign](https://github.com/sigstore/cosign) to verify that **troubleshoot-sbom.tgz** has
not been tampered with.
```shell
$ cosign verify-blob -key key.pub -signature troubleshoot-sbom.tgz.sig troubleshoot-sbom.tgz
Verified OK
```
+65
View File
@@ -0,0 +1,65 @@
package cli
import (
"os"
"strings"
"github.com/replicatedhq/troubleshoot/pkg/k8sutil"
"github.com/replicatedhq/troubleshoot/pkg/logger"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
func RootCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "collect [url]",
Args: cobra.MinimumNArgs(1),
Short: "Run a collector",
Long: `Run a collector and output the results.`,
SilenceUsage: true,
PreRun: func(cmd *cobra.Command, args []string) {
viper.BindPFlags(cmd.Flags())
},
RunE: func(cmd *cobra.Command, args []string) error {
v := viper.GetViper()
logger.SetQuiet(v.GetBool("quiet"))
return runCollect(v, args[0])
},
}
cobra.OnInitialize(initConfig)
cmd.AddCommand(VersionCmd())
cmd.Flags().StringSlice("redactors", []string{}, "names of the additional redactors to use")
cmd.Flags().Bool("redact", true, "enable/disable default redactions")
cmd.Flags().String("format", "json", "output format, one of json or raw.")
cmd.Flags().String("collector-image", "", "the full name of the collector image to use")
cmd.Flags().String("collector-pull-policy", "", "the pull policy of the collector image")
cmd.Flags().String("selector", "", "selector (label query) to filter remote collection nodes on.")
cmd.Flags().Bool("collect-without-permissions", false, "always generate a support bundle, even if it some require additional permissions")
// hidden in favor of the `insecure-skip-tls-verify` flag
cmd.Flags().Bool("allow-insecure-connections", false, "when set, do not verify TLS certs when retrieving spec and reporting results")
cmd.Flags().MarkHidden("allow-insecure-connections")
viper.BindPFlags(cmd.Flags())
viper.SetEnvKeyReplacer(strings.NewReplacer("-", "_"))
k8sutil.AddFlags(cmd.Flags())
return cmd
}
func InitAndExecute() {
if err := RootCmd().Execute(); err != nil {
os.Exit(1)
}
}
func initConfig() {
viper.SetEnvPrefix("TROUBLESHOOT")
viper.AutomaticEnv()
}
+184
View File
@@ -0,0 +1,184 @@
package cli
import (
"fmt"
"io/ioutil"
"net/http"
"os"
"os/signal"
"strings"
"time"
"github.com/pkg/errors"
"github.com/replicatedhq/troubleshoot/cmd/util"
troubleshootv1beta2 "github.com/replicatedhq/troubleshoot/pkg/apis/troubleshoot/v1beta2"
"github.com/replicatedhq/troubleshoot/pkg/client/troubleshootclientset/scheme"
troubleshootclientsetscheme "github.com/replicatedhq/troubleshoot/pkg/client/troubleshootclientset/scheme"
"github.com/replicatedhq/troubleshoot/pkg/collect"
"github.com/replicatedhq/troubleshoot/pkg/docrewrite"
"github.com/replicatedhq/troubleshoot/pkg/k8sutil"
"github.com/replicatedhq/troubleshoot/pkg/specs"
"github.com/replicatedhq/troubleshoot/pkg/supportbundle"
"github.com/spf13/viper"
"k8s.io/apimachinery/pkg/labels"
)
const (
defaultTimeout = 30 * time.Second
)
func runCollect(v *viper.Viper, arg string) error {
go func() {
signalChan := make(chan os.Signal, 1)
signal.Notify(signalChan, os.Interrupt)
<-signalChan
os.Exit(0)
}()
var collectorContent []byte
var err error
if strings.HasPrefix(arg, "secret/") {
// format secret/namespace-name/secret-name
pathParts := strings.Split(arg, "/")
if len(pathParts) != 3 {
return errors.Errorf("path %s must have 3 components", arg)
}
spec, err := specs.LoadFromSecret(pathParts[1], pathParts[2], "collect-spec")
if err != nil {
return errors.Wrap(err, "failed to get spec from secret")
}
collectorContent = spec
} else if _, err = os.Stat(arg); err == nil {
b, err := ioutil.ReadFile(arg)
if err != nil {
return err
}
collectorContent = b
} else {
if !util.IsURL(arg) {
return fmt.Errorf("%s is not a URL and was not found (err %s)", arg, err)
}
req, err := http.NewRequest("GET", arg, nil)
if err != nil {
return err
}
req.Header.Set("User-Agent", "Replicated_Collect/v1beta2")
resp, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return err
}
collectorContent = body
}
collectorContent, err = docrewrite.ConvertToV1Beta2(collectorContent)
if err != nil {
return errors.Wrap(err, "failed to convert to v1beta2")
}
multidocs := strings.Split(string(collectorContent), "\n---\n")
troubleshootclientsetscheme.AddToScheme(scheme.Scheme)
decode := scheme.Codecs.UniversalDeserializer().Decode
additionalRedactors := &troubleshootv1beta2.Redactor{}
for idx, redactor := range v.GetStringSlice("redactors") {
redactorObj, err := supportbundle.GetRedactorFromURI(redactor)
if err != nil {
return errors.Wrapf(err, "failed to get redactor spec %s, #%d", redactor, idx)
}
if redactorObj != nil {
additionalRedactors.Spec.Redactors = append(additionalRedactors.Spec.Redactors, redactorObj.Spec.Redactors...)
}
}
for i, additionalDoc := range multidocs {
if i == 0 {
continue
}
additionalDoc, err := docrewrite.ConvertToV1Beta2([]byte(additionalDoc))
if err != nil {
return errors.Wrap(err, "failed to convert to v1beta2")
}
obj, _, err := decode(additionalDoc, nil, nil)
if err != nil {
return errors.Wrapf(err, "failed to parse additional doc %d", i)
}
multidocRedactors, ok := obj.(*troubleshootv1beta2.Redactor)
if !ok {
continue
}
additionalRedactors.Spec.Redactors = append(additionalRedactors.Spec.Redactors, multidocRedactors.Spec.Redactors...)
}
// make sure we don't block any senders
progressCh := make(chan interface{})
defer close(progressCh)
go func() {
for range progressCh {
}
}()
restConfig, err := k8sutil.GetRESTConfig()
if err != nil {
return errors.Wrap(err, "failed to convert kube flags to rest config")
}
labelSelector, err := labels.Parse(v.GetString("selector"))
if err != nil {
return errors.Wrap(err, "unable to parse selector")
}
namespace := v.GetString("namespace")
if namespace == "" {
namespace = "default"
}
timeout := v.GetDuration("request-timeout")
if timeout == 0 {
timeout = defaultTimeout
}
createOpts := collect.CollectorRunOpts{
CollectWithoutPermissions: v.GetBool("collect-without-permissions"),
KubernetesRestConfig: restConfig,
Image: v.GetString("collector-image"),
PullPolicy: v.GetString("collector-pullpolicy"),
LabelSelector: labelSelector.String(),
Namespace: namespace,
Timeout: timeout,
ProgressChan: progressCh,
}
// we only support HostCollector or RemoteCollector kinds.
hostCollector, err := collect.ParseHostCollectorFromDoc([]byte(multidocs[0]))
if err == nil {
results, err := collect.CollectHost(hostCollector, additionalRedactors, createOpts)
if err != nil {
return errors.Wrap(err, "failed to collect from host")
}
return showHostStdoutResults(v.GetString("format"), hostCollector.Name, results)
}
remoteCollector, err := collect.ParseRemoteCollectorFromDoc([]byte(multidocs[0]))
if err == nil {
results, err := collect.CollectRemote(remoteCollector, additionalRedactors, createOpts)
if err != nil {
return errors.Wrap(err, "failed to collect from remote host(s)")
}
return showRemoteStdoutResults(v.GetString("format"), remoteCollector.Name, results)
}
return errors.New("failed to parse hostCollector or remoteCollector")
}
+103
View File
@@ -0,0 +1,103 @@
package cli
import (
"encoding/json"
"fmt"
"github.com/pkg/errors"
"github.com/replicatedhq/troubleshoot/pkg/collect"
)
const (
// FormatJSON is intended for CLI output.
FormatJSON = "json"
// FormatRaw is intended for consumption by a remote collector. Output is a
// string of quoted JSON.
FormatRaw = "raw"
)
func showHostStdoutResults(format string, collectName string, results *collect.HostCollectResult) error {
switch format {
case FormatJSON:
return showHostStdoutResultsJSON(collectName, results.AllCollectedData)
case FormatRaw:
return showHostStdoutResultsRaw(collectName, results.AllCollectedData)
default:
return errors.Errorf("unknown output format: %q", format)
}
}
func showRemoteStdoutResults(format string, collectName string, results *collect.RemoteCollectResult) error {
switch format {
case FormatJSON:
return showRemoteStdoutResultsJSON(collectName, results.AllCollectedData)
case FormatRaw:
return errors.Errorf("raw format not supported for remote collectors")
default:
return errors.Errorf("unknown output format: %q", format)
}
}
func showHostStdoutResultsJSON(collectName string, results map[string][]byte) error {
output := make(map[string]interface{})
for file, collectorResult := range results {
var collectedItems map[string]interface{}
if err := json.Unmarshal([]byte(collectorResult), &collectedItems); err != nil {
return errors.Wrap(err, "failed to marshal collector results")
}
output[file] = collectedItems
}
formatted, err := json.MarshalIndent(output, "", " ")
if err != nil {
return errors.Wrap(err, "failed to convert output to json")
}
fmt.Print(string(formatted))
return nil
}
// showHostStdoutResultsRaw outputs the collector output as a string of quoted json.
func showHostStdoutResultsRaw(collectName string, results map[string][]byte) error {
strData := map[string]string{}
for k, v := range results {
strData[k] = string(v)
}
formatted, err := json.MarshalIndent(strData, "", " ")
if err != nil {
return errors.Wrap(err, "failed to convert output to json")
}
fmt.Print(string(formatted))
return nil
}
func showRemoteStdoutResultsJSON(collectName string, results map[string][]byte) error {
type CollectorResult map[string]interface{}
type NodeResult map[string]CollectorResult
var output = make(map[string]NodeResult)
for node, result := range results {
var nodeResult map[string]string
if err := json.Unmarshal(result, &nodeResult); err != nil {
return errors.Wrap(err, "failed to marshal node results")
}
nr := make(NodeResult)
for file, collectorResult := range nodeResult {
var collectedItems map[string]interface{}
if err := json.Unmarshal([]byte(collectorResult), &collectedItems); err != nil {
return errors.Wrap(err, "failed to marshal collector results")
}
nr[file] = collectedItems
}
output[node] = nr
}
formatted, err := json.MarshalIndent(output, "", " ")
if err != nil {
return errors.Wrap(err, "failed to convert output to json")
}
fmt.Print(string(formatted))
return nil
}
+22
View File
@@ -0,0 +1,22 @@
package cli
import (
"fmt"
"github.com/replicatedhq/troubleshoot/pkg/version"
"github.com/spf13/cobra"
)
func VersionCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "version",
Short: "Print the current version and exit",
Long: `Print the current version and exit`,
RunE: func(cmd *cobra.Command, args []string) error {
fmt.Printf("Replicated Collect %s\n", version.Version())
return nil
},
}
return cmd
}
+10
View File
@@ -0,0 +1,10 @@
package main
import (
"github.com/replicatedhq/troubleshoot/cmd/collect/cli"
_ "k8s.io/client-go/plugin/pkg/client/auth"
)
func main() {
cli.InitAndExecute()
}
+2 -1
View File
@@ -34,7 +34,8 @@ that a cluster meets the requirements to run an application.`,
cmd.Flags().String("format", "human", "output format, one of human, json, yaml. only used when interactive is set to false")
cmd.Flags().String("collector-image", "", "the full name of the collector image to use")
cmd.Flags().String("collector-pullpolicy", "", "the pull policy of the collector image")
cmd.Flags().Bool("collect-without-permissions", false, "always run preflight checks even if some require permissions that preflight does not have")
cmd.Flags().Bool("collect-without-permissions", true, "always run preflight checks even if some require permissions that preflight does not have")
cmd.Flags().String("selector", "", "selector (label query) to filter remote collection nodes on.")
cmd.Flags().String("since-time", "", "force pod logs collectors to return logs after a specific date (RFC3339)")
cmd.Flags().String("since", "", "force pod logs collectors to return logs newer than a relative duration like 5s, 2m, or 3h.")
+149 -45
View File
@@ -1,10 +1,12 @@
package cli
import (
"context"
"fmt"
"io/ioutil"
"net/http"
"os"
"os/signal"
"strings"
"time"
@@ -12,6 +14,7 @@ import (
"github.com/fatih/color"
"github.com/pkg/errors"
"github.com/replicatedhq/troubleshoot/cmd/util"
analyzer "github.com/replicatedhq/troubleshoot/pkg/analyze"
troubleshootv1beta2 "github.com/replicatedhq/troubleshoot/pkg/apis/troubleshoot/v1beta2"
troubleshootclientsetscheme "github.com/replicatedhq/troubleshoot/pkg/client/troubleshootclientset/scheme"
"github.com/replicatedhq/troubleshoot/pkg/docrewrite"
@@ -20,13 +23,24 @@ import (
"github.com/replicatedhq/troubleshoot/pkg/specs"
"github.com/spf13/viper"
spin "github.com/tj/go-spin"
"golang.org/x/sync/errgroup"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/labels"
"k8s.io/client-go/kubernetes/scheme"
)
func runPreflights(v *viper.Viper, arg string) error {
fmt.Print(cursor.Hide())
defer fmt.Print(cursor.Show())
if v.GetBool("interactive") {
fmt.Print(cursor.Hide())
defer fmt.Print(cursor.Show())
}
go func() {
signalChan := make(chan os.Signal, 1)
signal.Notify(signalChan, os.Interrupt)
<-signalChan
os.Exit(0)
}()
var preflightContent []byte
var err error
@@ -86,54 +100,45 @@ func runPreflights(v *viper.Viper, arg string) error {
return errors.Wrapf(err, "failed to parse %s", arg)
}
var collectResults preflight.CollectResult
var collectResults []preflight.CollectResult
preflightSpecName := ""
finishedCh := make(chan bool, 1)
progressCh := make(chan interface{}, 0) // non-zero buffer will result in missed messages
s := spin.New()
go func() {
for {
select {
case msg, ok := <-progressCh:
if !ok {
continue
}
switch msg := msg.(type) {
case error:
c := color.New(color.FgHiRed)
c.Println(fmt.Sprintf("%s\r * %v", cursor.ClearEntireLine(), msg))
case string:
c := color.New(color.FgCyan)
c.Println(fmt.Sprintf("%s\r * %s", cursor.ClearEntireLine(), msg))
}
case <-time.After(time.Millisecond * 100):
fmt.Printf("\r \033[36mRunning Preflight checks\033[m %s ", s.Next())
case <-finishedCh:
fmt.Printf("\r%s\r", cursor.ClearEntireLine())
return
}
}
}()
progressCh := make(chan interface{})
defer close(progressCh)
defer func() {
close(finishedCh)
close(progressCh)
}()
ctx, stopProgressCollection := context.WithCancel(context.Background())
// make sure we shut down progress collection goroutines if an error occurs
defer stopProgressCollection()
progressCollection, ctx := errgroup.WithContext(ctx)
if v.GetBool("interactive") {
progressCollection.Go(collectInteractiveProgress(ctx, progressCh))
} else {
progressCollection.Go(collectNonInteractiveProgess(ctx, progressCh))
}
if preflightSpec, ok := obj.(*troubleshootv1beta2.Preflight); ok {
r, err := collectInCluster(preflightSpec, finishedCh, progressCh)
r, err := collectInCluster(preflightSpec, progressCh)
if err != nil {
return errors.Wrap(err, "failed to collect in cluster")
}
collectResults = *r
collectResults = append(collectResults, *r)
preflightSpecName = preflightSpec.Name
} else if hostPreflightSpec, ok := obj.(*troubleshootv1beta2.HostPreflight); ok {
r, err := collectHost(hostPreflightSpec, finishedCh, progressCh)
if err != nil {
return errors.Wrap(err, "failed to collect from host")
if len(hostPreflightSpec.Spec.Collectors) > 0 {
r, err := collectHost(hostPreflightSpec, progressCh)
if err != nil {
return errors.Wrap(err, "failed to collect from host")
}
collectResults = append(collectResults, *r)
}
if len(hostPreflightSpec.Spec.RemoteCollectors) > 0 {
r, err := collectRemote(hostPreflightSpec, progressCh)
if err != nil {
return errors.Wrap(err, "failed to collect remotely")
}
collectResults = append(collectResults, *r)
}
collectResults = *r
preflightSpecName = hostPreflightSpec.Name
}
@@ -141,7 +146,10 @@ func runPreflights(v *viper.Viper, arg string) error {
return errors.New("no results")
}
analyzeResults := collectResults.Analyze()
analyzeResults := []*analyzer.AnalyzeResult{}
for _, res := range collectResults {
analyzeResults = append(analyzeResults, res.Analyze()...)
}
if preflightSpec, ok := obj.(*troubleshootv1beta2.Preflight); ok {
if preflightSpec.Spec.UploadResultsTo != "" {
@@ -152,7 +160,8 @@ func runPreflights(v *viper.Viper, arg string) error {
}
}
finishedCh <- true
stopProgressCollection()
progressCollection.Wait()
if v.GetBool("interactive") {
if len(analyzeResults) == 0 {
@@ -164,7 +173,60 @@ func runPreflights(v *viper.Viper, arg string) error {
return showStdoutResults(v.GetString("format"), preflightSpecName, analyzeResults)
}
func collectInCluster(preflightSpec *troubleshootv1beta2.Preflight, finishedCh chan bool, progressCh chan interface{}) (*preflight.CollectResult, error) {
func collectInteractiveProgress(ctx context.Context, progressCh <-chan interface{}) func() error {
return func() error {
spinner := spin.New()
lastMsg := ""
errorTxt := color.New(color.FgHiRed)
infoTxt := color.New(color.FgCyan)
for {
select {
case msg := <-progressCh:
switch msg := msg.(type) {
case error:
errorTxt.Printf("%s\r * %v\n", cursor.ClearEntireLine(), msg)
case string:
if lastMsg == msg {
break
}
lastMsg = msg
infoTxt.Printf("%s\r * %s\n", cursor.ClearEntireLine(), msg)
}
case <-time.After(time.Millisecond * 100):
fmt.Printf("\r %s %s ", color.CyanString("Running Preflight Checks"), spinner.Next())
case <-ctx.Done():
fmt.Printf("\r%s\r", cursor.ClearEntireLine())
return nil
}
}
}
}
func collectNonInteractiveProgess(ctx context.Context, progressCh <-chan interface{}) func() error {
return func() error {
for {
select {
case msg := <-progressCh:
switch msg := msg.(type) {
case error:
fmt.Fprintf(os.Stderr, "error - %v\n", msg)
case string:
fmt.Fprintf(os.Stderr, "%s\n", msg)
case preflight.CollectProgress:
fmt.Fprintf(os.Stderr, "%s\n", msg.String())
}
case <-ctx.Done():
return nil
}
}
}
}
func collectInCluster(preflightSpec *troubleshootv1beta2.Preflight, progressCh chan interface{}) (*preflight.CollectResult, error) {
v := viper.GetViper()
restConfig, err := k8sutil.GetRESTConfig()
@@ -180,7 +242,7 @@ func collectInCluster(preflightSpec *troubleshootv1beta2.Preflight, finishedCh c
}
if v.GetString("since") != "" || v.GetString("since-time") != "" {
err := parseTimeFlags(v, progressCh, preflightSpec.Spec.Collectors)
err := parseTimeFlags(v, preflightSpec.Spec.Collectors)
if err != nil {
return nil, err
}
@@ -203,7 +265,49 @@ func collectInCluster(preflightSpec *troubleshootv1beta2.Preflight, finishedCh c
return &collectResults, nil
}
func collectHost(hostPreflightSpec *troubleshootv1beta2.HostPreflight, finishedCh chan bool, progressCh chan interface{}) (*preflight.CollectResult, error) {
func collectRemote(preflightSpec *troubleshootv1beta2.HostPreflight, progressCh chan interface{}) (*preflight.CollectResult, error) {
v := viper.GetViper()
restConfig, err := k8sutil.GetRESTConfig()
if err != nil {
return nil, errors.Wrap(err, "failed to convert kube flags to rest config")
}
labelSelector, err := labels.Parse(v.GetString("selector"))
if err != nil {
return nil, errors.Wrap(err, "unable to parse selector")
}
namespace := v.GetString("namespace")
if namespace == "" {
namespace = "default"
}
timeout := v.GetDuration("request-timeout")
if timeout == 0 {
timeout = 30 * time.Second
}
collectOpts := preflight.CollectOpts{
Namespace: namespace,
IgnorePermissionErrors: v.GetBool("collect-without-permissions"),
ProgressChan: progressCh,
KubernetesRestConfig: restConfig,
Image: v.GetString("collector-image"),
PullPolicy: v.GetString("collector-pullpolicy"),
LabelSelector: labelSelector.String(),
Timeout: timeout,
}
collectResults, err := preflight.CollectRemote(collectOpts, preflightSpec)
if err != nil {
return nil, errors.Wrap(err, "failed to collect from remote")
}
return &collectResults, nil
}
func collectHost(hostPreflightSpec *troubleshootv1beta2.HostPreflight, progressCh chan interface{}) (*preflight.CollectResult, error) {
collectOpts := preflight.CollectOpts{
ProgressChan: progressCh,
}
@@ -216,7 +320,7 @@ func collectHost(hostPreflightSpec *troubleshootv1beta2.HostPreflight, finishedC
return &collectResults, nil
}
func parseTimeFlags(v *viper.Viper, progressChan chan interface{}, collectors []*troubleshootv1beta2.Collect) error {
func parseTimeFlags(v *viper.Viper, collectors []*troubleshootv1beta2.Collect) error {
var (
sinceTime time.Time
err error
+12 -3
View File
@@ -11,7 +11,7 @@ import (
"github.com/pkg/errors"
"github.com/spf13/cobra"
"github.com/spf13/viper"
extensionsv1beta1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1"
extensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
extensionsscheme "k8s.io/apiextensions-apiserver/pkg/client/clientset/clientset/scheme"
"k8s.io/client-go/kubernetes/scheme"
)
@@ -129,9 +129,18 @@ func generateSchemaFromCRD(crd []byte, outfile string) error {
return errors.Wrap(err, "failed to decode crd")
}
customResourceDefinition := obj.(*extensionsv1beta1.CustomResourceDefinition)
customResourceDefinition := obj.(*extensionsv1.CustomResourceDefinition)
b, err := json.MarshalIndent(customResourceDefinition.Spec.Validation.OpenAPIV3Schema, "", " ")
if len(customResourceDefinition.Spec.Versions) == 0 {
return errors.New("no versions found for CRD")
}
crdSchema := customResourceDefinition.Spec.Versions[0].Schema
if crdSchema == nil {
return errors.New("CRD has a nil schema")
}
b, err := json.MarshalIndent(crdSchema.OpenAPIV3Schema, "", " ")
if err != nil {
return errors.Wrap(err, "failed to marshal json")
}
+2 -1
View File
@@ -38,7 +38,8 @@ from a server that can be used to assist when troubleshooting a Kubernetes clust
cmd.Flags().StringSlice("redactors", []string{}, "names of the additional redactors to use")
cmd.Flags().Bool("redact", true, "enable/disable default redactions")
cmd.Flags().Bool("collect-without-permissions", false, "always generate a support bundle, even if it some require additional permissions")
cmd.Flags().Bool("interactive", true, "enable/disable interactive mode")
cmd.Flags().Bool("collect-without-permissions", true, "always generate a support bundle, even if it some require additional permissions")
cmd.Flags().String("since-time", "", "force pod logs collectors to return logs after a specific date (RFC3339)")
cmd.Flags().String("since", "", "force pod logs collectors to return logs newer than a relative duration like 5s, 2m, or 3h.")
+191 -642
View File
@@ -1,20 +1,14 @@
package cli
import (
"archive/tar"
"bytes"
"compress/gzip"
"context"
"crypto/tls"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"net/http"
"net/url"
"os"
"os/signal"
"path/filepath"
"sort"
"strings"
"time"
@@ -23,47 +17,66 @@ import (
"github.com/manifoldco/promptui"
"github.com/mattn/go-isatty"
"github.com/pkg/errors"
"github.com/replicatedhq/troubleshoot/cmd/util"
analyzer "github.com/replicatedhq/troubleshoot/pkg/analyze"
troubleshootv1beta2 "github.com/replicatedhq/troubleshoot/pkg/apis/troubleshoot/v1beta2"
"github.com/replicatedhq/troubleshoot/pkg/client/troubleshootclientset/scheme"
troubleshootclientsetscheme "github.com/replicatedhq/troubleshoot/pkg/client/troubleshootclientset/scheme"
"github.com/replicatedhq/troubleshoot/pkg/collect"
"github.com/replicatedhq/troubleshoot/pkg/convert"
"github.com/replicatedhq/troubleshoot/pkg/docrewrite"
"github.com/replicatedhq/troubleshoot/pkg/httputil"
"github.com/replicatedhq/troubleshoot/pkg/k8sutil"
"github.com/replicatedhq/troubleshoot/pkg/redact"
"github.com/replicatedhq/troubleshoot/pkg/specs"
"github.com/replicatedhq/troubleshoot/pkg/supportbundle"
"github.com/spf13/viper"
spin "github.com/tj/go-spin"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
var (
httpClient *http.Client
"k8s.io/client-go/rest"
)
func runTroubleshoot(v *viper.Viper, arg string) error {
fmt.Print(cursor.Hide())
defer fmt.Print(cursor.Show())
interactive := v.GetBool("interactive") && isatty.IsTerminal(os.Stdout.Fd())
if v.GetBool("allow-insecure-connections") || v.GetBool("insecure-skip-tls-verify") {
httpClient = &http.Client{Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}}
} else {
httpClient = http.DefaultClient
if interactive {
fmt.Print(cursor.Hide())
defer fmt.Print(cursor.Show())
}
collectorContent, err := loadSpec(v, arg)
go func() {
signalChan := make(chan os.Signal, 1)
signal.Notify(signalChan, os.Interrupt)
<-signalChan
if interactive {
fmt.Print(cursor.Show())
}
os.Exit(0)
}()
restConfig, err := k8sutil.GetRESTConfig()
if err != nil {
return errors.Wrap(err, "failed to convert kube flags to rest config")
}
var sinceTime *time.Time
if v.GetString("since-time") != "" || v.GetString("since") != "" {
sinceTime, err = parseTimeFlags(v)
if err != nil {
return errors.Wrap(err, "failed parse since time")
}
}
if v.GetBool("allow-insecure-connections") || v.GetBool("insecure-skip-tls-verify") {
httputil.AddTransport(&http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
})
}
collectorContent, err := supportbundle.LoadSupportBundleSpec(arg)
if err != nil {
return errors.Wrap(err, "failed to load collector spec")
}
multidocs := strings.Split(string(collectorContent), "\n---\n")
// we suppory both raw collector kinds and supportbundle kinds here
supportBundleSpec, err := parseSupportBundleFromDoc([]byte(multidocs[0]))
// we support both raw collector kinds and supportbundle kinds here
supportBundle, err := supportbundle.ParseSupportBundleFromDoc([]byte(multidocs[0]))
if err != nil {
return errors.Wrap(err, "failed to parse collector")
}
@@ -73,24 +86,13 @@ func runTroubleshoot(v *viper.Viper, arg string) error {
additionalRedactors := &troubleshootv1beta2.Redactor{}
for idx, redactor := range v.GetStringSlice("redactors") {
redactorContent, err := loadSpec(v, redactor)
redactorObj, err := supportbundle.GetRedactorFromURI(redactor)
if err != nil {
return errors.Wrapf(err, "failed to load redactor spec #%d", idx)
return errors.Wrapf(err, "failed to get redactor spec %s, #%d", redactor, idx)
}
redactorContent, err = docrewrite.ConvertToV1Beta2(redactorContent)
if err != nil {
return errors.Wrap(err, "failed to convert to v1beta2")
}
obj, _, err := decode([]byte(redactorContent), nil, nil)
if err != nil {
return errors.Wrapf(err, "failed to parse redactors %s", redactor)
}
loopRedactors, ok := obj.(*troubleshootv1beta2.Redactor)
if !ok {
return fmt.Errorf("%s is not a troubleshootv1beta2 redactor type", redactor)
}
if loopRedactors != nil {
additionalRedactors.Spec.Redactors = append(additionalRedactors.Spec.Redactors, loopRedactors.Spec.Redactors...)
if redactorObj != nil {
additionalRedactors.Spec.Redactors = append(additionalRedactors.Spec.Redactors, redactorObj.Spec.Redactors...)
}
}
@@ -113,246 +115,173 @@ func runTroubleshoot(v *viper.Viper, arg string) error {
additionalRedactors.Spec.Redactors = append(additionalRedactors.Spec.Redactors, multidocRedactors.Spec.Redactors...)
}
s := spin.New()
var collectorCB func(chan interface{}, string)
progressChan := make(chan interface{}) // non-zero buffer can result in missed messages
finishedCh := make(chan bool, 1)
progressChan := make(chan interface{}, 0) // non-zero buffer can result in missed messages
isFinishedChClosed := false
go func() {
currentDir := ""
for {
select {
case msg := <-progressChan:
switch msg := msg.(type) {
case error:
c := color.New(color.FgHiRed)
c.Println(fmt.Sprintf("%s\r * %v", cursor.ClearEntireLine(), msg))
case string:
currentDir = filepath.Base(msg)
}
case <-finishedCh:
fmt.Printf("\r%s\r", cursor.ClearEntireLine())
return
case <-time.After(time.Millisecond * 100):
if currentDir == "" {
fmt.Printf("\r%s \033[36mCollecting support bundle\033[m %s", cursor.ClearEntireLine(), s.Next())
} else {
fmt.Printf("\r%s \033[36mCollecting support bundle\033[m %s %s", cursor.ClearEntireLine(), s.Next(), currentDir)
if !interactive {
// TODO (dans): custom warning handler to capture warning in `analysisOutput`
restConfig.WarningHandler = rest.NoWarnings{}
collectorCB = func(ch chan interface{}, name string) {
return
}
// TODO (dans): maybe log to file
go func() {
for {
select {
case _ = <-progressChan:
// do nothing
}
}
}
}()
defer func() {
if !isFinishedChClosed {
close(finishedCh)
}
}()
}()
} else {
s := spin.New()
go func() {
currentDir := ""
for {
select {
case msg := <-progressChan:
switch msg := msg.(type) {
case error:
c := color.New(color.FgHiRed)
c.Println(fmt.Sprintf("%s\r * %v", cursor.ClearEntireLine(), msg))
case string:
currentDir = filepath.Base(msg)
}
case <-finishedCh:
fmt.Printf("\r%s\r", cursor.ClearEntireLine())
return
case <-time.After(time.Millisecond * 100):
if currentDir == "" {
fmt.Printf("\r%s \033[36mCollecting support bundle\033[m %s", cursor.ClearEntireLine(), s.Next())
} else {
fmt.Printf("\r%s \033[36mCollecting support bundle\033[m %s %s", cursor.ClearEntireLine(), s.Next(), currentDir)
}
}
}
}()
defer func() {
if !isFinishedChClosed {
close(finishedCh)
}
}()
archivePath, err := runCollectors(v, supportBundleSpec.Spec.Collectors, additionalRedactors, progressChan)
collectorCB = func(c chan interface{}, msg string) {
c <- fmt.Sprintf("%s", msg)
}
}
createOpts := supportbundle.SupportBundleCreateOpts{
CollectorProgressCallback: collectorCB,
CollectWithoutPermissions: v.GetBool("collect-without-permissions"),
KubernetesRestConfig: restConfig,
Namespace: v.GetString("namespace"),
ProgressChan: progressChan,
SinceTime: sinceTime,
Redact: v.GetBool("redact"),
FromCLI: true,
}
nonInteractiveOutput := analysisOutput{}
if interactive {
c := color.New()
c.Println(fmt.Sprintf("\r%s\r", cursor.ClearEntireLine()))
}
response, err := supportbundle.CollectSupportBundleFromSpec(&supportBundle.Spec, additionalRedactors, createOpts)
if err != nil {
return errors.Wrap(err, "run collectors")
return errors.Wrap(err, "failed to run collect and analyze process")
}
fmt.Printf("\r%s\r", cursor.ClearEntireLine())
// upload if needed
fileUploaded := false
if len(supportBundleSpec.Spec.AfterCollection) > 0 {
for _, ac := range supportBundleSpec.Spec.AfterCollection {
if ac.UploadResultsTo != nil {
if err := uploadSupportBundle(ac.UploadResultsTo, archivePath); err != nil {
c := color.New(color.FgHiRed)
c.Printf("%s\r * Failed to upload support bundle: %v\n", cursor.ClearEntireLine(), err)
} else {
fileUploaded = true
}
} else if ac.Callback != nil {
if err := callbackSupportBundleAPI(ac.Callback, archivePath); err != nil {
c := color.New(color.FgHiRed)
c.Printf("%s\r * Failed to notify API that support bundle has been uploaded: %v\n", cursor.ClearEntireLine(), err)
}
}
}
}
// perform analysis, if possible
if len(supportBundleSpec.Spec.Analyzers) > 0 {
tmpDir, err := ioutil.TempDir("", "troubleshoot")
if err != nil {
c := color.New(color.FgHiRed)
c.Printf("%s\r * Failed to make directory for analysis: %v\n", cursor.ClearEntireLine(), err)
}
f, err := os.Open(archivePath)
if err != nil {
c := color.New(color.FgHiRed)
c.Printf("%s\r * Failed to open support bundle for analysis: %v\n", cursor.ClearEntireLine(), err)
}
if err := analyzer.ExtractTroubleshootBundle(f, tmpDir); err != nil {
c := color.New(color.FgHiRed)
c.Printf("%s\r * Failed to extract support bundle for analysis: %v\n", cursor.ClearEntireLine(), err)
}
analyzeResults, err := analyzer.AnalyzeLocal(tmpDir, supportBundleSpec.Spec.Analyzers)
if err != nil {
c := color.New(color.FgHiRed)
c.Printf("%s\r * Failed to analyze support bundle: %v\n", cursor.ClearEntireLine(), err)
}
interactive := isatty.IsTerminal(os.Stdout.Fd())
if len(response.AnalyzerResults) > 0 {
if interactive {
close(finishedCh) // this removes the spinner
isFinishedChClosed = true
if err := showInteractiveResults(supportBundleSpec.Name, analyzeResults); err != nil {
if err := showInteractiveResults(supportBundle.Name, response.AnalyzerResults); err != nil {
interactive = false
}
}
if !interactive {
data := convert.FromAnalyzerResult(analyzeResults)
formatted, err := json.MarshalIndent(data, "", " ")
if err != nil {
c := color.New(color.FgHiRed)
c.Printf("%s\r * Failed to format analysis: %v\n", cursor.ClearEntireLine(), err)
}
fmt.Printf("%s", formatted)
} else {
nonInteractiveOutput.Analysis = response.AnalyzerResults
}
}
if !fileUploaded {
msg := archivePath
if appName := supportBundleSpec.Labels["applicationName"]; appName != "" {
if !response.FileUploaded {
if appName := supportBundle.Labels["applicationName"]; appName != "" {
f := `A support bundle for %s has been created in this directory
named %s. Please upload it on the Troubleshoot page of
the %s Admin Console to begin analysis.`
msg = fmt.Sprintf(f, appName, archivePath, appName)
fmt.Printf(f, appName, response.ArchivePath, appName)
return nil
}
fmt.Printf("%s\n", msg)
if !interactive {
nonInteractiveOutput.ArchivePath = response.ArchivePath
output, err := nonInteractiveOutput.FormattedAnalysisOutput()
if err != nil {
return errors.Wrap(err, "failed to format non-interactive output")
}
fmt.Println(output)
return nil
}
fmt.Printf("%s\n", response.ArchivePath)
return nil
}
fmt.Printf("\r%s\r", cursor.ClearEntireLine())
if fileUploaded {
if interactive {
fmt.Printf("\r%s\r", cursor.ClearEntireLine())
}
if response.FileUploaded {
fmt.Printf("A support bundle has been created and uploaded to your cluster for analysis. Please visit the Troubleshoot page to continue.\n")
fmt.Printf("A copy of this support bundle was written to the current directory, named %q\n", archivePath)
fmt.Printf("A copy of this support bundle was written to the current directory, named %q\n", response.ArchivePath)
} else {
fmt.Printf("A support bundle has been created in the current directory named %q\n", archivePath)
fmt.Printf("A support bundle has been created in the current directory named %q\n", response.ArchivePath)
}
return nil
}
func loadSpec(v *viper.Viper, arg string) ([]byte, error) {
var err error
if strings.HasPrefix(arg, "secret/") {
// format secret/namespace-name/secret-name
pathParts := strings.Split(arg, "/")
if len(pathParts) != 3 {
return nil, errors.Errorf("path %s must have 3 components", arg)
}
spec, err := specs.LoadFromSecret(pathParts[1], pathParts[2], "support-bundle-spec")
if err != nil {
return nil, errors.Wrap(err, "failed to get spec from secret")
}
return spec, nil
}
if _, err = os.Stat(arg); err == nil {
b, err := ioutil.ReadFile(arg)
if err != nil {
return nil, errors.Wrap(err, "read spec file")
}
return b, nil
} else if !util.IsURL(arg) {
return nil, fmt.Errorf("%s is not a URL and was not found (err %s)", arg, err)
}
spec, err := loadSpecFromURL(v, arg)
func getExpectedContentType(uploadURL string) string {
parsedURL, err := url.Parse(uploadURL)
if err != nil {
return nil, errors.Wrap(err, "failed to get spec from URL")
return ""
}
return spec, nil
return parsedURL.Query().Get("Content-Type")
}
func loadSpecFromURL(v *viper.Viper, arg string) ([]byte, error) {
for {
req, err := http.NewRequest("GET", arg, nil)
func parseTimeFlags(v *viper.Viper) (*time.Time, error) {
var (
sinceTime time.Time
err error
)
if v.GetString("since-time") != "" {
if v.GetString("since") != "" {
return nil, errors.Errorf("at most one of `sinceTime` or `since` may be specified")
}
sinceTime, err = time.Parse(time.RFC3339, v.GetString("since-time"))
if err != nil {
return nil, errors.Wrap(err, "make request")
return nil, errors.Wrap(err, "unable to parse --since-time flag")
}
req.Header.Set("User-Agent", "Replicated_Troubleshoot/v1beta1")
req.Header.Set("Bundle-Upload-Host", fmt.Sprintf("%s://%s", req.URL.Scheme, req.URL.Host))
resp, err := httpClient.Do(req)
} else {
parsedDuration, err := time.ParseDuration(v.GetString("since"))
if err != nil {
if shouldRetryRequest(err) {
continue
}
return nil, errors.Wrap(err, "execute request")
return nil, errors.Wrap(err, "unable to parse --since flag")
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, errors.Wrap(err, "read responce body")
}
return body, nil
}
}
func parseSupportBundleFromDoc(doc []byte) (*troubleshootv1beta2.SupportBundle, error) {
doc, err := docrewrite.ConvertToV1Beta2(doc)
if err != nil {
return nil, errors.Wrap(err, "failed to convert to v1beta2")
now := time.Now()
sinceTime = now.Add(0 - parsedDuration)
}
troubleshootclientsetscheme.AddToScheme(scheme.Scheme)
decode := scheme.Codecs.UniversalDeserializer().Decode
obj, _, err := decode(doc, nil, nil)
if err != nil {
return nil, errors.Wrap(err, "failed to parse document")
}
collector, ok := obj.(*troubleshootv1beta2.Collector)
if ok {
supportBundle := troubleshootv1beta2.SupportBundle{
TypeMeta: metav1.TypeMeta{
APIVersion: "troubleshoot.sh/v1beta2",
Kind: "SupportBundle",
},
ObjectMeta: collector.ObjectMeta,
Spec: troubleshootv1beta2.SupportBundleSpec{
Collectors: collector.Spec.Collectors,
Analyzers: []*troubleshootv1beta2.Analyze{},
AfterCollection: collector.Spec.AfterCollection,
},
}
return &supportBundle, nil
}
supportBundle, ok := obj.(*troubleshootv1beta2.SupportBundle)
if ok {
return supportBundle, nil
}
return nil, errors.New("spec was not parseable as a troubleshoot kind")
return &sinceTime, nil
}
func shouldRetryRequest(err error) bool {
if strings.Contains(err.Error(), "x509") && httpClient == http.DefaultClient && canTryInsecure() {
httpClient = &http.Client{Transport: &http.Transport{
if strings.Contains(err.Error(), "x509") && canTryInsecure() {
httputil.AddTransport(&http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}}
})
return true
}
return false
@@ -368,410 +297,30 @@ func canTryInsecure() bool {
}
_, err := prompt.Run()
return err == nil
}
type analysisOutput struct {
Analysis []*analyzer.AnalyzeResult
ArchivePath string
}
func (a *analysisOutput) FormattedAnalysisOutput() (outputJson string, err error) {
type convertedOutput struct {
ConvertedAnalysis []*convert.Result `json:"analyzerResults"`
ArchivePath string `json:"archivePath"`
}
converted := convert.FromAnalyzerResult(a.Analysis)
o := convertedOutput{
ConvertedAnalysis: converted,
ArchivePath: a.ArchivePath,
}
formatted, err := json.MarshalIndent(o, "", " ")
if err != nil {
return false
return "", fmt.Errorf("\r * Failed to format analysis: %v\n", err)
}
return true
}
func runCollectors(v *viper.Viper, collectors []*troubleshootv1beta2.Collect, additionalRedactors *troubleshootv1beta2.Redactor, progressChan chan interface{}) (string, error) {
tmpDir, err := ioutil.TempDir("", "troubleshoot")
if err != nil {
return "", errors.Wrap(err, "create temp dir")
}
defer os.RemoveAll(tmpDir)
filename, err := findFileName("support-bundle-"+time.Now().Format("2006-01-02T15_04_05"), "tar.gz")
if err != nil {
return "", errors.Wrap(err, "find file name")
}
bundlePath := filepath.Join(tmpDir, strings.TrimSuffix(filename, ".tar.gz"))
if err := os.MkdirAll(bundlePath, 0777); err != nil {
return "", errors.Wrap(err, "create bundle dir")
}
if err = writeVersionFile(bundlePath); err != nil {
return "", errors.Wrap(err, "write version file")
}
collectSpecs := make([]*troubleshootv1beta2.Collect, 0, 0)
collectSpecs = append(collectSpecs, collectors...)
collectSpecs = ensureCollectorInList(collectSpecs, troubleshootv1beta2.Collect{ClusterInfo: &troubleshootv1beta2.ClusterInfo{}})
collectSpecs = ensureCollectorInList(collectSpecs, troubleshootv1beta2.Collect{ClusterResources: &troubleshootv1beta2.ClusterResources{}})
config, err := k8sutil.GetRESTConfig()
if err != nil {
return "", errors.Wrap(err, "failed to convert kube flags to rest config")
}
var cleanedCollectors collect.Collectors
for _, desiredCollector := range collectSpecs {
collector := collect.Collector{
Redact: true,
Collect: desiredCollector,
ClientConfig: config,
Namespace: v.GetString("namespace"),
PathPrefix: filepath.Base(bundlePath),
}
cleanedCollectors = append(cleanedCollectors, &collector)
}
if err := cleanedCollectors.CheckRBAC(context.Background()); err != nil {
return "", errors.Wrap(err, "failed to check RBAC for collectors")
}
foundForbidden := false
for _, c := range cleanedCollectors {
for _, e := range c.RBACErrors {
foundForbidden = true
progressChan <- e
}
}
if foundForbidden && !v.GetBool("collect-without-permissions") {
return "", errors.New("insufficient permissions to run all collectors")
}
globalRedactors := []*troubleshootv1beta2.Redact{}
if additionalRedactors != nil {
globalRedactors = additionalRedactors.Spec.Redactors
}
if v.GetString("since-time") != "" || v.GetString("since") != "" {
err := parseTimeFlags(v, progressChan, &cleanedCollectors)
if err != nil {
return "", err
}
}
// Run preflights collectors synchronously
for _, collector := range cleanedCollectors {
if len(collector.RBACErrors) > 0 {
// don't skip clusterResources collector due to RBAC issues
if collector.Collect.ClusterResources == nil {
progressChan <- fmt.Sprintf("skipping collector %s with insufficient RBAC permissions", collector.GetDisplayName())
continue
}
}
progressChan <- collector.GetDisplayName()
result, err := collector.RunCollectorSync(globalRedactors)
if err != nil {
progressChan <- fmt.Errorf("failed to run collector %q: %v", collector.GetDisplayName(), err)
continue
}
if result != nil {
// results already contain the bundle dir name in their paths
err = saveCollectorOutput(result, filepath.Dir(bundlePath), collector)
if err != nil {
progressChan <- fmt.Errorf("failed to parse collector spec %q: %v", collector.GetDisplayName(), err)
continue
}
}
}
if err := tarSupportBundleDir(bundlePath, filename); err != nil {
return "", errors.Wrap(err, "create bundle file")
}
return filename, nil
}
func saveCollectorOutput(output map[string][]byte, bundlePath string, c *collect.Collector) error {
for filename, maybeContents := range output {
if c.Collect.Copy != nil {
err := untarAndSave(maybeContents, filepath.Join(bundlePath, filepath.Dir(filename)))
if err != nil {
return errors.Wrap(err, "extract copied files")
}
continue
}
fileDir, fileName := filepath.Split(filename)
outPath := filepath.Join(bundlePath, fileDir)
if err := os.MkdirAll(outPath, 0777); err != nil {
return errors.Wrap(err, "create output file")
}
if err := writeFile(filepath.Join(outPath, fileName), maybeContents); err != nil {
return errors.Wrap(err, "write collector output")
}
}
return nil
}
func untarAndSave(tarFile []byte, bundlePath string) error {
keys := make([]string, 0)
dirs := make(map[string]*tar.Header)
files := make(map[string][]byte)
fileHeaders := make(map[string]*tar.Header)
tarReader := tar.NewReader(bytes.NewBuffer(tarFile))
//Extract and separate tar contentes in file and folders, keeping header info from each one.
for {
header, err := tarReader.Next()
if err != nil {
if err != io.EOF {
return err
}
break
}
switch header.Typeflag {
case tar.TypeDir:
dirs[header.Name] = header
case tar.TypeReg:
file := new(bytes.Buffer)
_, err = io.Copy(file, tarReader)
if err != nil {
return err
}
files[header.Name] = file.Bytes()
fileHeaders[header.Name] = header
default:
return fmt.Errorf("Tar file entry %s contained unsupported file type %v", header.Name, header.FileInfo().Mode())
}
}
//Create directories from base path: <namespace>/<pod name>/containerPath
if err := os.MkdirAll(filepath.Join(bundlePath), 0777); err != nil {
return errors.Wrap(err, "create output file")
}
//Order folders stored in variable keys to start always by parent folder. That way folder info is preserved.
for k := range dirs {
keys = append(keys, k)
}
sort.Strings(keys)
//Orderly create folders.
for _, k := range keys {
if err := os.Mkdir(filepath.Join(bundlePath, k), dirs[k].FileInfo().Mode().Perm()); err != nil {
return errors.Wrap(err, "create output file")
}
}
//Populate folders with respective files and its permissions stored in the header.
for k, v := range files {
if err := ioutil.WriteFile(filepath.Join(bundlePath, k), v, fileHeaders[k].FileInfo().Mode().Perm()); err != nil {
return err
}
}
return nil
}
func uploadSupportBundle(r *troubleshootv1beta2.ResultRequest, archivePath string) error {
contentType := getExpectedContentType(r.URI)
if contentType != "" && contentType != "application/tar+gzip" {
return fmt.Errorf("cannot upload content type %s", contentType)
}
for {
f, err := os.Open(archivePath)
if err != nil {
return errors.Wrap(err, "open file")
}
defer f.Close()
fileStat, err := f.Stat()
if err != nil {
return errors.Wrap(err, "stat file")
}
req, err := http.NewRequest(r.Method, r.URI, f)
if err != nil {
return errors.Wrap(err, "create request")
}
req.ContentLength = fileStat.Size()
if contentType != "" {
req.Header.Set("Content-Type", contentType)
}
resp, err := httpClient.Do(req)
if err != nil {
if shouldRetryRequest(err) {
continue
}
return errors.Wrap(err, "execute request")
}
if resp.StatusCode >= 300 {
return fmt.Errorf("unexpected status code %d", resp.StatusCode)
}
break
}
// send redaction report
if r.RedactURI != "" {
type PutSupportBundleRedactions struct {
Redactions redact.RedactionList `json:"redactions"`
}
redactBytes, err := json.Marshal(PutSupportBundleRedactions{Redactions: redact.GetRedactionList()})
if err != nil {
return errors.Wrap(err, "get redaction report")
}
for {
req, err := http.NewRequest("PUT", r.RedactURI, bytes.NewReader(redactBytes))
if err != nil {
return errors.Wrap(err, "create redaction report request")
}
req.ContentLength = int64(len(redactBytes))
resp, err := httpClient.Do(req)
if err != nil {
if shouldRetryRequest(err) {
continue
}
return errors.Wrap(err, "execute redaction request")
}
if resp.StatusCode >= 300 {
return fmt.Errorf("unexpected redaction status code %d", resp.StatusCode)
}
break
}
}
return nil
}
func getExpectedContentType(uploadURL string) string {
parsedURL, err := url.Parse(uploadURL)
if err != nil {
return ""
}
return parsedURL.Query().Get("Content-Type")
}
func callbackSupportBundleAPI(r *troubleshootv1beta2.ResultRequest, archivePath string) error {
for {
req, err := http.NewRequest(r.Method, r.URI, nil)
if err != nil {
return errors.Wrap(err, "create request")
}
resp, err := httpClient.Do(req)
if err != nil {
if shouldRetryRequest(err) {
continue
}
return errors.Wrap(err, "execute request")
}
if resp.StatusCode >= 300 {
return fmt.Errorf("unexpected status code %d", resp.StatusCode)
}
break
}
return nil
}
func tarSupportBundleDir(inputDir, outputFilename string) error {
fileWriter, err := os.Create(outputFilename)
if err != nil {
return errors.Wrap(err, "failed to create output file")
}
defer fileWriter.Close()
gzipWriter := gzip.NewWriter(fileWriter)
defer gzipWriter.Close()
tarWriter := tar.NewWriter(gzipWriter)
defer tarWriter.Close()
err = filepath.Walk(inputDir, func(filename string, info os.FileInfo, err error) error {
if err != nil {
return err
}
fileMode := info.Mode()
if !fileMode.IsRegular() { // support bundle can have only files
return nil
}
parentDirName := filepath.Dir(inputDir) // this is to have the files inside a subdirectory
nameInArchive, err := filepath.Rel(parentDirName, filename)
if err != nil {
return errors.Wrap(err, "failed to create relative file name")
}
// tar.FileInfoHeader call causes a crash in static builds
// https://github.com/golang/go/issues/24787
hdr := &tar.Header{
Name: nameInArchive,
ModTime: info.ModTime(),
Mode: int64(fileMode.Perm()),
Typeflag: tar.TypeReg,
Size: info.Size(),
}
err = tarWriter.WriteHeader(hdr)
if err != nil {
return errors.Wrap(err, "failed to write tar header")
}
err = func() error {
fileReader, err := os.Open(filename)
if err != nil {
return errors.Wrap(err, "failed to open source file")
}
defer fileReader.Close()
_, err = io.Copy(tarWriter, fileReader)
if err != nil {
return errors.Wrap(err, "failed to copy file into archive")
}
return nil
}()
if err != nil {
return err
}
return nil
})
if err != nil {
return errors.Wrap(err, "failed to walk source dir")
}
return nil
}
type CollectorFailure struct {
Collector *troubleshootv1beta2.Collect
Failure string
}
func parseTimeFlags(v *viper.Viper, progressChan chan interface{}, collectors *collect.Collectors) error {
var (
sinceTime time.Time
err error
)
if v.GetString("since-time") != "" {
if v.GetString("since") != "" {
return errors.Errorf("at most one of `sinceTime` or `since` may be specified")
}
sinceTime, err = time.Parse(time.RFC3339, v.GetString("since-time"))
if err != nil {
return errors.Wrap(err, "unable to parse --since-time flag")
}
} else {
parsedDuration, err := time.ParseDuration(v.GetString("since"))
if err != nil {
return errors.Wrap(err, "unable to parse --since flag")
}
now := time.Now()
sinceTime = now.Add(0 - parsedDuration)
}
for _, collector := range *collectors {
if collector.Collect.Logs != nil {
if collector.Collect.Logs.Limits == nil {
collector.Collect.Logs.Limits = new(troubleshootv1beta2.LogLimits)
}
collector.Collect.Logs.Limits.SinceTime = metav1.NewTime(sinceTime)
}
}
return nil
return string(formatted), nil
}
+9
View File
@@ -0,0 +1,9 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostCollector
metadata:
name: modules
spec:
collectors:
- cpu: {}
- memory: {}
# - kernelModules: {}
+9
View File
@@ -0,0 +1,9 @@
apiVersion: troubleshoot.sh/v1beta2
kind: RemoteCollector
metadata:
name: modules
spec:
collectors:
- cpu: {}
- memory: {}
# - kernelModules: {}
File diff suppressed because it is too large Load Diff
@@ -1,10 +1,10 @@
---
apiVersion: apiextensions.k8s.io/v1beta1
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.0
controller-gen.kubebuilder.io/version: v0.7.0
creationTimestamp: null
name: collectors.troubleshoot.replicated.com
spec:
@@ -15,368 +15,318 @@ spec:
plural: collectors
singular: collector
scope: Namespaced
validation:
openAPIV3Schema:
description: Collector is the Schema for the collectors API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: CollectorSpec defines the desired state of Collector
properties:
afterCollection:
items:
properties:
callback:
properties:
method:
type: string
redactUri:
type: string
uri:
type: string
required:
- method
- redactUri
- uri
type: object
uploadResultsTo:
properties:
method:
type: string
redactUri:
type: string
uri:
type: string
required:
- method
- redactUri
- uri
type: object
type: object
type: array
collectors:
items:
properties:
clusterInfo:
properties:
collectorName:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
type: object
clusterResources:
properties:
collectorName:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
type: object
copy:
properties:
collectorName:
type: string
containerName:
type: string
containerPath:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
name:
type: string
namespace:
type: string
selector:
items:
type: string
type: array
required:
- containerPath
- namespace
- selector
type: object
data:
properties:
collectorName:
type: string
data:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
name:
type: string
required:
- data
type: object
exec:
properties:
args:
items:
type: string
type: array
collectorName:
type: string
command:
items:
type: string
type: array
containerName:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
name:
type: string
namespace:
type: string
selector:
items:
type: string
type: array
timeout:
type: string
required:
- namespace
- selector
type: object
http:
properties:
collectorName:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
get:
properties:
headers:
additionalProperties:
type: string
type: object
insecureSkipVerify:
type: boolean
url:
type: string
required:
- url
type: object
name:
type: string
post:
properties:
body:
type: string
headers:
additionalProperties:
type: string
type: object
insecureSkipVerify:
type: boolean
url:
type: string
required:
- url
type: object
put:
properties:
body:
type: string
headers:
additionalProperties:
type: string
type: object
insecureSkipVerify:
type: boolean
url:
type: string
required:
- url
type: object
type: object
logs:
properties:
collectorName:
type: string
containerNames:
items:
type: string
type: array
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
limits:
properties:
maxAge:
type: string
maxLines:
format: int64
type: integer
type: object
name:
type: string
namespace:
type: string
selector:
items:
type: string
type: array
required:
- selector
type: object
mysql:
properties:
collectorName:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
uri:
type: string
required:
- uri
type: object
postgres:
properties:
collectorName:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
uri:
type: string
required:
- uri
type: object
redis:
properties:
collectorName:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
uri:
type: string
required:
- uri
type: object
run:
properties:
args:
items:
type: string
type: array
collectorName:
type: string
command:
items:
type: string
type: array
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
image:
type: string
imagePullPolicy:
type: string
name:
type: string
namespace:
type: string
timeout:
type: string
required:
- image
- namespace
type: object
secret:
properties:
collectorName:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
includeValue:
type: boolean
key:
type: string
name:
type: string
namespace:
type: string
required:
- name
type: object
type: object
type: array
type: object
status:
description: CollectorStatus defines the observed state of Collector
type: object
type: object
version: v1beta1
versions:
- name: v1beta1
schema:
openAPIV3Schema:
description: Collector is the Schema for the collectors API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: CollectorSpec defines the desired state of Collector
properties:
afterCollection:
items:
properties:
callback:
properties:
method:
type: string
redactUri:
type: string
uri:
type: string
required:
- method
- redactUri
- uri
type: object
uploadResultsTo:
properties:
method:
type: string
redactUri:
type: string
uri:
type: string
required:
- method
- redactUri
- uri
type: object
type: object
type: array
collectors:
items:
properties:
clusterInfo:
properties:
collectorName:
type: string
exclude:
type: BoolString
type: object
clusterResources:
properties:
collectorName:
type: string
exclude:
type: BoolString
type: object
copy:
properties:
collectorName:
type: string
containerName:
type: string
containerPath:
type: string
exclude:
type: BoolString
name:
type: string
namespace:
type: string
selector:
items:
type: string
type: array
required:
- containerPath
- namespace
- selector
type: object
data:
properties:
collectorName:
type: string
data:
type: string
exclude:
type: BoolString
name:
type: string
required:
- data
type: object
exec:
properties:
args:
items:
type: string
type: array
collectorName:
type: string
command:
items:
type: string
type: array
containerName:
type: string
exclude:
type: BoolString
name:
type: string
namespace:
type: string
selector:
items:
type: string
type: array
timeout:
type: string
required:
- namespace
- selector
type: object
http:
properties:
collectorName:
type: string
exclude:
type: BoolString
get:
properties:
headers:
additionalProperties:
type: string
type: object
insecureSkipVerify:
type: boolean
url:
type: string
required:
- url
type: object
name:
type: string
post:
properties:
body:
type: string
headers:
additionalProperties:
type: string
type: object
insecureSkipVerify:
type: boolean
url:
type: string
required:
- url
type: object
put:
properties:
body:
type: string
headers:
additionalProperties:
type: string
type: object
insecureSkipVerify:
type: boolean
url:
type: string
required:
- url
type: object
type: object
logs:
properties:
collectorName:
type: string
containerNames:
items:
type: string
type: array
exclude:
type: BoolString
limits:
properties:
maxAge:
type: string
maxLines:
format: int64
type: integer
type: object
name:
type: string
namespace:
type: string
selector:
items:
type: string
type: array
required:
- selector
type: object
mysql:
properties:
collectorName:
type: string
exclude:
type: BoolString
uri:
type: string
required:
- uri
type: object
postgres:
properties:
collectorName:
type: string
exclude:
type: BoolString
uri:
type: string
required:
- uri
type: object
redis:
properties:
collectorName:
type: string
exclude:
type: BoolString
uri:
type: string
required:
- uri
type: object
run:
properties:
args:
items:
type: string
type: array
collectorName:
type: string
command:
items:
type: string
type: array
exclude:
type: BoolString
image:
type: string
imagePullPolicy:
type: string
imagePullSecret:
properties:
data:
additionalProperties:
type: string
type: object
name:
type: string
type:
type: string
type: object
name:
type: string
namespace:
type: string
timeout:
type: string
required:
- image
- namespace
type: object
secret:
properties:
collectorName:
type: string
exclude:
type: BoolString
includeValue:
type: boolean
key:
type: string
name:
type: string
namespace:
type: string
required:
- name
type: object
type: object
type: array
type: object
status:
description: CollectorStatus defines the observed state of Collector
type: object
type: object
served: true
storage: true
status:
File diff suppressed because it is too large Load Diff
@@ -1,10 +1,10 @@
---
apiVersion: apiextensions.k8s.io/v1beta1
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.0
controller-gen.kubebuilder.io/version: v0.7.0
creationTimestamp: null
name: redactors.troubleshoot.replicated.com
spec:
@@ -15,69 +15,68 @@ spec:
plural: redactors
singular: redactor
scope: Namespaced
validation:
openAPIV3Schema:
description: Redactor is the Schema for the redaction API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: RedactorSpec defines the desired state of Redactor
properties:
redactors:
items:
properties:
fileSelector:
properties:
file:
type: string
files:
items:
type: string
type: array
type: object
name:
type: string
removals:
properties:
regex:
items:
properties:
redactor:
type: string
selector:
type: string
type: object
type: array
values:
items:
type: string
type: array
yamlPath:
items:
type: string
type: array
type: object
type: object
type: array
type: object
status:
description: RedactorStatus defines the observed state of Redactor
type: object
type: object
version: v1beta1
versions:
- name: v1beta1
schema:
openAPIV3Schema:
description: Redactor is the Schema for the redaction API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: RedactorSpec defines the desired state of Redactor
properties:
redactors:
items:
properties:
fileSelector:
properties:
file:
type: string
files:
items:
type: string
type: array
type: object
name:
type: string
removals:
properties:
regex:
items:
properties:
redactor:
type: string
selector:
type: string
type: object
type: array
values:
items:
type: string
type: array
yamlPath:
items:
type: string
type: array
type: object
type: object
type: array
type: object
status:
description: RedactorStatus defines the observed state of Redactor
type: object
type: object
served: true
storage: true
status:
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+496 -362
View File
@@ -1,10 +1,10 @@
---
apiVersion: apiextensions.k8s.io/v1beta1
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.0
controller-gen.kubebuilder.io/version: v0.7.0
creationTimestamp: null
name: collectors.troubleshoot.sh
spec:
@@ -15,368 +15,502 @@ spec:
plural: collectors
singular: collector
scope: Namespaced
validation:
openAPIV3Schema:
description: Collector is the Schema for the collectors API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: CollectorSpec defines the desired state of Collector
properties:
afterCollection:
items:
properties:
callback:
properties:
method:
type: string
redactUri:
type: string
uri:
type: string
required:
- method
- redactUri
- uri
type: object
uploadResultsTo:
properties:
method:
type: string
redactUri:
type: string
uri:
type: string
required:
- method
- redactUri
- uri
type: object
type: object
type: array
collectors:
items:
properties:
clusterInfo:
properties:
collectorName:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
type: object
clusterResources:
properties:
collectorName:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
type: object
copy:
properties:
collectorName:
type: string
containerName:
type: string
containerPath:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
name:
type: string
namespace:
type: string
selector:
items:
type: string
type: array
required:
- containerPath
- namespace
- selector
type: object
data:
properties:
collectorName:
type: string
data:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
name:
type: string
required:
- data
type: object
exec:
properties:
args:
items:
type: string
type: array
collectorName:
type: string
command:
items:
type: string
type: array
containerName:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
name:
type: string
namespace:
type: string
selector:
items:
type: string
type: array
timeout:
type: string
required:
- namespace
- selector
type: object
http:
properties:
collectorName:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
get:
properties:
headers:
additionalProperties:
type: string
type: object
insecureSkipVerify:
type: boolean
url:
type: string
required:
- url
type: object
name:
type: string
post:
properties:
body:
type: string
headers:
additionalProperties:
type: string
type: object
insecureSkipVerify:
type: boolean
url:
type: string
required:
- url
type: object
put:
properties:
body:
type: string
headers:
additionalProperties:
type: string
type: object
insecureSkipVerify:
type: boolean
url:
type: string
required:
- url
type: object
type: object
logs:
properties:
collectorName:
type: string
containerNames:
items:
type: string
type: array
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
limits:
properties:
maxAge:
type: string
maxLines:
format: int64
type: integer
type: object
name:
type: string
namespace:
type: string
selector:
items:
type: string
type: array
required:
- selector
type: object
mysql:
properties:
collectorName:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
uri:
type: string
required:
- uri
type: object
postgres:
properties:
collectorName:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
uri:
type: string
required:
- uri
type: object
redis:
properties:
collectorName:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
uri:
type: string
required:
- uri
type: object
run:
properties:
args:
items:
type: string
type: array
collectorName:
type: string
command:
items:
type: string
type: array
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
image:
type: string
imagePullPolicy:
type: string
name:
type: string
namespace:
type: string
timeout:
type: string
required:
- image
- namespace
type: object
secret:
properties:
collectorName:
type: string
exclude:
description: BoolOrString is a type that can hold an bool
or a string. When used in JSON or YAML marshalling and
unmarshalling, it produces or consumes the inner type. This
allows you to have, for example, a JSON field that can accept
a booolean string or raw bool.
type: BoolString
includeValue:
type: boolean
key:
type: string
name:
type: string
namespace:
type: string
required:
- name
type: object
type: object
type: array
type: object
status:
description: CollectorStatus defines the observed state of Collector
type: object
type: object
version: v1beta2
versions:
- name: v1beta2
schema:
openAPIV3Schema:
description: Collector is the Schema for the collectors API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: CollectorSpec defines the desired state of Collector
properties:
afterCollection:
items:
properties:
callback:
properties:
method:
type: string
redactUri:
type: string
uri:
type: string
required:
- method
- redactUri
- uri
type: object
uploadResultsTo:
properties:
method:
type: string
redactUri:
type: string
uri:
type: string
required:
- method
- redactUri
- uri
type: object
type: object
type: array
collectors:
items:
properties:
ceph:
properties:
collectorName:
type: string
exclude:
type: BoolString
namespace:
type: string
timeout:
type: string
required:
- namespace
type: object
clusterInfo:
properties:
collectorName:
type: string
exclude:
type: BoolString
type: object
clusterResources:
properties:
collectorName:
type: string
exclude:
type: BoolString
namespaces:
items:
type: string
type: array
type: object
collectd:
properties:
collectorName:
type: string
exclude:
type: BoolString
hostPath:
type: string
image:
type: string
imagePullPolicy:
type: string
imagePullSecret:
properties:
data:
additionalProperties:
type: string
type: object
name:
type: string
type:
type: string
type: object
namespace:
type: string
timeout:
type: string
required:
- hostPath
- image
- namespace
type: object
configMap:
properties:
collectorName:
type: string
exclude:
type: BoolString
includeAllData:
type: boolean
includeValue:
type: boolean
key:
type: string
name:
type: string
namespace:
type: string
selector:
items:
type: string
type: array
type: object
copy:
properties:
collectorName:
type: string
containerName:
type: string
containerPath:
type: string
exclude:
type: BoolString
extractArchive:
type: boolean
name:
type: string
namespace:
type: string
selector:
items:
type: string
type: array
required:
- containerPath
- namespace
- selector
type: object
copyFromHost:
properties:
collectorName:
type: string
exclude:
type: BoolString
extractArchive:
type: boolean
hostPath:
type: string
image:
type: string
imagePullPolicy:
type: string
imagePullSecret:
properties:
data:
additionalProperties:
type: string
type: object
name:
type: string
type:
type: string
type: object
name:
type: string
namespace:
type: string
timeout:
type: string
required:
- hostPath
- image
- namespace
type: object
data:
properties:
collectorName:
type: string
data:
type: string
exclude:
type: BoolString
name:
type: string
required:
- data
type: object
exec:
properties:
args:
items:
type: string
type: array
collectorName:
type: string
command:
items:
type: string
type: array
containerName:
type: string
exclude:
type: BoolString
name:
type: string
namespace:
type: string
selector:
items:
type: string
type: array
timeout:
type: string
required:
- namespace
- selector
type: object
http:
properties:
collectorName:
type: string
exclude:
type: BoolString
get:
properties:
headers:
additionalProperties:
type: string
type: object
insecureSkipVerify:
type: boolean
url:
type: string
required:
- url
type: object
name:
type: string
post:
properties:
body:
type: string
headers:
additionalProperties:
type: string
type: object
insecureSkipVerify:
type: boolean
url:
type: string
required:
- url
type: object
put:
properties:
body:
type: string
headers:
additionalProperties:
type: string
type: object
insecureSkipVerify:
type: boolean
url:
type: string
required:
- url
type: object
type: object
logs:
properties:
collectorName:
type: string
containerNames:
items:
type: string
type: array
exclude:
type: BoolString
limits:
properties:
maxAge:
type: string
maxLines:
format: int64
type: integer
sinceTime:
format: date-time
type: string
type: object
name:
type: string
namespace:
type: string
selector:
items:
type: string
type: array
required:
- selector
type: object
longhorn:
properties:
collectorName:
type: string
exclude:
type: BoolString
namespace:
type: string
timeout:
type: string
required:
- namespace
type: object
mysql:
properties:
collectorName:
type: string
exclude:
type: BoolString
uri:
type: string
required:
- uri
type: object
postgres:
properties:
collectorName:
type: string
exclude:
type: BoolString
uri:
type: string
required:
- uri
type: object
redis:
properties:
collectorName:
type: string
exclude:
type: BoolString
uri:
type: string
required:
- uri
type: object
registryImages:
properties:
collectorName:
type: string
exclude:
type: BoolString
imagePullSecret:
properties:
data:
additionalProperties:
type: string
type: object
name:
type: string
type:
type: string
type: object
images:
items:
type: string
type: array
namespace:
type: string
required:
- images
- namespace
type: object
run:
properties:
args:
items:
type: string
type: array
collectorName:
type: string
command:
items:
type: string
type: array
exclude:
type: BoolString
image:
type: string
imagePullPolicy:
type: string
imagePullSecret:
properties:
data:
additionalProperties:
type: string
type: object
name:
type: string
type:
type: string
type: object
name:
type: string
namespace:
type: string
timeout:
type: string
required:
- image
- namespace
type: object
secret:
properties:
collectorName:
type: string
exclude:
type: BoolString
includeValue:
type: boolean
key:
type: string
name:
type: string
namespace:
type: string
selector:
items:
type: string
type: array
type: object
sysctl:
properties:
collectorName:
type: string
exclude:
type: BoolString
image:
type: string
imagePullPolicy:
type: string
imagePullSecret:
properties:
data:
additionalProperties:
type: string
type: object
name:
type: string
type:
type: string
type: object
name:
type: string
namespace:
type: string
timeout:
type: string
required:
- image
- namespace
type: object
type: object
type: array
type: object
status:
description: CollectorStatus defines the observed state of Collector
type: object
type: object
served: true
storage: true
status:
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+62 -63
View File
@@ -1,10 +1,10 @@
---
apiVersion: apiextensions.k8s.io/v1beta1
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.0
controller-gen.kubebuilder.io/version: v0.7.0
creationTimestamp: null
name: redactors.troubleshoot.sh
spec:
@@ -15,69 +15,68 @@ spec:
plural: redactors
singular: redactor
scope: Namespaced
validation:
openAPIV3Schema:
description: Redactor is the Schema for the redaction API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: RedactorSpec defines the desired state of Redactor
properties:
redactors:
items:
properties:
fileSelector:
properties:
file:
type: string
files:
items:
type: string
type: array
type: object
name:
type: string
removals:
properties:
regex:
items:
properties:
redactor:
type: string
selector:
type: string
type: object
type: array
values:
items:
type: string
type: array
yamlPath:
items:
type: string
type: array
type: object
type: object
type: array
type: object
status:
description: RedactorStatus defines the observed state of Redactor
type: object
type: object
version: v1beta2
versions:
- name: v1beta2
schema:
openAPIV3Schema:
description: Redactor is the Schema for the redaction API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: RedactorSpec defines the desired state of Redactor
properties:
redactors:
items:
properties:
fileSelector:
properties:
file:
type: string
files:
items:
type: string
type: array
type: object
name:
type: string
removals:
properties:
regex:
items:
properties:
redactor:
type: string
selector:
type: string
type: object
type: array
values:
items:
type: string
type: array
yamlPath:
items:
type: string
type: array
type: object
type: object
type: array
type: object
status:
description: RedactorStatus defines the observed state of Redactor
type: object
type: object
served: true
storage: true
status:
@@ -0,0 +1,361 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.7.0
creationTimestamp: null
name: remotecollectors.troubleshoot.sh
spec:
group: troubleshoot.sh
names:
kind: RemoteCollector
listKind: RemoteCollectorList
plural: remotecollectors
singular: remotecollector
scope: Namespaced
versions:
- name: v1beta2
schema:
openAPIV3Schema:
description: RemoteCollector is the Schema for the remote collectors API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: RemoteCollectorSpec defines the desired state of the RemoteCollector
properties:
afterCollection:
items:
properties:
callback:
properties:
method:
type: string
redactUri:
type: string
uri:
type: string
required:
- method
- redactUri
- uri
type: object
uploadResultsTo:
properties:
method:
type: string
redactUri:
type: string
uri:
type: string
required:
- method
- redactUri
- uri
type: object
type: object
type: array
collectors:
items:
properties:
blockDevices:
properties:
collectorName:
type: string
exclude:
type: BoolString
type: object
certificate:
properties:
certificatePath:
type: string
collectorName:
type: string
exclude:
type: BoolString
keyPath:
type: string
required:
- certificatePath
- keyPath
type: object
cpu:
properties:
collectorName:
type: string
exclude:
type: BoolString
type: object
diskUsage:
properties:
collectorName:
type: string
exclude:
type: BoolString
path:
type: string
required:
- path
type: object
filesystemPerformance:
description: RemoteFilesystemPerformance benchmarks sequential
write latency on a single file. The optional background IOPS
feature attempts to mimic real-world conditions by running
read and write workloads prior to and during benchmark execution.
properties:
backgroundIOPSWarmupSeconds:
description: How long to run the background IOPS read and
write workloads prior to starting the benchmarks.
type: integer
backgroundReadIOPS:
description: The target read IOPS to run while benchmarking.
This is a limit and there is no guarantee it will be reached.
This is the total IOPS for all background read jobs.
type: integer
backgroundReadIOPSJobs:
description: Number of threads to use for background read
IOPS. This should be set high enough to reach the target
specified in BackgrounReadIOPS.
type: integer
backgroundWriteIOPS:
description: The target write IOPS to run while benchmarking.
This is a limit and there is no guarantee it will be reached.
This is the total IOPS for all background write jobs.
type: integer
backgroundWriteIOPSJobs:
description: 'Number of threads to use for background write
IOPS. This should be set high enough to reach the target
specified in BackgroundWriteIOPS. Example: If BackgroundWriteIOPS
is 100 and write latency is 10ms then a single job would
barely be able to reach 100 IOPS so this should be at
least 2.'
type: integer
collectorName:
type: string
datasync:
description: Whether to call datasync on the file after
each write. Skipped if Sync is also true. Does not apply
to background IOPS task.
type: boolean
directory:
description: The directory where the benchmark will create
files.
type: string
enableBackgroundIOPS:
description: Enable the background IOPS feature.
type: boolean
exclude:
type: BoolString
fileSize:
description: The size of the file used in the benchmark.
The number of IO operations for the benchmark will be
FileSize / OperationSizeBytes. Accepts valid Kubernetes
resource units such as Mi.
type: string
operationSize:
description: The size of each write operation performed
while benchmarking. This does not apply to the background
IOPS feature if enabled, since those must be fixed at
4096.
format: int64
type: integer
sync:
description: Whether to call sync on the file after each
write. Does not apply to background IOPS task.
type: boolean
timeout:
description: Total timeout, including background IOPS setup
and warmup if enabled.
type: string
required:
- backgroundIOPSWarmupSeconds
- backgroundReadIOPS
- backgroundReadIOPSJobs
- backgroundWriteIOPS
- backgroundWriteIOPSJobs
- enableBackgroundIOPS
type: object
hostServices:
properties:
collectorName:
type: string
exclude:
type: BoolString
type: object
http:
properties:
collectorName:
type: string
exclude:
type: BoolString
get:
properties:
headers:
additionalProperties:
type: string
type: object
insecureSkipVerify:
type: boolean
url:
type: string
required:
- url
type: object
post:
properties:
body:
type: string
headers:
additionalProperties:
type: string
type: object
insecureSkipVerify:
type: boolean
url:
type: string
required:
- url
type: object
put:
properties:
body:
type: string
headers:
additionalProperties:
type: string
type: object
insecureSkipVerify:
type: boolean
url:
type: string
required:
- url
type: object
type: object
httpLoadBalancer:
properties:
address:
type: string
collectorName:
type: string
exclude:
type: BoolString
path:
type: string
port:
type: integer
timeout:
type: string
required:
- address
- path
- port
type: object
ipv4Interfaces:
properties:
collectorName:
type: string
exclude:
type: BoolString
type: object
kernelModules:
properties:
collectorName:
type: string
exclude:
type: BoolString
type: object
memory:
properties:
collectorName:
type: string
exclude:
type: BoolString
type: object
systemPackages:
properties:
collectorName:
type: string
exclude:
type: BoolString
type: object
tcpConnect:
properties:
address:
type: string
collectorName:
type: string
exclude:
type: BoolString
timeout:
type: string
required:
- address
type: object
tcpLoadBalancer:
properties:
address:
type: string
collectorName:
type: string
exclude:
type: BoolString
port:
type: integer
timeout:
type: string
required:
- address
- port
type: object
tcpPortStatus:
properties:
collectorName:
type: string
exclude:
type: BoolString
interface:
type: string
port:
type: integer
required:
- port
type: object
time:
properties:
collectorName:
type: string
exclude:
type: BoolString
type: object
type: object
type: array
nodeSelector:
additionalProperties:
type: string
type: object
type: object
status:
description: CollectorStatus defines the observed state of Collector
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
File diff suppressed because it is too large Load Diff
@@ -15,7 +15,7 @@ spec:
# - name=nginx-ingress-microk8s
# namespace: default
# limits:
# maxAge: 30d
# maxAge: 720h # 30*24
# maxLines: 10000
# - run:
# collectorName: ping-google
+54 -22
View File
@@ -4,18 +4,6 @@ release:
owner: replicatedhq
name: troubleshoot
builds:
- id: so
goos:
- linux
goarch:
- amd64
env:
- CGO_ENABLED=1
- GO111MODULE=on
main: ffi/main.go
flags: -buildmode=c-shared
binary: troubleshoot.so
hooks: {}
- id: preflight
goos:
- linux
@@ -23,6 +11,7 @@ builds:
- windows
goarch:
- amd64
- arm64
env:
- CGO_ENABLED=0
main: cmd/preflight/main.go
@@ -31,7 +20,7 @@ builds:
-X github.com/replicatedhq/troubleshoot/pkg/version.gitSHA={{.Commit}}
-X github.com/replicatedhq/troubleshoot/pkg/version.buildTime={{.Date}}
-extldflags "-static"
flags: -tags netgo -installsuffix netgo
flags: -tags netgo -tags containers_image_ostree_stub -tags exclude_graphdriver_devicemapper -tags exclude_graphdriver_btrfs -tags containers_image_openpgp -installsuffix netgo
binary: preflight
hooks: {}
- id: support-bundle
@@ -41,6 +30,7 @@ builds:
- windows
goarch:
- amd64
- arm64
env:
- CGO_ENABLED=0
main: cmd/troubleshoot/main.go
@@ -49,15 +39,29 @@ builds:
-X github.com/replicatedhq/troubleshoot/pkg/version.gitSHA={{.Commit}}
-X github.com/replicatedhq/troubleshoot/pkg/version.buildTime={{.Date}}
-extldflags "-static"
flags: -tags netgo -installsuffix netgo
flags: -tags netgo -tags containers_image_ostree_stub -tags exclude_graphdriver_devicemapper -tags exclude_graphdriver_btrfs -tags containers_image_openpgp -installsuffix netgo
binary: support-bundle
hooks: {}
- id: collect
goos:
- linux
- darwin
- windows
goarch:
- amd64
- arm64
env:
- CGO_ENABLED=0
main: cmd/collect/main.go
ldflags: -s -w
-X github.com/replicatedhq/troubleshoot/pkg/version.version={{.Version}}
-X github.com/replicatedhq/troubleshoot/pkg/version.gitSHA={{.Commit}}
-X github.com/replicatedhq/troubleshoot/pkg/version.buildTime={{.Date}}
-extldflags "-static"
flags: -tags netgo -tags containers_image_ostree_stub -tags exclude_graphdriver_devicemapper -tags exclude_graphdriver_btrfs -tags containers_image_openpgp -installsuffix netgo
binary: collect
hooks: {}
archives:
- id: so
builds:
- so
format: tar.gz
name_template: 'troubleshoot.so_{{ .Os }}_{{ .Arch }}'
- id: preflight
builds:
- preflight
@@ -75,6 +79,9 @@ archives:
- README*
- changelog*
- CHANGELOG*
- src: 'sbom/assets/*'
dst: .
strip_parent: true # this is needed to make up for the way unzips work in krew v0.4.1
- id: support-bundle
builds:
- support-bundle
@@ -92,6 +99,29 @@ archives:
- README*
- changelog*
- CHANGELOG*
- src: 'sbom/assets/*'
dst: .
strip_parent: true # this is needed to make up for the way unzips work in krew v0.4.1
- id: collect
builds:
- collect
format: tar.gz
format_overrides:
- goos: windows
format: zip
name_template: 'collect_{{ .Os }}_{{ .Arch }}'
files:
- licence*
- LICENCE*
- license*
- LICENSE*
- readme*
- README*
- changelog*
- CHANGELOG*
- src: 'sbom/assets/*'
dst: .
strip_parent: true # this is needed to make up for the way unzips work in krew v0.4.1
dockers:
- dockerfile: ./deploy/Dockerfile.troubleshoot
image_templates:
@@ -99,15 +129,17 @@ dockers:
- "replicated/troubleshoot:{{ .Major }}"
- "replicated/troubleshoot:{{ .Major }}.{{ .Minor }}"
- "replicated/troubleshoot:{{ .Major }}.{{ .Minor }}.{{ .Patch }}"
binaries:
ids:
- support-bundle
- preflight
- collect
- dockerfile: ./deploy/Dockerfile.troubleshoot
image_templates:
- "replicated/preflight:latest"
- "replicated/preflight:{{ .Major }}"
- "replicated/preflight:{{ .Major }}.{{ .Minor }}"
- "replicated/preflight:{{ .Major }}.{{ .Minor }}.{{ .Patch }}"
binaries:
ids:
- support-bundle
- preflight
- preflight
- collect
+2 -1
View File
@@ -3,10 +3,11 @@ WORKDIR /
RUN apt-get -qq update \
&& apt-get -qq -y install \
ca-certificates
ca-certificates kmod
COPY support-bundle /troubleshoot/support-bundle
COPY preflight /troubleshoot/preflight
COPY collect /troubleshoot/collect
ENV PATH="/troubleshoot:${PATH}"
+11
View File
@@ -27,6 +27,17 @@ spec:
- from: LICENSE
to: .
bin: preflight
- selector:
matchLabels:
os: darwin
arch: arm64
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/preflight_darwin_arm64.tar.gz" .TagName }}
files:
- from: preflight
to: .
- from: LICENSE
to: .
bin: preflight
- selector:
matchLabels:
os: windows
+11
View File
@@ -27,6 +27,17 @@ spec:
- from: LICENSE
to: .
bin: support-bundle
- selector:
matchLabels:
os: darwin
arch: arm64
{{addURIAndSha "https://github.com/replicatedhq/troubleshoot/releases/download/{{ .TagName }}/support-bundle_darwin_arm64.tar.gz" .TagName }}
files:
- from: support-bundle
to: .
- from: LICENSE
to: .
bin: support-bundle
- selector:
matchLabels:
os: windows
+8
View File
@@ -0,0 +1,8 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostCollector
metadata:
name: block
spec:
collectors:
- blockDevices: {}
+9
View File
@@ -0,0 +1,9 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostCollector
metadata:
name: certificate
spec:
collectors:
- certificate:
certificatePath: /etc/ssl/corp.crt
keyPath: /etc/ssl/corp.key
+7
View File
@@ -0,0 +1,7 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostCollector
metadata:
name: cpu
spec:
collectors:
- cpu: {}
+9
View File
@@ -0,0 +1,9 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostCollector
metadata:
name: diskUsage
spec:
collectors:
- diskUsage:
collectorName: ephemeral
path: /var/lib/kubelet
@@ -0,0 +1,19 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostCollector
metadata:
name: fsperf
spec:
collectors:
- filesystemPerformance:
collectorName: etcd-perf
timeout: 2m
directory: /var/lib/etcd
fileSize: 22Mi
operationSizeBytes: 2300
datasync: true
enableBackgroundIOPS: true
backgroundIOPSWarmupSeconds: 10
backgroundWriteIOPS: 300
backgroundWriteIOPSJobs: 6
backgroundReadIOPS: 50
backgroundReadIOPSJobs: 1
@@ -0,0 +1,11 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostCollector
metadata:
name: httploadbalancer
spec:
collectors:
- httpLoadBalancer:
collectorName: httploadbalancer
port: 80
address: http://app.corporate.internal
timeout: 10s
+10
View File
@@ -0,0 +1,10 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostCollector
metadata:
name: http
spec:
collectors:
- http:
collectorName: registry
get:
url: https://registry.replicated.com
@@ -0,0 +1,7 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostCollector
metadata:
name: ipv4Interfaces
spec:
collectors:
- ipv4Interfaces: {}
@@ -0,0 +1,7 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostCollector
metadata:
name: modules
spec:
collectors:
- kernelModules: {}
+8
View File
@@ -0,0 +1,8 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostCollector
metadata:
name: memory
spec:
collectors:
- memory:
collectorName: memory
+7
View File
@@ -0,0 +1,7 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostCollector
metadata:
name: ntp
spec:
collectors:
- time: {}
+38
View File
@@ -0,0 +1,38 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostCollector
metadata:
name: example
spec:
collectors:
- blockDevices: {}
- certificate:
certificatePath: /etc/ssl/corp.crt
keyPath: /etc/ssl/corp.key
- cpu: {}
- diskUsage:
collectorName: ephemeral
path: /var/lib/kubelet
- httpLoadBalancer:
collectorName: httploadbalancer
port: 80
address: http://app.corporate.internal
timeout: 10s
- http:
collectorName: registry
get:
url: https://registry.replicated.com
- ipv4Interfaces: {}
- memory: {}
- time: {}
- tcpConnect:
collectorName: weave host 1
address: 10.128.0.2:6783
timeout: 2s
- tcpLoadBalancer:
collectorName: LB1
address: 10.128.0.20:6443
port: 6443
timeout: 5000ms
- tcpPortStatus:
collectorName: k8s
port: 6443
+9
View File
@@ -0,0 +1,9 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostCollector
metadata:
name: connect
spec:
collectors:
- tcpConnect:
collectorName: weave host 1
address: 10.128.0.2:6783
@@ -0,0 +1,10 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostCollector
metadata:
name: loadbalancer
spec:
collectors:
- tcpLoadBalancer:
collectorName: loadbalancer
port: 7443
address: 10.128.0.29:7444
+9
View File
@@ -0,0 +1,9 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostCollector
metadata:
name: port
spec:
collectors:
- tcpPortStatus:
collectorName: k8s
port: 7443
+7
View File
@@ -0,0 +1,7 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostCollector
metadata:
name: timezone
spec:
collectors:
- time: {}
@@ -0,0 +1,8 @@
apiVersion: troubleshoot.sh/v1beta2
kind: RemoteCollector
metadata:
name: block
spec:
collectors:
- blockDevices: {}
+9
View File
@@ -0,0 +1,9 @@
apiVersion: troubleshoot.sh/v1beta2
kind: RemoteCollector
metadata:
name: certificate
spec:
collectors:
- certificate:
certificatePath: /etc/ssl/corp.crt
keyPath: /etc/ssl/corp.key
+7
View File
@@ -0,0 +1,7 @@
apiVersion: troubleshoot.sh/v1beta2
kind: RemoteCollector
metadata:
name: cpu
spec:
collectors:
- cpu: {}
+9
View File
@@ -0,0 +1,9 @@
apiVersion: troubleshoot.sh/v1beta2
kind: RemoteCollector
metadata:
name: diskUsage
spec:
collectors:
- diskUsage:
collectorName: ephemeral
path: /var/lib/kubelet
@@ -0,0 +1,19 @@
apiVersion: troubleshoot.sh/v1beta2
kind: RemoteCollector
metadata:
name: fsperf
spec:
collectors:
- filesystemPerformance:
collectorName: etcd-perf
timeout: 2m
directory: /var/lib/etcd
fileSize: 22Mi
operationSizeBytes: 2300
datasync: true
enableBackgroundIOPS: true
backgroundIOPSWarmupSeconds: 10
backgroundWriteIOPS: 300
backgroundWriteIOPSJobs: 6
backgroundReadIOPS: 50
backgroundReadIOPSJobs: 1
@@ -0,0 +1,11 @@
apiVersion: troubleshoot.sh/v1beta2
kind: RemoteCollector
metadata:
name: httploadbalancer
spec:
collectors:
- httpLoadBalancer:
collectorName: httploadbalancer
port: 80
address: http://app.corporate.internal
timeout: 10s
+10
View File
@@ -0,0 +1,10 @@
apiVersion: troubleshoot.sh/v1beta2
kind: RemoteCollector
metadata:
name: http
spec:
collectors:
- http:
collectorName: registry
get:
url: https://registry.replicated.com
@@ -0,0 +1,7 @@
apiVersion: troubleshoot.sh/v1beta2
kind: RemoteCollector
metadata:
name: ipv4Interfaces
spec:
collectors:
- ipv4Interfaces: {}
@@ -0,0 +1,7 @@
apiVersion: troubleshoot.sh/v1beta2
kind: RemoteCollector
metadata:
name: modules
spec:
collectors:
- kernelModules: {}
+8
View File
@@ -0,0 +1,8 @@
apiVersion: troubleshoot.sh/v1beta2
kind: RemoteCollector
metadata:
name: memory
spec:
collectors:
- memory:
collectorName: memory
+7
View File
@@ -0,0 +1,7 @@
apiVersion: troubleshoot.sh/v1beta2
kind: RemoteCollector
metadata:
name: ntp
spec:
collectors:
- time: {}
+38
View File
@@ -0,0 +1,38 @@
apiVersion: troubleshoot.sh/v1beta2
kind: RemoteCollector
metadata:
name: example
spec:
collectors:
- blockDevices: {}
- certificate:
certificatePath: /etc/ssl/corp.crt
keyPath: /etc/ssl/corp.key
- cpu: {}
- diskUsage:
collectorName: ephemeral
path: /var/lib/kubelet
- httpLoadBalancer:
collectorName: httploadbalancer
port: 80
address: http://app.corporate.internal
timeout: 10s
- http:
collectorName: registry
get:
url: https://registry.replicated.com
- ipv4Interfaces: {}
- memory: {}
- time: {}
- tcpConnect:
collectorName: weave host 1
address: 10.128.0.2:6783
timeout: 2s
- tcpLoadBalancer:
collectorName: LB1
address: 10.128.0.20:6443
port: 6443
timeout: 5000ms
- tcpPortStatus:
collectorName: k8s
port: 6443
+9
View File
@@ -0,0 +1,9 @@
apiVersion: troubleshoot.sh/v1beta2
kind: RemoteCollector
metadata:
name: connect
spec:
collectors:
- tcpConnect:
collectorName: weave host 1
address: 10.128.0.2:6783
@@ -0,0 +1,10 @@
apiVersion: troubleshoot.sh/v1beta2
kind: RemoteCollector
metadata:
name: loadbalancer
spec:
collectors:
- tcpLoadBalancer:
collectorName: loadbalancer
port: 7443
address: 10.128.0.29:7444
+9
View File
@@ -0,0 +1,9 @@
apiVersion: troubleshoot.sh/v1beta2
kind: RemoteCollector
metadata:
name: port
spec:
collectors:
- tcpPortStatus:
collectorName: k8s
port: 7443
+7
View File
@@ -0,0 +1,7 @@
apiVersion: troubleshoot.sh/v1beta2
kind: RemoteCollector
metadata:
name: timezone
spec:
collectors:
- time: {}
@@ -6,26 +6,30 @@ spec:
collectors:
- filesystemPerformance:
collectorName: etcd-perf
timeout: 2m
directory: /var/lib/etcd
fileSize: 22Mi
operationSizeBytes: 2300
datasync: true
enableBackgroundIOPS: true
backgroundIOPSWarmupSeconds: 10
backgroundWriteIOPS: 300
backgroundWriteIOPSJobs: 6
backgroundReadIOPS: 50
backgroundReadIOPSJobs: 1
analyzers:
- filesystemPerformance:
collectorName: etcd-perf
outcomes:
- fail:
when: "iops < 50"
message: Insufficient random read IOPS
- pass:
when: "p99 < 3ms"
message: Write latency is great!
message: "Write latency is great! (p99: {{ .P99 }})"
- pass:
when: "p99 < 5ms"
message: Write latency is ok
message: "Write latency is ok (p99: {{ .P99 }})"
- warn:
when: "p99 < 8ms"
message: Write latency is high
message: "Write latency is high {{ .String }}"
- fail:
when: "p99 >= 8ms"
message: Write latency is too high
message: "Write latency is too high {{ .String }}"
+135
View File
@@ -0,0 +1,135 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostPreflight
metadata:
name: hostos
spec:
collectors:
- hostOS : {}
analyzers:
- hostOS:
outcomes:
- fail:
when: "ubuntu-16.04-kernel < 4.15"
message: unsupported distribution
- pass:
when: "ubuntu-18.04-kernel >= 4.15"
message: supported distribution
- warn:
when: "ubuntu == 16.04"
message: supported distribution
- pass:
when: "ubuntu == 18.04"
message: supported distribution
- pass:
when: "ubuntu == 20.04"
message: supported distribution
- warn:
when: "centos == 7.4"
message: supported distribution
- warn:
when: "centos == 7.5"
message: supported distribution
- warn:
when: "centos == 7.6"
message: supported distribution
- warn:
when: "centos == 7.7"
message: supported distribution
- warn:
when: "centos == 7.8"
message: supported distribution
- pass:
when: "centos == 7.9"
message: supported distribution
- warn:
when: "centos == 8.0"
message: supported distribution
- pass:
when: "centos == 8.1"
message: supported distribution
- pass:
when: "centos == 8.2"
message: supported distribution
- pass:
when: "centos == 8.3"
message: supported distribution
- pass:
when: "centos == 8.4"
message: supported distribution
- fail:
when: "centos > 8.4"
message: unsupported distribution
- warn:
when: "rhel == 7.4"
message: supported distribution
- warn:
when: "rhel == 7.5"
message: supported distribution
- warn:
when: "rhel == 7.6"
message: supported distribution
- warn:
when: "rhel == 7.7"
message: supported distribution
- warn:
when: "rhel == 7.8"
message: supported distribution
- pass:
when: "rhel == 7.9"
message: supported distribution
- warn:
when: "rhel == 8.0"
message: supported distribution
- pass:
when: "rhel == 8.1"
message: supported distribution
- pass:
when: "rhel == 8.2"
message: supported distribution
- pass:
when: "rhel == 8.3"
message: supported distribution
- pass:
when: "rhel == 8.4"
message: supported distribution
- fail:
when: "rhel > 8.4"
message: unsupported distribution
- warn:
when: "ol == 7.4"
message: supported distribution
- warn:
when: "ol == 7.5"
message: supported distribution
- warn:
when: "ol == 7.6"
message: supported distribution
- warn:
when: "ol == 7.7"
message: supported distribution
- warn:
when: "ol == 7.8"
message: supported distribution
- pass:
when: "ol == 7.9"
message: supported distribution
- warn:
when: "ol == 8.0"
message: supported distribution
- pass:
when: "ol == 8.1"
message: supported distribution
- pass:
when: "ol == 8.2"
message: supported distribution
- pass:
when: "ol == 8.3"
message: supported distribution
- pass:
when: "ol == 8.4"
message: supported distribution
- fail:
when: "ol > 8.4"
message: unsupported distribution
- fail:
message: unsupported distribution
@@ -0,0 +1,25 @@
apiVersion: troubleshoot.sh/v1beta2
kind: HostPreflight
metadata:
name: modules
spec:
collectors:
- kernelModules: {}
analyzers:
- kernelModules:
outcomes:
- fail:
when: "target_core_mod != loaded,loadable"
message: The 'target_core_mod' kernel module is not loaded or loadable
- fail:
when: "target_core_file != loaded,loadable"
message: The 'target_core_file' kernel module is not loaded or loadable
- fail:
when: "tcm_loop != loaded,loadable"
message: The 'tcm_loop' kernel module is not loaded or loadable
- warn:
when: "nvme != loaded"
message: The system is not using NVME storage, which will provide better performance
- pass:
when: "target_core_mod,target_core_file,tcm_loop == loaded,loadable"
message: The 'target_core_mod', target_core_file', and 'tcm_loop' kernel modules are loaded or loadable
+7
View File
@@ -17,6 +17,13 @@ spec:
- warn:
when: "ntp == synchronized+inactive"
message: NTP not active
- warn:
when: "timezone != UTC"
message: "Non UTC timezone can interfere with system function"
- pass:
when: "ntp == synchronized+active"
message: System clock is synchronized
- pass:
when: "timezone == UTC"
message: "timezone set to UTC"
+1
View File
@@ -235,4 +235,5 @@ spec:
when: "timezone == UTC"
message: Timezone is UTC
- fail:
when: "timezone != UTC"
message: Timezone is not UTC

Some files were not shown because too many files have changed in this diff Show More