Compare commits

..

20 Commits

Author SHA1 Message Date
Igor Gov
afa81c7ec2 Fixing bad conflict resolution 2021-08-19 13:33:14 +03:00
Igor Gov
e84c7d3310 Merge branch 'develop' 2021-08-19 13:18:06 +03:00
Igor Gov
7d0a90cb78 Merge branch 'main' into develop
# Conflicts:
#	cli/config/configStruct.go
#	cli/mizu/config.go
#	tap/http_reader.go
2021-08-19 13:16:19 +03:00
Nimrod Gilboa Markevich
24f79922e9 Add to periodic stats print in tapper (#221)
#patch
2021-08-16 15:50:04 +03:00
RoyUP9
c3995009ee Hotfix - ignore not allowed set flags (#192)
#patch
2021-08-10 14:21:16 +03:00
RoyUP9
6e9fe2986e removed duplicate har page header (#187) 2021-08-09 13:31:53 +03:00
RoyUP9
603240fedb temp fix - ignore agent image in config command (#185) 2021-08-09 11:55:45 +03:00
Igor Gov
e61871a68e Merge pull request #182 from up9inc/develop
Release 2021-08-08
2021-08-08 14:50:30 +03:00
nimrod-up9
379af59f07 Merge pull request #121 from up9inc/develop
Missing request body (#120)
2021-07-19 13:53:49 +03:00
gadotroee
ef9afe31a4 Merge pull request #119 from up9inc/develop
Mizu release
2021-07-18 16:54:31 +03:00
gadotroee
dca636b0fd Merge pull request #94 from up9inc/develop
Mizu release
2021-07-06 21:05:40 +03:00
Roee Gadot
9b72cc7aa6 Merge branch 'develop' into main
# Conflicts:
#	README.md
#	api/main.go
#	api/pkg/api/main.go
#	api/pkg/models/models.go
#	api/pkg/resolver/resolver.go
#	cli/Makefile
#	cli/cmd/tap.go
#	cli/cmd/tapRunner.go
#	tap/http_matcher.go
#	tap/http_reader.go
#	tap/tcp_stream_factory.go
2021-06-29 11:16:47 +03:00
Alex Haiut
d3c023b3ba mizu release 2021-06-21 (#79)
* Show pod name and namespace (#61)

* WIP

* Update main.go, consts.go, and 2 more files...

* Update messageSensitiveDataCleaner.go

* Update consts.go and messageSensitiveDataCleaner.go

* Update messageSensitiveDataCleaner.go

* Update main.go, consts.go, and 3 more files...

* WIP

* Update main.go, messageSensitiveDataCleaner.go, and 6 more files...

* Update main.go, messageSensitiveDataCleaner.go, and 3 more files...

* Update consts.go, messageSensitiveDataCleaner.go, and tap.go

* Update provider.go

* Update serializableRegexp.go

* Update tap.go

* TRA-3234 fetch with _source + no hard limit (#64)

* remove the HARD limit of 5000

* TRA-3299 Reduce footprint and Add Tolerances(#65)

* Use lib const for DNSClusterFirstWithHostNet.

* Whitespace.

* Break lines.

* Added affinity to pod names.

* Added tolerations to NoExecute and NoSchedule taints.

* Implementation of Mizu view command

* .

* .

* Update main.go and messageSensitiveDataCleaner.go

* Update main.go

* String and not pointers (#68)

* TRA-3318 - Cookies not null and fix har file names  (#69)

* no message

* TRA-3212 Passive-Tapper and Mizu share code (#70)

* Use log in tap package instead of fmt.

* Moved api/pkg/tap to root.

* Added go.mod and go.sum for tap.

* Added replace for shared.

* api uses tap module instead of tap package.

* Removed dependency of tap in shared by moving env var out of tap.

* Fixed compilation bugs.

* Fixed: Forgot to export struct field HostMode.

* Removed unused flag.

* Close har output channel when done.

* Moved websocket out of mizu and into passive-tapper.

* Send connection details over har output channel.

* Fixed compilation errors.

* Removed unused info from request response cache.

* Renamed connection -> connectionID.

* Fixed rename bug.

* Export setters and getters for filter ips and ports.

* Added tap dependency to Dockerfile.

* Uncomment error messages.

* Renamed `filterIpAddresses` -> `filterAuthorities`.

* Renamed ConnectionID -> ConnectionInfo.

* Fixed: Missed one replace.

* TRA-3342 Mizu/tap dump to har directory fails on Linux (#71)

* Instead of saving incomplete temp har files in a temp dir, save them in the output dir with a *.har.tmp suffix.

* API only loads har from *.har files (by extension).

* Add export entries endpoint for better up9 connect funcionality  (#72)

* no message
* no message
* no message

* Filter 'cookie' header

* Release action  (#73)

* Create main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* trying new approach

* no message

* yaml error

* no message

* no message

* no message

* missing )

* no message

* no message

* remove main.yml and fix branches

* Create tag-temp.yaml

* Update tag-temp.yaml

* Update tag-temp.yaml

* no message

* no message

* no message

* no message

* no message

* no message

* no message

* #minor

* no message

* no message

* added checksum calc to CLI makefile

* fixed build error - created bin directory upfront

* using markdown for release text

* use separate checksum files

* fixed release readme

* #minor

* readme updated

Co-authored-by: Alex Haiut <alex@up9.com>

* TRA-3360 Fix: Mizu ignores -n namespace flag and records traffic from all pods (#75)

Do not tap pods in namespaces which were not requested.

* added apple/m1 binary, updated readme (#77)

Co-authored-by: Alex Haiut <alex@up9.com>

* Update README.md (#78)

Co-authored-by: lirazyehezkel <61656597+lirazyehezkel@users.noreply.github.com>
Co-authored-by: RamiBerm <rami.berman@up9.com>
Co-authored-by: RamiBerm <54766858+RamiBerm@users.noreply.github.com>
Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
Co-authored-by: nimrod-up9 <59927337+nimrod-up9@users.noreply.github.com>
Co-authored-by: Igor Gov <igor.govorov1@gmail.com>
Co-authored-by: Alex Haiut <alex@up9.com>
2021-06-21 15:17:31 +03:00
Roee Gadot
5f2a4deb19 remove file 2021-05-26 18:08:37 +03:00
Roee Gadot
91f290987e Merge branch 'develop' into main
# Conflicts:
#	cli/cmd/tap.go
#	cli/cmd/version.go
#	cli/kubernetes/provider.go
#	cli/mizu/consts.go
#	cli/mizu/mizuRunner.go
#	debug.Dockerfile
#	ui/src/components/HarPage.tsx
2021-05-26 17:58:17 +03:00
gadotroee
2f3215b71a Fix mizu image parameter (#53) 2021-05-23 13:34:32 +03:00
Alex Haiut
2e87a01346 end of week - develop to master (#50)
* Provide cli version as git hash from makefile

* Update Makefile, version.go, and 3 more files...

* Update mizuRunner.go

* Update README.md, resolver.go, and 2 more files...

* Update provider.go

* Feature/UI/light theme (#44)

* light theme

* css polish

* unused code

* css

* text shadow

* footer style

* Update mizuRunner.go

* Handle nullable vars (#47)

* Decode gRPC body (#48)

* Decode grpc.

* Better variable names.

* Added protobuf-decoder dependency.

* Updated protobuf-decoder's version.

Co-authored-by: RamiBerm <rami.berman@up9.com>
Co-authored-by: RamiBerm <54766858+RamiBerm@users.noreply.github.com>
Co-authored-by: lirazyehezkel <61656597+lirazyehezkel@users.noreply.github.com>
Co-authored-by: nimrod-up9 <59927337+nimrod-up9@users.noreply.github.com>
2021-05-13 20:29:31 +03:00
gadotroee
453003bf14 remove leftovers (#43) 2021-05-10 17:35:59 +03:00
Roee Gadot
80ca377668 Merge branch 'develop' into main
# Conflicts:
#	Dockerfile
#	Makefile
#	api/go.mod
#	api/go.sum
#	api/main.go
#	api/pkg/controllers/entries_controller.go
#	api/pkg/inserter/main.go
#	api/pkg/models/models.go
#	api/pkg/tap/grpc_assembler.go
#	api/pkg/tap/har_writer.go
#	api/pkg/tap/http_matcher.go
#	api/pkg/tap/http_reader.go
#	api/pkg/tap/passive_tapper.go
#	api/pkg/utils/utils.go
#	cli/Makefile
#	cli/cmd/tap.go
#	cli/cmd/version.go
#	cli/config/config.go
#	cli/kubernetes/provider.go
#	cli/mizu/mizuRunner.go
2021-05-10 17:27:32 +03:00
gadotroee
d21297bc9c 0.9 (#37)
* Update .gitignore

* WIP

* WIP

* Update README.md, root.go, and 4 more files...

* Update README.md

* Update README.md

* Update root.go

* Update provider.go

* Update provider.go

* Update root.go, go.mod, and go.sum

* Update mizu.go

* Update go.sum and provider.go

* Update portForward.go, watch.go, and mizu.go

* Update README.md

* Update watch.go

* Update mizu.go

* Update mizu.go

* no message

* no message

* remove unused things and use external for object id (instead of copy)

* no message

* Update mizu.go

* Update go.mod, go.sum, and 2 more files...

* no message

* Update README.md, go.mod, and resolver.go

* Update README.md

* Update go.mod

* Update loader.go

* some refactor

* Update loader.go

* no message

* status to statusCode

* return data directly

* Traffic viewer

* cleaning

* css

* no message

* Clean warnings

* Makefile - first draft

* Update Makefile

* Update Makefile

* Update Makefile, README.md, and 4 more files...

* Add api build and clean to makefile (files restructure) (#9)

* no message
* add clean api command

* no message

* stating with web socket

* Add tap as a separate executable (#10)

* Added tap.

* Ignore build directories.

* Added tapper build to Makefile.

* Improvements  (#12)

* no message

* no message

* Feature/makefile (#11)

* minor fixes

* makefile fixes - docker build

* minor fix in Makefile
Co-authored-by: Alex Haiut <alex@up9.com>

* Update Dockerfile, multi-runner.sh, and 31 more files...

* Update multi-runner.sh

* no message

* Update .dockerignore, Dockerfile, and 30 more files...

* Update cleaner.go, grpc_assembler.go, and 2 more files...

* start the pod with host network and privileged

* fix multi runner passive tapper command

* add HOST_MODE env var

* do not return true in the should tap function

* remove line in the end

* default value in api is input
fix description and pass the parameter in the multi runner script

* missing flag.parse

* no message

* fix image

* Create main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Small fixes - permission + har writing exception (#17)

* Select node by pod (#18)

* Select node by pod.

* Removed watch pod by regex. Irrelevant for now.

* Changed default image to develop:latest.

* Features/clifix (#19)

* makefile fixes - docker build

* readme update, CLI usage fix

* added chmod

Co-authored-by: Alex Haiut <alex@up9.com>

* meta information

* Only record traffic of the requested pod. Filtered by pod IP. (#21)

* fixed readme and reduced batch size to 5 (#22)

Co-authored-by: Alex Haiut <alex@up9.com>

* API and TAP in single process  (#24)

* no message
* no message

* CLI make --pod required flag and faster api image build (#25)

* makefile fixes - docker build

* readme update, CLI usage fix

* added chmod

* typo

* run example incorreect in makefile

* no message

* no message

* no message

Co-authored-by: Alex Haiut <alex@up9.com>

* Reduce delay between tap and UI - Skip dump to file (#26)

* Pass HARs between tap and api via channel.

* Fixed make docker commad.

* Various fixes.

* Added .DS_Store to .gitignore.

* Parse flags in Mizu main instead of in tap_output.go.

* Use channel to pass HAR by default instead of files.

* Infinite scroll (#28)

* no message

* infinite scroll + new ws implementation

* no message

* scrolling top

* fetch button

* more Backend changes

* fix go mod and sum

* mire fixes against develop

* unused code

* small ui refactor

Co-authored-by: Roee Gadot <roee.gadot@up9.com>

* Fix gRPC crash, display gRPC as base64, display gRPC URL and status code (#27)

* Added Method (POST) and URL (emtpy) to gRPC requests.

* Removed quickfix that skips writing HTTP/2 to HAR.

* Use HTTP/2 body to fill out http.Request and htt.Response.

* Make sure that in HARs request.postData.mimeType and response.content.mimeType are application/grpc in case of grpc.

* Comment.

* Add URL and status code for gRPC.

* Don't assume http scheme.

* Use http.Header.Set instead of manually acccessing the underlaying map.

* General stats api fix  (#29)

* refactor and validation

* Show gRPC as ASCII (#31)

* Moved try-catch up one block.

* Display grpc as ASCII.

* Better code in entries fetch endpoint (#30)

* no message
* no message

* Feature/UI/filters (#32)

* UI filters

* refactor

* Revert "refactor"

This reverts commit 70e7d4b6ac.

* remove recursive func

* CLI cleanup (#33)

* Moved cli root command to tap subcommand.

* tap subcommand works.

* Added view and fetch placeholders.

* Updated descriptions.

* Fixed indentation.

* Added versio subcommand.

* Removed version flag.

* gofmt.

* Changed pod from flag to arg.

* Commented out "all namespaces" flag.

* CLI cleanup 2 (#34)

* Renamed dashboard -> GUI/web interface.

* Commented out --quiet, removed unused config variables.

* Quiter output when calling unimplemented subcommands.

* Leftovers from PR #30 (#36)

Co-authored-by: up9-github <info@up9.com>
Co-authored-by: RamiBerm <54766858+RamiBerm@users.noreply.github.com>
Co-authored-by: Liraz Yehezkel <lirazy@up9.com>
Co-authored-by: Alex Haiut <alex@testr.io>
Co-authored-by: lirazyehezkel <61656597+lirazyehezkel@users.noreply.github.com>
Co-authored-by: Alex Haiut <alex@up9.com>
Co-authored-by: nimrod-up9 <59927337+nimrod-up9@users.noreply.github.com>
Co-authored-by: RamiBerm <rami.berman@up9.com>
Co-authored-by: Alex Haiut <alex.haiut@gmail.com>
2021-05-09 11:45:39 +03:00
31 changed files with 413 additions and 1662 deletions

View File

@@ -1,15 +1,9 @@
name: PR validation
on:
pull_request:
branches:
- 'develop'
- 'main'
concurrency:
group: mizu-pr-validation-${{ github.ref }}
cancel-in-progress: true
jobs:
build-cli:
name: Build CLI
@@ -44,3 +38,43 @@ jobs:
- name: Build Agent
run: make agent
run-tests-cli:
name: Run CLI tests
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.16
uses: actions/setup-go@v2
with:
go-version: '^1.16'
- name: Check out code into the Go module directory
uses: actions/checkout@v2
- name: Test
run: make test-cli
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v2
run-tests-agent:
name: Run Agent tests
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.16
uses: actions/setup-go@v2
with:
go-version: '^1.16'
- name: Check out code into the Go module directory
uses: actions/checkout@v2
- shell: bash
run: |
sudo apt-get install libpcap-dev
- name: Test
run: make test-agent
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v2

View File

@@ -1,56 +0,0 @@
name: tests validation
on:
pull_request:
branches:
- 'develop'
- 'main'
push:
branches:
- 'develop'
- 'main'
concurrency:
group: mizu-tests-validation-${{ github.ref }}
cancel-in-progress: true
jobs:
run-tests-cli:
name: Run CLI tests
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.16
uses: actions/setup-go@v2
with:
go-version: '^1.16'
- name: Check out code into the Go module directory
uses: actions/checkout@v2
- name: Test
run: make test-cli
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v2
run-tests-agent:
name: Run Agent tests
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.16
uses: actions/setup-go@v2
with:
go-version: '^1.16'
- name: Check out code into the Go module directory
uses: actions/checkout@v2
- shell: bash
run: |
sudo apt-get install libpcap-dev
- name: Test
run: make test-agent
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v2

View File

@@ -2,9 +2,7 @@
# The API Traffic Viewer for Kubernetes
A simple-yet-powerful API traffic viewer for Kubernetes enabling you to view all API communication between microservices to help your debug and troubleshoot regressions.
Think TCPDump and Chrome Dev Tools combined.
A simple-yet-powerful API traffic viewer for Kubernetes to help you troubleshoot and debug your microservices. Think TCPDump and Chrome Dev Tools combined
![Simple UI](assets/mizu-ui.png)
@@ -40,10 +38,8 @@ SHA256 checksums are available on the [Releases](https://github.com/up9inc/mizu/
### Development (unstable) Build
Pick one from the [Releases](https://github.com/up9inc/mizu/releases) page
## Kubeconfig & Permissions
While `mizu`most often works out of the box, you can influence its behavior:
1. [OPTIONAL] Set `KUBECONFIG` environment variable to your Kubernetes configuration. If this is not set, Mizu assumes that configuration is at `${HOME}/.kube/config`
## Prerequisites
1. Set `KUBECONFIG` environment variable to your Kubernetes configuration. If this is not set, Mizu assumes that configuration is at `${HOME}/.kube/config`
2. `mizu` assumes user running the command has permissions to create resources (such as pods, services, namespaces) on your Kubernetes cluster (no worries - `mizu` resources are cleaned up upon termination)
For detailed list of k8s permissions see [PERMISSIONS](PERMISSIONS.md) document

View File

@@ -1,2 +1,2 @@
test: ## Run acceptance tests.
@go test ./... -timeout 1h
@go test ./...

View File

@@ -1,283 +0,0 @@
package acceptanceTests
import (
"fmt"
"gopkg.in/yaml.v3"
"io/ioutil"
"os"
"os/exec"
"testing"
)
type tapConfig struct {
GuiPort uint16 `yaml:"gui-port"`
}
type configStruct struct {
Tap tapConfig `yaml:"tap"`
}
func TestConfigRegenerate(t *testing.T) {
if testing.Short() {
t.Skip("ignored acceptance test")
}
cliPath, cliPathErr := getCliPath()
if cliPathErr != nil {
t.Errorf("failed to get cli path, err: %v", cliPathErr)
return
}
configPath, configPathErr := getConfigPath()
if configPathErr != nil {
t.Errorf("failed to get config path, err: %v", cliPathErr)
return
}
configCmdArgs := getDefaultConfigCommandArgs()
configCmdArgs = append(configCmdArgs, "-r")
configCmd := exec.Command(cliPath, configCmdArgs...)
t.Logf("running command: %v", configCmd.String())
t.Cleanup(func() {
if err := os.Remove(configPath); err != nil {
t.Logf("failed to delete config file, err: %v", err)
}
})
if err := configCmd.Start(); err != nil {
t.Errorf("failed to start config command, err: %v", err)
return
}
if err := configCmd.Wait(); err != nil {
t.Errorf("failed to wait config command, err: %v", err)
return
}
_, readFileErr := ioutil.ReadFile(configPath)
if readFileErr != nil {
t.Errorf("failed to read config file, err: %v", readFileErr)
return
}
}
func TestConfigGuiPort(t *testing.T) {
if testing.Short() {
t.Skip("ignored acceptance test")
}
tests := []uint16{8898}
for _, guiPort := range tests {
t.Run(fmt.Sprintf("%d", guiPort), func(t *testing.T) {
cliPath, cliPathErr := getCliPath()
if cliPathErr != nil {
t.Errorf("failed to get cli path, err: %v", cliPathErr)
return
}
configPath, configPathErr := getConfigPath()
if configPathErr != nil {
t.Errorf("failed to get config path, err: %v", cliPathErr)
return
}
config := configStruct{}
config.Tap.GuiPort = guiPort
configBytes, marshalErr := yaml.Marshal(config)
if marshalErr != nil {
t.Errorf("failed to marshal config, err: %v", marshalErr)
return
}
if writeErr := ioutil.WriteFile(configPath, configBytes, 0644); writeErr != nil {
t.Errorf("failed to write config to file, err: %v", writeErr)
return
}
tapCmdArgs := getDefaultTapCommandArgs()
tapNamespace := getDefaultTapNamespace()
tapCmdArgs = append(tapCmdArgs, tapNamespace...)
tapCmd := exec.Command(cliPath, tapCmdArgs...)
t.Logf("running command: %v", tapCmd.String())
t.Cleanup(func() {
if err := cleanupCommand(tapCmd); err != nil {
t.Logf("failed to cleanup tap command, err: %v", err)
}
if err := os.Remove(configPath); err != nil {
t.Logf("failed to delete config file, err: %v", err)
}
})
if err := tapCmd.Start(); err != nil {
t.Errorf("failed to start tap command, err: %v", err)
return
}
apiServerUrl := getApiServerUrl(guiPort)
if err := waitTapPodsReady(apiServerUrl); err != nil {
t.Errorf("failed to start tap pods on time, err: %v", err)
return
}
})
}
}
func TestConfigSetGuiPort(t *testing.T) {
if testing.Short() {
t.Skip("ignored acceptance test")
}
tests := []struct {
ConfigFileGuiPort uint16
SetGuiPort uint16
}{
{ConfigFileGuiPort: 8898, SetGuiPort: 8897},
}
for _, guiPortStruct := range tests {
t.Run(fmt.Sprintf("%d", guiPortStruct.SetGuiPort), func(t *testing.T) {
cliPath, cliPathErr := getCliPath()
if cliPathErr != nil {
t.Errorf("failed to get cli path, err: %v", cliPathErr)
return
}
configPath, configPathErr := getConfigPath()
if configPathErr != nil {
t.Errorf("failed to get config path, err: %v", cliPathErr)
return
}
config := configStruct{}
config.Tap.GuiPort = guiPortStruct.ConfigFileGuiPort
configBytes, marshalErr := yaml.Marshal(config)
if marshalErr != nil {
t.Errorf("failed to marshal config, err: %v", marshalErr)
return
}
if writeErr := ioutil.WriteFile(configPath, configBytes, 0644); writeErr != nil {
t.Errorf("failed to write config to file, err: %v", writeErr)
return
}
tapCmdArgs := getDefaultTapCommandArgs()
tapNamespace := getDefaultTapNamespace()
tapCmdArgs = append(tapCmdArgs, tapNamespace...)
tapCmdArgs = append(tapCmdArgs, "--set", fmt.Sprintf("tap.gui-port=%v", guiPortStruct.SetGuiPort))
tapCmd := exec.Command(cliPath, tapCmdArgs...)
t.Logf("running command: %v", tapCmd.String())
t.Cleanup(func() {
if err := cleanupCommand(tapCmd); err != nil {
t.Logf("failed to cleanup tap command, err: %v", err)
}
if err := os.Remove(configPath); err != nil {
t.Logf("failed to delete config file, err: %v", err)
}
})
if err := tapCmd.Start(); err != nil {
t.Errorf("failed to start tap command, err: %v", err)
return
}
apiServerUrl := getApiServerUrl(guiPortStruct.SetGuiPort)
if err := waitTapPodsReady(apiServerUrl); err != nil {
t.Errorf("failed to start tap pods on time, err: %v", err)
return
}
})
}
}
func TestConfigFlagGuiPort(t *testing.T) {
if testing.Short() {
t.Skip("ignored acceptance test")
}
tests := []struct {
ConfigFileGuiPort uint16
FlagGuiPort uint16
}{
{ConfigFileGuiPort: 8898, FlagGuiPort: 8896},
}
for _, guiPortStruct := range tests {
t.Run(fmt.Sprintf("%d", guiPortStruct.FlagGuiPort), func(t *testing.T) {
cliPath, cliPathErr := getCliPath()
if cliPathErr != nil {
t.Errorf("failed to get cli path, err: %v", cliPathErr)
return
}
configPath, configPathErr := getConfigPath()
if configPathErr != nil {
t.Errorf("failed to get config path, err: %v", cliPathErr)
return
}
config := configStruct{}
config.Tap.GuiPort = guiPortStruct.ConfigFileGuiPort
configBytes, marshalErr := yaml.Marshal(config)
if marshalErr != nil {
t.Errorf("failed to marshal config, err: %v", marshalErr)
return
}
if writeErr := ioutil.WriteFile(configPath, configBytes, 0644); writeErr != nil {
t.Errorf("failed to write config to file, err: %v", writeErr)
return
}
tapCmdArgs := getDefaultTapCommandArgs()
tapNamespace := getDefaultTapNamespace()
tapCmdArgs = append(tapCmdArgs, tapNamespace...)
tapCmdArgs = append(tapCmdArgs, "-p", fmt.Sprintf("%v", guiPortStruct.FlagGuiPort))
tapCmd := exec.Command(cliPath, tapCmdArgs...)
t.Logf("running command: %v", tapCmd.String())
t.Cleanup(func() {
if err := cleanupCommand(tapCmd); err != nil {
t.Logf("failed to cleanup tap command, err: %v", err)
}
if err := os.Remove(configPath); err != nil {
t.Logf("failed to delete config file, err: %v", err)
}
})
if err := tapCmd.Start(); err != nil {
t.Errorf("failed to start tap command, err: %v", err)
return
}
apiServerUrl := getApiServerUrl(guiPortStruct.FlagGuiPort)
if err := waitTapPodsReady(apiServerUrl); err != nil {
t.Errorf("failed to start tap pods on time, err: %v", err)
return
}
})
}
}

View File

@@ -1,5 +1,3 @@
module github.com/up9inc/mizu/tests
go 1.16
require gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b

View File

@@ -1,4 +0,0 @@
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -26,21 +26,14 @@ fi
echo "Starting minikube..."
minikube start
echo "Creating mizu tests namespaces"
echo "Creating mizu tests namespace"
kubectl create namespace mizu-tests
kubectl create namespace mizu-tests2
echo "Creating httpbin deployments"
echo "Creating httpbin deployment"
kubectl create deployment httpbin --image=kennethreitz/httpbin -n mizu-tests
kubectl create deployment httpbin2 --image=kennethreitz/httpbin -n mizu-tests
kubectl create deployment httpbin --image=kennethreitz/httpbin -n mizu-tests2
echo "Creating httpbin services"
echo "Creating httpbin service"
kubectl expose deployment httpbin --type=NodePort --port=80 -n mizu-tests
kubectl expose deployment httpbin2 --type=NodePort --port=80 -n mizu-tests
kubectl expose deployment httpbin --type=NodePort --port=80 -n mizu-tests2
echo "Starting proxy"
kubectl proxy --port=8080 &

View File

@@ -1,13 +1,9 @@
package acceptanceTests
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"os/exec"
"strings"
"testing"
"time"
)
@@ -17,26 +13,22 @@ func TestTapAndFetch(t *testing.T) {
t.Skip("ignored acceptance test")
}
tests := []int{50}
tests := []int{1, 100}
for _, entriesCount := range tests {
t.Run(fmt.Sprintf("%d", entriesCount), func(t *testing.T) {
cliPath, cliPathErr := getCliPath()
cliPath, cliPathErr := GetCliPath()
if cliPathErr != nil {
t.Errorf("failed to get cli path, err: %v", cliPathErr)
return
}
tapCmdArgs := getDefaultTapCommandArgs()
tapNamespace := getDefaultTapNamespace()
tapCmdArgs = append(tapCmdArgs, tapNamespace...)
tapCmdArgs := GetDefaultTapCommandArgs()
tapCmd := exec.Command(cliPath, tapCmdArgs...)
t.Logf("running command: %v", tapCmd.String())
t.Cleanup(func() {
if err := cleanupCommand(tapCmd); err != nil {
if err := CleanupCommand(tapCmd); err != nil {
t.Logf("failed to cleanup tap command, err: %v", err)
}
})
@@ -46,709 +38,88 @@ func TestTapAndFetch(t *testing.T) {
return
}
apiServerUrl := getApiServerUrl(defaultApiServerPort)
time.Sleep(30 * time.Second)
if err := waitTapPodsReady(apiServerUrl); err != nil {
t.Errorf("failed to start tap pods on time, err: %v", err)
return
}
proxyUrl := getProxyUrl(defaultNamespaceName, defaultServiceName)
proxyUrl := "http://localhost:8080/api/v1/namespaces/mizu-tests/services/httpbin/proxy/get"
for i := 0; i < entriesCount; i++ {
if _, requestErr := executeHttpGetRequest(fmt.Sprintf("%v/get", proxyUrl)); requestErr != nil {
if _, requestErr := ExecuteHttpRequest(proxyUrl); requestErr != nil {
t.Errorf("failed to send proxy request, err: %v", requestErr)
return
}
}
entriesCheckFunc := func() error {
timestamp := time.Now().UnixNano() / int64(time.Millisecond)
time.Sleep(5 * time.Second)
timestamp := time.Now().UnixNano() / int64(time.Millisecond)
entriesUrl := fmt.Sprintf("%v/api/entries?limit=%v&operator=lt&timestamp=%v", apiServerUrl, entriesCount, timestamp)
requestResult, requestErr := executeHttpGetRequest(entriesUrl)
if requestErr != nil {
return fmt.Errorf("failed to get entries, err: %v", requestErr)
}
entries := requestResult.([]interface{})
if len(entries) == 0 {
return fmt.Errorf("unexpected entries result - Expected more than 0 entries")
}
entry := entries[0].(map[string]interface{})
entryUrl := fmt.Sprintf("%v/api/entries/%v", apiServerUrl, entry["id"])
requestResult, requestErr = executeHttpGetRequest(entryUrl)
if requestErr != nil {
return fmt.Errorf("failed to get entry, err: %v", requestErr)
}
if requestResult == nil {
return fmt.Errorf("unexpected nil entry result")
}
return nil
}
if err := retriesExecute(shortRetriesCount, entriesCheckFunc); err != nil {
t.Errorf("%v", err)
entriesUrl := fmt.Sprintf("http://localhost:8899/mizu/api/entries?limit=%v&operator=lt&timestamp=%v", entriesCount, timestamp)
requestResult, requestErr := ExecuteHttpRequest(entriesUrl)
if requestErr != nil {
t.Errorf("failed to get entries, err: %v", requestErr)
return
}
fetchCmdArgs := getDefaultFetchCommandArgs()
entries, ok := requestResult.([]interface{})
if !ok {
t.Errorf("invalid entries type")
return
}
if len(entries) != entriesCount {
t.Errorf("unexpected entries result - Expected: %v, actual: %v", entriesCount, len(entries))
return
}
entry, ok := entries[0].(map[string]interface{})
if !ok {
t.Errorf("invalid entry type")
return
}
entryUrl := fmt.Sprintf("http://localhost:8899/mizu/api/entries/%v", entry["id"])
requestResult, requestErr = ExecuteHttpRequest(entryUrl)
if requestErr != nil {
t.Errorf("failed to get entry, err: %v", requestErr)
return
}
if requestResult == nil {
t.Errorf("unexpected nil entry result")
return
}
fetchCmdArgs := GetDefaultFetchCommandArgs()
fetchCmd := exec.Command(cliPath, fetchCmdArgs...)
t.Logf("running command: %v", fetchCmd.String())
t.Cleanup(func() {
if err := CleanupCommand(fetchCmd); err != nil {
t.Logf("failed to cleanup fetch command, err: %v", err)
}
})
if err := fetchCmd.Start(); err != nil {
t.Errorf("failed to start fetch command, err: %v", err)
return
}
harCheckFunc := func() error {
harBytes, readFileErr := ioutil.ReadFile("./unknown_source.har")
if readFileErr != nil {
return fmt.Errorf("failed to read har file, err: %v", readFileErr)
}
time.Sleep(5 * time.Second)
harEntries, err := getEntriesFromHarBytes(harBytes)
if err != nil {
return fmt.Errorf("failed to get entries from har, err: %v", err)
}
if len(harEntries) == 0 {
return fmt.Errorf("unexpected har entries result - Expected more than 0 entries")
}
return nil
harBytes, readFileErr := ioutil.ReadFile("./unknown_source.har")
if readFileErr != nil {
t.Errorf("failed to read har file, err: %v", readFileErr)
return
}
if err := retriesExecute(shortRetriesCount, harCheckFunc); err != nil {
t.Errorf("%v", err)
harEntries, err := GetEntriesFromHarBytes(harBytes)
if err != nil {
t.Errorf("failed to get entries from har, err: %v", err)
return
}
if len(harEntries) != entriesCount {
t.Errorf("unexpected har entries result - Expected: %v, actual: %v", entriesCount, len(harEntries))
return
}
})
}
}
func TestTapGuiPort(t *testing.T) {
if testing.Short() {
t.Skip("ignored acceptance test")
}
tests := []uint16{8898}
for _, guiPort := range tests {
t.Run(fmt.Sprintf("%d", guiPort), func(t *testing.T) {
cliPath, cliPathErr := getCliPath()
if cliPathErr != nil {
t.Errorf("failed to get cli path, err: %v", cliPathErr)
return
}
tapCmdArgs := getDefaultTapCommandArgs()
tapNamespace := getDefaultTapNamespace()
tapCmdArgs = append(tapCmdArgs, tapNamespace...)
tapCmdArgs = append(tapCmdArgs, "-p", fmt.Sprintf("%d", guiPort))
tapCmd := exec.Command(cliPath, tapCmdArgs...)
t.Logf("running command: %v", tapCmd.String())
t.Cleanup(func() {
if err := cleanupCommand(tapCmd); err != nil {
t.Logf("failed to cleanup tap command, err: %v", err)
}
})
if err := tapCmd.Start(); err != nil {
t.Errorf("failed to start tap command, err: %v", err)
return
}
apiServerUrl := getApiServerUrl(guiPort)
if err := waitTapPodsReady(apiServerUrl); err != nil {
t.Errorf("failed to start tap pods on time, err: %v", err)
return
}
})
}
}
func TestTapAllNamespaces(t *testing.T) {
if testing.Short() {
t.Skip("ignored acceptance test")
}
expectedPods := []struct{
Name string
Namespace string
}{
{Name: "httpbin", Namespace: "mizu-tests"},
{Name: "httpbin", Namespace: "mizu-tests2"},
}
cliPath, cliPathErr := getCliPath()
if cliPathErr != nil {
t.Errorf("failed to get cli path, err: %v", cliPathErr)
return
}
tapCmdArgs := getDefaultTapCommandArgs()
tapCmdArgs = append(tapCmdArgs, "-A")
tapCmd := exec.Command(cliPath, tapCmdArgs...)
t.Logf("running command: %v", tapCmd.String())
t.Cleanup(func() {
if err := cleanupCommand(tapCmd); err != nil {
t.Logf("failed to cleanup tap command, err: %v", err)
}
})
if err := tapCmd.Start(); err != nil {
t.Errorf("failed to start tap command, err: %v", err)
return
}
apiServerUrl := getApiServerUrl(defaultApiServerPort)
if err := waitTapPodsReady(apiServerUrl); err != nil {
t.Errorf("failed to start tap pods on time, err: %v", err)
return
}
podsUrl := fmt.Sprintf("%v/api/tapStatus", apiServerUrl)
requestResult, requestErr := executeHttpGetRequest(podsUrl)
if requestErr != nil {
t.Errorf("failed to get tap status, err: %v", requestErr)
return
}
pods, err := getPods(requestResult)
if err != nil {
t.Errorf("failed to get pods, err: %v", err)
return
}
for _, expectedPod := range expectedPods {
podFound := false
for _, pod := range pods {
podNamespace := pod["namespace"].(string)
podName := pod["name"].(string)
if expectedPod.Namespace == podNamespace && strings.Contains(podName, expectedPod.Name) {
podFound = true
break
}
}
if !podFound {
t.Errorf("unexpected result - expected pod not found, pod namespace: %v, pod name: %v", expectedPod.Namespace, expectedPod.Name)
return
}
}
}
func TestTapMultipleNamespaces(t *testing.T) {
if testing.Short() {
t.Skip("ignored acceptance test")
}
expectedPods := []struct{
Name string
Namespace string
}{
{Name: "httpbin", Namespace: "mizu-tests"},
{Name: "httpbin2", Namespace: "mizu-tests"},
{Name: "httpbin", Namespace: "mizu-tests2"},
}
cliPath, cliPathErr := getCliPath()
if cliPathErr != nil {
t.Errorf("failed to get cli path, err: %v", cliPathErr)
return
}
tapCmdArgs := getDefaultTapCommandArgs()
var namespacesCmd []string
for _, expectedPod := range expectedPods {
namespacesCmd = append(namespacesCmd, "-n", expectedPod.Namespace)
}
tapCmdArgs = append(tapCmdArgs, namespacesCmd...)
tapCmd := exec.Command(cliPath, tapCmdArgs...)
t.Logf("running command: %v", tapCmd.String())
t.Cleanup(func() {
if err := cleanupCommand(tapCmd); err != nil {
t.Logf("failed to cleanup tap command, err: %v", err)
}
})
if err := tapCmd.Start(); err != nil {
t.Errorf("failed to start tap command, err: %v", err)
return
}
apiServerUrl := getApiServerUrl(defaultApiServerPort)
if err := waitTapPodsReady(apiServerUrl); err != nil {
t.Errorf("failed to start tap pods on time, err: %v", err)
return
}
podsUrl := fmt.Sprintf("%v/api/tapStatus", apiServerUrl)
requestResult, requestErr := executeHttpGetRequest(podsUrl)
if requestErr != nil {
t.Errorf("failed to get tap status, err: %v", requestErr)
return
}
pods, err := getPods(requestResult)
if err != nil {
t.Errorf("failed to get pods, err: %v", err)
return
}
if len(expectedPods) != len(pods) {
t.Errorf("unexpected result - expected pods length: %v, actual pods length: %v", len(expectedPods), len(pods))
return
}
for _, expectedPod := range expectedPods {
podFound := false
for _, pod := range pods {
podNamespace := pod["namespace"].(string)
podName := pod["name"].(string)
if expectedPod.Namespace == podNamespace && strings.Contains(podName, expectedPod.Name) {
podFound = true
break
}
}
if !podFound {
t.Errorf("unexpected result - expected pod not found, pod namespace: %v, pod name: %v", expectedPod.Namespace, expectedPod.Name)
return
}
}
}
func TestTapRegex(t *testing.T) {
if testing.Short() {
t.Skip("ignored acceptance test")
}
regexPodName := "httpbin2"
expectedPods := []struct{
Name string
Namespace string
}{
{Name: regexPodName, Namespace: "mizu-tests"},
}
cliPath, cliPathErr := getCliPath()
if cliPathErr != nil {
t.Errorf("failed to get cli path, err: %v", cliPathErr)
return
}
tapCmdArgs := getDefaultTapCommandArgsWithRegex(regexPodName)
tapNamespace := getDefaultTapNamespace()
tapCmdArgs = append(tapCmdArgs, tapNamespace...)
tapCmd := exec.Command(cliPath, tapCmdArgs...)
t.Logf("running command: %v", tapCmd.String())
t.Cleanup(func() {
if err := cleanupCommand(tapCmd); err != nil {
t.Logf("failed to cleanup tap command, err: %v", err)
}
})
if err := tapCmd.Start(); err != nil {
t.Errorf("failed to start tap command, err: %v", err)
return
}
apiServerUrl := getApiServerUrl(defaultApiServerPort)
if err := waitTapPodsReady(apiServerUrl); err != nil {
t.Errorf("failed to start tap pods on time, err: %v", err)
return
}
podsUrl := fmt.Sprintf("%v/api/tapStatus", apiServerUrl)
requestResult, requestErr := executeHttpGetRequest(podsUrl)
if requestErr != nil {
t.Errorf("failed to get tap status, err: %v", requestErr)
return
}
pods, err := getPods(requestResult)
if err != nil {
t.Errorf("failed to get pods, err: %v", err)
return
}
if len(expectedPods) != len(pods) {
t.Errorf("unexpected result - expected pods length: %v, actual pods length: %v", len(expectedPods), len(pods))
return
}
for _, expectedPod := range expectedPods {
podFound := false
for _, pod := range pods {
podNamespace := pod["namespace"].(string)
podName := pod["name"].(string)
if expectedPod.Namespace == podNamespace && strings.Contains(podName, expectedPod.Name) {
podFound = true
break
}
}
if !podFound {
t.Errorf("unexpected result - expected pod not found, pod namespace: %v, pod name: %v", expectedPod.Namespace, expectedPod.Name)
return
}
}
}
func TestTapDryRun(t *testing.T) {
if testing.Short() {
t.Skip("ignored acceptance test")
}
cliPath, cliPathErr := getCliPath()
if cliPathErr != nil {
t.Errorf("failed to get cli path, err: %v", cliPathErr)
return
}
tapCmdArgs := getDefaultTapCommandArgs()
tapNamespace := getDefaultTapNamespace()
tapCmdArgs = append(tapCmdArgs, tapNamespace...)
tapCmdArgs = append(tapCmdArgs, "--dry-run")
tapCmd := exec.Command(cliPath, tapCmdArgs...)
t.Logf("running command: %v", tapCmd.String())
if err := tapCmd.Start(); err != nil {
t.Errorf("failed to start tap command, err: %v", err)
return
}
resultChannel := make(chan string, 1)
go func() {
if err := tapCmd.Wait(); err != nil {
resultChannel <- "fail"
return
}
resultChannel <- "success"
}()
go func() {
time.Sleep(shortRetriesCount * time.Second)
resultChannel <- "fail"
}()
testResult := <- resultChannel
if testResult != "success" {
t.Errorf("unexpected result - dry run cmd not done")
}
}
func TestTapRedact(t *testing.T) {
if testing.Short() {
t.Skip("ignored acceptance test")
}
cliPath, cliPathErr := getCliPath()
if cliPathErr != nil {
t.Errorf("failed to get cli path, err: %v", cliPathErr)
return
}
tapCmdArgs := getDefaultTapCommandArgs()
tapNamespace := getDefaultTapNamespace()
tapCmdArgs = append(tapCmdArgs, tapNamespace...)
tapCmd := exec.Command(cliPath, tapCmdArgs...)
t.Logf("running command: %v", tapCmd.String())
t.Cleanup(func() {
if err := cleanupCommand(tapCmd); err != nil {
t.Logf("failed to cleanup tap command, err: %v", err)
}
})
if err := tapCmd.Start(); err != nil {
t.Errorf("failed to start tap command, err: %v", err)
return
}
apiServerUrl := getApiServerUrl(defaultApiServerPort)
if err := waitTapPodsReady(apiServerUrl); err != nil {
t.Errorf("failed to start tap pods on time, err: %v", err)
return
}
proxyUrl := getProxyUrl(defaultNamespaceName, defaultServiceName)
requestBody := map[string]string{"User": "Mizu"}
for i := 0; i < defaultEntriesCount; i++ {
if _, requestErr := executeHttpPostRequest(fmt.Sprintf("%v/post", proxyUrl), requestBody); requestErr != nil {
t.Errorf("failed to send proxy request, err: %v", requestErr)
return
}
}
redactCheckFunc := func() error {
timestamp := time.Now().UnixNano() / int64(time.Millisecond)
entriesUrl := fmt.Sprintf("%v/api/entries?limit=%v&operator=lt&timestamp=%v", apiServerUrl, defaultEntriesCount, timestamp)
requestResult, requestErr := executeHttpGetRequest(entriesUrl)
if requestErr != nil {
return fmt.Errorf("failed to get entries, err: %v", requestErr)
}
entries := requestResult.([]interface{})
firstEntry := entries[0].(map[string]interface{})
entryUrl := fmt.Sprintf("%v/api/entries/%v", apiServerUrl, firstEntry["id"])
requestResult, requestErr = executeHttpGetRequest(entryUrl)
if requestErr != nil {
return fmt.Errorf("failed to get entry, err: %v", requestErr)
}
entry := requestResult.(map[string]interface{})["entry"].(map[string]interface{})
entryRequest := entry["request"].(map[string]interface{})
headers := entryRequest["headers"].([]interface{})
for _, headerInterface := range headers {
header := headerInterface.(map[string]interface{})
if header["name"].(string) != "User-Agent" {
continue
}
userAgent := header["value"].(string)
if userAgent != "[REDACTED]" {
return fmt.Errorf("unexpected result - user agent is not redacted")
}
}
data := entryRequest["postData"].(map[string]interface{})
textDataStr := data["text"].(string)
var textData map[string]string
if parseErr := json.Unmarshal([]byte(textDataStr), &textData); parseErr != nil {
return fmt.Errorf("failed to parse text data, err: %v", parseErr)
}
if textData["User"] != "[REDACTED]" {
return fmt.Errorf("unexpected result - user in body is not redacted")
}
return nil
}
if err := retriesExecute(shortRetriesCount, redactCheckFunc); err != nil {
t.Errorf("%v", err)
return
}
}
func TestTapNoRedact(t *testing.T) {
if testing.Short() {
t.Skip("ignored acceptance test")
}
cliPath, cliPathErr := getCliPath()
if cliPathErr != nil {
t.Errorf("failed to get cli path, err: %v", cliPathErr)
return
}
tapCmdArgs := getDefaultTapCommandArgs()
tapNamespace := getDefaultTapNamespace()
tapCmdArgs = append(tapCmdArgs, tapNamespace...)
tapCmdArgs = append(tapCmdArgs, "--no-redact")
tapCmd := exec.Command(cliPath, tapCmdArgs...)
t.Logf("running command: %v", tapCmd.String())
t.Cleanup(func() {
if err := cleanupCommand(tapCmd); err != nil {
t.Logf("failed to cleanup tap command, err: %v", err)
}
})
if err := tapCmd.Start(); err != nil {
t.Errorf("failed to start tap command, err: %v", err)
return
}
apiServerUrl := getApiServerUrl(defaultApiServerPort)
if err := waitTapPodsReady(apiServerUrl); err != nil {
t.Errorf("failed to start tap pods on time, err: %v", err)
return
}
proxyUrl := getProxyUrl(defaultNamespaceName, defaultServiceName)
requestBody := map[string]string{"User": "Mizu"}
for i := 0; i < defaultEntriesCount; i++ {
if _, requestErr := executeHttpPostRequest(fmt.Sprintf("%v/post", proxyUrl), requestBody); requestErr != nil {
t.Errorf("failed to send proxy request, err: %v", requestErr)
return
}
}
redactCheckFunc := func() error {
timestamp := time.Now().UnixNano() / int64(time.Millisecond)
entriesUrl := fmt.Sprintf("%v/api/entries?limit=%v&operator=lt&timestamp=%v", apiServerUrl, defaultEntriesCount, timestamp)
requestResult, requestErr := executeHttpGetRequest(entriesUrl)
if requestErr != nil {
return fmt.Errorf("failed to get entries, err: %v", requestErr)
}
entries := requestResult.([]interface{})
firstEntry := entries[0].(map[string]interface{})
entryUrl := fmt.Sprintf("%v/api/entries/%v", apiServerUrl, firstEntry["id"])
requestResult, requestErr = executeHttpGetRequest(entryUrl)
if requestErr != nil {
return fmt.Errorf("failed to get entry, err: %v", requestErr)
}
entry := requestResult.(map[string]interface{})["entry"].(map[string]interface{})
entryRequest := entry["request"].(map[string]interface{})
headers := entryRequest["headers"].([]interface{})
for _, headerInterface := range headers {
header := headerInterface.(map[string]interface{})
if header["name"].(string) != "User-Agent" {
continue
}
userAgent := header["value"].(string)
if userAgent == "[REDACTED]" {
return fmt.Errorf("unexpected result - user agent is redacted")
}
}
data := entryRequest["postData"].(map[string]interface{})
textDataStr := data["text"].(string)
var textData map[string]string
if parseErr := json.Unmarshal([]byte(textDataStr), &textData); parseErr != nil {
return fmt.Errorf("failed to parse text data, err: %v", parseErr)
}
if textData["User"] == "[REDACTED]" {
return fmt.Errorf("unexpected result - user in body is redacted")
}
return nil
}
if err := retriesExecute(shortRetriesCount, redactCheckFunc); err != nil {
t.Errorf("%v", err)
return
}
}
func TestTapRegexMasking(t *testing.T) {
if testing.Short() {
t.Skip("ignored acceptance test")
}
cliPath, cliPathErr := getCliPath()
if cliPathErr != nil {
t.Errorf("failed to get cli path, err: %v", cliPathErr)
return
}
tapCmdArgs := getDefaultTapCommandArgs()
tapNamespace := getDefaultTapNamespace()
tapCmdArgs = append(tapCmdArgs, tapNamespace...)
tapCmdArgs = append(tapCmdArgs, "-r", "Mizu")
tapCmd := exec.Command(cliPath, tapCmdArgs...)
t.Logf("running command: %v", tapCmd.String())
t.Cleanup(func() {
if err := cleanupCommand(tapCmd); err != nil {
t.Logf("failed to cleanup tap command, err: %v", err)
}
})
if err := tapCmd.Start(); err != nil {
t.Errorf("failed to start tap command, err: %v", err)
return
}
apiServerUrl := getApiServerUrl(defaultApiServerPort)
if err := waitTapPodsReady(apiServerUrl); err != nil {
t.Errorf("failed to start tap pods on time, err: %v", err)
return
}
proxyUrl := getProxyUrl(defaultNamespaceName, defaultServiceName)
for i := 0; i < defaultEntriesCount; i++ {
response, requestErr := http.Post(fmt.Sprintf("%v/post", proxyUrl), "text/plain", bytes.NewBufferString("Mizu"))
if _, requestErr = executeHttpRequest(response, requestErr); requestErr != nil {
t.Errorf("failed to send proxy request, err: %v", requestErr)
return
}
}
redactCheckFunc := func() error {
timestamp := time.Now().UnixNano() / int64(time.Millisecond)
entriesUrl := fmt.Sprintf("%v/api/entries?limit=%v&operator=lt&timestamp=%v", apiServerUrl, defaultEntriesCount, timestamp)
requestResult, requestErr := executeHttpGetRequest(entriesUrl)
if requestErr != nil {
return fmt.Errorf("failed to get entries, err: %v", requestErr)
}
entries := requestResult.([]interface{})
firstEntry := entries[0].(map[string]interface{})
entryUrl := fmt.Sprintf("%v/api/entries/%v", apiServerUrl, firstEntry["id"])
requestResult, requestErr = executeHttpGetRequest(entryUrl)
if requestErr != nil {
return fmt.Errorf("failed to get entry, err: %v", requestErr)
}
entry := requestResult.(map[string]interface{})["entry"].(map[string]interface{})
entryRequest := entry["request"].(map[string]interface{})
data := entryRequest["postData"].(map[string]interface{})
textData := data["text"].(string)
if textData != "[REDACTED]" {
return fmt.Errorf("unexpected result - body is not redacted")
}
return nil
}
if err := retriesExecute(shortRetriesCount, redactCheckFunc); err != nil {
t.Errorf("%v", err)
return
}
}

View File

@@ -1,8 +1,8 @@
package acceptanceTests
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io/ioutil"
"net/http"
@@ -10,19 +10,9 @@ import (
"os/exec"
"path"
"syscall"
"time"
)
const (
longRetriesCount = 100
shortRetriesCount = 10
defaultApiServerPort = 8899
defaultNamespaceName = "mizu-tests"
defaultServiceName = "httpbin"
defaultEntriesCount = 50
)
func getCliPath() (string, error) {
func GetCliPath() (string, error) {
dir, filePathErr := os.Getwd()
if filePathErr != nil {
return "", filePathErr
@@ -32,101 +22,34 @@ func getCliPath() (string, error) {
return cliPath, nil
}
func getConfigPath() (string, error) {
home, homeDirErr := os.UserHomeDir()
if homeDirErr != nil {
return "", homeDirErr
}
return path.Join(home, ".mizu", "config.yaml"), nil
}
func getProxyUrl(namespace string, service string) string {
return fmt.Sprintf("http://localhost:8080/api/v1/namespaces/%v/services/%v/proxy", namespace, service)
}
func getApiServerUrl(port uint16) string {
return fmt.Sprintf("http://localhost:%v/mizu", port)
}
func getDefaultCommandArgs() []string {
func GetDefaultCommandArgs() []string {
setFlag := "--set"
telemetry := "telemetry=false"
return []string{setFlag, telemetry}
}
func GetDefaultTapCommandArgs() []string {
tapCommand := "tap"
setFlag := "--set"
namespaces := "tap.namespaces=mizu-tests"
agentImage := "agent-image=gcr.io/up9-docker-hub/mizu/ci:0.0.0"
imagePullPolicy := "image-pull-policy=Never"
return []string{setFlag, telemetry, setFlag, agentImage, setFlag, imagePullPolicy}
defaultCmdArgs := GetDefaultCommandArgs()
return append([]string{tapCommand, setFlag, namespaces, setFlag, agentImage, setFlag, imagePullPolicy}, defaultCmdArgs...)
}
func getDefaultTapCommandArgs() []string {
tapCommand := "tap"
defaultCmdArgs := getDefaultCommandArgs()
func GetDefaultFetchCommandArgs() []string {
tapCommand := "fetch"
defaultCmdArgs := GetDefaultCommandArgs()
return append([]string{tapCommand}, defaultCmdArgs...)
}
func getDefaultTapCommandArgsWithRegex(regex string) []string {
tapCommand := "tap"
defaultCmdArgs := getDefaultCommandArgs()
return append([]string{tapCommand, regex}, defaultCmdArgs...)
}
func getDefaultTapNamespace() []string {
return []string{"-n", "mizu-tests"}
}
func getDefaultFetchCommandArgs() []string {
fetchCommand := "fetch"
defaultCmdArgs := getDefaultCommandArgs()
return append([]string{fetchCommand}, defaultCmdArgs...)
}
func getDefaultConfigCommandArgs() []string {
configCommand := "config"
defaultCmdArgs := getDefaultCommandArgs()
return append([]string{configCommand}, defaultCmdArgs...)
}
func retriesExecute(retriesCount int, executeFunc func() error) error {
var lastError error
for i := 0; i < retriesCount; i++ {
if err := executeFunc(); err != nil {
lastError = err
time.Sleep(1 * time.Second)
continue
}
return nil
}
return fmt.Errorf("reached max retries count, retries count: %v, last err: %v", retriesCount, lastError)
}
func waitTapPodsReady(apiServerUrl string) error {
resolvingUrl := fmt.Sprintf("%v/status/tappersCount", apiServerUrl)
tapPodsReadyFunc := func() error {
requestResult, requestErr := executeHttpGetRequest(resolvingUrl)
if requestErr != nil {
return requestErr
}
tappersCount := requestResult.(float64)
if tappersCount == 0 {
return fmt.Errorf("no tappers running")
}
return nil
}
return retriesExecute(longRetriesCount, tapPodsReadyFunc)
}
func jsonBytesToInterface(jsonBytes []byte) (interface{}, error) {
func JsonBytesToInterface(jsonBytes []byte) (interface{}, error) {
var result interface{}
if parseErr := json.Unmarshal(jsonBytes, &result); parseErr != nil {
return nil, parseErr
@@ -135,39 +58,23 @@ func jsonBytesToInterface(jsonBytes []byte) (interface{}, error) {
return result, nil
}
func executeHttpRequest(response *http.Response, requestErr error) (interface{}, error) {
func ExecuteHttpRequest(url string) (interface{}, error) {
response, requestErr := http.Get(url)
if requestErr != nil {
return nil, requestErr
} else if response.StatusCode != 200 {
return nil, fmt.Errorf("invalid status code %v", response.StatusCode)
}
defer func() { response.Body.Close() }()
data, readErr := ioutil.ReadAll(response.Body)
if readErr != nil {
return nil, readErr
}
return jsonBytesToInterface(data)
return JsonBytesToInterface(data)
}
func executeHttpGetRequest(url string) (interface{}, error) {
response, requestErr := http.Get(url)
return executeHttpRequest(response, requestErr)
}
func executeHttpPostRequest(url string, body interface{}) (interface{}, error) {
requestBody, jsonErr := json.Marshal(body)
if jsonErr != nil {
return nil, jsonErr
}
response, requestErr := http.Post(url, "application/json", bytes.NewBuffer(requestBody))
return executeHttpRequest(response, requestErr)
}
func cleanupCommand(cmd *exec.Cmd) error {
func CleanupCommand(cmd *exec.Cmd) error {
if err := cmd.Process.Signal(syscall.SIGQUIT); err != nil {
return err
}
@@ -179,27 +86,28 @@ func cleanupCommand(cmd *exec.Cmd) error {
return nil
}
func getEntriesFromHarBytes(harBytes []byte) ([]interface{}, error) {
harInterface, convertErr := jsonBytesToInterface(harBytes)
func GetEntriesFromHarBytes(harBytes []byte) ([]interface{}, error){
harInterface, convertErr := JsonBytesToInterface(harBytes)
if convertErr != nil {
return nil, convertErr
}
har := harInterface.(map[string]interface{})
harLog := har["log"].(map[string]interface{})
harEntries := harLog["entries"].([]interface{})
har, ok := harInterface.(map[string]interface{})
if !ok {
return nil, errors.New("invalid har type")
}
harLogInterface := har["log"]
harLog, ok := harLogInterface.(map[string]interface{})
if !ok {
return nil, errors.New("invalid har log type")
}
harEntriesInterface := harLog["entries"]
harEntries, ok := harEntriesInterface.([]interface{})
if !ok {
return nil, errors.New("invalid har entries type")
}
return harEntries, nil
}
func getPods(tapStatusInterface interface{}) ([]map[string]interface{}, error) {
tapStatus := tapStatusInterface.(map[string]interface{})
podsInterface := tapStatus["pods"].([]interface{})
var pods []map[string]interface{}
for _, podInterface := range podsInterface {
pods = append(pods, podInterface.(map[string]interface{}))
}
return pods, nil
}

View File

@@ -28,7 +28,6 @@ func init() {
func (h *RoutesEventHandlers) WebSocketConnect(socketId int, isTapper bool) {
if isTapper {
rlog.Infof("Websocket event - Tapper connected, socket ID: %d", socketId)
providers.TapperAdded()
} else {
rlog.Infof("Websocket event - Browser socket connected, socket ID: %d", socketId)
socketListLock.Lock()
@@ -40,7 +39,6 @@ func (h *RoutesEventHandlers) WebSocketConnect(socketId int, isTapper bool) {
func (h *RoutesEventHandlers) WebSocketDisconnect(socketId int, isTapper bool) {
if isTapper {
rlog.Infof("Websocket event - Tapper disconnected, socket ID: %d", socketId)
providers.TapperRemoved()
} else {
rlog.Infof("Websocket event - Browser socket disconnected, socket ID: %d", socketId)
socketListLock.Lock()

View File

@@ -30,7 +30,3 @@ func PostTappedPods(c *gin.Context) {
api.BroadcastToBrowserClients(jsonBytes)
}
}
func GetTappersCount(c *gin.Context) {
c.JSON(http.StatusOK, providers.TappersCount)
}

View File

@@ -4,18 +4,14 @@ import (
"github.com/patrickmn/go-cache"
"github.com/up9inc/mizu/shared"
"github.com/up9inc/mizu/tap"
"sync"
"time"
)
const tlsLinkRetainmentTime = time.Minute * 15
var (
TappersCount int
TapStatus shared.TapStatus
TapStatus shared.TapStatus
RecentTLSLinks = cache.New(tlsLinkRetainmentTime, tlsLinkRetainmentTime)
tappersCountLock = sync.Mutex{}
)
func GetAllRecentTLSAddresses() []string {
@@ -30,15 +26,3 @@ func GetAllRecentTLSAddresses() []string {
return recentTLSLinks
}
func TapperAdded() {
tappersCountLock.Lock()
TappersCount++
tappersCountLock.Unlock()
}
func TapperRemoved() {
tappersCountLock.Lock()
TappersCount--
tappersCountLock.Unlock()
}

View File

@@ -9,6 +9,4 @@ func StatusRoutes(ginApp *gin.Engine) {
routeGroup := ginApp.Group("/status")
routeGroup.POST("/tappedPods", controllers.PostTappedPods)
routeGroup.GET("/tappersCount", controllers.GetTappersCount)
}

View File

@@ -1,168 +0,0 @@
package apiserver
import (
"archive/zip"
"bytes"
"encoding/json"
"fmt"
"github.com/up9inc/mizu/cli/logger"
"github.com/up9inc/mizu/cli/uiUtils"
"github.com/up9inc/mizu/shared"
"io/ioutil"
core "k8s.io/api/core/v1"
"net/http"
"net/url"
"time"
)
type apiServerProvider struct {
url string
isReady bool
}
var Provider = apiServerProvider{}
func (provider *apiServerProvider) InitAndTestConnection(url string, retries int) error {
healthUrl := fmt.Sprintf("%s/", url)
retriesLeft := retries
for retriesLeft > 0 {
if response, err := http.Get(healthUrl); err != nil {
logger.Log.Debugf("[ERROR] failed connecting to api server %v", err)
} else if response.StatusCode != 200 {
logger.Log.Debugf("can't connect to api server yet, response status code %v", response.StatusCode)
} else {
logger.Log.Debugf("connection test to api server passed successfully")
break
}
retriesLeft -= 1
time.Sleep(time.Second)
}
if retriesLeft == 0 {
provider.isReady = false
return fmt.Errorf("couldn't reach the api server after %v retries", retries)
}
provider.url = url
provider.isReady = true
return nil
}
func (provider *apiServerProvider) ReportTappedPods(pods []core.Pod) error {
if !provider.isReady {
return fmt.Errorf("trying to reach api server when not initialized yet")
}
tappedPodsUrl := fmt.Sprintf("%s/status/tappedPods", provider.url)
podInfos := make([]shared.PodInfo, 0)
for _, pod := range pods {
podInfos = append(podInfos, shared.PodInfo{Name: pod.Name, Namespace: pod.Namespace})
}
tapStatus := shared.TapStatus{Pods: podInfos}
if jsonValue, err := json.Marshal(tapStatus); err != nil {
return fmt.Errorf("failed Marshal the tapped pods %w", err)
} else {
if response, err := http.Post(tappedPodsUrl, "application/json", bytes.NewBuffer(jsonValue)); err != nil {
return fmt.Errorf("failed sending to API server the tapped pods %w", err)
} else if response.StatusCode != 200 {
return fmt.Errorf("failed sending to API server the tapped pods, response status code %v", response.StatusCode)
} else {
logger.Log.Debugf("Reported to server API about %d taped pods successfully", len(podInfos))
return nil
}
}
}
func (provider *apiServerProvider) RequestAnalysis(analysisDestination string, sleepIntervalSec int) error {
if !provider.isReady {
return fmt.Errorf("trying to reach api server when not initialized yet")
}
urlPath := fmt.Sprintf("http://%s/api/uploadEntries?dest=%s&interval=%v", provider.url, url.QueryEscape(analysisDestination), sleepIntervalSec)
u, parseErr := url.ParseRequestURI(urlPath)
if parseErr != nil {
logger.Log.Fatal("Failed parsing the URL (consider changing the analysis dest URL), err: %v", parseErr)
}
logger.Log.Debugf("Analysis url %v", u.String())
if response, requestErr := http.Get(u.String()); requestErr != nil {
return fmt.Errorf("failed to notify agent for analysis, err: %w", requestErr)
} else if response.StatusCode != 200 {
return fmt.Errorf("failed to notify agent for analysis, status code: %v", response.StatusCode)
} else {
logger.Log.Infof(uiUtils.Purple, "Traffic is uploading to UP9 for further analysis")
return nil
}
}
func (provider *apiServerProvider) GetGeneralStats() (map[string]interface{}, error) {
if !provider.isReady {
return nil, fmt.Errorf("trying to reach api server when not initialized yet")
}
generalStatsUrl := fmt.Sprintf("%s/api/generalStats", provider.url)
response, requestErr := http.Get(generalStatsUrl)
if requestErr != nil {
return nil, fmt.Errorf("failed to get general stats for telemetry, err: %w", requestErr)
} else if response.StatusCode != 200 {
return nil, fmt.Errorf("failed to get general stats for telemetry, status code: %v", response.StatusCode)
}
defer func() { _ = response.Body.Close() }()
data, readErr := ioutil.ReadAll(response.Body)
if readErr != nil {
return nil, fmt.Errorf("failed to read general stats for telemetry, err: %v", readErr)
}
var generalStats map[string]interface{}
if parseErr := json.Unmarshal(data, &generalStats); parseErr != nil {
return nil, fmt.Errorf("failed to parse general stats for telemetry, err: %v", parseErr)
}
return generalStats, nil
}
func (provider *apiServerProvider) GetHars(fromTimestamp int, toTimestamp int) (*zip.Reader, error) {
if !provider.isReady {
return nil, fmt.Errorf("trying to reach api server when not initialized yet")
}
resp, err := http.Get(fmt.Sprintf("%s/api/har?from=%v&to=%v", provider.url, fromTimestamp, toTimestamp))
if err != nil {
return nil, fmt.Errorf("failed getting har from api server %w", err)
}
defer func() { _ = resp.Body.Close() }()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("failed reading hars %w", err)
}
zipReader, err := zip.NewReader(bytes.NewReader(body), int64(len(body)))
if err != nil {
return nil, fmt.Errorf("failed craeting zip reader %w", err)
}
return zipReader, nil
}
func (provider *apiServerProvider) GetVersion() (string, error) {
if !provider.isReady {
return "", fmt.Errorf("trying to reach api server when not initialized yet")
}
versionUrl, _ := url.Parse(fmt.Sprintf("%s/metadata/version", provider.url))
req := &http.Request{
Method: http.MethodGet,
URL: versionUrl,
}
statusResp, err := http.DefaultClient.Do(req)
if err != nil {
return "", err
}
defer statusResp.Body.Close()
versionResponse := &shared.VersionResponse{}
if err := json.NewDecoder(statusResp.Body).Decode(&versionResponse); err != nil {
return "", err
}
return versionResponse.SemVer, nil
}

View File

@@ -1,45 +0,0 @@
package cmd
import (
"context"
"fmt"
"github.com/up9inc/mizu/cli/config"
"github.com/up9inc/mizu/cli/config/configStructs"
"github.com/up9inc/mizu/cli/errormessage"
"github.com/up9inc/mizu/cli/kubernetes"
"github.com/up9inc/mizu/cli/logger"
"github.com/up9inc/mizu/cli/mizu"
"github.com/up9inc/mizu/cli/uiUtils"
"os"
"os/signal"
"syscall"
)
func GetApiServerUrl() string {
return fmt.Sprintf("http://%s", kubernetes.GetMizuApiServerProxiedHostAndPath(config.Config.Tap.GuiPort))
}
func startProxyReportErrorIfAny(kubernetesProvider *kubernetes.Provider, cancel context.CancelFunc) {
err := kubernetes.StartProxy(kubernetesProvider, config.Config.Tap.GuiPort, config.Config.MizuResourcesNamespace, mizu.ApiServerPodName)
if err != nil {
logger.Log.Errorf(uiUtils.Error, fmt.Sprintf("Error occured while running k8s proxy %v\n"+
"Try setting different port by using --%s", errormessage.FormatError(err), configStructs.GuiPortTapName))
cancel()
}
}
func waitForFinish(ctx context.Context, cancel context.CancelFunc) {
logger.Log.Debugf("waiting for finish...")
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM, syscall.SIGQUIT)
// block until ctx cancel is called or termination signal is received
select {
case <-ctx.Done():
logger.Log.Debugf("ctx done")
break
case <-sigChan:
logger.Log.Debugf("Got termination signal, canceling execution...")
cancel()
}
}

View File

@@ -25,11 +25,11 @@ var configCmd = &cobra.Command{
}
if config.Config.Config.Regenerate {
data := []byte(template)
if err := ioutil.WriteFile(config.Config.ConfigFilePath, data, 0644); err != nil {
if err := ioutil.WriteFile(config.GetConfigFilePath(), data, 0644); err != nil {
logger.Log.Errorf("Failed writing config %v", err)
return nil
}
logger.Log.Infof(fmt.Sprintf("Template File written to %s", fmt.Sprintf(uiUtils.Purple, config.Config.ConfigFilePath)))
logger.Log.Infof(fmt.Sprintf("Template File written to %s", fmt.Sprintf(uiUtils.Purple, config.GetConfigFilePath())))
} else {
logger.Log.Debugf("Writing template config.\n%v", template)
fmt.Printf("%v", template)
@@ -41,8 +41,8 @@ var configCmd = &cobra.Command{
func init() {
rootCmd.AddCommand(configCmd)
defaultConfig := config.ConfigStruct{}
defaults.Set(&defaultConfig)
defaultConfigConfig := configStructs.ConfigConfig{}
defaults.Set(&defaultConfigConfig)
configCmd.Flags().BoolP(configStructs.RegenerateConfigName, "r", defaultConfig.Config.Regenerate, fmt.Sprintf("Regenerate the config file with default values to path %s or to chosen path using --%s", defaultConfig.ConfigFilePath, config.ConfigFilePathCommandName))
configCmd.Flags().BoolP(configStructs.RegenerateConfigName, "r", defaultConfigConfig.Regenerate, fmt.Sprintf("Regenerate the config file with default values %s", config.GetConfigFilePath()))
}

View File

@@ -3,13 +3,10 @@ package cmd
import (
"github.com/creasty/defaults"
"github.com/spf13/cobra"
"github.com/up9inc/mizu/cli/apiserver"
"github.com/up9inc/mizu/cli/config"
"github.com/up9inc/mizu/cli/config/configStructs"
"github.com/up9inc/mizu/cli/logger"
"github.com/up9inc/mizu/cli/mizu/version"
"github.com/up9inc/mizu/cli/telemetry"
"github.com/up9inc/mizu/cli/uiUtils"
)
var fetchCmd = &cobra.Command{
@@ -18,12 +15,7 @@ var fetchCmd = &cobra.Command{
RunE: func(cmd *cobra.Command, args []string) error {
go telemetry.ReportRun("fetch", config.Config.Fetch)
if err := apiserver.Provider.InitAndTestConnection(GetApiServerUrl(), 1); err != nil {
logger.Log.Errorf(uiUtils.Error, "Couldn't connect to API server, make sure one running")
return nil
}
if isCompatible, err := version.CheckVersionCompatibility(); err != nil {
if isCompatible, err := version.CheckVersionCompatibility(config.Config.Fetch.GuiPort); err != nil {
return err
} else if !isCompatible {
return nil

View File

@@ -1,25 +1,96 @@
package cmd
import (
"github.com/up9inc/mizu/cli/apiserver"
"archive/zip"
"bytes"
"fmt"
"github.com/up9inc/mizu/cli/config"
"github.com/up9inc/mizu/cli/kubernetes"
"github.com/up9inc/mizu/cli/logger"
"github.com/up9inc/mizu/cli/mizu/fsUtils"
"github.com/up9inc/mizu/cli/uiUtils"
"io"
"io/ioutil"
"log"
"net/http"
"os"
"path/filepath"
"strings"
)
func RunMizuFetch() {
if err := apiserver.Provider.InitAndTestConnection(GetApiServerUrl(), 5); err != nil {
logger.Log.Errorf(uiUtils.Error, "Couldn't connect to API server, check logs")
}
zipReader, err := apiserver.Provider.GetHars(config.Config.Fetch.FromTimestamp, config.Config.Fetch.ToTimestamp)
mizuProxiedUrl := kubernetes.GetMizuApiServerProxiedHostAndPath(config.Config.Fetch.GuiPort)
resp, err := http.Get(fmt.Sprintf("http://%s/api/har?from=%v&to=%v", mizuProxiedUrl, config.Config.Fetch.FromTimestamp, config.Config.Fetch.ToTimestamp))
if err != nil {
logger.Log.Errorf("Failed fetch data from API server %v", err)
return
log.Fatal(err)
}
if err := fsUtils.Unzip(zipReader, config.Config.Fetch.Directory); err != nil {
logger.Log.Debugf("[ERROR] failed unzip %v", err)
defer func() { _ = resp.Body.Close() }()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatal(err)
}
zipReader, err := zip.NewReader(bytes.NewReader(body), int64(len(body)))
if err != nil {
log.Fatal(err)
}
_ = Unzip(zipReader, config.Config.Fetch.Directory)
}
func Unzip(reader *zip.Reader, dest string) error {
dest, _ = filepath.Abs(dest)
_ = os.MkdirAll(dest, os.ModePerm)
// Closure to address file descriptors issue with all the deferred .Close() methods
extractAndWriteFile := func(f *zip.File) error {
rc, err := f.Open()
if err != nil {
return err
}
defer func() {
if err := rc.Close(); err != nil {
panic(err)
}
}()
path := filepath.Join(dest, f.Name)
// Check for ZipSlip (Directory traversal)
if !strings.HasPrefix(path, filepath.Clean(dest)+string(os.PathSeparator)) {
return fmt.Errorf("illegal file path: %s", path)
}
if f.FileInfo().IsDir() {
_ = os.MkdirAll(path, f.Mode())
} else {
_ = os.MkdirAll(filepath.Dir(path), f.Mode())
logger.Log.Infof("writing HAR file [ %v ]", path)
f, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, f.Mode())
if err != nil {
return err
}
defer func() {
if err := f.Close(); err != nil {
panic(err)
}
logger.Log.Info(" done")
}()
_, err = io.Copy(f, rc)
if err != nil {
return err
}
}
return nil
}
for _, f := range reader.File {
err := extractAndWriteFile(f)
if err != nil {
return err
}
}
return nil
}

View File

@@ -2,7 +2,6 @@ package cmd
import (
"fmt"
"github.com/creasty/defaults"
"github.com/spf13/cobra"
"github.com/up9inc/mizu/cli/config"
"github.com/up9inc/mizu/cli/logger"
@@ -26,11 +25,7 @@ Further info is available at https://github.com/up9inc/mizu`,
}
func init() {
defaultConfig := config.ConfigStruct{}
defaults.Set(&defaultConfig)
rootCmd.PersistentFlags().StringSlice(config.SetCommandName, []string{}, fmt.Sprintf("Override values using --%s", config.SetCommandName))
rootCmd.PersistentFlags().String(config.ConfigFilePathCommandName, defaultConfig.ConfigFilePath, fmt.Sprintf("Override config file path using --%s", config.ConfigFilePathCommandName))
}
func printNewVersionIfNeeded(versionChan chan string) {

View File

@@ -1,28 +1,35 @@
package cmd
import (
"bytes"
"context"
"encoding/json"
"fmt"
"github.com/up9inc/mizu/cli/apiserver"
"github.com/up9inc/mizu/cli/config"
"github.com/up9inc/mizu/cli/config/configStructs"
"github.com/up9inc/mizu/cli/errormessage"
"github.com/up9inc/mizu/cli/kubernetes"
"github.com/up9inc/mizu/cli/logger"
"github.com/up9inc/mizu/cli/mizu"
"github.com/up9inc/mizu/cli/mizu/fsUtils"
"github.com/up9inc/mizu/cli/mizu/goUtils"
"github.com/up9inc/mizu/cli/telemetry"
"net/http"
"net/url"
"os"
"os/signal"
"path"
"regexp"
"strings"
"syscall"
"time"
"github.com/up9inc/mizu/cli/errormessage"
"github.com/up9inc/mizu/cli/kubernetes"
"github.com/up9inc/mizu/cli/mizu"
"github.com/up9inc/mizu/cli/uiUtils"
"github.com/up9inc/mizu/shared"
"github.com/up9inc/mizu/shared/debounce"
yaml "gopkg.in/yaml.v3"
core "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/util/wait"
"path"
"regexp"
"strings"
"time"
)
const (
@@ -53,6 +60,7 @@ func RunMizuTap() {
return
}
}
kubernetesProvider, err := kubernetes.NewProvider(config.Config.KubeConfigPath())
if err != nil {
logger.Log.Error(err)
@@ -100,13 +108,13 @@ func RunMizuTap() {
nodeToTappedPodIPMap := getNodeHostToTappedPodIpsMap(state.currentlyTappedPods)
defer finishMizuExecution(kubernetesProvider)
defer cleanUpMizu(kubernetesProvider)
if err := createMizuResources(ctx, kubernetesProvider, nodeToTappedPodIPMap, mizuApiFilteringOptions, mizuValidationRules); err != nil {
logger.Log.Errorf(uiUtils.Error, fmt.Sprintf("Error creating resources: %v", errormessage.FormatError(err)))
return
}
go goUtils.HandleExcWrapper(watchApiServerPod, ctx, kubernetesProvider, cancel)
go goUtils.HandleExcWrapper(createProxyToApiServerPod, ctx, kubernetesProvider, cancel)
go goUtils.HandleExcWrapper(watchPodsForTapping, ctx, kubernetesProvider, targetNamespaces, cancel)
//block until exit signal or error
@@ -253,26 +261,23 @@ func updateMizuTappers(ctx context.Context, kubernetesProvider *kubernetes.Provi
return nil
}
func finishMizuExecution(kubernetesProvider *kubernetes.Provider) {
telemetry.ReportAPICalls()
func cleanUpMizu(kubernetesProvider *kubernetes.Provider) {
telemetry.ReportAPICalls(config.Config.Tap.GuiPort)
cleanUpMizuResources(kubernetesProvider)
}
func cleanUpMizuResources(kubernetesProvider *kubernetes.Provider) {
removalCtx, cancel := context.WithTimeout(context.Background(), cleanupTimeout)
defer cancel()
dumpLogsIfNeeded(kubernetesProvider, removalCtx)
cleanUpMizuResources(kubernetesProvider, removalCtx, cancel)
}
func dumpLogsIfNeeded(kubernetesProvider *kubernetes.Provider, removalCtx context.Context) {
if !config.Config.DumpLogs {
return
if config.Config.DumpLogs {
mizuDir := mizu.GetMizuFolderPath()
filePath := path.Join(mizuDir, fmt.Sprintf("mizu_logs_%s.zip", time.Now().Format("2006_01_02__15_04_05")))
if err := fsUtils.DumpLogs(kubernetesProvider, removalCtx, filePath); err != nil {
logger.Log.Errorf("Failed dump logs %v", err)
}
}
mizuDir := mizu.GetMizuFolderPath()
filePath := path.Join(mizuDir, fmt.Sprintf("mizu_logs_%s.zip", time.Now().Format("2006_01_02__15_04_05")))
if err := fsUtils.DumpLogs(kubernetesProvider, removalCtx, filePath); err != nil {
logger.Log.Errorf("Failed dump logs %v", err)
}
}
func cleanUpMizuResources(kubernetesProvider *kubernetes.Provider, removalCtx context.Context, cancel context.CancelFunc) {
logger.Log.Infof("\nRemoving mizu resources\n")
if !config.Config.IsNsRestrictedMode() {
@@ -337,7 +342,7 @@ func waitUntilNamespaceDeleted(ctx context.Context, cancel context.CancelFunc, k
if err := kubernetesProvider.WaitUtilNamespaceDeleted(ctx, config.Config.MizuResourcesNamespace); err != nil {
switch {
case ctx.Err() == context.Canceled:
logger.Log.Debugf("Do nothing. User interrupted the wait")
// Do nothing. User interrupted the wait.
case err == wait.ErrWaitTimeout:
logger.Log.Errorf(uiUtils.Error, fmt.Sprintf("Timeout while removing Namespace %s", config.Config.MizuResourcesNamespace))
default:
@@ -346,6 +351,29 @@ func waitUntilNamespaceDeleted(ctx context.Context, cancel context.CancelFunc, k
}
}
func reportTappedPods() {
mizuProxiedUrl := kubernetes.GetMizuApiServerProxiedHostAndPath(config.Config.Tap.GuiPort)
tappedPodsUrl := fmt.Sprintf("http://%s/status/tappedPods", mizuProxiedUrl)
podInfos := make([]shared.PodInfo, 0)
for _, pod := range state.currentlyTappedPods {
podInfos = append(podInfos, shared.PodInfo{Name: pod.Name, Namespace: pod.Namespace})
}
tapStatus := shared.TapStatus{Pods: podInfos}
if jsonValue, err := json.Marshal(tapStatus); err != nil {
logger.Log.Debugf("[ERROR] failed Marshal the tapped pods %v", err)
} else {
if response, err := http.Post(tappedPodsUrl, "application/json", bytes.NewBuffer(jsonValue)); err != nil {
logger.Log.Debugf("[ERROR] failed sending to API server the tapped pods %v", err)
} else if response.StatusCode != 200 {
logger.Log.Debugf("[ERROR] failed sending to API server the tapped pods, response status code %v", response.StatusCode)
} else {
logger.Log.Debugf("Reported to server API about %d taped pods successfully", len(podInfos))
}
}
}
func watchPodsForTapping(ctx context.Context, kubernetesProvider *kubernetes.Provider, targetNamespaces []string, cancel context.CancelFunc) {
added, modified, removed, errorChan := kubernetes.FilteredWatch(ctx, kubernetesProvider, targetNamespaces, config.Config.Tap.PodRegex())
@@ -361,9 +389,7 @@ func watchPodsForTapping(ctx context.Context, kubernetesProvider *kubernetes.Pro
return
}
if err := apiserver.Provider.ReportTappedPods(state.currentlyTappedPods); err != nil {
logger.Log.Debugf("[Error] failed update tapped pods %v", err)
}
reportTappedPods()
nodeToTappedPodIPMap := getNodeHostToTappedPodIpsMap(state.currentlyTappedPods)
if err != nil {
@@ -470,7 +496,7 @@ func getMissingPods(pods1 []core.Pod, pods2 []core.Pod) []core.Pod {
return missingPods
}
func watchApiServerPod(ctx context.Context, kubernetesProvider *kubernetes.Provider, cancel context.CancelFunc) {
func createProxyToApiServerPod(ctx context.Context, kubernetesProvider *kubernetes.Provider, cancel context.CancelFunc) {
podExactRegex := regexp.MustCompile(fmt.Sprintf("^%s$", mizu.ApiServerPodName))
added, modified, removed, errorChan := kubernetes.FilteredWatch(ctx, kubernetesProvider, []string{config.Config.MizuResourcesNamespace}, podExactRegex)
isPodReady := false
@@ -496,17 +522,10 @@ func watchApiServerPod(ctx context.Context, kubernetesProvider *kubernetes.Provi
if modifiedPod.Status.Phase == core.PodRunning && !isPodReady {
isPodReady = true
go startProxyReportErrorIfAny(kubernetesProvider, cancel)
if err := apiserver.Provider.InitAndTestConnection(GetApiServerUrl(), 20); err != nil {
logger.Log.Errorf(uiUtils.Error, "Couldn't connect to API server, check logs")
cancel()
break
}
logger.Log.Infof("Mizu is available at %s\n", GetApiServerUrl())
requestForAnalysisIfNeeded()
if err := apiserver.Provider.ReportTappedPods(state.currentlyTappedPods); err != nil {
logger.Log.Debugf("[Error] failed update tapped pods %v", err)
}
logger.Log.Infof("Mizu is available at http://%s\n", kubernetes.GetMizuApiServerProxiedHostAndPath(config.Config.Tap.GuiPort))
time.Sleep(time.Second * 5) // Waiting to be sure the proxy is ready
requestForAnalysis()
reportTappedPods()
}
case <-timeAfter:
if !isPodReady {
@@ -520,12 +539,34 @@ func watchApiServerPod(ctx context.Context, kubernetesProvider *kubernetes.Provi
}
}
func requestForAnalysisIfNeeded() {
func startProxyReportErrorIfAny(kubernetesProvider *kubernetes.Provider, cancel context.CancelFunc) {
err := kubernetes.StartProxy(kubernetesProvider, config.Config.Tap.GuiPort, config.Config.MizuResourcesNamespace, mizu.ApiServerPodName)
if err != nil {
logger.Log.Errorf(uiUtils.Error, fmt.Sprintf("Error occured while running k8s proxy %v\n"+
"Try setting different port by using --%s", errormessage.FormatError(err), configStructs.GuiPortTapName))
cancel()
}
}
func requestForAnalysis() {
if !config.Config.Tap.Analysis {
return
}
if err := apiserver.Provider.RequestAnalysis(config.Config.Tap.AnalysisDestination, config.Config.Tap.SleepIntervalSec); err != nil {
logger.Log.Debugf("[Error] failed requesting for analysis %v", err)
mizuProxiedUrl := kubernetes.GetMizuApiServerProxiedHostAndPath(config.Config.Tap.GuiPort)
urlPath := fmt.Sprintf("http://%s/api/uploadEntries?dest=%s&interval=%v", mizuProxiedUrl, url.QueryEscape(config.Config.Tap.AnalysisDestination), config.Config.Tap.SleepIntervalSec)
u, parseErr := url.ParseRequestURI(urlPath)
if parseErr != nil {
logger.Log.Fatal("Failed parsing the URL (consider changing the analysis dest URL), err: %v", parseErr)
}
logger.Log.Debugf("Sending get request to %v", u.String())
if response, requestErr := http.Get(u.String()); requestErr != nil {
logger.Log.Errorf("Failed to notify agent for analysis, err: %v", requestErr)
} else if response.StatusCode != 200 {
logger.Log.Errorf("Failed to notify agent for analysis, status code: %v", response.StatusCode)
} else {
logger.Log.Infof(uiUtils.Purple, "Traffic is uploading to UP9 for further analysis")
}
}
@@ -557,11 +598,24 @@ func getNodeHostToTappedPodIpsMap(tappedPods []core.Pod) map[string][]string {
return nodeToTappedPodIPMap
}
func waitForFinish(ctx context.Context, cancel context.CancelFunc) {
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM, syscall.SIGQUIT)
// block until ctx cancel is called or termination signal is received
select {
case <-ctx.Done():
break
case <-sigChan:
cancel()
}
}
func getNamespaces(kubernetesProvider *kubernetes.Provider) []string {
if config.Config.Tap.AllNamespaces {
return []string{mizu.K8sAllNamespaces}
} else if len(config.Config.Tap.Namespaces) > 0 {
return mizu.Unique(config.Config.Tap.Namespaces)
return config.Config.Tap.Namespaces
} else {
return []string{kubernetesProvider.CurrentNamespace()}
}

View File

@@ -3,14 +3,13 @@ package cmd
import (
"context"
"fmt"
"github.com/up9inc/mizu/cli/apiserver"
"github.com/up9inc/mizu/cli/config"
"github.com/up9inc/mizu/cli/kubernetes"
"github.com/up9inc/mizu/cli/logger"
"github.com/up9inc/mizu/cli/mizu"
"github.com/up9inc/mizu/cli/mizu/version"
"github.com/up9inc/mizu/cli/uiUtils"
"net/http"
"time"
)
func runMizuView() {
@@ -35,21 +34,19 @@ func runMizuView() {
return
}
response, err := http.Get(fmt.Sprintf("%s/", GetApiServerUrl()))
if err == nil && response.StatusCode == 200 {
mizuProxiedUrl := kubernetes.GetMizuApiServerProxiedHostAndPath(config.Config.View.GuiPort)
_, err = http.Get(fmt.Sprintf("http://%s/", mizuProxiedUrl))
if err == nil {
logger.Log.Infof("Found a running service %s and open port %d", mizu.ApiServerPodName, config.Config.View.GuiPort)
return
}
logger.Log.Infof("Establishing connection to k8s cluster...")
go startProxyReportErrorIfAny(kubernetesProvider, cancel)
if err := apiserver.Provider.InitAndTestConnection(GetApiServerUrl(), 10); err != nil {
logger.Log.Errorf(uiUtils.Error, "Couldn't connect to API server, check logs")
return
}
time.Sleep(time.Second * 5) // Waiting to be sure the proxy is ready
logger.Log.Infof("Mizu is available at %s\n", GetApiServerUrl())
if isCompatible, err := version.CheckVersionCompatibility(); err != nil {
logger.Log.Infof("Mizu is available at http://%s\n", kubernetes.GetMizuApiServerProxiedHostAndPath(config.Config.View.GuiPort))
if isCompatible, err := version.CheckVersionCompatibility(config.Config.View.GuiPort); err != nil {
logger.Log.Errorf("Failed to check versions compatibility %v", err)
cancel()
return

View File

@@ -7,6 +7,7 @@ import (
"github.com/up9inc/mizu/cli/mizu"
"io/ioutil"
"os"
"path"
"reflect"
"strconv"
"strings"
@@ -26,7 +27,7 @@ const (
)
var (
Config = ConfigStruct{}
Config = ConfigStruct{}
cmdName string
)
@@ -37,11 +38,9 @@ func InitConfig(cmd *cobra.Command) error {
return err
}
configFilePath := cmd.Flags().Lookup(ConfigFilePathCommandName).Value.String()
if err := mergeConfigFile(configFilePath); err != nil {
return fmt.Errorf("invalid config, %w\n"+
"you can regenerate the file by removing it (%v) and using `mizu config -r`", err, configFilePath)
if err := mergeConfigFile(); err != nil {
return fmt.Errorf("invalid config, %w\n" +
"you can regenerate the file by removing it (%v) and using `mizu config -r`", err, GetConfigFilePath())
}
cmd.Flags().Visit(initFlag)
@@ -64,8 +63,12 @@ func GetConfigWithDefaults() (string, error) {
return uiUtils.PrettyYaml(defaultConf)
}
func mergeConfigFile(configFilePath string) error {
reader, openErr := os.Open(configFilePath)
func GetConfigFilePath() string {
return path.Join(mizu.GetMizuFolderPath(), "config.yaml")
}
func mergeConfigFile() error {
reader, openErr := os.Open(GetConfigFilePath())
if openErr != nil {
return nil
}
@@ -86,12 +89,7 @@ func mergeConfigFile(configFilePath string) error {
func initFlag(f *pflag.Flag) {
configElemValue := reflect.ValueOf(&Config).Elem()
var flagPath []string
if mizu.Contains([]string{ConfigFilePathCommandName}, f.Name) {
flagPath = []string{f.Name}
} else {
flagPath = []string{cmdName, f.Name}
}
flagPath := []string {cmdName, f.Name}
sliceValue, isSliceValue := f.Value.(pflag.SliceValue)
if !isSliceValue {

View File

@@ -7,13 +7,11 @@ import (
v1 "k8s.io/api/core/v1"
"k8s.io/client-go/util/homedir"
"os"
"path"
"path/filepath"
)
const (
MizuResourcesNamespaceConfigName = "mizu-resources-namespace"
ConfigFilePathCommandName = "config-path"
)
type ConfigStruct struct {
@@ -29,12 +27,10 @@ type ConfigStruct struct {
Telemetry bool `yaml:"telemetry" default:"true"`
DumpLogs bool `yaml:"dump-logs" default:"false"`
KubeConfigPathStr string `yaml:"kube-config-path"`
ConfigFilePath string `yaml:"config-path,omitempty" readonly:""`
}
func (config *ConfigStruct) SetDefaults() {
config.AgentImage = fmt.Sprintf("gcr.io/up9-docker-hub/mizu/%s:%s", mizu.Branch, mizu.SemVer)
config.ConfigFilePath = path.Join(mizu.GetMizuFolderPath(), "config.yaml")
}
func (config *ConfigStruct) ImagePullPolicy() v1.PullPolicy {

View File

@@ -1,7 +1,6 @@
package config_test
import (
"fmt"
"github.com/up9inc/mizu/cli/config"
"reflect"
"strings"
@@ -17,8 +16,7 @@ func TestConfigWriteIgnoresReadonlyFields(t *testing.T) {
configWithDefaults, _ := config.GetConfigWithDefaults()
for _, readonlyField := range readonlyFields {
t.Run(readonlyField, func(t *testing.T) {
readonlyFieldToCheck := fmt.Sprintf("\n%s:", readonlyField)
if strings.Contains(configWithDefaults, readonlyFieldToCheck) {
if strings.Contains(configWithDefaults, readonlyField) {
t.Errorf("unexpected result - readonly field: %v, config: %v", readonlyField, configWithDefaults)
}
})

View File

@@ -45,10 +45,10 @@ func DumpLogs(provider *kubernetes.Provider, ctx context.Context, filePath strin
logger.Log.Debugf("Successfully added log length %d from pod: %s.%s", len(logs), pod.Namespace, pod.Name)
}
}
if err := AddFileToZip(zipWriter, config.Config.ConfigFilePath); err != nil {
if err := AddFileToZip(zipWriter, config.GetConfigFilePath()); err != nil {
logger.Log.Debugf("Failed write file, %v", err)
} else {
logger.Log.Debugf("Successfully added file %s", config.Config.ConfigFilePath)
logger.Log.Debugf("Successfully added file %s", config.GetConfigFilePath())
}
if err := AddFileToZip(zipWriter, logger.GetLogFilePath()); err != nil {
logger.Log.Debugf("Failed write file, %v", err)

View File

@@ -3,11 +3,9 @@ package fsUtils
import (
"archive/zip"
"fmt"
"github.com/up9inc/mizu/cli/logger"
"io"
"os"
"path/filepath"
"strings"
)
func AddFileToZip(zipWriter *zip.Writer, filename string) error {
@@ -55,60 +53,3 @@ func AddStrToZip(writer *zip.Writer, logs string, fileName string) error {
}
return nil
}
func Unzip(reader *zip.Reader, dest string) error {
dest, _ = filepath.Abs(dest)
_ = os.MkdirAll(dest, os.ModePerm)
// Closure to address file descriptors issue with all the deferred .Close() methods
extractAndWriteFile := func(f *zip.File) error {
rc, err := f.Open()
if err != nil {
return err
}
defer func() {
if err := rc.Close(); err != nil {
panic(err)
}
}()
path := filepath.Join(dest, f.Name)
// Check for ZipSlip (Directory traversal)
if !strings.HasPrefix(path, filepath.Clean(dest)+string(os.PathSeparator)) {
return fmt.Errorf("illegal file path: %s", path)
}
if f.FileInfo().IsDir() {
_ = os.MkdirAll(path, f.Mode())
} else {
_ = os.MkdirAll(filepath.Dir(path), f.Mode())
logger.Log.Infof("writing HAR file [ %v ]", path)
f, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, f.Mode())
if err != nil {
return err
}
defer func() {
if err := f.Close(); err != nil {
panic(err)
}
logger.Log.Info(" done")
}()
_, err = io.Copy(f, rc)
if err != nil {
return err
}
}
return nil
}
for _, f := range reader.File {
err := extractAndWriteFile(f)
if err != nil {
return err
}
}
return nil
}

View File

@@ -9,17 +9,3 @@ func Contains(slice []string, containsValue string) bool {
return false
}
func Unique(slice []string) []string {
keys := make(map[string]bool)
var list []string
for _, entry := range slice {
if _, value := keys[entry]; !value {
keys[entry] = true
list = append(list, entry)
}
}
return list
}

View File

@@ -1,9 +1,7 @@
package mizu_test
import (
"fmt"
"github.com/up9inc/mizu/cli/mizu"
"reflect"
"testing"
)
@@ -90,41 +88,3 @@ func TestContainsNilSlice(t *testing.T) {
})
}
}
func TestUniqueNoDuplicateValues(t *testing.T) {
tests := []struct {
Slice []string
Expected []string
}{
{Slice: []string{"apple", "orange", "banana", "grapes"}, Expected: []string{"apple", "orange", "banana", "grapes"}},
{Slice: []string{"dog", "cat", "mouse"}, Expected: []string{"dog", "cat", "mouse"}},
}
for index, test := range tests {
t.Run(fmt.Sprintf("%v", index), func(t *testing.T) {
actual := mizu.Unique(test.Slice)
if !reflect.DeepEqual(test.Expected, actual) {
t.Errorf("unexpected result - Expected: %v, actual: %v", test.Expected, actual)
}
})
}
}
func TestUniqueDuplicateValues(t *testing.T) {
tests := []struct {
Slice []string
Expected []string
}{
{Slice: []string{"apple", "apple", "orange", "orange", "banana", "banana", "grapes", "grapes"}, Expected: []string{"apple", "orange", "banana", "grapes"}},
{Slice: []string{"dog", "cat", "cat", "mouse"}, Expected: []string{"dog", "cat", "mouse"}},
}
for index, test := range tests {
t.Run(fmt.Sprintf("%v", index), func(t *testing.T) {
actual := mizu.Unique(test.Slice)
if !reflect.DeepEqual(test.Expected, actual) {
t.Errorf("unexpected result - Expected: %v, actual: %v", test.Expected, actual)
}
})
}
}

View File

@@ -2,21 +2,43 @@ package version
import (
"context"
"encoding/json"
"fmt"
"github.com/up9inc/mizu/cli/apiserver"
"github.com/up9inc/mizu/cli/logger"
"github.com/up9inc/mizu/cli/mizu"
"io/ioutil"
"net/http"
"net/url"
"time"
"github.com/google/go-github/v37/github"
"github.com/up9inc/mizu/cli/uiUtils"
"github.com/up9inc/mizu/shared"
"github.com/up9inc/mizu/shared/semver"
)
func CheckVersionCompatibility() (bool, error) {
apiSemVer, err := apiserver.Provider.GetVersion()
func getApiVersion(port uint16) (string, error) {
versionUrl, _ := url.Parse(fmt.Sprintf("http://localhost:%d/mizu/metadata/version", port))
req := &http.Request{
Method: http.MethodGet,
URL: versionUrl,
}
statusResp, err := http.DefaultClient.Do(req)
if err != nil {
return "", err
}
defer statusResp.Body.Close()
versionResponse := &shared.VersionResponse{}
if err := json.NewDecoder(statusResp.Body).Decode(&versionResponse); err != nil {
return "", err
}
return versionResponse.SemVer, nil
}
func CheckVersionCompatibility(port uint16) (bool, error) {
apiSemVer, err := getApiVersion(port)
if err != nil {
return false, err
}

View File

@@ -5,10 +5,11 @@ import (
"encoding/json"
"fmt"
"github.com/denisbrodbeck/machineid"
"github.com/up9inc/mizu/cli/apiserver"
"github.com/up9inc/mizu/cli/config"
"github.com/up9inc/mizu/cli/kubernetes"
"github.com/up9inc/mizu/cli/logger"
"github.com/up9inc/mizu/cli/mizu"
"io/ioutil"
"net/http"
)
@@ -34,15 +35,35 @@ func ReportRun(cmd string, args interface{}) {
logger.Log.Debugf("successfully reported telemetry for cmd %v", cmd)
}
func ReportAPICalls() {
func ReportAPICalls(mizuPort uint16) {
if !shouldRunTelemetry() {
logger.Log.Debugf("not reporting telemetry")
return
}
generalStats, err := apiserver.Provider.GetGeneralStats()
if err != nil {
logger.Log.Debugf("[ERROR] failed get general stats from api server %v", err)
mizuProxiedUrl := kubernetes.GetMizuApiServerProxiedHostAndPath(mizuPort)
generalStatsUrl := fmt.Sprintf("http://%s/api/generalStats", mizuProxiedUrl)
response, requestErr := http.Get(generalStatsUrl)
if requestErr != nil {
logger.Log.Debugf("ERROR: failed to get general stats for telemetry, err: %v", requestErr)
return
} else if response.StatusCode != 200 {
logger.Log.Debugf("ERROR: failed to get general stats for telemetry, status code: %v", response.StatusCode)
return
}
defer func() { _ = response.Body.Close() }()
data, readErr := ioutil.ReadAll(response.Body)
if readErr != nil {
logger.Log.Debugf("ERROR: failed to read general stats for telemetry, err: %v", readErr)
return
}
var generalStats map[string]interface{}
if parseErr := json.Unmarshal(data, &generalStats); parseErr != nil {
logger.Log.Debugf("ERROR: failed to parse general stats for telemetry, err: %v", parseErr)
return
}