Compare commits

..

39 Commits

Author SHA1 Message Date
Roy Island
4badaadcc1 Merge remote-tracking branch 'origin/develop' 2021-11-07 12:31:42 +02:00
RamiBerm
a866576cfc TRA-3903 more docs (#438)
* Update README.md and PERMISSIONS.md

* Update PERMISSIONS.md

* Update README.md

* Update PERMISSIONS.md

* Update PERMISSIONS.md
2021-11-07 12:23:15 +02:00
M. Mert Yıldıran
6811bd5050 Remove the ERROR level logs in loadOAS and let the caller handle them in INFO level (#434) 2021-11-07 12:40:57 +03:00
M. Mert Yıldıran
4009386d82 Set a default value for SEM_VER build-time variable in Dockerfile (#435)
Also fix a runtime error that happens when the API server's version is not a valid SemVer.
2021-11-07 12:36:02 +03:00
RamiBerm
a6ebc460b0 TRA-3903 add daemon flag readme doc (#437)
* add daemon flag readme doc

* Update README.md
2021-11-07 09:33:19 +02:00
Alex Haiut
3f01f20f0c upgrade alpine base image (#413) 2021-10-28 17:00:42 +03:00
RoyUP9
1fbb00f8f0 Merge pull request #398 from up9inc/develop
Develop -> Main #patch
2021-10-25 13:15:41 +03:00
Igor Gov
da7d3590fc Merge pull request #394 from up9inc/develop
Develop -> main
2021-10-24 14:35:06 +03:00
gadotroee
256006ca3e Merge pull request #332 - update download link fix
#minor
2021-10-07 19:45:05 +03:00
Roee Gadot
213528c619 no message 2021-10-07 19:41:51 +03:00
Igor Gov
8b47dba05d Merge pull request #326 from up9inc/develop
Develop -> Main
2021-10-07 12:28:21 +03:00
RoyUP9
5e5d5de91a Merge pull request #297 from up9inc/develop
Develop -> main
2021-09-22 12:14:07 +03:00
Igor Gov
680ea71958 Merge branch 'develop'
# Conflicts:
#	acceptanceTests/tap_test.go
#	cli/apiserver/provider.go
#	cli/cmd/common.go
#	cli/cmd/fetch.go
#	cli/cmd/fetchRunner.go
#	cli/cmd/tapRunner.go
#	cli/cmd/viewRunner.go
#	cli/config/config.go
#	cli/mizu/fsUtils/mizuLogsUtils.go
2021-09-02 12:17:57 +03:00
Igor Gov
5fb5dbbbf5 Fixing call to analysis (#248) 2021-08-30 11:16:55 +03:00
RoyUP9
b3fe448ff1 added custom config path option (#247) 2021-08-30 11:16:55 +03:00
RoyUP9
101a54e8da added tap acceptance tests, fixed duplicate namespace problem (#244) 2021-08-30 11:16:55 +03:00
Igor Gov
3308cab826 Introducing API server provider (#243) 2021-08-30 11:16:55 +03:00
RoyUP9
5fdd8288f4 added tapper count route and wait time for tappers in test (#226) 2021-08-30 11:16:55 +03:00
Alon Girmonsky
4cb32b40e6 some changes in the read me (#241)
change prerequisite to permissions and kubeconfig. These are more FYIs as Mizu requires very little prerequisites. 
Change the description to match getmizu.io
2021-08-30 11:16:55 +03:00
Igor Gov
afa81c7ec2 Fixing bad conflict resolution 2021-08-19 13:33:14 +03:00
Igor Gov
e84c7d3310 Merge branch 'develop' 2021-08-19 13:18:06 +03:00
Igor Gov
7d0a90cb78 Merge branch 'main' into develop
# Conflicts:
#	cli/config/configStruct.go
#	cli/mizu/config.go
#	tap/http_reader.go
2021-08-19 13:16:19 +03:00
Nimrod Gilboa Markevich
24f79922e9 Add to periodic stats print in tapper (#221)
#patch
2021-08-16 15:50:04 +03:00
RoyUP9
c3995009ee Hotfix - ignore not allowed set flags (#192)
#patch
2021-08-10 14:21:16 +03:00
RoyUP9
6e9fe2986e removed duplicate har page header (#187) 2021-08-09 13:31:53 +03:00
RoyUP9
603240fedb temp fix - ignore agent image in config command (#185) 2021-08-09 11:55:45 +03:00
Igor Gov
e61871a68e Merge pull request #182 from up9inc/develop
Release 2021-08-08
2021-08-08 14:50:30 +03:00
nimrod-up9
379af59f07 Merge pull request #121 from up9inc/develop
Missing request body (#120)
2021-07-19 13:53:49 +03:00
gadotroee
ef9afe31a4 Merge pull request #119 from up9inc/develop
Mizu release
2021-07-18 16:54:31 +03:00
gadotroee
dca636b0fd Merge pull request #94 from up9inc/develop
Mizu release
2021-07-06 21:05:40 +03:00
Roee Gadot
9b72cc7aa6 Merge branch 'develop' into main
# Conflicts:
#	README.md
#	api/main.go
#	api/pkg/api/main.go
#	api/pkg/models/models.go
#	api/pkg/resolver/resolver.go
#	cli/Makefile
#	cli/cmd/tap.go
#	cli/cmd/tapRunner.go
#	tap/http_matcher.go
#	tap/http_reader.go
#	tap/tcp_stream_factory.go
2021-06-29 11:16:47 +03:00
Alex Haiut
d3c023b3ba mizu release 2021-06-21 (#79)
* Show pod name and namespace (#61)

* WIP

* Update main.go, consts.go, and 2 more files...

* Update messageSensitiveDataCleaner.go

* Update consts.go and messageSensitiveDataCleaner.go

* Update messageSensitiveDataCleaner.go

* Update main.go, consts.go, and 3 more files...

* WIP

* Update main.go, messageSensitiveDataCleaner.go, and 6 more files...

* Update main.go, messageSensitiveDataCleaner.go, and 3 more files...

* Update consts.go, messageSensitiveDataCleaner.go, and tap.go

* Update provider.go

* Update serializableRegexp.go

* Update tap.go

* TRA-3234 fetch with _source + no hard limit (#64)

* remove the HARD limit of 5000

* TRA-3299 Reduce footprint and Add Tolerances(#65)

* Use lib const for DNSClusterFirstWithHostNet.

* Whitespace.

* Break lines.

* Added affinity to pod names.

* Added tolerations to NoExecute and NoSchedule taints.

* Implementation of Mizu view command

* .

* .

* Update main.go and messageSensitiveDataCleaner.go

* Update main.go

* String and not pointers (#68)

* TRA-3318 - Cookies not null and fix har file names  (#69)

* no message

* TRA-3212 Passive-Tapper and Mizu share code (#70)

* Use log in tap package instead of fmt.

* Moved api/pkg/tap to root.

* Added go.mod and go.sum for tap.

* Added replace for shared.

* api uses tap module instead of tap package.

* Removed dependency of tap in shared by moving env var out of tap.

* Fixed compilation bugs.

* Fixed: Forgot to export struct field HostMode.

* Removed unused flag.

* Close har output channel when done.

* Moved websocket out of mizu and into passive-tapper.

* Send connection details over har output channel.

* Fixed compilation errors.

* Removed unused info from request response cache.

* Renamed connection -> connectionID.

* Fixed rename bug.

* Export setters and getters for filter ips and ports.

* Added tap dependency to Dockerfile.

* Uncomment error messages.

* Renamed `filterIpAddresses` -> `filterAuthorities`.

* Renamed ConnectionID -> ConnectionInfo.

* Fixed: Missed one replace.

* TRA-3342 Mizu/tap dump to har directory fails on Linux (#71)

* Instead of saving incomplete temp har files in a temp dir, save them in the output dir with a *.har.tmp suffix.

* API only loads har from *.har files (by extension).

* Add export entries endpoint for better up9 connect funcionality  (#72)

* no message
* no message
* no message

* Filter 'cookie' header

* Release action  (#73)

* Create main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* trying new approach

* no message

* yaml error

* no message

* no message

* no message

* missing )

* no message

* no message

* remove main.yml and fix branches

* Create tag-temp.yaml

* Update tag-temp.yaml

* Update tag-temp.yaml

* no message

* no message

* no message

* no message

* no message

* no message

* no message

* #minor

* no message

* no message

* added checksum calc to CLI makefile

* fixed build error - created bin directory upfront

* using markdown for release text

* use separate checksum files

* fixed release readme

* #minor

* readme updated

Co-authored-by: Alex Haiut <alex@up9.com>

* TRA-3360 Fix: Mizu ignores -n namespace flag and records traffic from all pods (#75)

Do not tap pods in namespaces which were not requested.

* added apple/m1 binary, updated readme (#77)

Co-authored-by: Alex Haiut <alex@up9.com>

* Update README.md (#78)

Co-authored-by: lirazyehezkel <61656597+lirazyehezkel@users.noreply.github.com>
Co-authored-by: RamiBerm <rami.berman@up9.com>
Co-authored-by: RamiBerm <54766858+RamiBerm@users.noreply.github.com>
Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
Co-authored-by: nimrod-up9 <59927337+nimrod-up9@users.noreply.github.com>
Co-authored-by: Igor Gov <igor.govorov1@gmail.com>
Co-authored-by: Alex Haiut <alex@up9.com>
2021-06-21 15:17:31 +03:00
Roee Gadot
5f2a4deb19 remove file 2021-05-26 18:08:37 +03:00
Roee Gadot
91f290987e Merge branch 'develop' into main
# Conflicts:
#	cli/cmd/tap.go
#	cli/cmd/version.go
#	cli/kubernetes/provider.go
#	cli/mizu/consts.go
#	cli/mizu/mizuRunner.go
#	debug.Dockerfile
#	ui/src/components/HarPage.tsx
2021-05-26 17:58:17 +03:00
gadotroee
2f3215b71a Fix mizu image parameter (#53) 2021-05-23 13:34:32 +03:00
Alex Haiut
2e87a01346 end of week - develop to master (#50)
* Provide cli version as git hash from makefile

* Update Makefile, version.go, and 3 more files...

* Update mizuRunner.go

* Update README.md, resolver.go, and 2 more files...

* Update provider.go

* Feature/UI/light theme (#44)

* light theme

* css polish

* unused code

* css

* text shadow

* footer style

* Update mizuRunner.go

* Handle nullable vars (#47)

* Decode gRPC body (#48)

* Decode grpc.

* Better variable names.

* Added protobuf-decoder dependency.

* Updated protobuf-decoder's version.

Co-authored-by: RamiBerm <rami.berman@up9.com>
Co-authored-by: RamiBerm <54766858+RamiBerm@users.noreply.github.com>
Co-authored-by: lirazyehezkel <61656597+lirazyehezkel@users.noreply.github.com>
Co-authored-by: nimrod-up9 <59927337+nimrod-up9@users.noreply.github.com>
2021-05-13 20:29:31 +03:00
gadotroee
453003bf14 remove leftovers (#43) 2021-05-10 17:35:59 +03:00
Roee Gadot
80ca377668 Merge branch 'develop' into main
# Conflicts:
#	Dockerfile
#	Makefile
#	api/go.mod
#	api/go.sum
#	api/main.go
#	api/pkg/controllers/entries_controller.go
#	api/pkg/inserter/main.go
#	api/pkg/models/models.go
#	api/pkg/tap/grpc_assembler.go
#	api/pkg/tap/har_writer.go
#	api/pkg/tap/http_matcher.go
#	api/pkg/tap/http_reader.go
#	api/pkg/tap/passive_tapper.go
#	api/pkg/utils/utils.go
#	cli/Makefile
#	cli/cmd/tap.go
#	cli/cmd/version.go
#	cli/config/config.go
#	cli/kubernetes/provider.go
#	cli/mizu/mizuRunner.go
2021-05-10 17:27:32 +03:00
gadotroee
d21297bc9c 0.9 (#37)
* Update .gitignore

* WIP

* WIP

* Update README.md, root.go, and 4 more files...

* Update README.md

* Update README.md

* Update root.go

* Update provider.go

* Update provider.go

* Update root.go, go.mod, and go.sum

* Update mizu.go

* Update go.sum and provider.go

* Update portForward.go, watch.go, and mizu.go

* Update README.md

* Update watch.go

* Update mizu.go

* Update mizu.go

* no message

* no message

* remove unused things and use external for object id (instead of copy)

* no message

* Update mizu.go

* Update go.mod, go.sum, and 2 more files...

* no message

* Update README.md, go.mod, and resolver.go

* Update README.md

* Update go.mod

* Update loader.go

* some refactor

* Update loader.go

* no message

* status to statusCode

* return data directly

* Traffic viewer

* cleaning

* css

* no message

* Clean warnings

* Makefile - first draft

* Update Makefile

* Update Makefile

* Update Makefile, README.md, and 4 more files...

* Add api build and clean to makefile (files restructure) (#9)

* no message
* add clean api command

* no message

* stating with web socket

* Add tap as a separate executable (#10)

* Added tap.

* Ignore build directories.

* Added tapper build to Makefile.

* Improvements  (#12)

* no message

* no message

* Feature/makefile (#11)

* minor fixes

* makefile fixes - docker build

* minor fix in Makefile
Co-authored-by: Alex Haiut <alex@up9.com>

* Update Dockerfile, multi-runner.sh, and 31 more files...

* Update multi-runner.sh

* no message

* Update .dockerignore, Dockerfile, and 30 more files...

* Update cleaner.go, grpc_assembler.go, and 2 more files...

* start the pod with host network and privileged

* fix multi runner passive tapper command

* add HOST_MODE env var

* do not return true in the should tap function

* remove line in the end

* default value in api is input
fix description and pass the parameter in the multi runner script

* missing flag.parse

* no message

* fix image

* Create main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Small fixes - permission + har writing exception (#17)

* Select node by pod (#18)

* Select node by pod.

* Removed watch pod by regex. Irrelevant for now.

* Changed default image to develop:latest.

* Features/clifix (#19)

* makefile fixes - docker build

* readme update, CLI usage fix

* added chmod

Co-authored-by: Alex Haiut <alex@up9.com>

* meta information

* Only record traffic of the requested pod. Filtered by pod IP. (#21)

* fixed readme and reduced batch size to 5 (#22)

Co-authored-by: Alex Haiut <alex@up9.com>

* API and TAP in single process  (#24)

* no message
* no message

* CLI make --pod required flag and faster api image build (#25)

* makefile fixes - docker build

* readme update, CLI usage fix

* added chmod

* typo

* run example incorreect in makefile

* no message

* no message

* no message

Co-authored-by: Alex Haiut <alex@up9.com>

* Reduce delay between tap and UI - Skip dump to file (#26)

* Pass HARs between tap and api via channel.

* Fixed make docker commad.

* Various fixes.

* Added .DS_Store to .gitignore.

* Parse flags in Mizu main instead of in tap_output.go.

* Use channel to pass HAR by default instead of files.

* Infinite scroll (#28)

* no message

* infinite scroll + new ws implementation

* no message

* scrolling top

* fetch button

* more Backend changes

* fix go mod and sum

* mire fixes against develop

* unused code

* small ui refactor

Co-authored-by: Roee Gadot <roee.gadot@up9.com>

* Fix gRPC crash, display gRPC as base64, display gRPC URL and status code (#27)

* Added Method (POST) and URL (emtpy) to gRPC requests.

* Removed quickfix that skips writing HTTP/2 to HAR.

* Use HTTP/2 body to fill out http.Request and htt.Response.

* Make sure that in HARs request.postData.mimeType and response.content.mimeType are application/grpc in case of grpc.

* Comment.

* Add URL and status code for gRPC.

* Don't assume http scheme.

* Use http.Header.Set instead of manually acccessing the underlaying map.

* General stats api fix  (#29)

* refactor and validation

* Show gRPC as ASCII (#31)

* Moved try-catch up one block.

* Display grpc as ASCII.

* Better code in entries fetch endpoint (#30)

* no message
* no message

* Feature/UI/filters (#32)

* UI filters

* refactor

* Revert "refactor"

This reverts commit 70e7d4b6ac.

* remove recursive func

* CLI cleanup (#33)

* Moved cli root command to tap subcommand.

* tap subcommand works.

* Added view and fetch placeholders.

* Updated descriptions.

* Fixed indentation.

* Added versio subcommand.

* Removed version flag.

* gofmt.

* Changed pod from flag to arg.

* Commented out "all namespaces" flag.

* CLI cleanup 2 (#34)

* Renamed dashboard -> GUI/web interface.

* Commented out --quiet, removed unused config variables.

* Quiter output when calling unimplemented subcommands.

* Leftovers from PR #30 (#36)

Co-authored-by: up9-github <info@up9.com>
Co-authored-by: RamiBerm <54766858+RamiBerm@users.noreply.github.com>
Co-authored-by: Liraz Yehezkel <lirazy@up9.com>
Co-authored-by: Alex Haiut <alex@testr.io>
Co-authored-by: lirazyehezkel <61656597+lirazyehezkel@users.noreply.github.com>
Co-authored-by: Alex Haiut <alex@up9.com>
Co-authored-by: nimrod-up9 <59927337+nimrod-up9@users.noreply.github.com>
Co-authored-by: RamiBerm <rami.berman@up9.com>
Co-authored-by: Alex Haiut <alex.haiut@gmail.com>
2021-05-09 11:45:39 +03:00
7 changed files with 24 additions and 9 deletions

View File

@@ -29,7 +29,7 @@ RUN go list -f '{{.Path}}@{{.Version}}' -m all | sed 1d | grep -e 'go-cache' -e
ARG COMMIT_HASH
ARG GIT_BRANCH
ARG BUILD_TIMESTAMP
ARG SEM_VER
ARG SEM_VER=0.0.0
# Copy and build agent code
COPY shared ../shared

View File

@@ -180,6 +180,16 @@ tap
proxy-host: 0.0.0.0
and when changed it will support accessing by IP
### Run in daemon mode
Mizu can be ran detached from the cli using the daemon flag: `mizu tap --daemon`. This type of mizu instance will run indefinitely in the cluster.
Please note that daemon mode requires you to have RBAC creation permissions, see the [permissions](docs/PERMISSIONS.md) doc for more details.
In order to access a daemon mizu you will have to run `mizu view` after running the `tap --daemon` command.
To stop the detached mizu instance and clean all cluster side resources, run `mizu clean`
## How to Run local UI

View File

@@ -13,7 +13,6 @@ import (
legacyrouter "github.com/getkin/kin-openapi/routers/legacy"
"github.com/up9inc/mizu/shared"
"github.com/up9inc/mizu/shared/logger"
"github.com/up9inc/mizu/tap/api"
)
@@ -27,7 +26,6 @@ func loadOAS(ctx context.Context) (doc *openapi3.T, contractContent string, rout
path := fmt.Sprintf("%s%s", shared.ConfigDirPath, shared.ContractFileName)
bytes, err := ioutil.ReadFile(path)
if err != nil {
logger.Log.Error(err.Error())
return
}
contractContent = string(bytes)
@@ -35,7 +33,6 @@ func loadOAS(ctx context.Context) (doc *openapi3.T, contractContent string, rout
doc, _ = loader.LoadFromData(bytes)
err = doc.Validate(ctx)
if err != nil {
logger.Log.Error(err.Error())
return
}
router, _ = legacyrouter.NewRouter(doc)

View File

@@ -5,12 +5,13 @@ import (
"encoding/json"
"errors"
"fmt"
"github.com/up9inc/mizu/shared/kubernetes"
"io/ioutil"
"net/http"
"net/url"
"time"
"github.com/up9inc/mizu/shared/kubernetes"
"github.com/up9inc/mizu/cli/config"
"github.com/up9inc/mizu/shared"
"github.com/up9inc/mizu/shared/logger"
@@ -28,7 +29,7 @@ const DefaultTimeout = 5 * time.Second
func NewProvider(url string, retries int, timeout time.Duration) *Provider {
return &Provider{
url: url,
url: url,
retries: config.GetIntEnvConfig(config.ApiServerRetries, retries),
client: &http.Client{
Timeout: timeout,

View File

@@ -24,6 +24,11 @@ func CheckVersionCompatibility(apiServerProvider *apiserver.Provider) (bool, err
return false, err
}
if !semver.SemVersion(apiSemVer).IsValid() {
logger.Log.Errorf(uiUtils.Red, fmt.Sprintf("api version (%s) is not a valid SemVer", apiSemVer))
return false, nil
}
if semver.SemVersion(apiSemVer).Major() == semver.SemVersion(mizu.SemVer).Major() &&
semver.SemVersion(apiSemVer).Minor() == semver.SemVersion(mizu.SemVer).Minor() {
return true, nil

View File

@@ -28,7 +28,7 @@ RUN go list -f '{{.Path}}@{{.Version}}' -m all | sed 1d | grep -e 'go-cache' -e
ARG COMMIT_HASH
ARG GIT_BRANCH
ARG BUILD_TIMESTAMP
ARG SEM_VER
ARG SEM_VER=0.0.0
# Copy and build agent code
COPY shared ../shared

View File

@@ -57,9 +57,11 @@ Mizu needs following permissions on your Kubernetes cluster to run properly
- get
```
## Permissions required for service / pod name resolving (opt)
## Permissions required running with --daemon flag or (optional) for service / pod name resolving
Optionally, for proper resolving of IP addresses to Kubernetes service name, Mizu needs below permissions:
Mandatory permissions for running with `--daemon` flag.
Optional for service/pod name resolving in non daemon mode
```yaml
- apiGroups: