Compare commits

..

8 Commits
0.5.1 ... 0.3.0

Author SHA1 Message Date
Alex Haiut
d3c023b3ba mizu release 2021-06-21 (#79)
* Show pod name and namespace (#61)

* WIP

* Update main.go, consts.go, and 2 more files...

* Update messageSensitiveDataCleaner.go

* Update consts.go and messageSensitiveDataCleaner.go

* Update messageSensitiveDataCleaner.go

* Update main.go, consts.go, and 3 more files...

* WIP

* Update main.go, messageSensitiveDataCleaner.go, and 6 more files...

* Update main.go, messageSensitiveDataCleaner.go, and 3 more files...

* Update consts.go, messageSensitiveDataCleaner.go, and tap.go

* Update provider.go

* Update serializableRegexp.go

* Update tap.go

* TRA-3234 fetch with _source + no hard limit (#64)

* remove the HARD limit of 5000

* TRA-3299 Reduce footprint and Add Tolerances(#65)

* Use lib const for DNSClusterFirstWithHostNet.

* Whitespace.

* Break lines.

* Added affinity to pod names.

* Added tolerations to NoExecute and NoSchedule taints.

* Implementation of Mizu view command

* .

* .

* Update main.go and messageSensitiveDataCleaner.go

* Update main.go

* String and not pointers (#68)

* TRA-3318 - Cookies not null and fix har file names  (#69)

* no message

* TRA-3212 Passive-Tapper and Mizu share code (#70)

* Use log in tap package instead of fmt.

* Moved api/pkg/tap to root.

* Added go.mod and go.sum for tap.

* Added replace for shared.

* api uses tap module instead of tap package.

* Removed dependency of tap in shared by moving env var out of tap.

* Fixed compilation bugs.

* Fixed: Forgot to export struct field HostMode.

* Removed unused flag.

* Close har output channel when done.

* Moved websocket out of mizu and into passive-tapper.

* Send connection details over har output channel.

* Fixed compilation errors.

* Removed unused info from request response cache.

* Renamed connection -> connectionID.

* Fixed rename bug.

* Export setters and getters for filter ips and ports.

* Added tap dependency to Dockerfile.

* Uncomment error messages.

* Renamed `filterIpAddresses` -> `filterAuthorities`.

* Renamed ConnectionID -> ConnectionInfo.

* Fixed: Missed one replace.

* TRA-3342 Mizu/tap dump to har directory fails on Linux (#71)

* Instead of saving incomplete temp har files in a temp dir, save them in the output dir with a *.har.tmp suffix.

* API only loads har from *.har files (by extension).

* Add export entries endpoint for better up9 connect funcionality  (#72)

* no message
* no message
* no message

* Filter 'cookie' header

* Release action  (#73)

* Create main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* trying new approach

* no message

* yaml error

* no message

* no message

* no message

* missing )

* no message

* no message

* remove main.yml and fix branches

* Create tag-temp.yaml

* Update tag-temp.yaml

* Update tag-temp.yaml

* no message

* no message

* no message

* no message

* no message

* no message

* no message

* #minor

* no message

* no message

* added checksum calc to CLI makefile

* fixed build error - created bin directory upfront

* using markdown for release text

* use separate checksum files

* fixed release readme

* #minor

* readme updated

Co-authored-by: Alex Haiut <alex@up9.com>

* TRA-3360 Fix: Mizu ignores -n namespace flag and records traffic from all pods (#75)

Do not tap pods in namespaces which were not requested.

* added apple/m1 binary, updated readme (#77)

Co-authored-by: Alex Haiut <alex@up9.com>

* Update README.md (#78)

Co-authored-by: lirazyehezkel <61656597+lirazyehezkel@users.noreply.github.com>
Co-authored-by: RamiBerm <rami.berman@up9.com>
Co-authored-by: RamiBerm <54766858+RamiBerm@users.noreply.github.com>
Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
Co-authored-by: nimrod-up9 <59927337+nimrod-up9@users.noreply.github.com>
Co-authored-by: Igor Gov <igor.govorov1@gmail.com>
Co-authored-by: Alex Haiut <alex@up9.com>
2021-06-21 15:17:31 +03:00
Roee Gadot
5f2a4deb19 remove file 2021-05-26 18:08:37 +03:00
Roee Gadot
91f290987e Merge branch 'develop' into main
# Conflicts:
#	cli/cmd/tap.go
#	cli/cmd/version.go
#	cli/kubernetes/provider.go
#	cli/mizu/consts.go
#	cli/mizu/mizuRunner.go
#	debug.Dockerfile
#	ui/src/components/HarPage.tsx
2021-05-26 17:58:17 +03:00
gadotroee
2f3215b71a Fix mizu image parameter (#53) 2021-05-23 13:34:32 +03:00
Alex Haiut
2e87a01346 end of week - develop to master (#50)
* Provide cli version as git hash from makefile

* Update Makefile, version.go, and 3 more files...

* Update mizuRunner.go

* Update README.md, resolver.go, and 2 more files...

* Update provider.go

* Feature/UI/light theme (#44)

* light theme

* css polish

* unused code

* css

* text shadow

* footer style

* Update mizuRunner.go

* Handle nullable vars (#47)

* Decode gRPC body (#48)

* Decode grpc.

* Better variable names.

* Added protobuf-decoder dependency.

* Updated protobuf-decoder's version.

Co-authored-by: RamiBerm <rami.berman@up9.com>
Co-authored-by: RamiBerm <54766858+RamiBerm@users.noreply.github.com>
Co-authored-by: lirazyehezkel <61656597+lirazyehezkel@users.noreply.github.com>
Co-authored-by: nimrod-up9 <59927337+nimrod-up9@users.noreply.github.com>
2021-05-13 20:29:31 +03:00
gadotroee
453003bf14 remove leftovers (#43) 2021-05-10 17:35:59 +03:00
Roee Gadot
80ca377668 Merge branch 'develop' into main
# Conflicts:
#	Dockerfile
#	Makefile
#	api/go.mod
#	api/go.sum
#	api/main.go
#	api/pkg/controllers/entries_controller.go
#	api/pkg/inserter/main.go
#	api/pkg/models/models.go
#	api/pkg/tap/grpc_assembler.go
#	api/pkg/tap/har_writer.go
#	api/pkg/tap/http_matcher.go
#	api/pkg/tap/http_reader.go
#	api/pkg/tap/passive_tapper.go
#	api/pkg/utils/utils.go
#	cli/Makefile
#	cli/cmd/tap.go
#	cli/cmd/version.go
#	cli/config/config.go
#	cli/kubernetes/provider.go
#	cli/mizu/mizuRunner.go
2021-05-10 17:27:32 +03:00
gadotroee
d21297bc9c 0.9 (#37)
* Update .gitignore

* WIP

* WIP

* Update README.md, root.go, and 4 more files...

* Update README.md

* Update README.md

* Update root.go

* Update provider.go

* Update provider.go

* Update root.go, go.mod, and go.sum

* Update mizu.go

* Update go.sum and provider.go

* Update portForward.go, watch.go, and mizu.go

* Update README.md

* Update watch.go

* Update mizu.go

* Update mizu.go

* no message

* no message

* remove unused things and use external for object id (instead of copy)

* no message

* Update mizu.go

* Update go.mod, go.sum, and 2 more files...

* no message

* Update README.md, go.mod, and resolver.go

* Update README.md

* Update go.mod

* Update loader.go

* some refactor

* Update loader.go

* no message

* status to statusCode

* return data directly

* Traffic viewer

* cleaning

* css

* no message

* Clean warnings

* Makefile - first draft

* Update Makefile

* Update Makefile

* Update Makefile, README.md, and 4 more files...

* Add api build and clean to makefile (files restructure) (#9)

* no message
* add clean api command

* no message

* stating with web socket

* Add tap as a separate executable (#10)

* Added tap.

* Ignore build directories.

* Added tapper build to Makefile.

* Improvements  (#12)

* no message

* no message

* Feature/makefile (#11)

* minor fixes

* makefile fixes - docker build

* minor fix in Makefile
Co-authored-by: Alex Haiut <alex@up9.com>

* Update Dockerfile, multi-runner.sh, and 31 more files...

* Update multi-runner.sh

* no message

* Update .dockerignore, Dockerfile, and 30 more files...

* Update cleaner.go, grpc_assembler.go, and 2 more files...

* start the pod with host network and privileged

* fix multi runner passive tapper command

* add HOST_MODE env var

* do not return true in the should tap function

* remove line in the end

* default value in api is input
fix description and pass the parameter in the multi runner script

* missing flag.parse

* no message

* fix image

* Create main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Small fixes - permission + har writing exception (#17)

* Select node by pod (#18)

* Select node by pod.

* Removed watch pod by regex. Irrelevant for now.

* Changed default image to develop:latest.

* Features/clifix (#19)

* makefile fixes - docker build

* readme update, CLI usage fix

* added chmod

Co-authored-by: Alex Haiut <alex@up9.com>

* meta information

* Only record traffic of the requested pod. Filtered by pod IP. (#21)

* fixed readme and reduced batch size to 5 (#22)

Co-authored-by: Alex Haiut <alex@up9.com>

* API and TAP in single process  (#24)

* no message
* no message

* CLI make --pod required flag and faster api image build (#25)

* makefile fixes - docker build

* readme update, CLI usage fix

* added chmod

* typo

* run example incorreect in makefile

* no message

* no message

* no message

Co-authored-by: Alex Haiut <alex@up9.com>

* Reduce delay between tap and UI - Skip dump to file (#26)

* Pass HARs between tap and api via channel.

* Fixed make docker commad.

* Various fixes.

* Added .DS_Store to .gitignore.

* Parse flags in Mizu main instead of in tap_output.go.

* Use channel to pass HAR by default instead of files.

* Infinite scroll (#28)

* no message

* infinite scroll + new ws implementation

* no message

* scrolling top

* fetch button

* more Backend changes

* fix go mod and sum

* mire fixes against develop

* unused code

* small ui refactor

Co-authored-by: Roee Gadot <roee.gadot@up9.com>

* Fix gRPC crash, display gRPC as base64, display gRPC URL and status code (#27)

* Added Method (POST) and URL (emtpy) to gRPC requests.

* Removed quickfix that skips writing HTTP/2 to HAR.

* Use HTTP/2 body to fill out http.Request and htt.Response.

* Make sure that in HARs request.postData.mimeType and response.content.mimeType are application/grpc in case of grpc.

* Comment.

* Add URL and status code for gRPC.

* Don't assume http scheme.

* Use http.Header.Set instead of manually acccessing the underlaying map.

* General stats api fix  (#29)

* refactor and validation

* Show gRPC as ASCII (#31)

* Moved try-catch up one block.

* Display grpc as ASCII.

* Better code in entries fetch endpoint (#30)

* no message
* no message

* Feature/UI/filters (#32)

* UI filters

* refactor

* Revert "refactor"

This reverts commit 70e7d4b6ac.

* remove recursive func

* CLI cleanup (#33)

* Moved cli root command to tap subcommand.

* tap subcommand works.

* Added view and fetch placeholders.

* Updated descriptions.

* Fixed indentation.

* Added versio subcommand.

* Removed version flag.

* gofmt.

* Changed pod from flag to arg.

* Commented out "all namespaces" flag.

* CLI cleanup 2 (#34)

* Renamed dashboard -> GUI/web interface.

* Commented out --quiet, removed unused config variables.

* Quiter output when calling unimplemented subcommands.

* Leftovers from PR #30 (#36)

Co-authored-by: up9-github <info@up9.com>
Co-authored-by: RamiBerm <54766858+RamiBerm@users.noreply.github.com>
Co-authored-by: Liraz Yehezkel <lirazy@up9.com>
Co-authored-by: Alex Haiut <alex@testr.io>
Co-authored-by: lirazyehezkel <61656597+lirazyehezkel@users.noreply.github.com>
Co-authored-by: Alex Haiut <alex@up9.com>
Co-authored-by: nimrod-up9 <59927337+nimrod-up9@users.noreply.github.com>
Co-authored-by: RamiBerm <rami.berman@up9.com>
Co-authored-by: Alex Haiut <alex.haiut@gmail.com>
2021-05-09 11:45:39 +03:00
38 changed files with 224 additions and 844 deletions

View File

@@ -1,5 +1,5 @@
# 水 mizu
A simple-yet-powerful API traffic viewer for Kubernetes to help you troubleshoot and debug your microservices. Think TCPDump and Chrome Dev Tools combined.
standalone web app traffic viewer for Kubernetes
## Download
@@ -13,6 +13,13 @@ curl -Lo mizu \
https://github.com/up9inc/mizu/releases/latest/download/mizu_darwin_amd64 \
&& chmod 755 mizu
```
* for MacOS - Apple Silicon
```
curl -Lo mizu \
https://github.com/up9inc/mizu/releases/latest/download/mizu_darwin_arm64 \
&& chmod 755 mizu
```
* for Linux - Intel 64bit
```
@@ -21,6 +28,7 @@ https://github.com/up9inc/mizu/releases/latest/download/mizu_linux_amd64 \
&& chmod 755 mizu
```
SHA256 checksums are available on the [Releases](https://github.com/up9inc/mizu/releases) page.
### Development (unstable) build
@@ -29,7 +37,7 @@ Pick one from the [Releases](https://github.com/up9inc/mizu/releases) page.
## How to run
1. Find pod you'd like to tap to in your Kubernetes cluster
2. Run `mizu tap PODNAME` or `mizu tap REGEX`
2. Run `mizu PODNAME` or `mizu REGEX`
3. Open browser on `http://localhost:8899` as instructed ..
4. Watch the WebAPI traffic flowing ..
5. Type ^C to stop

View File

@@ -11,12 +11,16 @@ require (
github.com/go-playground/universal-translator v0.17.0
github.com/go-playground/validator/v10 v10.5.0
github.com/gofiber/fiber/v2 v2.8.0
github.com/google/gopacket v1.1.19
github.com/google/martian v2.1.0+incompatible
github.com/gorilla/websocket v1.4.2
github.com/leodido/go-urn v1.2.1 // indirect
github.com/orcaman/concurrent-map v0.0.0-20210106121528-16402b402231
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/up9inc/mizu/shared v0.0.0
github.com/up9inc/mizu/tap v0.0.0
go.mongodb.org/mongo-driver v1.5.1
golang.org/x/net v0.0.0-20210421230115-4e50805a0758
gorm.io/driver/sqlite v1.1.4
gorm.io/gorm v1.21.8
k8s.io/api v0.21.0
@@ -25,5 +29,4 @@ require (
)
replace github.com/up9inc/mizu/shared v0.0.0 => ../shared
replace github.com/up9inc/mizu/tap v0.0.0 => ../tap

View File

@@ -251,6 +251,7 @@ github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c/go.mod h1:C1qb7wdrVGGV
github.com/onsi/gomega v1.7.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/orcaman/concurrent-map v0.0.0-20210106121528-16402b402231 h1:fa50YL1pzKW+1SsBnJDOHppJN9stOEwS+CRWyUtyYGU=
github.com/orcaman/concurrent-map v0.0.0-20210106121528-16402b402231/go.mod h1:Lu3tH6HLW3feq74c2GC+jIMS/K2CFcDWnWD9XkenwhI=
github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
github.com/pelletier/go-toml v1.7.0/go.mod h1:vwGMzjaWMwyfHwgIBhI2YUM4fB6nL6lVAvS1LBMMhTE=
github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU=

View File

@@ -37,7 +37,7 @@ func main() {
harOutputChannel, outboundLinkOutputChannel := tap.StartPassiveTapper(tapOpts)
filteredHarChannel := make(chan *tap.OutputChannelItem)
go filterHarItems(harOutputChannel, filteredHarChannel, getTrafficFilteringOptions())
go filterHarHeaders(harOutputChannel, filteredHarChannel, getTrafficFilteringOptions())
go api.StartReadingEntries(filteredHarChannel, nil)
go api.StartReadingOutbound(outboundLinkOutputChannel)
@@ -66,8 +66,8 @@ func main() {
socketHarOutChannel := make(chan *tap.OutputChannelItem, 1000)
filteredHarChannel := make(chan *tap.OutputChannelItem)
go filterHarItems(socketHarOutChannel, filteredHarChannel, getTrafficFilteringOptions())
go api.StartReadingEntries(filteredHarChannel, nil)
go filterHarHeaders(socketHarOutChannel, filteredHarChannel, getTrafficFilteringOptions())
hostApi(socketHarOutChannel)
}
@@ -125,14 +125,9 @@ func getTrafficFilteringOptions() *shared.TrafficFilteringOptions {
return &filteringOptions
}
func filterHarItems(inChannel <- chan *tap.OutputChannelItem, outChannel chan *tap.OutputChannelItem, filterOptions *shared.TrafficFilteringOptions) {
func filterHarHeaders(inChannel <- chan *tap.OutputChannelItem, outChannel chan *tap.OutputChannelItem, filterOptions *shared.TrafficFilteringOptions) {
for message := range inChannel {
if message.ConnectionInfo.IsOutgoing && api.CheckIsServiceIP(message.ConnectionInfo.ServerIP) {
continue
}
sensitiveDataFiltering.FilterSensitiveInfoFromHarRequest(message, filterOptions)
outChannel <- message
}
}

View File

@@ -5,6 +5,10 @@ import (
"context"
"encoding/json"
"fmt"
"mizuserver/pkg/database"
"mizuserver/pkg/models"
"mizuserver/pkg/resolver"
"mizuserver/pkg/utils"
"net/url"
"os"
"path"
@@ -15,11 +19,6 @@ import (
"github.com/google/martian/har"
"github.com/up9inc/mizu/tap"
"go.mongodb.org/mongo-driver/bson/primitive"
"mizuserver/pkg/database"
"mizuserver/pkg/models"
"mizuserver/pkg/resolver"
"mizuserver/pkg/utils"
)
var k8sResolver *resolver.Resolver
@@ -85,14 +84,7 @@ func startReadingFiles(workingDir string) {
for _, entry := range inputHar.Log.Entries {
time.Sleep(time.Millisecond * 250)
connectionInfo := &tap.ConnectionInfo{
ClientIP: fileInfo.Name(),
ClientPort: "",
ServerIP: "",
ServerPort: "",
IsOutgoing: false,
}
saveHarToDb(entry, connectionInfo)
saveHarToDb(entry, fileInfo.Name())
}
rmErr := os.Remove(inputFilePath)
utils.CheckErr(rmErr)
@@ -105,7 +97,7 @@ func startReadingChannel(outputItems <-chan *tap.OutputChannelItem) {
}
for item := range outputItems {
saveHarToDb(item.HarEntry, item.ConnectionInfo)
saveHarToDb(item.HarEntry, item.ConnectionInfo.ClientIP)
}
}
@@ -117,17 +109,17 @@ func StartReadingOutbound(outboundLinkChannel <-chan *tap.OutboundLink) {
}
func saveHarToDb(entry *har.Entry, connectionInfo *tap.ConnectionInfo) {
func saveHarToDb(entry *har.Entry, sender string) {
entryBytes, _ := json.Marshal(entry)
serviceName, urlPath := getServiceNameFromUrl(entry.Request.URL)
serviceName, urlPath, serviceHostName := getServiceNameFromUrl(entry.Request.URL)
entryId := primitive.NewObjectID().Hex()
var (
resolvedSource string
resolvedDestination string
)
if k8sResolver != nil {
resolvedSource = k8sResolver.Resolve(connectionInfo.ClientIP)
resolvedDestination = k8sResolver.Resolve(fmt.Sprintf("%s:%s", connectionInfo.ServerIP, connectionInfo.ServerPort))
resolvedSource = k8sResolver.Resolve(sender)
resolvedDestination = k8sResolver.Resolve(serviceHostName)
}
mizuEntry := models.MizuEntry{
EntryId: entryId,
@@ -137,28 +129,20 @@ func saveHarToDb(entry *har.Entry, connectionInfo *tap.ConnectionInfo) {
Path: urlPath,
Method: entry.Request.Method,
Status: entry.Response.Status,
RequestSenderIp: connectionInfo.ClientIP,
RequestSenderIp: sender,
Timestamp: entry.StartedDateTime.UnixNano() / int64(time.Millisecond),
ResolvedSource: resolvedSource,
ResolvedDestination: resolvedDestination,
IsOutgoing: connectionInfo.IsOutgoing,
}
database.GetEntriesTable().Create(&mizuEntry)
baseEntry := models.BaseEntryDetails{}
if err := models.GetEntry(&mizuEntry, &baseEntry); err != nil {
return
}
baseEntry := utils.GetResolvedBaseEntry(mizuEntry)
baseEntryBytes, _ := models.CreateBaseEntryWebSocketMessage(&baseEntry)
broadcastToBrowserClients(baseEntryBytes)
}
func getServiceNameFromUrl(inputUrl string) (string, string) {
func getServiceNameFromUrl(inputUrl string) (string, string, string) {
parsed, err := url.Parse(inputUrl)
utils.CheckErr(err)
return fmt.Sprintf("%s://%s", parsed.Scheme, parsed.Host), parsed.Path
}
func CheckIsServiceIP(address string) bool {
return k8sResolver.CheckIsServiceIP(address)
return fmt.Sprintf("%s://%s", parsed.Scheme, parsed.Host), parsed.Path, parsed.Host
}

View File

@@ -9,7 +9,6 @@ import (
"mizuserver/pkg/controllers"
"mizuserver/pkg/models"
"mizuserver/pkg/routes"
"mizuserver/pkg/up9"
)
var browserClientSocketUUIDs = make([]string, 0)
@@ -19,9 +18,6 @@ type RoutesEventHandlers struct {
SocketHarOutChannel chan<- *tap.OutputChannelItem
}
func init() {
go up9.UpdateAnalyzeStatus(broadcastToBrowserClients)
}
func (h *RoutesEventHandlers) WebSocketConnect(ep *ikisocket.EventPayload) {
if ep.Kws.GetAttribute("is_tapper") == true {
@@ -88,6 +84,7 @@ func (h *RoutesEventHandlers) WebSocketMessage(ep *ikisocket.EventPayload) {
}
}
func removeSocketUUIDFromBrowserSlice(uuidToRemove string) {
newUUIDSlice := make([]string, 0, len(browserClientSocketUUIDs))
for _, uuid := range browserClientSocketUUIDs {

View File

@@ -7,13 +7,29 @@ import (
"github.com/google/martian/har"
"mizuserver/pkg/database"
"mizuserver/pkg/models"
"mizuserver/pkg/up9"
"mizuserver/pkg/utils"
"mizuserver/pkg/validation"
"strings"
"time"
)
const (
OrderDesc = "desc"
OrderAsc = "asc"
LT = "lt"
GT = "gt"
)
var (
operatorToSymbolMapping = map[string]string{
LT: "<",
GT: ">",
}
operatorToOrderMapping = map[string]string{
LT: OrderDesc,
GT: OrderAsc,
}
)
func GetEntries(c *fiber.Ctx) error {
entriesFilter := &models.EntriesFilter{}
@@ -25,8 +41,8 @@ func GetEntries(c *fiber.Ctx) error {
return c.Status(fiber.StatusBadRequest).JSON(err)
}
order := database.OperatorToOrderMapping[entriesFilter.Operator]
operatorSymbol := database.OperatorToSymbolMapping[entriesFilter.Operator]
order := operatorToOrderMapping[entriesFilter.Operator]
operatorSymbol := operatorToSymbolMapping[entriesFilter.Operator]
var entries []models.MizuEntry
database.GetEntriesTable().
Order(fmt.Sprintf("timestamp %s", order)).
@@ -35,18 +51,15 @@ func GetEntries(c *fiber.Ctx) error {
Limit(entriesFilter.Limit).
Find(&entries)
if len(entries) > 0 && order == database.OrderDesc {
if len(entries) > 0 && order == OrderDesc {
// the entries always order from oldest to newest so we should revers
utils.ReverseSlice(entries)
}
baseEntries := make([]models.BaseEntryDetails, 0)
for _, data := range entries {
harEntry := models.BaseEntryDetails{}
if err := models.GetEntry(&data, &harEntry); err != nil {
continue
}
baseEntries = append(baseEntries, harEntry)
// Convert to base entries
baseEntries := make([]models.BaseEntryDetails, 0, entriesFilter.Limit)
for _, entry := range entries {
baseEntries = append(baseEntries, utils.GetResolvedBaseEntry(entry))
}
return c.Status(fiber.StatusOK).JSON(baseEntries)
@@ -54,7 +67,7 @@ func GetEntries(c *fiber.Ctx) error {
func GetHARs(c *fiber.Ctx) error {
entriesFilter := &models.HarFetchRequestBody{}
order := database.OrderDesc
order := OrderDesc
if err := c.QueryParser(entriesFilter); err != nil {
return c.Status(fiber.StatusBadRequest).JSON(err)
}
@@ -92,20 +105,9 @@ func GetHARs(c *fiber.Ctx) error {
for _, entryData := range entries {
var harEntry har.Entry
_ = json.Unmarshal([]byte(entryData.Entry), &harEntry)
if entryData.ResolvedDestination != "" {
harEntry.Request.URL = utils.SetHostname(harEntry.Request.URL, entryData.ResolvedDestination)
}
var fileName string
sourceOfEntry := entryData.ResolvedSource
if sourceOfEntry != "" {
// naively assumes the proper service source is http
sourceOfEntry = fmt.Sprintf("http://%s", sourceOfEntry)
//replace / from the file name cause they end up creating a corrupted folder
fileName = fmt.Sprintf("%s.har", strings.ReplaceAll(sourceOfEntry, "/", "_"))
} else {
fileName = "unknown_source.har"
}
fileName := fmt.Sprintf("%s.har", sourceOfEntry)
if harOfSource, ok := harsObject[fileName]; ok {
harOfSource.Log.Entries = append(harOfSource.Log.Entries, &harEntry)
} else {
@@ -119,14 +121,11 @@ func GetHARs(c *fiber.Ctx) error {
Name: "mizu",
Version: "0.0.2",
},
Source: sourceOfEntry,
},
Entries: entriesHar,
},
}
// leave undefined when no source is present, otherwise modeler assumes source is empty string ""
if sourceOfEntry != "" {
harsObject[fileName].Log.Creator.Source = &sourceOfEntry
}
}
}
@@ -139,25 +138,9 @@ func GetHARs(c *fiber.Ctx) error {
return c.Status(fiber.StatusOK).SendStream(buffer)
}
func UploadEntries(c *fiber.Ctx) error {
uploadRequestBody := &models.UploadEntriesRequestBody{}
if err := c.QueryParser(uploadRequestBody); err != nil {
return c.Status(fiber.StatusBadRequest).JSON(err)
}
if err := validation.Validate(uploadRequestBody); err != nil {
return c.Status(fiber.StatusBadRequest).JSON(err)
}
if up9.GetAnalyzeInfo().IsAnalyzing {
return c.Status(fiber.StatusBadRequest).SendString("Cannot analyze, mizu is already analyzing")
}
token, _ := up9.CreateAnonymousToken(uploadRequestBody.Dest)
go up9.UploadEntriesImpl(token.Token, token.Model, uploadRequestBody.Dest)
return c.Status(fiber.StatusOK).SendString("OK")
}
func GetFullEntries(c *fiber.Ctx) error {
entriesFilter := &models.HarFetchRequestBody{}
order := OrderDesc
if err := c.QueryParser(entriesFilter); err != nil {
return c.Status(fiber.StatusBadRequest).JSON(err)
}
@@ -179,32 +162,41 @@ func GetFullEntries(c *fiber.Ctx) error {
timestampTo = entriesFilter.To
}
entriesArray := database.GetEntriesFromDb(timestampFrom, timestampTo)
result := make([]models.FullEntryDetails, 0)
for _, data := range entriesArray {
harEntry := models.FullEntryDetails{}
if err := models.GetEntry(&data, &harEntry); err != nil {
continue
}
result = append(result, harEntry)
var entries []models.MizuEntry
database.GetEntriesTable().
Where(fmt.Sprintf("timestamp BETWEEN %v AND %v", timestampFrom, timestampTo)).
Order(fmt.Sprintf("timestamp %s", order)).
Find(&entries)
if len(entries) > 0 {
// the entries always order from oldest to newest so we should revers
utils.ReverseSlice(entries)
}
return c.Status(fiber.StatusOK).JSON(result)
entriesArray := make([]har.Entry, 0)
for _, entryData := range entries {
var harEntry har.Entry
_ = json.Unmarshal([]byte(entryData.Entry), &harEntry)
entriesArray = append(entriesArray, harEntry)
}
return c.Status(fiber.StatusOK).JSON(entriesArray)
}
func GetEntry(c *fiber.Ctx) error {
var entryData models.MizuEntry
var entryData models.EntryData
database.GetEntriesTable().
Select("entry", "resolvedDestination").
Where(map[string]string{"entryId": c.Params("entryId")}).
First(&entryData)
fullEntry := models.FullEntryDetails{}
if err := models.GetEntry(&entryData, &fullEntry); err != nil {
return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{
"error": true,
"msg": "Can't get entry details",
})
var fullEntry har.Entry
unmarshallErr := json.Unmarshal([]byte(entryData.Entry), &fullEntry)
utils.CheckErr(unmarshallErr)
if entryData.ResolvedDestination != "" {
fullEntry.Request.URL = utils.SetHostname(fullEntry.Request.URL, entryData.ResolvedDestination)
}
return c.Status(fiber.StatusOK).JSON(fullEntry)
}

View File

@@ -3,7 +3,6 @@ package controllers
import (
"github.com/gofiber/fiber/v2"
"github.com/up9inc/mizu/shared"
"mizuserver/pkg/up9"
)
var TapStatus shared.TapStatus
@@ -11,7 +10,3 @@ var TapStatus shared.TapStatus
func GetTappingStatus(c *fiber.Ctx) error {
return c.Status(fiber.StatusOK).JSON(TapStatus)
}
func AnalyzeInformation(c *fiber.Ctx) error {
return c.Status(fiber.StatusOK).JSON(up9.GetAnalyzeInfo())
}

View File

@@ -1,11 +1,9 @@
package database
import (
"fmt"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
"mizuserver/pkg/models"
"mizuserver/pkg/utils"
)
const (
@@ -16,24 +14,6 @@ var (
DB = initDataBase(DBPath)
)
const (
OrderDesc = "desc"
OrderAsc = "asc"
LT = "lt"
GT = "gt"
)
var (
OperatorToSymbolMapping = map[string]string{
LT: "<",
GT: ">",
}
OperatorToOrderMapping = map[string]string{
LT: OrderDesc,
GT: OrderAsc,
}
)
func GetEntriesTable() *gorm.DB {
return DB.Table("mizu_entries")
}
@@ -43,20 +23,3 @@ func initDataBase(databasePath string) *gorm.DB {
_ = temp.AutoMigrate(&models.MizuEntry{}) // this will ensure table is created
return temp
}
func GetEntriesFromDb(timestampFrom int64, timestampTo int64) []models.MizuEntry {
order := OrderDesc
var entries []models.MizuEntry
GetEntriesTable().
Where(fmt.Sprintf("timestamp BETWEEN %v AND %v", timestampFrom, timestampTo)).
Order(fmt.Sprintf("timestamp %s", order)).
Find(&entries)
if len(entries) > 0 {
// the entries always order from oldest to newest so we should revers
utils.ReverseSlice(entries)
}
return entries
}

View File

@@ -5,18 +5,9 @@ import (
"github.com/google/martian/har"
"github.com/up9inc/mizu/shared"
"github.com/up9inc/mizu/tap"
"mizuserver/pkg/utils"
"time"
)
type DataUnmarshaler interface {
UnmarshalData(*MizuEntry) error
}
func GetEntry(r *MizuEntry, v DataUnmarshaler) error {
return v.UnmarshalData(r)
}
type MizuEntry struct {
ID uint `gorm:"primarykey"`
CreatedAt time.Time
@@ -32,7 +23,6 @@ type MizuEntry struct {
Path string `json:"path" gorm:"column:path"`
ResolvedSource string `json:"resolvedSource,omitempty" gorm:"column:resolvedSource"`
ResolvedDestination string `json:"resolvedDestination,omitempty" gorm:"column:resolvedDestination"`
IsOutgoing bool `json:"isOutgoing,omitempty" gorm:"column:isOutgoing"`
}
type BaseEntryDetails struct {
@@ -44,64 +34,8 @@ type BaseEntryDetails struct {
StatusCode int `json:"statusCode,omitempty"`
Method string `json:"method,omitempty"`
Timestamp int64 `json:"timestamp,omitempty"`
IsOutgoing bool `json:"isOutgoing,omitempty"`
}
type FullEntryDetails struct {
har.Entry
}
type FullEntryDetailsExtra struct {
har.Entry
}
func (bed *BaseEntryDetails) UnmarshalData(entry *MizuEntry) error {
entryUrl := entry.Url
service := entry.Service
if entry.ResolvedDestination != "" {
entryUrl = utils.SetHostname(entryUrl, entry.ResolvedDestination)
service = utils.SetHostname(service, entry.ResolvedDestination)
}
bed.Id = entry.EntryId
bed.Url = entryUrl
bed.Service = service
bed.Path = entry.Path
bed.StatusCode = entry.Status
bed.Method = entry.Method
bed.Timestamp = entry.Timestamp
bed.RequestSenderIp = entry.RequestSenderIp
bed.IsOutgoing = entry.IsOutgoing
return nil
}
func (fed *FullEntryDetails) UnmarshalData(entry *MizuEntry) error {
if err := json.Unmarshal([]byte(entry.Entry), &fed.Entry); err != nil {
return err
}
if entry.ResolvedDestination != "" {
fed.Entry.Request.URL = utils.SetHostname(fed.Entry.Request.URL, entry.ResolvedDestination)
}
return nil
}
func (fedex *FullEntryDetailsExtra) UnmarshalData(entry *MizuEntry) error {
if err := json.Unmarshal([]byte(entry.Entry), &fedex.Entry); err != nil {
return err
}
if entry.ResolvedSource != "" {
fedex.Entry.Request.Headers = append(fedex.Request.Headers, har.Header{Name: "x-mizu-source", Value: entry.ResolvedSource})
}
if entry.ResolvedDestination != "" {
fedex.Entry.Request.Headers = append(fedex.Request.Headers, har.Header{Name: "x-mizu-destination", Value: entry.ResolvedDestination})
fedex.Entry.Request.URL = utils.SetHostname(fedex.Entry.Request.URL, entry.ResolvedDestination)
}
return nil
}
type EntryData struct {
Entry string `json:"entry,omitempty"`
ResolvedDestination string `json:"resolvedDestination,omitempty" gorm:"column:resolvedDestination"`
@@ -113,10 +47,6 @@ type EntriesFilter struct {
Timestamp int64 `query:"timestamp" validate:"required,min=1"`
}
type UploadEntriesRequestBody struct {
Dest string `query:"dest"`
}
type HarFetchRequestBody struct {
From int64 `query:"from"`
To int64 `query:"to"`
@@ -169,5 +99,5 @@ type ExtendedLog struct {
type ExtendedCreator struct {
*har.Creator
Source *string `json:"_source"`
Source string `json:"_source"`
}

View File

@@ -21,7 +21,7 @@ func NewFromInCluster(errOut chan error) (*Resolver, error) {
if err != nil {
return nil, err
}
return &Resolver{clientConfig: config, clientSet: clientset, nameMap: make(map[string]string), serviceMap: make(map[string]string), errOut: errOut}, nil
return &Resolver{clientConfig: config, clientSet: clientset, nameMap: make(map[string]string), errOut: errOut}, nil
}
func NewFromOutOfCluster(kubeConfigPath string, errOut chan error) (*Resolver, error) {
@@ -53,9 +53,9 @@ func NewFromOutOfCluster(kubeConfigPath string, errOut chan error) (*Resolver, e
return nil, err
}
return &Resolver{clientConfig: clientConfig, clientSet: clientset, nameMap: make(map[string]string), serviceMap: make(map[string]string), errOut: errOut}, nil
return &Resolver{clientConfig: clientConfig, clientSet: clientset, nameMap: make(map[string]string), errOut: errOut}, nil
}
func NewFromExisting(clientConfig *restclient.Config, clientSet *kubernetes.Clientset, errOut chan error) *Resolver {
return &Resolver{clientConfig: clientConfig, clientSet: clientSet, nameMap: make(map[string]string), serviceMap: make(map[string]string), errOut: errOut}
return &Resolver{clientConfig: clientConfig, clientSet: clientSet, nameMap: make(map[string]string), errOut: errOut}
}

View File

@@ -20,7 +20,6 @@ type Resolver struct {
clientConfig *restclient.Config
clientSet *kubernetes.Clientset
nameMap map[string]string
serviceMap map[string]string
isStarted bool
errOut chan error
}
@@ -42,11 +41,6 @@ func (resolver *Resolver) Resolve(name string) string {
return resolvedName
}
func (resolver *Resolver) CheckIsServiceIP(address string) bool {
_, isFound := resolver.serviceMap[address]
return isFound
}
func (resolver *Resolver) watchPods(ctx context.Context) error {
// empty namespace makes the client watch all namespaces
watcher, err := resolver.clientSet.CoreV1().Pods("").Watch(ctx, metav1.ListOptions{Watch: true})
@@ -130,7 +124,6 @@ func (resolver *Resolver) watchServices(ctx context.Context) error {
serviceHostname := fmt.Sprintf("%s.%s", service.Name, service.Namespace)
if service.Spec.ClusterIP != "" && service.Spec.ClusterIP != kubClientNullString {
resolver.saveResolvedName(service.Spec.ClusterIP, serviceHostname, event.Type)
resolver.saveServiceIP(service.Spec.ClusterIP, serviceHostname, event.Type)
}
if service.Status.LoadBalancer.Ingress != nil {
for _, ingress := range service.Status.LoadBalancer.Ingress {
@@ -154,14 +147,6 @@ func (resolver *Resolver) saveResolvedName(key string, resolved string, eventTyp
}
}
func (resolver *Resolver) saveServiceIP(key string, resolved string, eventType watch.EventType) {
if eventType == watch.Deleted {
delete(resolver.serviceMap, key)
} else {
resolver.serviceMap[key] = resolved
}
}
func (resolver *Resolver) infiniteErrorHandleRetryFunc(ctx context.Context, fun func(ctx context.Context) error) {
for {
err := fun(ctx)

View File

@@ -12,7 +12,7 @@ func EntriesRoutes(fiberApp *fiber.App) {
routeGroup.Get("/entries", controllers.GetEntries) // get entries (base/thin entries)
routeGroup.Get("/entries/:entryId", controllers.GetEntry) // get single (full) entry
routeGroup.Get("/exportEntries", controllers.GetFullEntries)
routeGroup.Get("/uploadEntries", controllers.UploadEntries)
routeGroup.Get("/har", controllers.GetHARs)
@@ -20,5 +20,4 @@ func EntriesRoutes(fiberApp *fiber.App) {
routeGroup.Get("/generalStats", controllers.GetGeneralStats) // get general stats about entries in DB
routeGroup.Get("/tapStatus", controllers.GetTappingStatus) // get tapping status
routeGroup.Get("/analyzeStatus", controllers.AnalyzeInformation)
}

View File

@@ -1,188 +0,0 @@
package up9
import (
"bytes"
"compress/zlib"
"encoding/json"
"fmt"
"github.com/up9inc/mizu/shared"
"io/ioutil"
"log"
"mizuserver/pkg/database"
"mizuserver/pkg/models"
"net/http"
"net/url"
"time"
)
const (
AnalyzeCheckSleepTime = 5 * time.Second
)
type GuestToken struct {
Token string `json:"token"`
Model string `json:"model"`
}
type ModelStatus struct {
LastMajorGeneration float64 `json:"lastMajorGeneration"`
}
func getGuestToken(url string, target *GuestToken) error {
resp, err := http.Get(url)
if err != nil {
return err
}
defer resp.Body.Close()
return json.NewDecoder(resp.Body).Decode(target)
}
func CreateAnonymousToken(envPrefix string) (*GuestToken, error) {
tokenUrl := fmt.Sprintf("https://trcc.%v/anonymous/token", envPrefix)
token := &GuestToken{}
if err := getGuestToken(tokenUrl, token); err != nil {
fmt.Println(err)
return nil, err
}
return token, nil
}
func GetRemoteUrl(analyzeDestination string, analyzeToken string) string {
return fmt.Sprintf("https://%s/share/%s", analyzeDestination, analyzeToken)
}
func CheckIfModelReady(analyzeDestination string, analyzeModel string, analyzeToken string) bool {
statusUrl, _ := url.Parse(fmt.Sprintf("https://trcc.%s/models/%s/status", analyzeDestination, analyzeModel))
req := &http.Request{
Method: http.MethodGet,
URL: statusUrl,
Header: map[string][]string{
"Content-Type": {"application/json"},
"Guest-Auth": {analyzeToken},
},
}
statusResp, err := http.DefaultClient.Do(req)
if err != nil {
return false
}
defer statusResp.Body.Close()
target := &ModelStatus{}
_ = json.NewDecoder(statusResp.Body).Decode(&target)
return target.LastMajorGeneration > 0
}
func GetTrafficDumpUrl(analyzeDestination string, analyzeModel string) *url.URL {
postUrl, _ := url.Parse(fmt.Sprintf("https://traffic.%s/dumpTrafficBulk/%s", analyzeDestination, analyzeModel))
return postUrl
}
type AnalyzeInformation struct {
IsAnalyzing bool
AnalyzedModel string
AnalyzeToken string
AnalyzeDestination string
}
func (info *AnalyzeInformation) Reset() {
info.IsAnalyzing = false
info.AnalyzedModel = ""
info.AnalyzeToken = ""
info.AnalyzeDestination = ""
}
var analyzeInformation = &AnalyzeInformation{}
func GetAnalyzeInfo() *shared.AnalyzeStatus {
return &shared.AnalyzeStatus{
IsAnalyzing: analyzeInformation.IsAnalyzing,
RemoteUrl: GetRemoteUrl(analyzeInformation.AnalyzeDestination, analyzeInformation.AnalyzeToken),
IsRemoteReady: CheckIfModelReady(analyzeInformation.AnalyzeDestination, analyzeInformation.AnalyzedModel, analyzeInformation.AnalyzeToken),
}
}
func UploadEntriesImpl(token string, model string, envPrefix string) {
analyzeInformation.IsAnalyzing = true
analyzeInformation.AnalyzedModel = model
analyzeInformation.AnalyzeToken = token
analyzeInformation.AnalyzeDestination = envPrefix
sleepTime := time.Second * 10
var timestampFrom int64 = 0
for {
timestampTo := time.Now().UnixNano() / int64(time.Millisecond)
fmt.Printf("Getting entries from %v, to %v\n", timestampFrom, timestampTo)
entriesArray := database.GetEntriesFromDb(timestampFrom, timestampTo)
if len(entriesArray) > 0 {
fullEntriesExtra := make([]models.FullEntryDetailsExtra, 0)
for _, data := range entriesArray {
harEntry := models.FullEntryDetailsExtra{}
if err := models.GetEntry(&data, &harEntry); err != nil {
continue
}
fullEntriesExtra = append(fullEntriesExtra, harEntry)
}
fmt.Printf("About to upload %v entries\n", len(fullEntriesExtra))
body, jMarshalErr := json.Marshal(fullEntriesExtra)
if jMarshalErr != nil {
analyzeInformation.Reset()
fmt.Println("Stopping analyzing")
log.Fatal(jMarshalErr)
}
var in bytes.Buffer
w := zlib.NewWriter(&in)
_, _ = w.Write(body)
_ = w.Close()
reqBody := ioutil.NopCloser(bytes.NewReader(in.Bytes()))
req := &http.Request{
Method: http.MethodPost,
URL: GetTrafficDumpUrl(envPrefix, model),
Header: map[string][]string{
"Content-Encoding": {"deflate"},
"Content-Type": {"application/octet-stream"},
"Guest-Auth": {token},
},
Body: reqBody,
}
if _, postErr := http.DefaultClient.Do(req); postErr != nil {
analyzeInformation.Reset()
log.Println("Stopping analyzing")
log.Fatal(postErr)
}
fmt.Printf("Finish uploading %v entries to %s\n", len(entriesArray), GetTrafficDumpUrl(envPrefix, model))
} else {
fmt.Println("Nothing to upload")
}
fmt.Printf("Sleeping for %v...\n", sleepTime)
time.Sleep(sleepTime)
timestampFrom = timestampTo
}
}
func UpdateAnalyzeStatus(callback func(data []byte)) {
for {
if !analyzeInformation.IsAnalyzing {
time.Sleep(AnalyzeCheckSleepTime)
continue
}
analyzeStatus := GetAnalyzeInfo()
socketMessage := shared.CreateWebSocketMessageTypeAnalyzeStatus(*analyzeStatus)
jsonMessage, _ := json.Marshal(socketMessage)
callback(jsonMessage)
time.Sleep(AnalyzeCheckSleepTime)
}
}

View File

@@ -1,9 +1,11 @@
package utils
import (
"encoding/json"
"fmt"
"github.com/gofiber/fiber/v2"
"log"
"mizuserver/pkg/models"
"net/url"
"os"
"os/signal"
@@ -59,3 +61,27 @@ func SetHostname(address, newHostname string) string {
return replacedUrl.String()
}
func GetResolvedBaseEntry(entry models.MizuEntry) models.BaseEntryDetails {
entryUrl := entry.Url
service := entry.Service
if entry.ResolvedDestination != "" {
entryUrl = SetHostname(entryUrl, entry.ResolvedDestination)
service = SetHostname(service, entry.ResolvedDestination)
}
return models.BaseEntryDetails{
Id: entry.EntryId,
Url: entryUrl,
Service: service,
Path: entry.Path,
StatusCode: entry.Status,
Method: entry.Method,
Timestamp: entry.Timestamp,
RequestSenderIp: entry.RequestSenderIp,
}
}
func GetBytesFromStruct(v interface{}) []byte{
a, _ := json.Marshal(v)
return a
}

View File

@@ -26,10 +26,10 @@ build-all: ## build for all supported platforms
@mkdir -p bin && echo "SHA256 checksums available for compiled binaries \n\nRun \`shasum -a 256 -c mizu_OS_ARCH.sha256\` to verify\n\n" > bin/README.md
@$(MAKE) build GOOS=darwin GOARCH=amd64
@$(MAKE) build GOOS=linux GOARCH=amd64
@# $(MAKE) build GOOS=darwin GOARCH=arm64
@# $(MAKE) GOOS=windows GOARCH=amd64
@# $(MAKE) GOOS=linux GOARCH=386
@# $(MAKE) GOOS=windows GOARCH=386
@$(MAKE) GOOS=darwin GOARCH=arm64
@# $(MAKE) GOOS=linux GOARCH=arm64
@# $(MAKE) GOOS=windows GOARCH=arm64
@echo "---------"

View File

@@ -3,10 +3,8 @@ package cmd
import (
"errors"
"fmt"
"regexp"
"strings"
"github.com/up9inc/mizu/cli/mizu"
"regexp"
"github.com/spf13/cobra"
)
@@ -15,23 +13,20 @@ type MizuTapOptions struct {
GuiPort uint16
Namespace string
AllNamespaces bool
Analyze bool
AnalyzeDestination string
KubeConfigPath string
MizuImage string
MizuPodPort uint16
PlainTextFilterRegexes []string
TapOutgoing bool
}
var mizuTapOptions = &MizuTapOptions{}
var direction string
var tapCmd = &cobra.Command{
Use: "tap [POD REGEX]",
Short: "Record ingoing traffic of a kubernetes pod",
Long: `Record the ingoing traffic of a kubernetes pod.
Supported protocols are HTTP and gRPC.`,
Supported protocols are HTTP and gRPC.`,
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) == 0 {
return errors.New("POD REGEX argument is required")
@@ -44,15 +39,6 @@ Supported protocols are HTTP and gRPC.`,
return errors.New(fmt.Sprintf("%s is not a valid regex %s", args[0], err))
}
directionLowerCase := strings.ToLower(direction)
if directionLowerCase == "any" {
mizuTapOptions.TapOutgoing = true
} else if directionLowerCase == "in" {
mizuTapOptions.TapOutgoing = false
} else {
return errors.New(fmt.Sprintf("%s is not a valid value for flag --direction. Acceptable values are in/any.", direction))
}
RunMizuTap(regex, mizuTapOptions)
return nil
},
@@ -63,12 +49,9 @@ func init() {
tapCmd.Flags().Uint16VarP(&mizuTapOptions.GuiPort, "gui-port", "p", 8899, "Provide a custom port for the web interface webserver")
tapCmd.Flags().StringVarP(&mizuTapOptions.Namespace, "namespace", "n", "", "Namespace selector")
tapCmd.Flags().BoolVar(&mizuTapOptions.Analyze, "analyze", false, "Uploads traffic to UP9 cloud for further analysis (Beta)")
tapCmd.Flags().StringVar(&mizuTapOptions.AnalyzeDestination, "dest", "up9.app", "Destination environment")
tapCmd.Flags().BoolVarP(&mizuTapOptions.AllNamespaces, "all-namespaces", "A", false, "Tap all namespaces")
tapCmd.Flags().StringVarP(&mizuTapOptions.KubeConfigPath, "kube-config", "k", "", "Path to kube-config file")
tapCmd.Flags().StringVarP(&mizuTapOptions.MizuImage, "mizu-image", "", fmt.Sprintf("gcr.io/up9-docker-hub/mizu/%s:latest", mizu.Branch), "Custom image for mizu collector")
tapCmd.Flags().Uint16VarP(&mizuTapOptions.MizuPodPort, "mizu-port", "", 8899, "Port which mizu cli will attempt to forward from the mizu collector pod")
tapCmd.Flags().StringArrayVarP(&mizuTapOptions.PlainTextFilterRegexes, "regex-masking", "r", nil, "List of regex expressions that are used to filter matching values from text/plain http bodies")
tapCmd.Flags().StringVarP(&direction, "direction", "", "in", "Record traffic that goes in this direction (relative to the tapped pod): in/any")
}

View File

@@ -3,15 +3,13 @@ package cmd
import (
"context"
"fmt"
"net/http"
"github.com/up9inc/mizu/shared"
"os"
"os/signal"
"regexp"
"syscall"
"time"
"github.com/up9inc/mizu/shared"
core "k8s.io/api/core/v1"
"github.com/up9inc/mizu/cli/debounce"
@@ -47,22 +45,6 @@ func RunMizuTap(podRegexQuery *regexp.Regexp, tappingOptions *MizuTapOptions) {
currentlyTappedPods = matchingPods
}
var namespacesStr string
if targetNamespace != mizu.K8sAllNamespaces {
namespacesStr = fmt.Sprintf("namespace \"%s\"", targetNamespace)
} else {
namespacesStr = "all namespaces"
}
fmt.Printf("Tapping pods in %s\n", namespacesStr)
if len(currentlyTappedPods) == 0 {
var suggestionStr string
if targetNamespace != mizu.K8sAllNamespaces {
suggestionStr = "\nSelect a different namespace with -n or tap all namespaces with -A"
}
fmt.Printf("Did not find any pods matching the regex argument%s\n", suggestionStr)
}
nodeToTappedPodIPMap, err := getNodeHostToTappedPodIpsMap(currentlyTappedPods)
if err != nil {
return
@@ -78,15 +60,16 @@ func RunMizuTap(podRegexQuery *regexp.Regexp, tappingOptions *MizuTapOptions) {
//block until exit signal or error
waitForFinish(ctx, cancel)
}
// TODO handle incoming traffic from tapper using a channel
}
func createMizuResources(ctx context.Context, kubernetesProvider *kubernetes.Provider, nodeToTappedPodIPMap map[string][]string, tappingOptions *MizuTapOptions, mizuApiFilteringOptions *shared.TrafficFilteringOptions) error {
if err := createMizuAggregator(ctx, kubernetesProvider, tappingOptions, mizuApiFilteringOptions); err != nil {
return err
}
if err := updateMizuTappers(ctx, kubernetesProvider, nodeToTappedPodIPMap, tappingOptions); err != nil {
if err := createMizuTappers(ctx, kubernetesProvider, nodeToTappedPodIPMap, tappingOptions); err != nil {
return err
}
@@ -130,27 +113,19 @@ func getMizuApiFilteringOptions(tappingOptions *MizuTapOptions) (*shared.Traffic
return &shared.TrafficFilteringOptions{PlainTextMaskingRegexes: compiledRegexSlice}, nil
}
func updateMizuTappers(ctx context.Context, kubernetesProvider *kubernetes.Provider, nodeToTappedPodIPMap map[string][]string, tappingOptions *MizuTapOptions) error {
if len(nodeToTappedPodIPMap) > 0 {
if err := kubernetesProvider.ApplyMizuTapperDaemonSet(
ctx,
mizu.ResourcesNamespace,
mizu.TapperDaemonSetName,
tappingOptions.MizuImage,
mizu.TapperPodName,
fmt.Sprintf("%s.%s.svc.cluster.local", aggregatorService.Name, aggregatorService.Namespace),
nodeToTappedPodIPMap,
mizuServiceAccountExists,
tappingOptions.TapOutgoing,
); err != nil {
fmt.Printf("Error creating mizu tapper daemonset: %v\n", err)
return err
}
} else {
if err := kubernetesProvider.RemoveDaemonSet(ctx, mizu.ResourcesNamespace, mizu.TapperDaemonSetName); err != nil {
fmt.Printf("Error deleting mizu tapper daemonset: %v\n", err)
return err
}
func createMizuTappers(ctx context.Context, kubernetesProvider *kubernetes.Provider, nodeToTappedPodIPMap map[string][]string, tappingOptions *MizuTapOptions) error {
if err := kubernetesProvider.ApplyMizuTapperDaemonSet(
ctx,
mizu.ResourcesNamespace,
mizu.TapperDaemonSetName,
tappingOptions.MizuImage,
mizu.TapperPodName,
fmt.Sprintf("%s.%s.svc.cluster.local", aggregatorService.Name, aggregatorService.Namespace),
nodeToTappedPodIPMap,
mizuServiceAccountExists,
); err != nil {
fmt.Printf("Error creating mizu tapper daemonset: %v\n", err)
return err
}
return nil
@@ -190,7 +165,7 @@ func watchPodsForTapping(ctx context.Context, kubernetesProvider *kubernetes.Pro
cancel()
}
if err := updateMizuTappers(ctx, kubernetesProvider, nodeToTappedPodIPMap, tappingOptions); err != nil {
if err := createMizuTappers(ctx, kubernetesProvider, nodeToTappedPodIPMap, tappingOptions); err != nil {
fmt.Printf("Error updating daemonset: %s (%v,%+v)\n", err, err, err)
cancel()
}
@@ -200,10 +175,10 @@ func watchPodsForTapping(ctx context.Context, kubernetesProvider *kubernetes.Pro
for {
select {
case newTarget := <-added:
fmt.Printf(mizu.Green, fmt.Sprintf("+%s\n", newTarget.Name))
fmt.Printf("+%s\n", newTarget.Name)
case removedTarget := <-removed:
fmt.Printf(mizu.Red, fmt.Sprintf("-%s\n", removedTarget.Name))
fmt.Printf("-%s\n", removedTarget.Name)
restartTappersDebouncer.SetOn()
case modifiedTarget := <-modified:
@@ -243,21 +218,12 @@ func portForwardApiPod(ctx context.Context, kubernetesProvider *kubernetes.Provi
case modifiedPod := <-modified:
if modifiedPod.Status.Phase == "Running" && !isPodReady {
isPodReady = true
var portForwardCreateError error
if portForward, portForwardCreateError = kubernetes.NewPortForward(kubernetesProvider, mizu.ResourcesNamespace, mizu.AggregatorPodName, tappingOptions.GuiPort, tappingOptions.MizuPodPort, cancel); portForwardCreateError != nil {
fmt.Printf("error forwarding port to pod %s\n", portForwardCreateError)
var err error
portForward, err = kubernetes.NewPortForward(kubernetesProvider, mizu.ResourcesNamespace, mizu.AggregatorPodName, tappingOptions.GuiPort, tappingOptions.MizuPodPort, cancel)
fmt.Printf("Web interface is now available at http://localhost:%d\n", tappingOptions.GuiPort)
if err != nil {
fmt.Printf("error forwarding port to pod %s\n", err)
cancel()
} else {
fmt.Printf("Web interface is now available at http://localhost:%d\n", tappingOptions.GuiPort)
time.Sleep(time.Second * 5) // Waiting to be sure port forwarding finished
if tappingOptions.Analyze {
if _, err := http.Get(fmt.Sprintf("http://localhost:%d/api/uploadEntries?dest=%s", tappingOptions.GuiPort, tappingOptions.AnalyzeDestination)); err != nil {
fmt.Println(err)
} else {
fmt.Printf(mizu.Purple, "Traffic is uploading to UP9 cloud for further analsys")
fmt.Println()
}
}
}
}

View File

@@ -102,6 +102,7 @@ func (provider *Provider) CreateMizuAggregatorPod(ctx context.Context, namespace
},
DNSPolicy: core.DNSClusterFirstWithHostNet,
TerminationGracePeriodSeconds: new(int64),
// Affinity: TODO: define node selector for all relevant nodes for this mizu instance
},
}
//define the service account only when it exists to prevent pod crash
@@ -214,117 +215,30 @@ func (provider *Provider) CreateMizuRBAC(ctx context.Context, namespace string,
}
func (provider *Provider) RemovePod(ctx context.Context, namespace string, podName string) error {
if isFound, err := provider.CheckPodExists(ctx, namespace, podName);
err != nil {
return err
} else if !isFound {
return nil
}
return provider.clientSet.CoreV1().Pods(namespace).Delete(ctx, podName, metav1.DeleteOptions{})
}
func (provider *Provider) RemoveService(ctx context.Context, namespace string, serviceName string) error {
if isFound, err := provider.CheckServiceExists(ctx, namespace, serviceName);
err != nil {
return err
} else if !isFound {
return nil
}
return provider.clientSet.CoreV1().Services(namespace).Delete(ctx, serviceName, metav1.DeleteOptions{})
}
func (provider *Provider) RemoveDaemonSet(ctx context.Context, namespace string, daemonSetName string) error {
if isFound, err := provider.CheckDaemonSetExists(ctx, namespace, daemonSetName);
err != nil {
return err
} else if !isFound {
return nil
}
return provider.clientSet.AppsV1().DaemonSets(namespace).Delete(ctx, daemonSetName, metav1.DeleteOptions{})
}
func (provider *Provider) CheckPodExists(ctx context.Context, namespace string, name string) (bool, error) {
listOptions := metav1.ListOptions{
FieldSelector: fmt.Sprintf("metadata.name=%s", name),
Limit: 1,
}
resourceList, err := provider.clientSet.CoreV1().Pods(namespace).List(ctx, listOptions)
if err != nil {
return false, err
}
if len(resourceList.Items) > 0 {
return true, nil
}
return false, nil
}
func (provider *Provider) CheckServiceExists(ctx context.Context, namespace string, name string) (bool, error) {
listOptions := metav1.ListOptions{
FieldSelector: fmt.Sprintf("metadata.name=%s", name),
Limit: 1,
}
resourceList, err := provider.clientSet.CoreV1().Services(namespace).List(ctx, listOptions)
if err != nil {
return false, err
}
if len(resourceList.Items) > 0 {
return true, nil
}
return false, nil
}
func (provider *Provider) CheckDaemonSetExists(ctx context.Context, namespace string, name string) (bool, error) {
listOptions := metav1.ListOptions{
FieldSelector: fmt.Sprintf("metadata.name=%s", name),
Limit: 1,
}
resourceList, err := provider.clientSet.AppsV1().DaemonSets(namespace).List(ctx, listOptions)
if err != nil {
return false, err
}
if len(resourceList.Items) > 0 {
return true, nil
}
return false, nil
}
func (provider *Provider) ApplyMizuTapperDaemonSet(ctx context.Context, namespace string, daemonSetName string, podImage string, tapperPodName string, aggregatorPodIp string, nodeToTappedPodIPMap map[string][]string, linkServiceAccount bool, tapOutgoing bool) error {
if len(nodeToTappedPodIPMap) == 0 {
return fmt.Errorf("Daemon set %s must tap at least 1 pod", daemonSetName)
}
func (provider *Provider) ApplyMizuTapperDaemonSet(ctx context.Context, namespace string, daemonSetName string, podImage string, tapperPodName string, aggregatorPodIp string, nodeToTappedPodIPMap map[string][]string, linkServiceAccount bool) error {
nodeToTappedPodIPMapJsonStr, err := json.Marshal(nodeToTappedPodIPMap)
if err != nil {
return err
}
mizuCmd := []string{
"./mizuagent",
"-i", "any",
"--tap",
"--hardump",
"--aggregator-address", fmt.Sprintf("ws://%s/wsTapper", aggregatorPodIp),
}
if tapOutgoing {
mizuCmd = append(mizuCmd, "--anydirection")
}
privileged := true
agentContainer := applyconfcore.Container()
agentContainer.WithName(tapperPodName)
agentContainer.WithImage(podImage)
agentContainer.WithImagePullPolicy(core.PullAlways)
agentContainer.WithSecurityContext(applyconfcore.SecurityContext().WithPrivileged(privileged))
agentContainer.WithCommand(mizuCmd...)
agentContainer.WithCommand("./mizuagent", "-i", "any", "--tap", "--hardump", "--aggregator-address", fmt.Sprintf("ws://%s/wsTapper", aggregatorPodIp))
agentContainer.WithEnv(
applyconfcore.EnvVar().WithName(shared.HostModeEnvVar).WithValue("1"),
applyconfcore.EnvVar().WithName(shared.TappedAddressesPerNodeDictEnvVar).WithValue(string(nodeToTappedPodIPMapJsonStr)),

View File

@@ -15,14 +15,3 @@ const (
TapperPodName = "mizu-tapper"
K8sAllNamespaces = ""
)
const (
Black = "\033[1;30m%s\033[0m"
Red = "\033[1;31m%s\033[0m"
Green = "\033[1;32m%s\033[0m"
Yellow = "\033[1;33m%s\033[0m"
Purple = "\033[1;34m%s\033[0m"
Magenta = "\033[1;35m%s\033[0m"
Teal = "\033[1;36m%s\033[0m"
White = "\033[1;37m%s\033[0m"
)

View File

@@ -1,41 +1,28 @@
package shared
type WebSocketMessageType string
const (
WebSocketMessageTypeEntry WebSocketMessageType = "entry"
WebSocketMessageTypeTappedEntry WebSocketMessageType = "tappedEntry"
WebSocketMessageTypeUpdateStatus WebSocketMessageType = "status"
WebSocketMessageTypeAnalyzeStatus WebSocketMessageType = "analyzeStatus"
WebSocketMessageTypeEntry WebSocketMessageType = "entry"
WebSocketMessageTypeTappedEntry WebSocketMessageType = "tappedEntry"
WebSocketMessageTypeUpdateStatus WebSocketMessageType = "status"
)
type WebSocketMessageMetadata struct {
MessageType WebSocketMessageType `json:"messageType,omitempty"`
}
type WebSocketAnalyzeStatusMessage struct {
*WebSocketMessageMetadata
AnalyzeStatus AnalyzeStatus `json:"analyzeStatus"`
}
type AnalyzeStatus struct {
IsAnalyzing bool `json:"isAnalyzing"`
RemoteUrl string `json:"remoteUrl"`
IsRemoteReady bool `json:"isRemoteReady"`
}
type WebSocketStatusMessage struct {
*WebSocketMessageMetadata
TappingStatus TapStatus `json:"tappingStatus"`
}
type TapStatus struct {
Pods []PodInfo `json:"pods"`
Pods []PodInfo `json:"pods"`
}
type PodInfo struct {
Namespace string `json:"namespace"`
Name string `json:"name"`
Name string `json:"name"`
}
func CreateWebSocketStatusMessage(tappingStatus TapStatus) WebSocketStatusMessage {
@@ -47,15 +34,6 @@ func CreateWebSocketStatusMessage(tappingStatus TapStatus) WebSocketStatusMessag
}
}
func CreateWebSocketMessageTypeAnalyzeStatus(analyzeStatus AnalyzeStatus) WebSocketAnalyzeStatusMessage {
return WebSocketAnalyzeStatusMessage{
WebSocketMessageMetadata: &WebSocketMessageMetadata{
MessageType: WebSocketMessageTypeAnalyzeStatus,
},
AnalyzeStatus: analyzeStatus,
}
}
type TrafficFilteringOptions struct {
PlainTextMaskingRegexes []*SerializableRegexp
}

View File

@@ -14,10 +14,18 @@ type requestResponsePair struct {
Response httpMessage `json:"response"`
}
type ConnectionInfo struct {
ClientIP string
ClientPort string
ServerIP string
ServerPort string
}
type httpMessage struct {
isRequest bool
captureTime time.Time
orig interface{}
connectionInfo ConnectionInfo
}
@@ -36,10 +44,18 @@ func (matcher *requestResponseMatcher) registerRequest(ident string, request *ht
split := splitIdent(ident)
key := genKey(split)
connectionInfo := &ConnectionInfo{
ClientIP: split[0],
ClientPort: split[2],
ServerIP: split[1],
ServerPort: split[3],
}
requestHTTPMessage := httpMessage{
isRequest: true,
captureTime: captureTime,
orig: request,
connectionInfo: *connectionInfo,
}
if response, found := matcher.openMessagesMap.Pop(key); found {

View File

@@ -24,14 +24,6 @@ type tcpID struct {
dstPort string
}
type ConnectionInfo struct {
ClientIP string
ClientPort string
ServerIP string
ServerPort string
IsOutgoing bool
}
func (tid *tcpID) String() string {
return fmt.Sprintf("%s->%s %s->%s", tid.srcIP, tid.dstIP, tid.srcPort, tid.dstPort)
}
@@ -46,7 +38,6 @@ type httpReader struct {
tcpID tcpID
isClient bool
isHTTP2 bool
isOutgoing bool
msgQueue chan httpReaderDataMsg // Channel of captured reassembled tcp payload
data []byte
captureTime time.Time
@@ -130,28 +121,13 @@ func (h *httpReader) handleHTTP2Stream() error {
}
var reqResPair *requestResponsePair
var connectionInfo *ConnectionInfo
switch messageHTTP1 := messageHTTP1.(type) {
case http.Request:
ident := fmt.Sprintf("%s->%s %s->%s %d", h.tcpID.srcIP, h.tcpID.dstIP, h.tcpID.srcPort, h.tcpID.dstPort, streamID)
connectionInfo = &ConnectionInfo{
ClientIP: h.tcpID.srcIP,
ClientPort: h.tcpID.srcPort,
ServerIP: h.tcpID.dstIP,
ServerPort: h.tcpID.dstPort,
IsOutgoing: h.isOutgoing,
}
reqResPair = reqResMatcher.registerRequest(ident, &messageHTTP1, h.captureTime)
case http.Response:
ident := fmt.Sprintf("%s->%s %s->%s %d", h.tcpID.dstIP, h.tcpID.srcIP, h.tcpID.dstPort, h.tcpID.srcPort, streamID)
connectionInfo = &ConnectionInfo{
ClientIP: h.tcpID.dstIP,
ClientPort: h.tcpID.dstPort,
ServerIP: h.tcpID.srcIP,
ServerPort: h.tcpID.srcPort,
IsOutgoing: h.isOutgoing,
}
reqResPair = reqResMatcher.registerResponse(ident, &messageHTTP1, h.captureTime)
}
@@ -164,7 +140,7 @@ func (h *httpReader) handleHTTP2Stream() error {
reqResPair.Request.captureTime,
reqResPair.Response.orig.(*http.Response),
reqResPair.Response.captureTime,
connectionInfo,
&reqResPair.Request.connectionInfo,
)
}
}
@@ -203,13 +179,7 @@ func (h *httpReader) handleHTTP1ClientStream(b *bufio.Reader) error {
reqResPair.Request.captureTime,
reqResPair.Response.orig.(*http.Response),
reqResPair.Response.captureTime,
&ConnectionInfo{
ClientIP: h.tcpID.srcIP,
ClientPort: h.tcpID.srcPort,
ServerIP: h.tcpID.dstIP,
ServerPort: h.tcpID.dstPort,
IsOutgoing: h.isOutgoing,
},
&reqResPair.Request.connectionInfo,
)
}
}
@@ -269,13 +239,7 @@ func (h *httpReader) handleHTTP1ServerStream(b *bufio.Reader) error {
reqResPair.Request.captureTime,
reqResPair.Response.orig.(*http.Response),
reqResPair.Response.captureTime,
&ConnectionInfo{
ClientIP: h.tcpID.dstIP,
ClientPort: h.tcpID.dstPort,
ServerIP: h.tcpID.srcIP,
ServerPort: h.tcpID.srcPort,
IsOutgoing: h.isOutgoing,
},
&reqResPair.Request.connectionInfo,
)
}
}

View File

@@ -27,15 +27,13 @@ func (factory *tcpStreamFactory) New(net, transport gopacket.Flow, tcp *layers.T
SupportMissingEstablishment: *allowmissinginit,
}
Debug("Current App Ports: %v", gSettings.filterPorts)
srcIp := net.Src().String()
dstIp := net.Dst().String()
dstPort := int(tcp.DstPort)
if factory.shouldNotifyOnOutboundLink(dstIp, dstPort) {
factory.outbountLinkWriter.WriteOutboundLink(net.Src().String(), dstIp, dstPort)
}
props := factory.getStreamProps(srcIp, dstIp, dstPort)
isHTTP := props.isTapTarget
isHTTP := factory.shouldTap(dstIp, dstPort)
stream := &tcpStream{
net: net,
transport: transport,
@@ -59,7 +57,6 @@ func (factory *tcpStreamFactory) New(net, transport gopacket.Flow, tcp *layers.T
hexdump: *hexdump,
parent: stream,
isClient: true,
isOutgoing: props.isOutgoing,
harWriter: factory.harWriter,
}
stream.server = httpReader{
@@ -73,7 +70,6 @@ func (factory *tcpStreamFactory) New(net, transport gopacket.Flow, tcp *layers.T
},
hexdump: *hexdump,
parent: stream,
isOutgoing: props.isOutgoing,
harWriter: factory.harWriter,
}
factory.wg.Add(2)
@@ -88,29 +84,28 @@ func (factory *tcpStreamFactory) WaitGoRoutines() {
factory.wg.Wait()
}
func (factory *tcpStreamFactory) getStreamProps(srcIP string, dstIP string, dstPort int) *streamProps {
func (factory *tcpStreamFactory) shouldTap(dstIP string, dstPort int) bool {
if hostMode {
if inArrayString(gSettings.filterAuthorities, fmt.Sprintf("%s:%d", dstIP, dstPort)) == true {
return &streamProps{isTapTarget: true, isOutgoing: false}
return true
} else if inArrayString(gSettings.filterAuthorities, dstIP) == true {
return &streamProps{isTapTarget: true, isOutgoing: false}
} else if *anydirection && inArrayString(gSettings.filterAuthorities, srcIP) == true {
return &streamProps{isTapTarget: true, isOutgoing: true}
return true
}
return &streamProps{isTapTarget: false}
return false
} else {
isTappedPort := dstPort == 80 || (gSettings.filterPorts != nil && (inArrayInt(gSettings.filterPorts, dstPort)))
if !isTappedPort {
return &streamProps{isTapTarget: false, isOutgoing: false}
return false
}
isOutgoing := !inArrayString(ownIps, dstIP)
if !*anydirection && isOutgoing {
return &streamProps{isTapTarget: false, isOutgoing: isOutgoing}
if !*anydirection {
isDirectedHere := inArrayString(ownIps, dstIP)
if !isDirectedHere {
return false
}
}
return &streamProps{isTapTarget: true}
return true
}
}
@@ -121,9 +116,3 @@ func (factory *tcpStreamFactory) shouldNotifyOnOutboundLink(dstIP string, dstPor
}
return true
}
type streamProps struct {
isTapTarget bool
isOutgoing bool
}

View File

@@ -10,8 +10,6 @@
display: flex
align-items: center
padding-left: 24px
padding-right: 24px
justify-content: space-between
.title
font-size: 45px

View File

@@ -1,41 +1,18 @@
import React, {useState} from 'react';
import React from 'react';
import {HarPage} from "./components/HarPage";
import './App.sass';
import logo from './components/assets/Mizu.svg';
import {Button} from "@material-ui/core";
import {HarPage} from "./components/HarPage";
const App = () => {
const [analyzeStatus, setAnalyzeStatus] = useState(null);
return (
<div className="mizuApp">
<div className="header">
<div style={{display: "flex", alignItems: "center"}}>
<div className="title"><img src={logo} alt="logo"/></div>
<div className="description">Traffic viewer for Kubernetes</div>
</div>
<div>
{analyzeStatus?.isAnalyzing &&
<div
title={!analyzeStatus?.isRemoteReady ? "Analysis is not ready yet" : "Go To see further analysis"}>
<Button
variant="contained"
color="primary"
disabled={!analyzeStatus?.isRemoteReady}
onClick={() => {
window.open(analyzeStatus?.remoteUrl)
}}>
Analysis
</Button>
</div>
}
</div>
</div>
<HarPage setAnalyzeStatus={setAnalyzeStatus}/>
return (
<div className="mizuApp">
<div className="header">
<div className="title"><img src={logo} alt="logo"/></div>
<div className="description">Traffic viewer for Kubernetes</div>
</div>
);
<HarPage/>
</div>
);
}
export default App;

View File

@@ -1,13 +1,7 @@
import React from "react";
import styles from './style/HarEntry.module.sass';
import StatusCode, {getClassification, StatusCodeClassification} from "./StatusCode";
import StatusCode from "./StatusCode";
import {EndpointPath} from "./EndpointPath";
import ingoingIconSuccess from "./assets/ingoing-traffic-success.svg"
import ingoingIconFailure from "./assets/ingoing-traffic-failure.svg"
import ingoingIconNeutral from "./assets/ingoing-traffic-neutral.svg"
import outgoingIconSuccess from "./assets/outgoing-traffic-success.svg"
import outgoingIconFailure from "./assets/outgoing-traffic-failure.svg"
import outgoingIconNeutral from "./assets/outgoing-traffic-neutral.svg"
interface HAREntry {
method?: string,
@@ -18,7 +12,6 @@ interface HAREntry {
url?: string;
isCurrentRevision?: boolean;
timestamp: Date;
isOutgoing?: boolean;
}
interface HAREntryProps {
@@ -28,26 +21,6 @@ interface HAREntryProps {
}
export const HarEntry: React.FC<HAREntryProps> = ({entry, setFocusedEntryId, isSelected}) => {
const classification = getClassification(entry.statusCode)
let ingoingIcon;
let outgoingIcon;
switch(classification) {
case StatusCodeClassification.SUCCESS: {
ingoingIcon = ingoingIconSuccess;
outgoingIcon = outgoingIconSuccess;
break;
}
case StatusCodeClassification.FAILURE: {
ingoingIcon = ingoingIconFailure;
outgoingIcon = outgoingIconFailure;
break;
}
case StatusCodeClassification.NEUTRAL: {
ingoingIcon = ingoingIconNeutral;
outgoingIcon = outgoingIconNeutral;
break;
}
}
return <>
<div id={entry.id} className={`${styles.row} ${isSelected ? styles.rowSelected : ''}`} onClick={() => setFocusedEntryId(entry.id)}>
@@ -60,14 +33,7 @@ export const HarEntry: React.FC<HAREntryProps> = ({entry, setFocusedEntryId, isS
{entry.service}
</div>
</div>
<div className={styles.directionContainer}>
{entry.isOutgoing ?
<img src={outgoingIcon} alt="outgoing traffic" title="outgoing"/>
:
<img src={ingoingIcon} alt="ingoing traffic" title="ingoing"/>
}
</div>
<div className={styles.timestamp}>{new Date(+entry.timestamp)?.toLocaleString()}</div>
</div>
</>
};
};

View File

@@ -35,11 +35,7 @@ enum ConnectionStatus {
Paused
}
interface HarPageProps {
setAnalyzeStatus: (status: any) => void;
}
export const HarPage: React.FC<HarPageProps> = ({setAnalyzeStatus}) => {
export const HarPage: React.FC = () => {
const classes = useLayoutStyles();
@@ -64,21 +60,21 @@ export const HarPage: React.FC<HarPageProps> = ({setAnalyzeStatus}) => {
ws.current.onclose = () => setConnection(ConnectionStatus.Closed);
}
if (ws.current) {
if(ws.current) {
ws.current.onmessage = e => {
if (!e?.data) return;
if(!e?.data) return;
const message = JSON.parse(e.data);
switch (message.messageType) {
case "entry":
const entry = message.data
if (connection === ConnectionStatus.Paused) {
if(connection === ConnectionStatus.Paused) {
setNoMoreDataBottom(false)
return;
}
if (!focusedEntryId) setFocusedEntryId(entry.id)
if(!focusedEntryId) setFocusedEntryId(entry.id)
let newEntries = [...entries];
if (entries.length === 1000) {
if(entries.length === 1000) {
newEntries = newEntries.splice(1);
setNoMoreDataTop(false);
}
@@ -87,9 +83,6 @@ export const HarPage: React.FC<HarPageProps> = ({setAnalyzeStatus}) => {
case "status":
setTappingStatus(message.tappingStatus);
break
case "analyzeStatus":
setAnalyzeStatus(message.analyzeStatus);
break
default:
console.error(`unsupported websocket message type, Got: ${message.messageType}`)
}
@@ -101,23 +94,19 @@ export const HarPage: React.FC<HarPageProps> = ({setAnalyzeStatus}) => {
fetch(`http://localhost:8899/api/tapStatus`)
.then(response => response.json())
.then(data => setTappingStatus(data));
fetch(`http://localhost:8899/api/analyzeStatus`)
.then(response => response.json())
.then(data => setAnalyzeStatus(data));
}, []);
useEffect(() => {
if (!focusedEntryId) return;
if(!focusedEntryId) return;
setSelectedHarEntry(null)
fetch(`http://localhost:8899/api/entries/${focusedEntryId}`)
.then(response => response.json())
.then(data => setSelectedHarEntry(data));
}, [focusedEntryId])
},[focusedEntryId])
const toggleConnection = () => {
setConnection(connection === ConnectionStatus.Connected ? ConnectionStatus.Paused : ConnectionStatus.Connected);
setConnection(connection === ConnectionStatus.Connected ? ConnectionStatus.Paused : ConnectionStatus.Connected );
}
const getConnectionStatusClass = (isContainer) => {
@@ -146,12 +135,11 @@ export const HarPage: React.FC<HarPageProps> = ({setAnalyzeStatus}) => {
return (
<div className="HarPage">
<div className="harPageHeader">
<img style={{cursor: "pointer", marginRight: 15, height: 30}} alt="pause"
src={connection === ConnectionStatus.Connected ? pauseIcon : playIcon} onClick={toggleConnection}/>
<img style={{cursor: "pointer", marginRight: 15, height: 30}} alt="pause" src={connection === ConnectionStatus.Connected ? pauseIcon : playIcon} onClick={toggleConnection}/>
<div className="connectionText">
{getConnectionTitle()}
<div className={"indicatorContainer " + getConnectionStatusClass(true)}>
<div className={"indicator " + getConnectionStatusClass(false)}/>
<div className={"indicator " + getConnectionStatusClass(false)} />
</div>
</div>
</div>
@@ -181,8 +169,7 @@ export const HarPage: React.FC<HarPageProps> = ({setAnalyzeStatus}) => {
</div>
</div>
<div className={classes.details}>
{selectedHarEntry &&
<HAREntryDetailed harEntry={selectedHarEntry} classes={{root: classes.harViewer}}/>}
{selectedHarEntry && <HAREntryDetailed harEntry={selectedHarEntry} classes={{root: classes.harViewer}}/>}
</div>
</div>}
{tappingStatus?.pods != null && <StatusBar tappingStatus={tappingStatus}/>}

View File

@@ -1,7 +1,7 @@
import React from "react";
import styles from './style/StatusCode.module.sass';
export enum StatusCodeClassification {
enum StatusCodeClassification {
SUCCESS = "success",
FAILURE = "failure",
NEUTRAL = "neutral"
@@ -14,12 +14,6 @@ interface HAREntryProps {
const StatusCode: React.FC<HAREntryProps> = ({statusCode}) => {
const classification = getClassification(statusCode)
return <span className={`${styles[classification]} ${styles.base}`}>{statusCode}</span>
};
export function getClassification(statusCode: number): string {
let classification = StatusCodeClassification.NEUTRAL;
if (statusCode >= 200 && statusCode <= 399) {
@@ -28,7 +22,7 @@ export function getClassification(statusCode: number): string {
classification = StatusCodeClassification.FAILURE;
}
return classification
}
return <span className={`${styles[classification]} ${styles.base}`}>{statusCode}</span>
};
export default StatusCode;
export default StatusCode;

View File

@@ -1,5 +0,0 @@
<svg width="20" height="20" viewBox="0 0 20 20" fill="none" xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" clip-rule="evenodd" d="M16.5175 11.1465C16.8392 10.8869 17 10.4434 17 10C17 9.55657 16.8392 9.11314 16.5175 8.85348L12.5425 5.64459C13.2682 5.23422 14.1067 5 15 5C17.7614 5 20 7.23858 20 10C20 12.7614 17.7614 15 15 15C14.1067 15 13.2682 14.7658 12.5425 14.3554L16.5175 11.1465Z" fill="#BCCEFD"/>
<path d="M16 10C16 10.3167 15.8749 10.6335 15.6247 10.8189L10.1706 14.8624C9.65543 15.2444 9 14.7858 9 14.0435V5.95652C9 5.21417 9.65543 4.75564 10.1706 5.13758L15.6247 9.18106C15.8749 9.36653 16 9.68326 16 10Z" fill="#EB5757"/>
<path d="M0 10C0 8.89543 0.895431 8 2 8H10C11.1046 8 12 8.89543 12 10C12 11.1046 11.1046 12 10 12H2C0.895431 12 0 11.1046 0 10Z" fill="#EB5757"/>
</svg>

Before

Width:  |  Height:  |  Size: 800 B

View File

@@ -1,5 +0,0 @@
<svg width="20" height="20" viewBox="0 0 20 20" fill="none" xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" clip-rule="evenodd" d="M16.5175 11.1465C16.8392 10.8869 17 10.4434 17 10C17 9.55657 16.8392 9.11314 16.5175 8.85348L12.5425 5.64459C13.2682 5.23422 14.1067 5 15 5C17.7614 5 20 7.23858 20 10C20 12.7614 17.7614 15 15 15C14.1067 15 13.2682 14.7658 12.5425 14.3554L16.5175 11.1465Z" fill="#BCCEFD"/>
<path d="M16 10C16 10.3167 15.8749 10.6335 15.6247 10.8189L10.1706 14.8624C9.65543 15.2444 9 14.7858 9 14.0435V5.95652C9 5.21417 9.65543 4.75564 10.1706 5.13758L15.6247 9.18106C15.8749 9.36653 16 9.68326 16 10Z" fill="gray"/>
<path d="M0 10C0 8.89543 0.895431 8 2 8H10C11.1046 8 12 8.89543 12 10C12 11.1046 11.1046 12 10 12H2C0.895431 12 0 11.1046 0 10Z" fill="gray"/>
</svg>

Before

Width:  |  Height:  |  Size: 794 B

View File

@@ -1,5 +0,0 @@
<svg width="20" height="20" viewBox="0 0 20 20" fill="none" xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" clip-rule="evenodd" d="M16.5175 11.1465C16.8392 10.8869 17 10.4434 17 10C17 9.55657 16.8392 9.11314 16.5175 8.85348L12.5425 5.64459C13.2682 5.23422 14.1067 5 15 5C17.7614 5 20 7.23858 20 10C20 12.7614 17.7614 15 15 15C14.1067 15 13.2682 14.7658 12.5425 14.3554L16.5175 11.1465Z" fill="#BCCEFD"/>
<path d="M16 10C16 10.3167 15.8749 10.6335 15.6247 10.8189L10.1706 14.8624C9.65543 15.2444 9 14.7858 9 14.0435V5.95652C9 5.21417 9.65543 4.75564 10.1706 5.13758L15.6247 9.18106C15.8749 9.36653 16 9.68326 16 10Z" fill="#27AE60"/>
<path d="M0 10C0 8.89543 0.895431 8 2 8H10C11.1046 8 12 8.89543 12 10C12 11.1046 11.1046 12 10 12H2C0.895431 12 0 11.1046 0 10Z" fill="#27AE60"/>
</svg>

Before

Width:  |  Height:  |  Size: 800 B

View File

@@ -1,5 +0,0 @@
<svg width="20" height="20" viewBox="0 0 20 20" fill="none" xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" clip-rule="evenodd" d="M15 15C17.7614 15 20 12.7615 20 10C20 7.23861 17.7614 5.00003 15 5.00003C13.3642 5.00003 11.9118 5.78558 10.9996 7.00003H14C15.6569 7.00003 17 8.34318 17 10C17 11.6569 15.6569 13 14 13H10.9996C11.9118 14.2145 13.3642 15 15 15Z" fill="#BCCEFD"/>
<rect x="4" y="8.00003" width="12" height="4" rx="2" fill="#EB5757"/>
<path d="M5.96244e-08 10C6.34015e-08 9.68329 0.125088 9.36656 0.375266 9.18109L5.82939 5.13761C6.34457 4.75567 7 5.2142 7 5.95655L7 14.0435C7 14.7859 6.34457 15.2444 5.82939 14.8625L0.375266 10.819C0.125088 10.6335 5.58474e-08 10.3168 5.96244e-08 10Z" fill="#EB5757"/>
</svg>

Before

Width:  |  Height:  |  Size: 736 B

View File

@@ -1,5 +0,0 @@
<svg width="20" height="20" viewBox="0 0 20 20" fill="none" xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" clip-rule="evenodd" d="M15 15C17.7614 15 20 12.7615 20 10C20 7.23861 17.7614 5.00003 15 5.00003C13.3642 5.00003 11.9118 5.78558 10.9996 7.00003H14C15.6569 7.00003 17 8.34318 17 10C17 11.6569 15.6569 13 14 13H10.9996C11.9118 14.2145 13.3642 15 15 15Z" fill="#BCCEFD"/>
<rect x="4" y="8.00003" width="12" height="4" rx="2" fill="gray"/>
<path d="M5.96244e-08 10C6.34015e-08 9.68329 0.125088 9.36656 0.375266 9.18109L5.82939 5.13761C6.34457 4.75567 7 5.2142 7 5.95655L7 14.0435C7 14.7859 6.34457 15.2444 5.82939 14.8625L0.375266 10.819C0.125088 10.6335 5.58474e-08 10.3168 5.96244e-08 10Z" fill="gray"/>
</svg>

Before

Width:  |  Height:  |  Size: 730 B

View File

@@ -1,5 +0,0 @@
<svg width="20" height="20" viewBox="0 0 20 20" fill="none" xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" clip-rule="evenodd" d="M15 15C17.7614 15 20 12.7615 20 10C20 7.23861 17.7614 5.00003 15 5.00003C13.3642 5.00003 11.9118 5.78558 10.9996 7.00003H14C15.6569 7.00003 17 8.34318 17 10C17 11.6569 15.6569 13 14 13H10.9996C11.9118 14.2145 13.3642 15 15 15Z" fill="#BCCEFD"/>
<rect x="4" y="8.00003" width="12" height="4" rx="2" fill="#27AE60"/>
<path d="M5.96244e-08 10C6.34015e-08 9.68329 0.125088 9.36656 0.375266 9.18109L5.82939 5.13761C6.34457 4.75567 7 5.2142 7 5.95655L7 14.0435C7 14.7859 6.34457 15.2444 5.82939 14.8625L0.375266 10.819C0.125088 10.6335 5.58474e-08 10.3168 5.96244e-08 10Z" fill="#27AE60"/>
</svg>

Before

Width:  |  Height:  |  Size: 736 B

View File

@@ -37,10 +37,9 @@
.timestamp
font-size: 12px
color: $secondary-font-color
padding-left: 12px
padding-left: 8px
padding-right: 8px
flex-shrink: 0
width: 145px
text-align: left
.endpointServiceContainer
display: flex
@@ -48,10 +47,4 @@
overflow: hidden
padding-right: 10px
padding-left: 10px
flex-grow: 1
.directionContainer
display: flex
border-right: 1px solid $data-background-color
padding: 4px
padding-right: 12px
flex-grow: 1

View File

@@ -14,6 +14,12 @@
align-items: center
background-color: $header-background-color
.harPageHeader
padding: 20px 24px
display: flex
align-items: center
background-color: $header-background-color
.HarPage-Header
display: flex
height: 2.5%

View File

@@ -19,4 +19,4 @@ $blue-gray: #494677;
successColor: $success-color;
failureColor: $failure-color;
blueGray: $blue-gray;
}
}