Commit Graph

43 Commits

Author SHA1 Message Date
lirazyehezkel
b47718e094 TRA-4442 Improve UI performance (#960)
* Move ws entry listener to entriesList component

* unused code
2022-04-03 15:51:20 +03:00
Igor Gov
e867b7d0f1 Build ui-common part of CI (#914)
* Build ui-common always locally
2022-03-29 14:14:52 +03:00
lirazyehezkel
bf8d5ed069 Support multiple workspaces (TRA-4365) (#945)
* support multiple workspaces

* reopen by websocket url dep

* open websocket only when websocketURL is changed

* upgrade common version

Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
2022-03-29 09:56:59 +03:00
Igor Gov
0a9be1884a Traffic viewer list item ebpf to lock icon (#939)
* Traffic viewer changing ebpf to lock icon
2022-03-28 11:10:54 +03:00
AmitUp9
10dffd9331 TRA-4419 remove headers from OAS select (#936) 2022-03-28 10:01:57 +03:00
leon-up9
0a800e8d8a passed containerId in the object config (#935)
* passed containerId in the object config

* pkg version

* tmp: run acceptance tests, revert before merge

* reverte

Co-authored-by: Leon <>
2022-03-27 19:41:51 +03:00
RamiBerm
068a4ff86e TRA-4422 line numbers fix (#934)
* fix line numbering in body syntax highlighter

* Packages
2022-03-27 16:17:34 +03:00
leon-up9
c45a869b75 Fix/regration-fixes-24.3 (#933)
* link open in a new tab

* fixed double toast

* added const

* javiers new icon

* epbg img fix

* pkg update

* changed import

* formated

Co-authored-by: Leon <>
2022-03-27 15:47:46 +03:00
leon-up9
0a793cd9e0 added props (#924) 2022-03-27 11:31:40 +03:00
AmitUp9
e07e04377f mizu and ui-common version update (#923) 2022-03-24 12:31:08 +02:00
AmitUp9
3c8f63ed92 Feature/UI/tra 4390 status bar banner demo env (#921)
* no message

* banner styling condition added and button text transform disabled

* added prps to container

* build fix
2022-03-24 12:14:31 +02:00
AmitUp9
0f4710918f TRA-4122 oas gui grooming (#915)
* oas styling
* oas button text change
* ui-common version update
* font fix
* version updating
Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
2022-03-23 22:15:04 +02:00
leon-up9
2513e9099f UI/tra 4406 add an information icon (#913)
* information icon added

* new icon and position

* icon changed

* new Ver

* space removed

Co-authored-by: Adam Kol <adam@up9.com>
Co-authored-by: Leon <>
2022-03-23 14:54:51 +02:00
lirazyehezkel
bc7776cbd3 Fix ws errors and warnings (#908)
* Fix ws errors and warnings

* versioning
2022-03-22 15:43:05 +02:00
M. Mert Yıldıran
308fa78955 TRA-4383 Calculate request and response sizes and display them instead of BodySize field (#897)
* Define `ReadProgress` struct and update `Dissector` interface such that the `bufio.Reader` progress can be learned on item emitting

* Display the `requestSize` and `responseSize` fields in the UI

* Update the tests

* publish ui-common version 1.0.130 and bump to this version in ui/package.json file

Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
Co-authored-by: Roee Gadot <roee.gadot@up9.com>
2022-03-21 19:34:59 +02:00
Adam Kol
7893b4596d closing ws on modal open (#904)
Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
2022-03-21 16:44:40 +02:00
gadotroee
a4d0e250c9 Fixing acceptance tests (#900) 2022-03-21 14:21:08 +02:00
leon-up9
9430e291b4 UI Splitting to common components (#883)
* initial commit

* removing files

* after pr

* move StatusBar to common

* last changes from dev

* update common-ui

* webSocket was added to TrafficViewerApi

* useWS

* actionButtons added to TrafficV

* comment clean

* api clean up

* api clean up

* statusbar position changed

* Checkbox changed

* AnalyzeButton exported to common

* CustomModal added from Ent

* oas modal exported to common

* removed redundant

* oasmodal usage

* es6 function

* api changed

* removed react-scripts

Co-authored-by: Leon <>
2022-03-15 18:45:43 +02:00
RoyUP9
edbe4ab00b Removed all code related to install (#892) 2022-03-15 16:41:42 +02:00
lirazyehezkel
2e5cf13b3f Mizu start dev commands (#726)
* start dev commands

* env files

* no message
2022-01-31 12:05:50 +02:00
lirazyehezkel
80418f1802 TRA-4205 React router (#684)
* React router

* removed comment
2022-01-25 10:45:07 +02:00
RoyUP9
3067bf5eaf Fixed ui split (#690) 2022-01-24 18:11:45 +02:00
lirazyehezkel
39f0b74897 Split UI build (#681)
* Split ui build into build and build-ent folders

* remove is_standalone var

Co-authored-by: RoyUP9 <87927115+RoyUP9@users.noreply.github.com>
2022-01-24 10:02:35 +02:00
lirazyehezkel
18d90cdf36 Support node 16 (#673)
* upgrade node-sass

* upgrade axios

* update dockerfile

Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
2022-01-20 18:41:00 +02:00
Gustavo Massaneiro
d5fd2ff1da Service Map (#623)
* debug builds and gcflags

* update dockerfile for debug

* service map routes and controller

* service map graph structure

* service map interface and new methods

* adding service map edges from mizu entries

* new service map count methods

* implementing the status endpoint

* ServiceMapResponse and ServiceMapEdge models

* service map get endpoint logic

* reset logic and endpoint

* fixed service map get status

* improvements to graph node structure

* front-end implementation and service map buttons

* new render endpoint to render the graph in real time

* spinner sass

* new ServiceMapModal component

* testing react-force-graph-2d lib

* Improvements to service map graph structure, added node id and updated edge source/destination type

* Revert "testing react-force-graph-2d lib"

This reverts commit 1153938386.

* testing react-graph-vis lib

* updated to work with react-graph-vis lib

* removed render endpoint

* go mod tidy

* serviceMap config flag

* using the serviceMap config flag

* passing mizu config to service map as a dependency

* service map tests

* Removed print functions

* finished service map tests

* new service property

* service map controller tests

* moved service map reset button to service map modal
reset closes the modal

* service map modal refresh button and logic

* reset button resets data and refresh

* service map modal close button

* node size/edge size based on the count value
edge color based on protocol

* nodes and edges shadow

* enabled physics to avoid node overlap, changed kafka protocol color to dark green

* showing edges count values and fixed bidirectional edges overlap

* go mod tidy

* removed console.log

* Using the destination node protocol instead of the source node protocol

* Revert "debug builds and gcflags"
Addressed by #624 and #626

This reverts commit 17ecaece3e.

* Revert "update dockerfile for debug"
Addressed by #635

This reverts commit 5dfc15b140.

* using the entire tap Protocol struct instead of only the protocol name

* using the backend protocol background color for node colors

* fixed test, the node list order can change

* re-factoring to get 100% coverage

* using protocol colors just for edges

* re-factored service map to use TCP Entry data. Node key is the entry ip-address instead of the name

* fallback to ip-address when entry name is unresolved

* re-factored front-end

* adjustment to main div style

* added support for multiple protocols for the same edge

* using the item protocol instead of the extension variable

* fixed controller tests

* displaying service name and ip-address on graph nodes

* fixed service map test, we cannot guarantee the slice order

* auth middleware

* created a new pkg for the service map

* re-factoring

* re-factored front-end

* reverting the import order as previous

* Aligning with other UI feature flags handling

* we don't need to get the status anymore, we have window["isServiceMapEnabled"]

* small adjustments

* renamed from .tsx to .ts

* button styles and minor improvements

* moved service map modal from trafficPage to app component

Co-authored-by: Igor Gov <igor.govorov1@gmail.com>
2022-01-19 15:27:12 -03:00
Igor Gov
7477f867f9 TRA-4122 - OAS dialog 2 (#637)
* parent d97d481392
author Amit Fainholts <amit@up9.com> 1641398982 +0200
committer Igor Gov <igor.govorov1@gmail.com> 1642070154 +0200

init

* revert

* small fixes

* api request to oas added

* redoc version downgraded and remove redundant package-lock

* remove redundant package json

* redoc updated to latest

* libraries updated to prevent vulnerabilities

* pr fixes

* remove ! from global var

* update useEffect in OasModal and other pr fixes

* change errors messages

Co-authored-by: Amit Fainholts <amit@up9.com>
Co-authored-by: Igor Gov <igor.govorov1@gmail.com>
Co-authored-by: lirazyehezkel <61656597+lirazyehezkel@users.noreply.github.com>
2022-01-19 13:06:39 +02:00
lirazyehezkel
5fed5808d2 TRA-4159 Mizu state management (#631)
* initiate recoil state management with entPage and tappingStatus

* first recoil selector

* insert entries and focusedEntryId into recoil

* ws connection, entry data

* manage query by recoil

* identifier for cypress

* conflicts fix

* css fix

* cr fixes

Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
2022-01-16 15:27:09 +02:00
lirazyehezkel
cb5344090a TRA-4089 Mizu enterprise frame (#594)
* ent app

* mizu ent frame

* apis

* design settings modal by Javier

* fix warnings

* fix warnings

* text change

* redirect after logout

* cr fixes
2022-01-06 14:50:50 +02:00
M. Mert Yıldıran
3a83531590 Use react-lowlight to highlight and json-beautify, xml-formatter to prettify the EntryBodySection (#554)
* Use `react-lowlight` to highlight and `json-beautify` to prettify the `EntryBodySection`

* Bring back the line numbers

* Make the Base64 decoding optional but make it `true` by default

* Align line numbers to right and don't have a dot character

* Make line numbers semi transparent

* Make `markers` code more elegant

* Prettify XML as well
2021-12-26 16:12:17 +03:00
M. Mert Yıldıran
0a2aacfb02 Include milliseconds information into the timestamps in the UI (#524)
* Include milliseconds information into the timestamps in the UI

* Upgrade Basenine version from `0.2.16` to `0.2.17`

* Increase the `width` of timestamp
2021-12-10 18:03:17 +03:00
M. Mert Yıldıran
9f1e311689 TRA-4017 Bring back getOldEntries method using fetch API and always start streaming from now (#518)
* Bring back `getOldEntries` method using fetch API

* Determine no more data on top based on `leftOff` value

* Remove `entriesBuffer` state

* Always open WebSocket with some `leftOff` value

* Rename `leftOff` state to `leftOffBottom`

* Don't set the `focusedEntryId` through WebSocket if the WebSocket is closed

* Call `setQueriedCurrent` with addition

* Close WebSocket upon reaching to top

* Open WebSocket upon snapping to bottom

* Close the WebSocket on snap broken event instead

* Set queried current value to zero upon filter submit

* Upgrade `react-scrollable-feed-virtualized` version and use `scrollToIndex` function

* Change the footer text format

* Improve no more data top logic

* Fix `closeWebSocket()` call logic in `onSnapBrokenEvent` and handle `data.meta` being `null` in `getOldEntries`

* Fix the issues around fetching old records

* Clean up `EntriesList.module.sass`

* Decrement initial `leftOffTop` value by `2`

* Fix the order of `incomingEntries` in `getOldEntries`

* Request `leftOffTop - 1` from `fetchEntries`

* Limit the front-end total entries fetched through WebSocket count to `10000`

* Lose the UI performance gain that's provided by #452

* Revert "Fix the selected entry behavior by propagating the `focusedEntryId` through WebSocket (before #452) TRA-3983 (#513)"

This reverts commit 873f252544.

* Fix the issues caused by 09371f141f

* Upgrade Basenine version from `0.2.13` to `0.2.14`

* Upgrade Basenine version from `0.2.14` to `0.2.15`

* Fix the condition of "Fetch old records" button visibility

* Upgrade Basenine version from `0.2.15` to `0.2.16` and fix the UI code related to fetching old records

* Make `newEntries` constant
2021-12-08 15:19:35 +03:00
M. Mert Yıldıran
618cb3a409 Optimize UI entry feed performance (#452)
* Optimize the React code for feeding the entries

By building `EntryItem` only once and updating the `entries` state on meta query messages.

* Upgrade `react-scrollable-feed-virtualized` version from `1.4.3` to `1.4.8`

* Fix the `isSelected` state

* Set the query text before deciding the background to prevent lags while typing

* Upgrade Basenine version from `0.2.6` to `0.2.7`

* Set the query background color only if the query is same after the HTTP request and use `useEffect` instead

* Upgrade Basenine version from `0.2.7` to `0.2.8`

* Use `CancelToken` of `axios` instead of trying to check the query state

* Turn `updateQuery` function into a state hook

* Update the macro for `http`

* Do the `source.cancel()` call in `axios.CancelToken`

* Reduce client-side logging
2021-11-15 17:32:05 +03:00
M. Mert Yıldıran
d2fe3f6620 Migrate from SQLite to Basenine and introduce a new filtering syntax (#279)
* Fix the OOMKilled error by calling `debug.FreeOSMemory` periodically

* Remove `MAX_NUMBER_OF_GOROUTINES` environment variable

* Change the line

* Increase the default value of `TCP_STREAM_CHANNEL_TIMEOUT_MS` to `10000`

* Write the client and integrate to the new real-time database

* Refactor the WebSocket implementaiton for `/ws`

* Adapt the UI to the new filtering system

* Fix the rest of the issues in the UI

* Increase the buffer of the scanner

* Implement accessing single records

* Increase the buffer of another scanner

* Populate `Request` and `Response` fields of `MizuEntry`

* Add syntax highlighting for the query

* Add database to `Dockerfile`

* Fix some issues

* Update the `realtime_dbms` Git module commit hash

* Upgrade Gin version and print the query string

* Revert "Upgrade Gin version and print the query string"

This reverts commit aa09f904ee.

* Use WebSocket's itself to query instead of the query string

* Fix some errors related to conversion to HAR

* Fix the issues caused by the latest merge

* Fix the build error

* Fix PR validation GitHub workflow

* Replace the git submodule with latest Basenine version `0.1.0`

Remove `realtime_client.go` and use the official client library `github.com/up9inc/basenine/client/go` instead.

* Move Basenine host and port constants to `shared` module

* Reliably execute and wait for Basenine to become available

* Upgrade Basenine version

* Properly close WebSocket and data channel

* Fix the issues caused by the recent merge commit

* Clean up the TypeScript code

* Update `.gitignore`

* Limit the database size

* Add `Macros` method signature to `Dissector` interface and set the macros provided by the protocol extensions

* Run `go mod tidy` on `agent`

* Upgrade `github.com/up9inc/basenine/client/go` version

* Implement a mechanism to update the query using click events in the UI and use it for protocol macros

* Update the query on click to timestamps

* Fix some issues in the WebSocket and channel handling

* Update the query on clicks to status code

* Update the query on clicks to method, path and service

* Update the query on clicks to is outgoing, source and destination ports

* Add an API endpoint to validate the query against syntax errors

* Move the query background color state into `TrafficPage`

* Fix the logic in `setQuery`

* Display a toast message in case of a syntax error in the query

* Remove a call to `fmt.Printf`

* Upgrade Basenine version to `0.1.3`

* Fix an issue related to getting `MAX_ENTRIES_DB_BYTES` environment variable

* Have the `path` key in request details, in HTTP

* Rearrange the HTTP headers for the querying

* Do the same thing for `cookies` and `queryString`

* Update the query on click to table elements

Add the selectors for `TABLE` type representations in HTTP extension.

* Update the query on click to `bodySize` and `elapsedTime` in `EntryTitle`

* Add the selectors for `TABLE` type representations in AMQP extension

* Add the selectors for `TABLE` type representations in Kafka extension

* Add the selectors for `TABLE` type representations in Redis extension

* Define a struct in `tap/api.go` for the section representation data

* Add the selectors for `BODY` type representations

* Add `request.path` to the HTTP request details

* Change the summary string's field name from `path` to `summary`

* Introduce `queryable` CSS class for queryable UI elements and underline them on hover

* Instead of `N requests` at the bottom, make it `Displaying N results (queried X/Y)` and live update the values

Upgrade Basenine version to `0.2.0`.

* Verify the sha256sum of Basenine executable inside `Dockerfile`

* Pass the start time to web UI through WebSocket and always show the `EntriesList` footer

* Pipe the `stderr` of Basenine as well

* Fix the layout issues related to `CodeEditor` in the UI

* Use the correct `shasum` command in `Dockerfile`

* Upgrade Basenine version to `0.2.1`

* Limit the height of `CodeEditor` container

* Remove `Paused` enum `ConnectionStatus` in UI

* Fix the issue caused by the recent merge

* Add the filtering guide (cheatsheet)

* Update open cheatsheet button's title

* Update cheatsheet content

* Remove the old SQLite code, adapt the `--analyze` related code to Basenine

* Change the method signature of `NewEntry`

* Change the method signature of `Represent`

* Introduce `HTTPPair` field in `MizuEntry` specific to HTTP

* Remove `Entry`, `EntryId` and `EstimatedSizeBytes` fields from `MizuEntry`

Also remove the `getEstimatedEntrySizeBytes` method.

* Remove `gorm.io/gorm` dependency

* Remove unused `sensitiveDataFiltering` folder

* Increase the left margin of open cheatsheet button

* Add `overflow: auto` to the cheatsheet `Modal`

* Fix `GetEntry` method

* Fix the macro for gRPC

* Fix an interface conversion in case of AMQP

* Fix two more interface conversion errors in AMQP

* Make the `syncEntriesImpl` method blocking

* Fix a grammar mistake in the cheatsheet

* Adapt to the changes in the recent merge commit

* Improve the cheatsheet text

* Always display the timestamp in `en-US`

* Upgrade Basenine version to `0.2.2`

* Fix the order of closing Basenine connections and channels

* Don't close the Basenine channels at all

* Upgrade Basenine version to `0.2.3`

* Set the initial filter to `rlimit(100)`

* Make Basenine persistent

* Upgrade Basenine version to `0.2.4`

* Update `debug.Dockerfile`

* Fix a failing test

* Upgrade Basenine version to `0.2.5`

* Revert "Do not show play icon when disconnected (#428)"

This reverts commit 8af2e562f8.

* Upgrade Basenine version to `0.2.6`

* Make all non-informative things informative

* Make `100` a constant

* Use `===` in JavaScript no matter what

* Remove a forgotten `console.log`

* Add a comment and update the `query` in `syncEntriesImpl`

* Don't call `panic` in `GetEntry`

* Replace `panic` calls in `startBasenineServer` with `logger.Log.Panicf`

* Remove unnecessary `\n` characters in the logs
2021-11-09 19:54:48 +03:00
M. Mert Yıldıran
e8ea93cb64 Upgrade react-scrollable-feed-virtualized version from 1.4.2 to 1.4.3 (#299) 2021-09-23 08:36:44 +03:00
M. Mert Yıldıran
616eccb2cf Make ScrollableFeed virtualized by replacing react-scrollable-feed with react-scrollable-feed-virtualized (#268)
* Make `ScrollableFeed` virtualized by replacing `react-scrollable-feed` with `react-scrollable-feed-virtualized`

* fix get new entries button

* Fix the not populated `Protocol` struct in case of `GetEntries` endpoint is called

Co-authored-by: Liraz Yehezkel <lirazy@up9.com>
2021-09-13 16:18:43 +03:00
Selton Fiuza
dea223bfe1 Feature/tra 3349 validation rules merged with develop (#148)
* Implemented validation rules, based on: https://up9.atlassian.net/browse/TRA-3349

* Color on Entry based on rules

* Background red/green based on rules

* Change flag --validation-rules to --test-rules

* rules tab UI updated

* rules tab font and background-color is changed for objects

* Merged with develop

* Fixed compilation issues.

* Renamed fullEntry -> harEntry where appropriate.

* Change green/red logic

* Update models.go

* Fix latency bug and alignment

* Merge Conflicts fix

* Working after merge

* Working on Nimrod comments

* Resolving conflicts

* Resolving conflicts

* Resolving conflicts

* Nimrod Comments pt.3

* Log Error on configmap creation if the user doesn't have permission.

* Checking configmap permission to ignore --test-rules

* Revert time for mizu to get ready

* Nimrod comments pt 4 && merge develop pt3

* Nimrod comments pt 4 && merge develop pt3

* Const rulePolicyPath and filename

Co-authored-by: Neim <elezin9@gmail.com>
Co-authored-by: nimrod-up9 <nimrod@up9.com>
2021-08-04 09:21:36 -03:00
RamiBerm
ceb8d714e3 TRA-3420 inform user of tls traffic (#149)
* Update passive_tapper.go and tls_utils.go

* Update go.mod, go.sum, and 18 more files...

* go fmt

* Update http_reader.go, passive_tapper.go, and 3 more files...

* Update status_controller.go and status_provider.go

Co-authored-by: RamiBerm <rami.berman@up9.com>
2021-08-01 14:57:43 +03:00
lirazyehezkel
a2150b4a78 UI api helper (#131)
* Api helper

* cr fixes
2021-07-22 15:41:09 +03:00
nimrod-up9
1ddc7f2f6b Decode gRPC body (#48)
* Decode grpc.

* Better variable names.

* Added protobuf-decoder dependency.

* Updated protobuf-decoder's version.
2021-05-12 18:42:42 +03:00
lirazyehezkel
377fc79315 Infinite scroll (#28)
* no message

* infinite scroll + new ws implementation

* no message

* scrolling top

* fetch button

* more Backend changes

* fix go mod and sum

* mire fixes against develop

* unused code

* small ui refactor

Co-authored-by: Roee Gadot <roee.gadot@up9.com>
2021-05-04 18:23:58 +03:00
Liraz Yehezkel
a8c72ceb69 Live traffic web socket 2021-04-28 16:15:13 +03:00
Liraz Yehezkel
5f71fcfb9f stating with web socket 2021-04-28 13:49:07 +03:00
Liraz Yehezkel
0e0b51b42c Traffic viewer 2021-04-27 14:48:22 +03:00