Skip to content

Latest commit

 

History

History
491 lines (340 loc) · 16.7 KB

Readme.md

File metadata and controls

491 lines (340 loc) · 16.7 KB
Raw

Observe Entity Explorer

Observe Entity Explorer allows discovery of dependencies between Observe objects (datasets, worksheets, dashboards, monitors, stages, parameters).

Dataset Graph

NOTE: This tool uses the Observe GraphQL API, which is not [yet] a publicly documented API. Therefore this tool, or any application that you develop that is derived from it may stop working without notice in the future. If you have specific API needs that are not covered by our currently documented API found at https://developer.observeinc.com/, please contact us for more information.

Install Application

Observe Entity Explorer can run on Windows, Mac or Linux. Compiled are in Releases.

Install on OSX

  1. Install .NET 8.0 SDK

  2. Download Releases\ observe-entity-explorer.osx-x64.<version>.zip for Intel or observe-entity-explorer.osx-arm64.<version>.zip for M1/M2 machines, but do not extract the archive yet.

  3. Remove the quarantine attribute that will otherwise stop the application from running this command in your terminal:

    xattr -d com.apple.quarantine observe-entity-explorer.osx*.zip

  4. Extract the archive:

    unzip observe-entity-explorer.osx*.zip

  5. Run the application from the extracted folder:

    dotnet ./observe-entity-explorer.dll

Install on Windows

  1. Download Releases\ observe-entity-explorer.win-x64.<version>.zip

  2. Extract the archive using File Explorer or:

    Expand-Archive observe-entity-explorer.win*.zip

  3. Run the application from the extracted folder:

    .\observe-entity-explorer.exe

Install on Linux

  1. Download Releases\ observe-entity-explorer.linux-x64.<version>.zip

  2. Extract the archive:

    unzip observe-entity-explorer.linux*.zip

  3. Run the application from the extracted folder:

    ./observe-entity-explorer

Run from Source

You can also run it from source.

First, install .NET 8.0 SDK from Microsoft Downloads web site.

Then clone, build and run:

# Clone repo to local folder
git clone https://github.com/observeinc/entity-explorer entity-explorer

# Change to local folder
cd entity-explorer

# For Win and Linux, build the project into single self-contained binary. Choose the right runtime
dotnet publish observe-entity-explorer.csproj --self-contained --runtime <linux-x64|win-x64> -p:PublishSingleFile=true -o bin/publish/entity-explorer

# For OSX, build the project into multiple files to be ran by dotnet command later. Choose the right runtime
dotnet publish observe-entity-explorer.csproj --self-contained --runtime <osx-x64|osx-arm64> -p:PublishSingleFile=false -o bin/publish/entity-explorer

# Run the application from where it was published to
bin/publish/entity-explorer/observe-entity-explorer &

Usage

Open the Web Site

When application starts, it opens a web interface at http://localhost:50110. Open that web site in your web browser:

./observe-entity-explorer

info: Microsoft.Hosting.Lifetime[0]
      Now listening on: http://localhost:50110
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Production
info: Microsoft.Hosting.Lifetime[0]
      Content root path: C:\observe\observe-entity-explorer-releases\2023.10.18.0\observe-entity-explorer.win.2023.10.18.0

Connect Options

Authenticate options:

  • Account
    • Full URL of the environment https://#####.observeinc.com
    • Just customer ID of the environment #####. Production URL observeinc.com is assumed
    • Host name #####.observeinc.com without https://
  • Username
    • Typically it is your email
  • Password
    • Specify your password
  • Token
    • Specify your API token if you have one issued to you
    • OR
    • Specify token taken from the GraphQL Bearer authentication in your browser devtools
  • Delegate (must be > Reader)
    • Log into Observe UI to approve your sign-in
    • Return to this UI to complete sign-in

Connect

Connect with Delegate Login

To connect with delegation, you must have > Reader role. If you have it:

  • Specify account and username
  • Click "Connect with Observe UI" button
  • Click on "Go to Observe to approve login request link"
  • In new browser window, authenticate into Observe
  • Confirm your sign-in request on "Accounts Settings" page
  • Back in Observe Entity Explorer, click "Check for Completion and Connect" button to finish signing in

Connect Delegate

Connect with Multiple Accounts

Your connections are saved. You can switch between them any time.

You can open new tab with different connection ID.

Multiple Connections

Entity Summary

Summary screen shows basic statistics about types of entities in your Observe environment and links to the Dataset List Dashboard List for getting more details.

You can also see the entire tree of relationships of all objects in All Relationships button.

Environment Summary

Dataset List

Select Dataset screen shows all datasets grouped by type (DataStream, Event, Resource, Log, Metric, etc).

Column Description
Details Open the Dataset Detail page
View Open object page in Observe
Edit Open object page in Observe in edit mode
Type Type of object (Event, Resource, Interval)
Origin What created this object (System, App, Terraform, User, DataStream)
ID Object ID
Name Object Name
Description Description label, if present
Parts Fields, primary and foreign keys, related keys
Uses What this object uses as data inputs and data links
Used By What uses this object (Data, Dashboard, Monitor)
Created Who and when created this object
Updated Who and when updates this object last
Transform 1h credits_transform in last 1 hour
Transform 1d credits_transform in last 1 day
Transform 1w credits_transform in last 1 week
Query 1h credits_adhoc_query in last 1 hour
Query 1d credits_adhoc_query in last 1 day
Query 1w credits_adhoc_query in last 1 week
Accel Conf. Configured staleness target of the dataset
Accel Eff. The target staleness of this dataset when taking downstream datasets
Accel Actl. Staleness of the dataset (averaged over some moving window)
Accel Range Actual accelerated range of the dataset

Dataset List

Dashboard List

Select Dashboard screen shows all dashboards.

Column Description
Details Links to the Dashboard Detail page
View Open object page in Observe
Edit Open object page in Observe in edit mode
Origin What created this object (System, App, Terraform, User, DataStream)
ID Object ID
Name Object Name
Sections Number of sections in dashboard
Widgets Number of widgets in dashboard
Stages Number of stages in dashboard
Parameters Number of parameters in dashboard
Uses What this object uses as data inputs and data links
Created Who and when created this object
Updated Who and when updates this object last

Dashboard List

Monitor List

Select Monitor screen shows all monitors.

Column Description
Details Links to the Monitor Detail page
Notif Open notification list page in Observe
Edit Open object page in Observe in edit mode
Type Type of the monitor (Count, Promote, Threshold, Facet, Log)
Origin What created this object (System, App, Terraform, User, DataStream)
ID Object ID
Name Object Name
Comm. Comment if it exists
Enabled Is the monitor enabled
Actions Number of actions in monitor
Stages Number of stages in monitor
Uses What this object uses as data inputs and data links
Created Who and when created this object
Updated Who and when updates this object last
Transform 1h credits_monitor in last 1 hour
Transform 1d credits_monitor in last 1 day
Transform 1w credits_monitor in last 1 week
Accel Conf. Configured staleness target of the dataset
Accel Eff. The target staleness of this dataset when taking downstream datasets
Accel Actl. Staleness of the dataset (averaged over some moving window)
Accel Range Actual accelerated range of the dataset

Monitor List

Dataset Detail

Summary - Dataset

Summary information about the dataset.

Section Description
Observe Open object page in Observe
Name Object name and ID
Description Dataset description
Type Type of dataset
Icon Icon of the object
Origin What created this object (System, App, Terraform, User, DataStream)
Created Who and when created this object
Updated Who and when updates this object last
Credits credits_transform and credits_adhoc_query for 1 hour, 1 day, 1 week
Query Credits credits_adhoc_query for each user for 1 hour, 1 day, 1 week
Important Fields From/To/Label designated fields
Fields Table of fields and their datatypes
Related Key List of related keys
Foreign Key List of foreign keys

Dataset Details

Acceleration - Dataset

Dataset acceleration settings.

Dataset Acceleration

Ancestors - Dataset

List of all datasets feeding data into this dataset.

Dataset Ancestors

Descendants - Dataset

List of all datasets using data from this dataset.

Dataset Descendants

Related Datasets Graph - Dataset

Visual diagram showing relationship between datasets and dashboards related to this dataset.

Dataset Graph

Stages - Dataset

Information about each stage in the dataset.

Section Description
Details Link to the Stage in the subsequent List
Type Type of the stage (table, graph etc)
Name Name of the stage
ID ID of the stage
Uses Objects providing data and links to this stage
Used By Objects this stage is providing data and links to

Dataset Stages List

This table is followed by the repeated sections for each stage, containing:

Section Description
Name Object name and ID
Inputs Objects providing data and links to this stage
Used By Objects this stage is providing data and links to
OPAL The OPAL pipeline for this stage

Dataset Stage Detail

You can click "Show/Hide input and output tables" to remove details and just focus on OPAL code.

Stages Graph - Dataset

Visual diagram showing relationship between inputs and stages of this dataset.

Dataset Stages Graph

Related Dashboards

List of all dashboards using this dataset.

Dataset Related Dashboards

Related Monitors

List of all monitors using this dataset.

Dataset Related Dashboards

Dashboard Detail

Summary - Dashboard

Summary information about the dashboard.

Section Description
Observe Open object page in Observe
Name Object name and ID
Source What created this object (System, App, Terraform, User, DataStream)
Components Number of various components in dashboard
Created Who and when created this object
Updated Who and when updates this object last

Dashboard Details

Ancestors/Inputs - Dashboard

List of all datasets feeding data into this dashboard.

Related Datasets - Dashboard

Visual diagram showing relationship between datasets related to this dashboard.

Dataset Graph

Parameters - Dashboard

Information about each parameter in the dashboard.

Section Description
Name Display name of parameter
ID ID of parameter
Type Type of parameter (resource-input, single-select, text, numeric, input)
Data Type Data Type of parameter ()
Source Type What kind of object is providing data to this parameter (Dataset, Stage or empty for text entry)
Source Object Link to the object that is used as input
Source Column The name of the column in the dataset or stage that is being used as value in the drop-down
Default Value Default value, if specified
Allow Empty Is empty value allowed

Dashboard Parameters

Stages - Dashboard

Information about each stage in the dashboard.

Section Description
Details Link to the stage in the subsequent list
Type Type of the stage (table, graph etc)
Name Name of the stage
ID ID of the stage
Uses Objects providing data and links to this stage
Used By Objects this stage is providing data and links to
Params Which parameters are being used by the stage

Dashboard Stages

This table is followed by the repeated sections for each stage, containing:

Section Description
Name Object name and ID
Inputs Objects providing data and links to this stage
Used By Objects this stage is providing data and links to
OPAL The OPAL pipeline for this stage

Dashboard Stage Detail

You can click "Show/Hide input and output tables" to remove details and just focus on OPAL code.

You can click "Show/Hide dashboard widget image preview" to remove images and just focus on OPAL code.

Stages Graph - Dashboard

Visual diagram showing relationship between inputs and stages of this dashboard.

Dashboard Stages Graph

Monitor Detail

Summary - Monitor

Summary information about the monitor.

Section Description
Observe Open object page in Observe
Name Object name and ID
Comment Comment if present
Type Type of the monitor
Source What created this object (System, App, Terraform, User, DataStream)
Components Number of various components in dashboard
Enabled Is this monitor enabled
Template Is this monitor template
Created Who and when created this object
Updated Who and when updates this object last
Settings Monitor settings
Notifications Monitor notification settings
Info Monitor detail information

Monitor Details

Acceleration - Monitor

Monitor acceleration settings similar to those of Dataset

Ancestors - Monitor

List of all datasets providing data to this monitor.

Monitor Ancestors

Supporting - Monitor

List of all datasets supporting this monitor.

Monitor Ancestors

Related Datasets Graph - Monitor

Visual diagram showing relationship between datasets related to this monitor.

Dataset Graph

Stages - Monitor

Information about each stage in the monitor.

Logging

Logs are in /logs subfolder below where your application is installed with new files created every day:

logs\Observe.EntityExplorer.Console.2023-10-18.log
logs\Observe.EntityExplorer.Main.2023-10-18.log
logs\Observe.EntityExplorer.ObserveConnection.2023-10-18.log