feat(indexer): Add grafana and alert service to local development #7257

roninjin10 · 2023-09-15T02:58:47Z

add prometheus to docker-compose
add grafana to docker-compose
add new folder for holding grafana provisioning and configuration
Add alert service
Configure alert service
Configure gafana dashboard for api
Configure grafana dashboard for indexer
remove gateway/backend service
make docker-compose run against more networks than just goerli

roninjin10 · 2023-09-15T02:58:59Z

Current dependencies on/for this PR:

develop
- PR fix(indexer): Add migrations file to docker image" #7260
  - PR feat(indexer): Add grafana and alert service to local development #7257 👈

This comment was auto-generated by Graphite.

semgrep-app · 2023-09-15T02:59:31Z

indexer/docker-compose.yml

+    indexer:
+      condition: service_healthy
+
+  grafana:


Service 'grafana' is running with a writable root filesystem. This may allow malicious applications to download and run additional payloads, or modify container files. If an application inside a container has to save something temporarily consider using a tmpfs. Add 'read_only: true' to this service to prevent this.
_{Ignore this finding from writable-filesystem-service.}

semgrep-app · 2023-09-15T02:59:33Z

indexer/docker-compose.yml

    depends_on:
      postgres:
        condition: service_healthy

-  ui:
+  prometheus:


Service 'prometheus' is running with a writable root filesystem. This may allow malicious applications to download and run additional payloads, or modify container files. If an application inside a container has to save something temporarily consider using a tmpfs. Add 'read_only: true' to this service to prevent this.
_{Ignore this finding from writable-filesystem-service.}

semgrep-app · 2023-09-15T02:59:34Z

indexer/docker-compose.yml

+    indexer:
+      condition: service_healthy
+
+  grafana:


Service 'grafana' allows for privilege escalation via setuid or setgid binaries. Add 'no-new-privileges:true' in 'security_opt' to prevent this.
_{Ignore this finding from no-new-privileges.}

semgrep-app · 2023-09-15T02:59:35Z

indexer/docker-compose.yml

    depends_on:
      postgres:
        condition: service_healthy

-  ui:
+  prometheus:


Service 'prometheus' allows for privilege escalation via setuid or setgid binaries. Add 'no-new-privileges:true' in 'security_opt' to prevent this.
_{Ignore this finding from no-new-privileges.}

semgrep-app · 2023-09-15T03:07:21Z

indexer/docker-compose.yml

+    indexer:
+      condition: service_healthy
+
+  alertmanager:


Service 'alertmanager' is running with a writable root filesystem. This may allow malicious applications to download and run additional payloads, or modify container files. If an application inside a container has to save something temporarily consider using a tmpfs. Add 'read_only: true' to this service to prevent this.
_{Ignore this finding from writable-filesystem-service.}

semgrep-app · 2023-09-15T03:07:22Z

indexer/docker-compose.yml

+    indexer:
+      condition: service_healthy
+
+  alertmanager:


Service 'alertmanager' allows for privilege escalation via setuid or setgid binaries. Add 'no-new-privileges:true' in 'security_opt' to prevent this.
_{Ignore this finding from no-new-privileges.}

semgrep-app · 2023-09-15T03:32:03Z

Semgrep found 1 gorm-hardcoded-secret finding:

indexer/database/db.go: L51-55

A secret is hard-coded in the application. Secrets stored in source code, such as credentials, identifiers, and other types of sensitive data, can be leaked and used by internal or external malicious actors. Use environment variables to securely provide credentials and other secrets or retrieve them from a secure vault or Hardware Security Module (HSM).

View Dataflow Graph

flowchart LR
    classDef invis fill:white, stroke: none
    classDef default fill:#e7f5ff, color:#1c7fd6, stroke: none

    subgraph File0["<b>indexer/database/db.go</b>"]
        direction LR
        %% Source

        subgraph Source
            direction LR

            v0("<b>[Line: 49]</b> #quot; password=%s#quot;")
        end
        %% Intermediate

        subgraph Traces0[Traces]
            direction TB

            v2("<b>[Line: 49]</b> dsn")
        end
        %% Sink

        subgraph Sink
            direction LR

            v1("<b>[Line: 51]</b> gorm.Open(postgres.Open(dsn), &gorm.Config{<br>			// The indexer will explicitly manage the transactions<br>			SkipDefaultTransaction: true,<br>			Logger:                 logger.Default.LogMode(logger.Silent),<br>		})")
        end
    end
    %% Class Assignment
    Source:::invis
    Sink:::invis

    Traces0:::invis
    File0:::invis

    %% Connections

    Source --> Traces0
    Traces0 --> Sink

    %% Clickable

    click v0 href "https://github.com/ethereum-optimism/optimism/blob/07e6b6e53a4988118488d265a5ee7f175a29d351/indexer/database/db.go#L49" "View in source" _blank
    click v1 href "https://github.com/ethereum-optimism/optimism/blob/07e6b6e53a4988118488d265a5ee7f175a29d351/indexer/database/db.go#L51" "View in source" _blank
    click v2 href "https://github.com/ethereum-optimism/optimism/blob/07e6b6e53a4988118488d265a5ee7f175a29d351/indexer/database/db.go#L49" "View in source" _blank

_{Ignore this finding from gorm-hardcoded-secret.}

Semgrep found 1 gorm-empty-password finding:

indexer/database/db.go: L51-55

The application uses an empty credential. This can lead to unauthorized access by either an internal or external malicious actor. Use environment variables to securely provide credentials and other secrets or retrieve them from a secure vault or Hardware Security Module (HSM).

View Dataflow Graph

flowchart LR
    classDef invis fill:white, stroke: none
    classDef default fill:#e7f5ff, color:#1c7fd6, stroke: none

    subgraph File0["<b>indexer/database/db.go</b>"]
        direction LR
        %% Source

        subgraph Source
            direction LR

            v0("<b>[Line: 49]</b> #quot; password=%s#quot;")
        end
        %% Intermediate

        subgraph Traces0[Traces]
            direction TB

            v2("<b>[Line: 49]</b> dsn")
        end
        %% Sink

        subgraph Sink
            direction LR

            v1("<b>[Line: 51]</b> gorm.Open(postgres.Open(dsn), &gorm.Config{<br>			// The indexer will explicitly manage the transactions<br>			SkipDefaultTransaction: true,<br>			Logger:                 logger.Default.LogMode(logger.Silent),<br>		})")
        end
    end
    %% Class Assignment
    Source:::invis
    Sink:::invis

    Traces0:::invis
    File0:::invis

    %% Connections

    Source --> Traces0
    Traces0 --> Sink

    %% Clickable

    click v0 href "https://github.com/ethereum-optimism/optimism/blob/07e6b6e53a4988118488d265a5ee7f175a29d351/indexer/database/db.go#L49" "View in source" _blank
    click v1 href "https://github.com/ethereum-optimism/optimism/blob/07e6b6e53a4988118488d265a5ee7f175a29d351/indexer/database/db.go#L51" "View in source" _blank
    click v2 href "https://github.com/ethereum-optimism/optimism/blob/07e6b6e53a4988118488d265a5ee7f175a29d351/indexer/database/db.go#L49" "View in source" _blank

_{Ignore this finding from gorm-empty-password.}

feat: Add alert manager fix: Add chain preset id to docker compose fix: Pass in database env variables in docker compose

github-actions · 2023-09-30T01:45:32Z

This PR is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 5 days.

mergify bot added the A-indexer Area: indexer label Sep 15, 2023

semgrep-app bot reviewed Sep 15, 2023

View reviewed changes

roninjin10 requested a review from hamdiallam September 15, 2023 03:07

roninjin10 changed the title ~~feat(indexer): Add grafana to local development~~ feat(indexer): Add grafana and alert service to local development Sep 15, 2023

roninjin10 requested a review from spacesailor24 September 15, 2023 03:12

roninjin10 changed the base branch from develop to 09-01-feat_indexer_Add_sql_migrations_file September 15, 2023 03:31

roninjin10 force-pushed the 09-14-feat_indexer_Add_grafana_to_local_development branch from 0791421 to 07e6b6e Compare September 15, 2023 03:31

roninjin10 force-pushed the 09-14-feat_indexer_Add_grafana_to_local_development branch 2 times, most recently from 2cd7f9d to 7f3d0a1 Compare September 15, 2023 04:06

roninjin10 changed the base branch from 09-01-feat_indexer_Add_sql_migrations_file to 09-15-fix_Add_migrations_to_dockerfile September 15, 2023 13:59

roninjin10 force-pushed the 09-14-feat_indexer_Add_grafana_to_local_development branch from 7f3d0a1 to 4c5adcc Compare September 15, 2023 13:59

roninjin10 mentioned this pull request Sep 15, 2023

fix(indexer): Add migrations file to docker image" #7260

Closed

roninjin10 force-pushed the 09-15-fix_Add_migrations_to_dockerfile branch from d7cfe44 to 0809213 Compare September 15, 2023 14:12

roninjin10 force-pushed the 09-14-feat_indexer_Add_grafana_to_local_development branch from 4c5adcc to 2d80dd3 Compare September 15, 2023 14:12

roninjin10 mentioned this pull request Sep 15, 2023

🔧 feat: Add grafana #7221

Closed

Will Cory added 2 commits September 15, 2023 10:05

🐛 fix(indexer): add retries to db connection

6648dd0

🐛 fix: Copy foundry.toml in dockerfile

b0f5761

Will Cory and others added 4 commits September 15, 2023 10:05

✨ feat(indexer): Add sql migrations file

8330595

test: update them'

0e22fee

Update indexer/database/db.go

55be9f4

feat(indexer): Add grafana to local development

e961711

feat: Add alert manager fix: Add chain preset id to docker compose fix: Pass in database env variables in docker compose

roninjin10 force-pushed the 09-15-fix_Add_migrations_to_dockerfile branch from 0809213 to f25a591 Compare September 15, 2023 17:07

roninjin10 force-pushed the 09-14-feat_indexer_Add_grafana_to_local_development branch from 2d80dd3 to e961711 Compare September 15, 2023 17:07

github-actions bot added the Stale label Sep 30, 2023

github-actions bot closed this Oct 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(indexer): Add grafana and alert service to local development #7257

feat(indexer): Add grafana and alert service to local development #7257

roninjin10 commented Sep 15, 2023 •

edited

Loading

roninjin10 commented Sep 15, 2023 •

edited

Loading

semgrep-app bot Sep 15, 2023

semgrep-app bot Sep 15, 2023

semgrep-app bot Sep 15, 2023

semgrep-app bot Sep 15, 2023

semgrep-app bot Sep 15, 2023

semgrep-app bot Sep 15, 2023

semgrep-app bot commented Sep 15, 2023

github-actions bot commented Sep 30, 2023

feat(indexer): Add grafana and alert service to local development #7257

feat(indexer): Add grafana and alert service to local development #7257

Conversation

roninjin10 commented Sep 15, 2023 • edited Loading

roninjin10 commented Sep 15, 2023 • edited Loading

semgrep-app bot Sep 15, 2023

Choose a reason for hiding this comment

semgrep-app bot Sep 15, 2023

Choose a reason for hiding this comment

semgrep-app bot Sep 15, 2023

Choose a reason for hiding this comment

semgrep-app bot Sep 15, 2023

Choose a reason for hiding this comment

semgrep-app bot Sep 15, 2023

Choose a reason for hiding this comment

semgrep-app bot Sep 15, 2023

Choose a reason for hiding this comment

semgrep-app bot commented Sep 15, 2023

github-actions bot commented Sep 30, 2023

roninjin10 commented Sep 15, 2023 •

edited

Loading

roninjin10 commented Sep 15, 2023 •

edited

Loading