handle env variables in dynamic secret file #5835

alban-stourbe-wmx · 2024-11-18T14:07:37Z

Proposed changes

Add env variables in dynamic secret files.

Checklist

Pull request is created against the dev branch
All checks passed (lint, unit/integration/regression tests etc.) with my changes
I have added tests that prove my fix is effective or that my feature works
I have added necessary documentation (if appropriate)

GeorginaReeder · 2024-11-19T10:46:31Z

Thanks for your contribution @alban-stourbe-wmx ! :)

internal/runner/lazy.go

tarunKoyalwar

lgtm !

along with env secrets now secrets can also be specified from -v and -env-vars ex:

Env based secret

variables:
   - key: password
     value: $PASSWORD

Config file / Flag based secrets ( using -env-vars or -vars )

variables:
     - key: password
     - value: {{password}}

tarunKoyalwar

needs rebase from dev

alban-stourbe-wmx · 2024-11-21T09:27:18Z

needs rebase from dev

Hi @tarunKoyalwar, the rebase has been made. Thank for the code added ☝️

ehsandeep

Thank you @alban-stourbe-wmx for this update, FYI we are also working on #5841

alban-stourbe-wmx · 2024-11-21T11:08:35Z

Thank you @alban-stourbe-wmx for this update, FYI we are also working on #5841

Thanks to you and also congrats to @dwisiswant0 and @tarunKoyalwar

* projectdiscovery-main: (283 commits) upgrade ci(generate-docs): push w/o pull (projectdiscovery#5843) auto gen docs update handle env variables in dynamic secret file (projectdiscovery#5835) disable self-contained and file protocol templates as default (projectdiscovery#5825) chore(deps): bump github.com/projectdiscovery/hmap from 0.0.65 to 0.0.67 (projectdiscovery#5832) feat: Added time based delay analyzer to fuzzing implementation (projectdiscovery#5781) fix: data race at `protocolstate`, `contextargs` & some outdated test cases (projectdiscovery#5820) chore(deps): bump github.com/projectdiscovery/retryabledns chore(deps): bump github.com/projectdiscovery/goflags chore(deps): bump github.com/projectdiscovery/dsl from 0.3.0 to 0.3.3 chore(deps): bump github.com/projectdiscovery/rawhttp Batch JSONL output and add trailing commas (projectdiscovery#5705) ci: refactor workflows (projectdiscovery#5818) chore(deps): bump github.com/golang-jwt/jwt/v4 from 4.5.0 to 4.5.1 (projectdiscovery#5795) chore(deps): bump github.com/projectdiscovery/rawhttp (projectdiscovery#5809) chore(deps): bump github.com/projectdiscovery/wappalyzergo (projectdiscovery#5808) update version chore(deps): bump github.com/projectdiscovery/gologger chore(deps): bump github.com/projectdiscovery/useragent ... # Conflicts: # .github/ISSUE_TEMPLATE/feature_request.md # .github/ISSUE_TEMPLATE/issue-report.md # .run/DSLFunctionsIT.run.xml # DESIGN.md # README.md # README_CN.md # README_ID.md # README_KR.md # cmd/docgen/docgen.go # cmd/functional-test/main.go # cmd/functional-test/run.sh # cmd/integration-test/custom-dir.go # cmd/integration-test/dns.go # cmd/integration-test/file.go # cmd/integration-test/fuzz.go # cmd/integration-test/headless.go # cmd/integration-test/http.go # cmd/integration-test/integration-test.go # cmd/integration-test/library.go # cmd/integration-test/loader.go # cmd/integration-test/offline-http.go # cmd/integration-test/ssl.go # cmd/integration-test/template-dir.go # cmd/integration-test/template-path.go # cmd/integration-test/websocket.go # cmd/integration-test/whois.go # cmd/integration-test/workflow.go # cmd/nuclei/main.go # examples/advanced/advanced.go # examples/simple/simple.go # go.mod # go.sum # internal/colorizer/colorizer.go # internal/runner/banner.go # internal/runner/healthcheck.go # internal/runner/lazy.go # internal/runner/options.go # internal/runner/proxy.go # internal/runner/runner.go # internal/runner/runner_test.go # internal/runner/templates.go # lib/example_test.go # lib/sdk.go # lib/sdk_private.go # pkg/catalog/disk/find.go # pkg/catalog/loader/filter/path_filter.go # pkg/catalog/loader/loader.go # pkg/catalog/loader/loader_test.go # pkg/catalog/loader/remote_loader.go # pkg/core/engine.go # pkg/core/executors.go # pkg/core/workflow_execute.go # pkg/core/workflow_execute_test.go # pkg/external/customtemplates/azure_blob.go # pkg/external/customtemplates/github.go # pkg/external/customtemplates/github_test.go # pkg/external/customtemplates/gitlab.go # pkg/external/customtemplates/s3.go # pkg/external/customtemplates/templates_provider.go # pkg/fuzz/component/path.go # pkg/fuzz/fuzz.go # pkg/input/formats/swagger/swagger.go # pkg/input/provider/list/hmap.go # pkg/input/provider/list/hmap_test.go # pkg/input/transform.go # pkg/installer/template.go # pkg/installer/template_test.go # pkg/installer/util.go # pkg/installer/versioncheck.go # pkg/installer/versioncheck_test.go # pkg/js/compiler/compiler.go # pkg/js/compiler/init.go # pkg/js/global/scripts.go # pkg/js/libs/mssql/mssql.go # pkg/js/libs/postgres/postgres.go # pkg/loader/workflow/workflow_loader.go # pkg/model/model.go # pkg/model/model_test.go # pkg/model/types/severity/severities.go # pkg/model/types/stringslice/stringslice.go # pkg/operators/common/dsl/dsl.go # pkg/operators/extractors/compile.go # pkg/operators/extractors/extract.go # pkg/operators/matchers/compile.go # pkg/operators/matchers/match.go # pkg/operators/operators.go # pkg/output/format_screen.go # pkg/output/output.go # pkg/output/output_test.go # pkg/protocols/common/expressions/expressions.go # pkg/protocols/common/expressions/variables.go # pkg/protocols/common/generators/generators.go # pkg/protocols/common/generators/generators_test.go # pkg/protocols/common/generators/options.go # pkg/protocols/common/generators/validate.go # pkg/protocols/common/helpers/eventcreator/eventcreator.go # pkg/protocols/common/helpers/responsehighlighter/response_highlighter.go # pkg/protocols/common/helpers/responsehighlighter/response_highlighter_test.go # pkg/protocols/common/helpers/writer/writer.go # pkg/protocols/common/interactsh/interactsh.go # pkg/protocols/common/interactsh/options.go # pkg/protocols/common/protocolstate/state.go # pkg/protocols/common/replacer/replacer.go # pkg/protocols/common/uncover/uncover.go # pkg/protocols/common/utils/vardump/dump.go # pkg/protocols/common/variables/variables.go # pkg/protocols/dns/dns.go # pkg/protocols/dns/dns_test.go # pkg/protocols/dns/dnsclientpool/clientpool.go # pkg/protocols/dns/operators.go # pkg/protocols/dns/operators_test.go # pkg/protocols/dns/request.go # pkg/protocols/dns/request_test.go # pkg/protocols/file/file.go # pkg/protocols/file/find_test.go # pkg/protocols/file/operators.go # pkg/protocols/file/operators_test.go # pkg/protocols/file/request.go # pkg/protocols/file/request_test.go # pkg/protocols/headless/engine/engine.go # pkg/protocols/headless/engine/http_client.go # pkg/protocols/headless/engine/instance.go # pkg/protocols/headless/engine/page_actions.go # pkg/protocols/headless/engine/page_actions_test.go # pkg/protocols/headless/engine/util.go # pkg/protocols/headless/headless.go # pkg/protocols/headless/operators.go # pkg/protocols/headless/request.go # pkg/protocols/http/build_request.go # pkg/protocols/http/build_request_test.go # pkg/protocols/http/cluster.go # pkg/protocols/http/http.go # pkg/protocols/http/http_test.go # pkg/protocols/http/httpclientpool/clientpool.go # pkg/protocols/http/operators.go # pkg/protocols/http/operators_test.go # pkg/protocols/http/request.go # pkg/protocols/http/request_annotations.go # pkg/protocols/http/request_annotations_test.go # pkg/protocols/http/request_fuzz.go # pkg/protocols/http/request_generator.go # pkg/protocols/http/request_generator_test.go # pkg/protocols/http/signature.go # pkg/protocols/http/signer/signer.go # pkg/protocols/http/signerpool/signerpool.go # pkg/protocols/javascript/js.go # pkg/protocols/network/network.go # pkg/protocols/network/network_test.go # pkg/protocols/network/networkclientpool/clientpool.go # pkg/protocols/network/operators.go # pkg/protocols/network/operators_test.go # pkg/protocols/network/request.go # pkg/protocols/network/request_test.go # pkg/protocols/offlinehttp/find_test.go # pkg/protocols/offlinehttp/offlinehttp.go # pkg/protocols/offlinehttp/operators.go # pkg/protocols/offlinehttp/operators_test.go # pkg/protocols/offlinehttp/request.go # pkg/protocols/protocols.go # pkg/protocols/ssl/ssl.go # pkg/protocols/ssl/ssl_test.go # pkg/protocols/utils/http/variables.go # pkg/protocols/utils/http/variables_test.go # pkg/protocols/utils/utils.go # pkg/protocols/websocket/websocket.go # pkg/protocols/whois/rdapclientpool/clientpool.go # pkg/protocols/whois/whois.go # pkg/reporting/client.go # pkg/reporting/dedupe/dedupe.go # pkg/reporting/dedupe/dedupe_test.go # pkg/reporting/exporters/es/elasticsearch.go # pkg/reporting/exporters/jsonexporter/jsonexporter.go # pkg/reporting/exporters/markdown/markdown.go # pkg/reporting/exporters/sarif/sarif.go # pkg/reporting/exporters/splunk/splunkhec.go # pkg/reporting/format/format_utils.go # pkg/reporting/format/format_utils_test.go # pkg/reporting/options.go # pkg/reporting/reporting.go # pkg/reporting/trackers/jira/jira.go # pkg/scan/scan_context.go # pkg/templates/cluster.go # pkg/templates/compile.go # pkg/templates/compile_test.go # pkg/templates/parser_test.go # pkg/templates/tag_filter.go # pkg/templates/tag_filter_test.go # pkg/templates/templates.go # pkg/templates/templates_doc_examples.go # pkg/templates/types/types.go # pkg/templates/workflows.go # pkg/testutils/testutils.go # pkg/tmplexec/exec.go # pkg/tmplexec/flow/flow_executor.go # pkg/tmplexec/interface.go # pkg/tmplexec/multiproto/multi.go # pkg/tmplexec/multiproto/multi_test.go # pkg/types/interfaces.go # pkg/types/resume.go # pkg/types/types.go # pkg/utils/template_path.go # pkg/utils/utils.go # pkg/utils/yaml/preprocess.go # pkg/workflows/workflows.go # pkg/workflows/workflows_test.go

auto-assign bot requested a review from dwisiswant0 November 18, 2024 14:07

alban-stourbe-wmx mentioned this pull request Nov 18, 2024

[FEATURE] Add Env variables to dynamic secret file variables #5834

Closed

ehsandeep requested a review from tarunKoyalwar November 19, 2024 16:32

dwisiswant0 mentioned this pull request Nov 20, 2024

[FEATURE] Integrate Secret File with SOPS #5840

Open

dogancanbakir requested changes Nov 20, 2024

View reviewed changes

internal/runner/lazy.go Show resolved Hide resolved

tarunKoyalwar approved these changes Nov 20, 2024

View reviewed changes

tarunKoyalwar requested changes Nov 20, 2024

View reviewed changes

ehsandeep linked an issue Nov 20, 2024 that may be closed by this pull request

[FEATURE] Add Env variables to dynamic secret file variables #5834

Closed

alban-stourbe-wmx and others added 4 commits November 21, 2024 10:24

handle env variables in dynamic secret file

92d7286

inject more variables from -v and -env-vars

6f83d45

use expand with env

4fb62df

fix missing replacer

01c62c6

alban-stourbe-wmx force-pushed the feature/add-env-vars-into-variables-secret-files branch from c4aa997 to 01c62c6 Compare November 21, 2024 09:24

ehsandeep approved these changes Nov 21, 2024

View reviewed changes

ehsandeep merged commit 2b4b058 into projectdiscovery:dev Nov 21, 2024
17 checks passed

BrewTestBot mentioned this pull request Nov 21, 2024

nuclei 3.3.6 Homebrew/homebrew-core#198559

Merged

blsaccess mentioned this pull request Nov 22, 2024

Update nuclei to 3.3.6 blacklanternsecurity/bbot#2006

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

handle env variables in dynamic secret file #5835

handle env variables in dynamic secret file #5835

alban-stourbe-wmx commented Nov 18, 2024

GeorginaReeder commented Nov 19, 2024

tarunKoyalwar left a comment •

edited

Loading

tarunKoyalwar left a comment

alban-stourbe-wmx commented Nov 21, 2024

ehsandeep left a comment

alban-stourbe-wmx commented Nov 21, 2024

handle env variables in dynamic secret file #5835

handle env variables in dynamic secret file #5835

Conversation

alban-stourbe-wmx commented Nov 18, 2024

Proposed changes

Checklist

GeorginaReeder commented Nov 19, 2024

tarunKoyalwar left a comment • edited Loading

Choose a reason for hiding this comment

Env based secret

Config file / Flag based secrets ( using -env-vars or -vars )

tarunKoyalwar left a comment

Choose a reason for hiding this comment

alban-stourbe-wmx commented Nov 21, 2024

ehsandeep left a comment

Choose a reason for hiding this comment

alban-stourbe-wmx commented Nov 21, 2024

tarunKoyalwar left a comment •

edited

Loading