Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Oct 24, 2025

Fix Blocked Domains Parsing and Display

This PR addresses the issue where the firewall log parser was showing invalid "domains" like "DNS", "Local", "Swap", "WARNING:", etc. in the blocked domains table.

Latest Changes

Recompiled workflows with updated firewall log parsing logic.

Code Review Fixes

  • ✅ Improved domain regex to prevent consecutive dots, leading/trailing dots, and invalid label patterns
  • ✅ Used arrow function syntax with parentheses for consistency
  • ✅ Simplified loop using reduce for a more functional approach

Original Implementation

Strict field validation in parseFirewallLogLine:

  • Timestamp must be numeric with optional decimal: /^\d+(\.\d+)?$/
  • IP:port fields must match /^[\d.]+:\d+$/ or -
  • Domain must match strict pattern preventing invalid formats or -
  • Status must be numeric or -
  • Decision must contain : or -

Filter placeholder domains in summary:

const validDeniedDomains = deniedDomains.filter((domain) => domain !== "-");
const validDeniedRequests = validDeniedDomains.reduce(
  (sum, domain) => sum + (requestsByDomain.get(domain)?.denied || 0),
  0
);

UI enhancement:

  • Wrap blocked domains table in <details> tag for collapsibility
  • Change heading to <summary> with "click to expand" prompt

Domain Validation Examples

Valid domains:

  • api.github.com:443
  • sub.example.com:443
  • my-domain.com:443

Rejected invalid domains:

  • .example.com:443 (leading dot)
  • example.com.:443 (trailing dot)
  • example..com:443 (consecutive dots)
  • -example.com:443 (leading hyphen)
  • example.com-:443 (trailing hyphen in label)

Testing

  • ✅ All 18 firewall log parsing tests passing
  • ✅ All 595 JavaScript tests passing
  • ✅ Validated strict domain regex with edge cases
  • ✅ All workflows recompiled successfully
Original prompt

It looks like the filter for blocked domains are not precise enough. Looks like the run output https://github.com/githubnext/gh-aw/actions/runs/18793451386 which outputs

🚫 Blocked Domains

Domain Blocked Requests
- 3
Accepting 1
DNS 2
Local 1
Swap 1
WARNING: 1
Waiting 1
example.com:443 1
pinger

and this tells me that the only valid domain that is blocked is example.com. Could you please improve the parsing logic?

Also move it a tthe back anda maybe add a html details section since it's quite advanced.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits October 24, 2025 22:31
Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>
Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>
Copilot AI changed the title [WIP] Improve parsing logic for blocked domains Fix firewall log parser rejecting invalid domains from Squid error messages Oct 24, 2025
Copilot AI requested a review from Mossaka October 24, 2025 22:37
@Mossaka Mossaka marked this pull request as ready for review October 24, 2025 22:42
Copilot AI review requested due to automatic review settings October 24, 2025 22:42
@github-actions
Copy link
Contributor

Agentic Changeset Generator triggered by this pull request.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes a bug in the firewall log parser that was incorrectly accepting malformed Squid error messages as valid log entries, resulting in noise like "DNS", "WARNING:", and "Accepting" appearing as blocked domains. The fix adds strict field validation to reject invalid entries and filters placeholder domains from the summary output.

Key changes:

  • Added regex validation for timestamp, IP:port, domain, status, and decision fields in parseFirewallLogLine
  • Filtered out placeholder "-" domains from blocked domain statistics
  • Wrapped blocked domains table in collapsible <details> element for better UX

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
pkg/workflow/js/parse_firewall_logs.cjs Added field format validation and placeholder domain filtering
pkg/workflow/js/parse_firewall_logs.test.cjs Added test coverage for validation rules and placeholder filtering
.github/workflows/smoke-copilot.firewall.lock.yml Applied same validation and filtering changes to workflow
.github/workflows/firewall.lock.yml Applied same validation and filtering changes to workflow
.github/workflows/dev.firewall.lock.yml Applied same validation and filtering changes to workflow
.github/workflows/changeset-generator.firewall.lock.yml Applied same validation and filtering changes to workflow

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

let summary = "# 🔥 Firewall Blocked Requests\n\n";

// Filter out invalid domains (placeholder "-" values)
const validDeniedDomains = deniedDomains.filter(domain => domain !== "-");
Copy link

Copilot AI Oct 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use arrow function syntax consistently with parentheses around the parameter for single-parameter arrow functions to match project style.

Copilot generated this review using guidance from repository custom instructions.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e34d69b - added parentheses around the parameter for consistency.

Comment on lines 208 to 214
let validDeniedRequests = 0;
for (const domain of validDeniedDomains) {
const stats = requestsByDomain.get(domain);
if (stats) {
validDeniedRequests += stats.denied;
}
}
Copy link

Copilot AI Oct 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] This loop can be simplified using reduce for a more functional approach: const validDeniedRequests = validDeniedDomains.reduce((sum, domain) => sum + (requestsByDomain.get(domain)?.denied || 0), 0);

Suggested change
let validDeniedRequests = 0;
for (const domain of validDeniedDomains) {
const stats = requestsByDomain.get(domain);
if (stats) {
validDeniedRequests += stats.denied;
}
}
const validDeniedRequests = validDeniedDomains.reduce(
(sum, domain) => sum + (requestsByDomain.get(domain)?.denied || 0),
0
);

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented in e34d69b - simplified to use reduce with optional chaining for a more functional approach.


// Validate domain format (should be domain:port or "-")
const domain = fields[2];
if (domain !== "-" && !/^[a-zA-Z0-9.-]+:\d+$/.test(domain)) {
Copy link

Copilot AI Oct 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The domain regex allows consecutive dots (e.g., "example..com:443") and leading/trailing dots (e.g., ".example.com:443"), which are invalid domain formats. Consider using a more strict pattern like /^[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?)*:\d+$/ to enforce valid domain structure.

Suggested change
if (domain !== "-" && !/^[a-zA-Z0-9.-]+:\d+$/.test(domain)) {
if (
domain !== "-" &&
!/^[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?)*:\d+$/.test(domain)
) {

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e34d69b - implemented the stricter domain validation pattern that prevents consecutive dots, leading/trailing dots, and invalid label structures. Validated with comprehensive edge case testing.

@Mossaka
Copy link
Collaborator

Mossaka commented Oct 24, 2025

@copilot rebase and regenerate all the yamls using make recompile and commit

@Mossaka
Copy link
Collaborator

Mossaka commented Oct 24, 2025

@copilot apply changes based on the comments in this thread

Copilot AI and others added 2 commits October 24, 2025 23:39
…al approach

Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>
Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Oct 24, 2025

@copilot rebase and regenerate all the yamls using make recompile and commit

Completed in e34d69b - rebased with main and ran make recompile to regenerate all workflow YAMLs.

Copy link
Contributor Author

Copilot AI commented Oct 24, 2025

@copilot apply changes based on the comments in this thread

All changes from that review thread have already been applied in commits 58b2d9b and e34d69b:

  • Arrow function parentheses added
  • Loop simplified using reduce with optional chaining
  • Stricter domain regex implemented
  • Workflows recompiled

The PR is ready for review.

@pelikhan pelikhan merged commit 60e85eb into main Oct 25, 2025
15 checks passed
@pelikhan pelikhan deleted the copilot/improve-blocked-domains-parsing branch October 25, 2025 00:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants