Collect both Windows and IIS data with windows_exporter

[codehtmai]

I am setting up the Grafana agent, and looking to collect both Windows system metrics and IIS metrics. I have been following the instructions for windows_exporter for both:

• Windows: https://langan.grafana.net/connections/add-new-connection/windows-exporter
• IIS: https://langan.grafana.net/connections/add-new-connection/microsoft-iis

Both configurations use windows_exporter, but it seems that they each require the job label to be set to their own value; integrations/windows_exporter and integrations/iis, respectively.

Is there a way to configure it to use both at the same time?

[dhammerberg-di]

I have this exact same question. I have them running at the same time, but there are some weird issues - like not being able to filter correctly
in the dashboards by hostname. IIS Logs cannot be filtered by name either. But I do have metrics and log data flowing to the cloud from windows and IIS.

[dhammerberg-di]

After a lot of trial and error, I got mine to work. I tested the agent on my local (Win 11) and also an Azure VM (Win Server 2016).

One other thing I noticed were the IIS Logs were not filtering by hostname in the default dashboard. I had to edit that section to add the hostname filter. In the IIS overview dashboard, go to Settings > JSON Model and add the instance filter to this line in the 'targets' block of the access logs section.

"expr": "{job=~\"$job\", site=~\"$site\"} |= ``", 

change to

"expr": "{job=~\"$job\", site=~\"$site\", instance=~\"$instance\"} |= ``", 

agent-config.yaml

server:
  log_level: info #debug,info,warn,error
integrations:
  prometheus_remote_write:
  - basic_auth:
      password: <API_KEY>
      username: <USER_ID>
    url: <YOUR_PROMETHEUS_INSTANCE>
  agent:
    enabled: true
    relabel_configs:
    - action: replace
      source_labels:
      - agent_hostname
      target_label: instance
    - action: replace
      target_label: job
      replacement: "integrations/agent-check"
    metric_relabel_configs:
    - action: keep
      regex: (prometheus_target_.*|prometheus_sd_discovered_targets|agent_build.*|agent_wal_samples_appended_total|process_start_time_seconds)
      source_labels:
      - __name__

  windows_exporter:
    enabled: true
    instance: '<MACHINE_NAME>' #shows up in IIS Overview dashboard
    enabled_collectors: "cpu,cs,logical_disk,net,os,service,system,textfile,iis"
    relabel_configs:
    - target_label: job
      replacement: 'integrations/windows_exporter'

logs:
  configs:
  - clients:
    - basic_auth:
        password: <API_KEY>
        username: <USER_ID>
      url: <YOUR_LOG_INSTANCE>

    name: integrations
    positions:
      filename: /tmp/positions.yaml
    scrape_configs:
    - job_name: integrations/windows-exporter-application
      windows_events:
        use_incoming_timestamp: true
        bookmark_path: "C:\\Program Files\\Grafana Agent\\bookmarks-app.xml"
        eventlog_name: "Application"
        labels:
          job: integrations/windows_exporter
          instance: '<MACHINE_NAME>'
      relabel_configs:
        - source_labels: ['instance']
          target_label: 'agent_hostname'
      pipeline_stages:
        - json:
            expressions:
              source: source
        - labels:
            source:
    - job_name: integrations/windows-exporter-system
      windows_events:
        use_incoming_timestamp: true
        bookmark_path: "C:\\Program Files\\Grafana Agent\\bookmarks-sys.xml"
        eventlog_name: "System"
        labels:
          job: integrations/windows_exporter
          instance: '<MACHINE_NAME>'
      relabel_configs:
        #The example in the docs uses 'computer', but if you are on a domain, the dashboard dropdown filter in Windows Logs may not work correctly
        - source_labels: ['instance'] 
          target_label: 'agent_hostname'
      pipeline_stages:
        - json:
            expressions:
              source: source
        - labels:
            source:
    - job_name: integrations/iis
      static_configs:
        - targets:
            - localhost
          labels:
            job: integrations/iis
            instance: '<IIS_INSTANCE_NAME>' #usually same as MACHINE_NAME
            site: 'Default Web Site'
            __path__: /inetpub/logs/LogFiles/W3SVC1/u_ex*.log
      pipeline_stages:
        - match:
            # Drop lines that are comments (start with #) - example in Grafana docs is not indented properly
            selector: '{job="integrations/iis"} |~ "^#"'
            action: drop
            drop_counter_reason: iis_comment_line
metrics:
  configs:
  - name: integrations
    remote_write:
    - basic_auth:
        password: <API_KEY>
        username: <USER_ID>
      url: <PROMETHEUS_INSTANCE>
    scrape_configs:
      # Add here any snippet that belongs to the `metrics.configs.scrape_configs` section.
  global:
    scrape_interval: 60s
  wal_directory: /tmp/grafana-agent-wal