How to trigger a website's favicon.ico request in chrome Headless mode

[Fly-Playgroud]

Rod Version: v0.108.2
In chrome Headless mode, the website's favicon.ico request is not triggered, resulting in a missing favicon.ico.

[ysmood]

How about use golang http lib to simulate a favicon request after the page navigation?

[Fly-Playgroud]

How about use golang http lib to simulate a favicon request after the page navigation?

@ysmood
This is not elegant enough in my opinion, and this is a solution I thought of before. I think since we are already using Headless chrome, it is better to just get the traffic of favicon.ico without requesting the favicon.ico url again.

[Fly-Playgroud]

Now I have a more elegant solution, but I also need an API to determine if chrome has headless mode on. In other words, I'd like to ask if there is an API to determine if chrome is in headless mode.

How about use golang http lib to simulate a favicon request after the page navigation?

@ysmood

[ysmood]

package main

import (
	"fmt"
	"strings"

	"github.com/go-rod/rod"
	"github.com/go-rod/rod/lib/proto"
	"github.com/go-rod/rod/lib/utils"
)

func main() {
	browser := rod.New().MustConnect()

	res, err := proto.BrowserGetBrowserCommandLine{}.Call(browser)
	utils.E(err)

	for _, v := range res.Arguments {
		if strings.Contains(v, "headless") {
			fmt.Println("headless mode")
			return
		}
	}
	fmt.Println("headful mode")
}

[kurimi1]

Hi，Could you share the elegant solution?🙂

[Fly-Playgroud]

Hi，Could you share the elegant solution?🙂

When my testing is stable, I'll propose a PR as an API for rod to incorporate

[kurimi1]

Hi，Could you share the elegant solution?🙂

When my testing is stable, I'll propose a PR as an API for rod to incorporate

Thanks, bro. I appreciate it.

[kurimi1]

There are my tow solution, but not working!

errRod = rod.Try(func() {
		ico := s.Page.Timeout(time.Second * 3).MustElement("link[rel=\"shortcut icon\"]")
		logrus.Info("ico html:", ico.MustHTML())
		logrus.Info("ico url:", ico.MustProperty("href").String())
		// ico not in resource
		bin, err := s.Page.GetResource(ico.MustProperty("href").String())
		if err != nil {
			logrus.Error("get ico error:", err)
		}
		scraped.Favicon = string(bin)

		// it not work too
		res, _ := proto.PageGetManifestIcons{}.Call(s.Page)
		scraped.Favicon = string(res.PrimaryIcon)
	})

[Fly-Playgroud]

func (b *Browser) IsHeadless() bool {
	res, err := proto.BrowserGetBrowserCommandLine{}.Call(b)
	utils.E(err)
	for _, v := range res.Arguments {
		if strings.Contains(v, "headless") {
			return true
		}
	}
	return false
}

func (p *Page) TriggerFavicon() error {
	if !p.browser.IsHeadless() {
		return errors.New("Browser is headful")
	}
	wait := p.EachEvent(func(e *proto.PageFrameNavigated) bool {
		return e.Frame.ID == p.FrameID
	})
	js := `() => {
    const faviconElement = document.querySelector("link[rel~=icon]");
    const href = (faviconElement && faviconElement.href) || "/favicon.ico";
    const faviconUrl = new URL(href,window.location).toString();
    const xhr = new XMLHttpRequest();
    xhr.open("GET",faviconUrl);
    xhr.send();
    xhr.addEventListener("readystatechange",function () {
        if (xhr.readyState === 4) {
            if (xhr.status>=200 && xhr.status <= 300) {
                return true;
            }
        }
    });
}`
	_, err := p.Evaluate(Eval(js).ByUser())
	if err != nil {
		return err
	}
	wait()
	p.unsetJSCtxID()
	return nil

}

Here's my solution: the idea is, let's do what Chrome does for it, and inject this JS instead of headless Chrome rendering requests.
@ysmood @kurimi1

[Fly-Playgroud]

@ysmood Is it possible to combine these two APIs(TriggerFavicon() IsHeadless()) into rod master?

[ysmood]

Sure, but you need to add the js to helper.js file, and use promise wrap it, so the caller knows when the ico is actually available for browser.