Code from "third_party" folder missing from results

[jlisee]

In our private repository on cs.github.com I can browse to files in our the third_party folder in the root of our repository but searches do not return any content there. I can even visit a file and see a detected symbol then click "search for .... in this repo" and it finds it everywhere except the file I was just in.

Is there some global filtering control that is taking effect here that can be adjusted?

[colinwm]

Hey @jlisee, we currently don't index all code, see https://cs.github.com/about/faq#indexed-content. Code in the third_party directory is getting detected as vendored, so it isn't indexed.

We're working on removing that restriction, but for now it's not searchable. Sorry about that!

[LiteBrite82]

I couldn't find the equivalent, I think I came close though: About GitHub Code Search: Limitations. Or maybe Finding files on GitHub

I also found a blog with an in depth explanation: The technology behind GitHub’s new code search. It may be a bit TL;DR though...

If you really want to get in depth, check out "Lessons from building GitHub code search"

[traversaro]

Thanks @LiteBrite82 ! I may be wrong, but I could not find any reference to third_party folders being excluded from search results in those links. In the first link it is mentioned that "Vendored and generated code is excluded", but they do not specify how "vendored" code is identified. I guess third_party folder are classified as "vendored", but it would be great to have a clear documentation on what is considerd "vendored" code and what not.

[LiteBrite82]

I fully agree, Im still searching.

[usmonster]

Possibly related & possibly more serious: #130254.

[jlisee]

Thanks for the quick response, I read over that page but not closely enough.

Just so you can understand the use case our third_party folder has some external code, but also a lot of build and other configuration related code we wrote. If I see a reference in the main part of the tree to external_repo_foo it's great to be able to search and find that external_repo_foo is defined in the file we wrote third_repo/external_repo_foo/BUILD.

Eventually our third_party folder we consistent only of files like this, so at least for us the ability to simply force include a few paths no matter what Enry or a successor system thinks would be great.

[ghost]

+1 to indexing all code. I had to do some deep digging to find some error messages today; if we index all the vendor code it'll save me a lot of time (!).