How to get all calls to a FunctionValue in python

[BaiChienKao]

Hello, I am looking for a correct way to get a caller.
I am using the class FunctionValue to enumerate all the functions in a file as below:

from FunctionValue val
where  val.getScope().getLocation().getFile().toString().matches("%WebServer.py")
select val

and it gives me four functions:
1 Function helloHandler.do_GET
2 Function helloHandler.do_POST
3 Function helloHandler.respond
4 Function main

However, when I would like to get if there is a call calls those functions by "getACall()" as below:

from FunctionValue val
where  val.getScope().getLocation().getFile().toString().matches("%WebServer.py")
select val, val.getACall()

it only return one result as below.
1 Function main | ControlFlowNode for main()

I am pretty sure there are calls in function do_GET and do_POST call the FunvtionValue "Function helloHandler.respond".
But not present in the result.
I was wondering how to get the calls correctly.

My python code sample is like below:

from http.server import HTTPServer,BaseHTTPRequestHandler

class helloHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        if...
            self.respond(response)
        else:
            ...
    def do_POST(self):
        ...
            self.respond(VulnList)

    def respond(self, response, status=200):
		...
		
def main():
		...
		
if __name__ == '__main__':
    main()

The complete code can be found in https://github.com/BaiChienKao/SbomTestP1_01262023/blob/main/WebServer.py

Thanks for the help.

[tausbn]

I think the reason we're not resolving the calls to do_GET and do_POST in this case is that these are handled using a form of introspection internally in http.server. (See here: https://github.com/python/cpython/blob/3.11/Lib/http/server.py#L413-L420)
Because of this indirection, going via getattr, we are unable to tell (in terms of the call graph) that GET requests are handled by do_GET.

If what you want is to simply locate these request handlers, then I would suggest using API graphs instead. Something like

moduleImport("http")
  .getMember('server')
  .getMember('BaseHTTPRequestHandler')
  .getASubclass*()
  .getMember('do_GET')

(untested) should be enough to locate the do_GET request handler. To match all request handlers, you can bind the name of the request with an existential variable, and check that it has the prefix do_. This would look something like this:

exists(string method_name | 
  moduleImport("http")
    .getMember('server')
    .getMember('BaseHTTPRequestHandler')
    .getASubclass*()
    .getMember(method_name) and 
  method_name.matches("do\_%")
)

At any rate, I would not recommend using FunctionValue (or any other parts of the ...Value API), as these are in the process of being deprecated.

[BaiChienKao]

Hello tausbn, Thank you for the reply.
I did not want to locate the do_GET/POST handlers.
I already enumerated those four functions in this case and I want to know if there is a call calls them.
for example:
I already know there is a FunctionValue: helloHandler.respond.
Then how can I find the call: "self.respond(response)" in the code (which is in do_GET as well as in do_POST).
FunctionValue.getACall() didn't give the two CallNodes to me, so I was wondering how to query it.

By the way, if you don't recommend the ...Value API then what can I use for the replacement?
Thank you.

[tausbn]

Okay, so first of all, as I mentioned we're unlikely to be able to resolve the calls to do_GET and do_POST, as they go via getattr.

The remaining calls to respond and main should be possible to resolve using the new call-graph (which was merged last week). Unfortunately, the public-facing API for the call-graph has not been stabilised yet, so there's no easy way to access this data. Until we get the API settled, you can access it as follows:

private import semmle.python.dataflow.new.internal.DataFlowDispatch

predicate resolvedCall(CallNode call, Function callable) {
  exists(DataFlowCallable dfCallable, DataFlowCall dfCall |
    dfCallable.getScope() = callable and
    dfCall.getNode() = call and
    dfCallable = viableCallable(dfCall)
  )
}

Finally, may I ask what it is you're actually trying to do? For most queries, you shouldn't have to directly query the call-graph, but rather use things like data-flow and taint tracking (which both use the improved call-graph that was recently merged).

[BaiChienKao]

Hi tausbn,
I am trying to use the script you provided. Somehow I can not import semmle.python.dataflow.new.internal.DataFlowDispatch (no module) as well as cannot resolve the the classes DataFlowCallable and DataFlowCall, and the predicate viableCallable.
Also, I cannot find the documentation and information of those module, classes and predicate via the CodeQL library for Python https://codeql.github.com/codeql-standard-libraries/python/. (I saw viableCallable in Java but didn't see in python )
I will try to upgrade the version I am using, at the same time, I was wondering if you could please tell me where can I find the newest documentation that describe the detail information of the items above.

The reason I didn't use data-flow or taint tracking because based on my understanding it only tell me if there is a flow from source to sink, but not tell me what is the path and how many nodes/edges between them.
Also there are scenarios that no data/arguments passed via an invocation, then based on my understanding the data-flow tracking cannot follow this call connection.

The thing that I trying to do is to create a call-graph for an application so that I can measure which node or function is most impactful. And, the call path is important information to do this.

If data-flow or taint tracking provides this kind of information please let me know, since I cannot use data-flow tracking to extract the call-path information.

Thank you so much.