[BUG] Internal Execution Errors in Apex Classes while running Graph Scan

[AnkitBhatnagar123]

Describe the bug
We are getting error in different apex classes :
Graph Engine identified your source and sink, but you must manually verify that you have a sanitizer in this path. Then, add an engine directive to skip the path. Next, create a Github issue for the Code Analyzer team that includes the error and stack trace. After we fix this issue, check the Code Analyzer release notes for more info. Error and stacktrace: UnexpectedException: null: com.salesforce.graph.vertex.SFVertexFactory.load(SFVertexFactory.java:245);com.salesforce.graph.vertex.SFVertexFactory.access$000(SFVertexFactory.java:56);com.salesforce.graph.vertex.SFVertexFactory$1.apply(SFVertexFactory.java:186);com.salesforce.graph.vertex.SFVertexFactory$1.apply(SFVertexFactory.java:183);com.salesforce.graph.cache.AbstractVertexCacheImpl.get(AbstractVertexCacheImpl.java:101);com.salesforce.graph.vertex.SFVertexFactory.loadVertices(SFVertexFactory.java:180)

To Reproduce
Getting these errors on the first line of method Definitions. Eg: Getting the error on public static void revokeUserAccess(String userId) in code :

public static void revokeUserAccess(String userId) {
String access_token = NDAuthController.getAccessToken(userId);

    //revoke api access token
    HttpResponse res = NDAuthUtility.basicAuthCallout(access_token, 'POST', '/v1/oauth/revoke', 'token='+access_token, 'application/x-www-form-urlencoded');
    
    // delete saved user auth info
    delete NDAuthController.getOAuthInfo(userId);
}

"Urgency": We have to resolve this issues for our package's security review.

[jfeingold35]

@AnkitBhatnagar123 , are NetDocumentsUtility and NetDocumentsAuthController custom classes?
Also, could you please delete your ~/.sfdx-scanner/sfge.log, re-run the command, and then post the resulting logs here? That would give us more to go on.

[AnkitBhatnagar123]

Deleted the previous logs. This is the new log. I have also updated the issue description with the new class names
CodeAnalyzerDFA.csv

[jfeingold35]

@AnkitBhatnagar123 , that's actually the results file. I need the logs, which will live in ~/.sfdx-scanner/sfge.log.

[AnkitBhatnagar123]

sfge.log

[jfeingold35]

@AnkitBhatnagar123 , please provide the following information:

1. What's the exact command you're running?
2. What's happening at line 50 of NDUtility.cls? That's one of several locations that crops up pretty consistently before an error, so maybe there's something notable happening there.
   Thanks!

[vc4u]

Josh- I'll wait for my colleague to respond to your Ques #1. Though for Ques. #2, I can provide some insights.

Under that class at line 50- we're returning a ternary operator based statement that either returns a New instance of Custom metadata (when called under Apex Testing context) OR return Custom Metadata record via SOQL.

Format of that line 50 statement looks like this:

public static ND_Config__mdt getNDConfig() {
**LINE 50 --->**	return Test.isRunningTest() ? new ND_Config__mdt(
		DeveloperName = 'test',
		Label = 'Test',
		Location__c = getNDAppConfigsList()[0].Id,
		Location__r = getNDAppConfigsList()[0]
	) : [SELECT id, Location__c FROM ND_Config__mdt WHERE DeveloperName = 'My_Config'];
}
public static ND_AppConfigs__mdt[] getNDAppConfigsList() {
	return Test.isRunningTest() ? new List<ND_AppConfigs__mdt> { new ND_AppConfigs__mdt(
		DeveloperName = 'test',
		Label = 'Test'
	)} : [SELECT  Id, MasterLabel, QualifiedApiName, DeveloperName FROM ND_AppConfigs__mdt ORDER BY MasterLabel, DeveloperName];
}

[jfeingold35]

@AnkitBhatnagar123 , @vc4u , I've done some local testing, and I believe I've isolated the problem to the Location__r = getNDAppConfigsList()[0].
The following caused the same error you're seeing:

public class MyClass {
    @AuraEnabled
    public static CustomObj__c entrypointMethod() {
        CustomObj__c obj = new CustomObj__c(Name = 'asdf', Child__r = getChildArray()[0]);
        insert obj;
        return obj;
    }

    public static ChildObj__c[] getChildArray() {
        return [SELECT Name FROM ChildObj__c];
    }
}

After adjusting it to the following, I stopped seeing the error:

...
        CustomChild__c child = getChildArray()[0];
        CustomObj__c obj = new CustomObj__c(Name = 'asdf', Child__r = child);
...

Please let me know if making a similar adjustment in your code resolves the error.
Examining the logs indicates that the overwhelming majority of the exceptions are happening at lines 9, 25, and 50 of NDUtility.cls. If fixing those files doesn't resolve all the problems, I can examine for others.
Thanks for your patience.

[AnkitBhatnagar123]

Hi @jfeingold35
We made the changes suggested by you. Then our error count reduced. We also fixed some timout errors by adding params to our command. Now, we're getting :Path evaluation timed out after 900000 ms even after adding timout params.
sfge.log
Command that we're using now is : sfdx scanner:run:dfa --format=csv --outfile=CodeAnalyzerDFA.csv --target="./" --projectdir="./" --category="Security" --pathexplimit -1 --rule-thread-timeout=300000

[jfeingold35]

Recommendations:

• You probably don't need to use --pathexplimit, and definitely shouldn't set it to 1 as this will cause errors basically immediately.
• Your value for --rule-thread-timeout appears to be 300,000, which is shorter than the default value of 900,000. Try using a larger value (e.g., 1,800,000, which would be 30 minutes).

[git2gus]

This issue has been linked to a new work item: W-14080002

[jfeingold35]

@AnkitBhatnagar123 , @vc4u , let me know if the recommendations I provided work.
In the meantime, the issue with Location__r is a genuine bug, so I'm adding it to our backlog to fix, along with the note that it has a workaround.

[AnkitBhatnagar123]

After Using : sfdx scanner:run:dfa --format=csv --outfile=CodeAnalyzerDFA.csv --target="./" --projectdir="./" --category="Security" --rule-thread-timeout=1800000

"Graph Engine reached the path expansion upper limit (743). The analysis preemptively stopped running on this path to prevent an OutOfMemory error. Rerun Graph Engine and target this entry method with a larger heap space." comes back.

[jfeingold35]

@AnkitBhatnagar123 , our documentation has some information about this.
Basically, that error means your memory isn't great enough to hold all the paths being expanded. This limit is dynamically calculated based on memory, and in your case it seems to be 743.
You can either:

1. Use method-level targeting to scan individual entrypoints, thereby decreasing the number of paths being expanded at once.
2. Use --sfgejvmargs to set a larger heap size, thereby allowing more paths to fit in memory.
3. Use --pathexplimit to increase the limit to some number greater than 743.

[AnkitBhatnagar123]

Hi @jfeingold35
We made some changes in the code and also ran the cmd as per your suggestions :
sfdx scanner:run:dfa --format=csv --outfile=CodeAnalyzerDFA.csv --target="./force-app" --projectdir="./force-app" --category="Security" --rule-thread-timeout=1800000 --sfgejvmargs "-Xmx60g"
This is the max heap size limit we are able to go with.
Still we are getting a couple of Internal Execution errors.

sfge (1).log.zip