Implement a whitelist for deobfuscation

[nitram84]

The deobfucator of jadx usually obfuscates packages or classes like

android.support.v4.*
android.support.v7.*
android.support.v4.os.*
android.support.annotation.Px
androidx.core.os.*
androidx.annotation.Px

Reproducible with any app (obfuscated or not) which uses support libraries.

Of course those names could be renamed again, but wouldn't it be easier with a configurable whitelist for deobfucation and reasonable defaults of well known classes and packages?

My default list may not be complete and if the list gets much longer in the future it could be better to switch to a file based whitelist.

This PR is a duplicate of #765 which was rejected, because jadx can't identify libraries and marked as duplicate of #66. In my opinion the opposite is now true: With this PR jadx would be able to identify more libraries correctly when deobfuscation is enabled:

@file:DependsOn("org.xerial:sqlite-jdbc:3.43.2.1")
// LibCkecker.jadx.kts
/**
* For script usage check https://github.com/skylot/jadx/blob/master/jadx-plugins/jadx-script/README.md
*/

import org.sqlite.JDBC
import java.sql.Connection
import java.sql.ResultSet
import java.sql.Statement
import java.util.Properties
import java.util.regex.Pattern

data class Rule(val id: Int, val name: String, val label: String, val isRegex: Boolean, val regex: String?)

fun loadLibCheckerDatabase(): List<Rule> {
	val props: Properties = System.getProperties()
	val conn: Connection = JDBC.createConnection("jdbc:sqlite::resource:https://github.com/LibChecker/LibChecker-Rules/raw/master/cloud/rules/v3/rules.db", props)
	val stmt: Statement = conn.createStatement()
	val res: ResultSet = stmt.executeQuery("select _id, name, label, isRegexRule, regexName from rules_table where type = 5") // Only packages are supported for now, other resources not accessible via public jadx scripting api
	val rules: MutableList<Rule> = mutableListOf()
	while (res.next()) {
		val rule: Rule = Rule(res.getInt(1), res.getString(2), res.getString(3), res.getBoolean(4), res.getString(5))
		rules.add(rule)
	}
	res.close()
	stmt.close()
	conn.close()
	log.debug { "libChecker database loaded" }
	return rules
}

fun getPackages(): Set<String> {
	val packages: MutableSet<String> = mutableSetOf()
	jadx.classes.forEach {
		var idx = it.fullName.lastIndexOf('.')
		var packageName = it.fullName.substring(0, idx)
		packages.add(packageName)
		idx = packageName.lastIndexOf('.')
		while (idx > 0) {
			packageName = packageName.substring(0, idx)
			if (packages.contains(packageName)) {
				break
			}
			packages.add(packageName)
			idx = packageName.lastIndexOf('.')
		}
	}
	return packages
}

val jadx = getJadxInstance()

jadx.afterLoad {

	val packages: Set<String> = getPackages()
	val rules: List<Rule> = loadLibCheckerDatabase()
	rules.forEach {
		val currentRule: Rule = it
		var rulePattern: Pattern? = null
		if (currentRule.isRegex) {
			rulePattern = Pattern.compile(currentRule.regex)
		}
		packages.forEach {
			if ((currentRule.isRegex && rulePattern != null && rulePattern.matcher(it).matches()) || it.equals(currentRule.name)) {
				log.info { "Lib " + currentRule.label + " found, rule id: " + currentRule.id }
			}
		}
	}
}

And library detection might be a first step towards automatic dependency detection as a precondition of #66.

Off topic question: Do you also accept also sample scripts as PR?

[skylot]

@nitram84 thanks for PR!
Some thoughts according to implementation:
Now rename condition for deobfuscator exposed through jadx API in (JadxArgs) and should allow modify or completely replace default implementation (DeobfCondition), so it is not good idea to modify DeobfuscatorVisitor.
Here are options to how handle this additional conditions:

1. Put new checks in DeobfCondition as it already contains checks for length conditions and TLD names whitelist (PR chore: do not deobfuscate known top level domains #1369 by @jpstotz)
2. Create a class/methods to combine several IRenameCondition and provide 'default conditions' list by splitting DeobfCondition into small parts. This should make these conditions modular and simplify adjustments though jadx API.

I like the second option, but it requires a bit of refactoring.
So, I will prepare these changes (can take several days) and after that merge this PR as is.

[skylot]

Off topic question: Do you also accept also sample scripts as PR?

Sure PRs are welcome

And LibChecker rules looks good, I definitly needs to check them, thanks 🙂

[skylot]

Ok, I merged PR and my refactoring commit.
I hope I didn't break anything, but it will be nice if someone checks the latest unstable build.

@nitram84 along with small fixes to your PR, I changed separator symbol to space for deobfuscation whitelist option, this way it is closer to similar options: excluded packages (now it is gui only) and slightly easier to read.
And I think we need to make a decent editor for such lists in jadx preferences 🙂

[nitram84]

You were right: having modular deobfucation conditions is/was definitively the better choice. Sorry for the work you had due to my PR. I'll soon fix the translations with the new separator symbol and propose more whitelist entries.