Remove rex_pcre dependency from core

[subnetmarco]

Use Lua patterns instead.

[thibaultcha]

👍

[thibaultcha]

I am actually not sure if this is going to be possible. Places where we need a regex, checked if a test case with a working pattern is provided:

• Validate a public_dns
• Validate a uuid for entities in the Cassandra DAO
• catching invalid target_url Catching invalid target urls #149 (needed soon)
• Any future other plugin that will have to rely on regexes
• Some places in the specs to extract strings of JSON

uuid

local x = "%x"
local pattern = "^"..table.concat({ x:rep(8), x:rep(4), x:rep(4), x:rep(4), x:rep(12) }, '%-').."$"

local guid = "3F2504E0-4F89-41D3-9A0C-0305E82C3301"
assert(guid:match(pattern))
guid = "fba88588-ffd0-4769-c782-7c537c92078d"
assert(guid:match(pattern))
guid = "fba88588-ffd0-4769-c782-7c537c92078dblah"
assert(not guid:match(pattern))
guid = "blahfba88588-ffd0-4769-c782-7c537c92078d"
assert(not guid:match(pattern))
guid = "3F2504E0-4F89-41D3-9A0C0305E82C3301"
assert(not guid:match(pattern))
guid = "304E0-4F89-41D3-9A0C0305E8^3301"
assert(not guid:match(pattern))
guid = "fba88588-féd0-4769-c782-7c537c92078d"
assert(not guid:match(pattern))
guid = "fba88588-(§è!)-4769-c782-7c537c92078d"
assert(not guid:match(pattern))
guid = "3F2504E0-4F89-41D3-9A0§-03(5E82C3301"
assert(not guid:match(pattern))
guid = "3F2504E0-4F89-41D3-9A0C0305é82C3301"
assert(not guid:match(pattern))
guid = "hello3F2504E0-4F89-41D3-9A0C0305é82C3301world"
assert(not guid:match(pattern))
guid = "abcd"
assert(not guid:match(pattern))
guid = ""
assert(not guid:match(pattern))

public_dns

local function is_valid_public_dns(str)
  -- check the format
  local pattern1 = "([A-Za-z0-9%-]*)%.*([A-Za-z0-9%-]*)%.([A-Za-z0-9][A-Za-z0-9]+)$"
  -- check all characters are alphanumeric
  local pattern2 = "^[%w%.%-]*$"
  -- both checks need to be true to be a valid public_dns
  return str:match(pattern1) and str:match(pattern2)
end

assert(is_valid_public_dns("mockbin.com"))
assert(is_valid_public_dns("foo-bar.mockbin.com"))
assert(is_valid_public_dns("mockbin.com"))
assert(is_valid_public_dns("mockbin.ninja"))
assert(is_valid_public_dns("api.mockbin.com"))
assert(not is_valid_public_dns("apiè!!.mockbin.com"))
assert(not is_valid_public_dns(""))
assert(not is_valid_public_dns("mockbincom"))
assert(not is_valid_public_dns( "mockbincom!!"))
assert(not is_valid_public_dns("mockbin.com!!"))
assert(not is_valid_public_dns("mockbin.com."))

specs

In the specs I was able to use a simple pattern to extract JSON string from a log line:

local json_str = line:match("(%{.*%})")

[subnetmarco]

If if it's not feasible to remove it from everywhere, we shouldn't force it. Just reduce regex usage to the places where it's absolutely necessary would be okay.

[thibaultcha]

It might be possible, we might also use a slightly more enhanced way of using the patterns (like a library: http://www.inf.puc-rio.br/~roberto/lpeg/lpeg.html)

[thibaultcha]

Just updated the issue with a working version of a public_dns check. Please do test it with other values you could think of.

[thibaultcha]

Updated for the specs part too

[thibaultcha]

Merged #167 which does some improvements and use patterns in most places instead of regexes. PCRE is not removed and the regex field is still available in schema definitions.