edit_distance stops at ~5sec

[adamsolomou]

Looking at latest performance commit edit distance stops at problem_size=8410 and accumulated time ~5sec << time budget (30sec).

csv file:

all.log :

However when I run the same code (master branch) on AWS (same instance) it runs for much greater input sizes with no issue and outputs are correct.

My test script:

My result (same for all input sizes):

Is this a problem related to my implementation or with the test scripts used by @m8pple ?

P.S: It seems that other groups have the same issue.

[m8pple]

One possibility is that it is related to the hacking needed to try and get around
instances freezing (see #47). The logfile from the auto run suggests that a timeout
is being observed at 33.91 seconds, but this is something printed out by a memory+time
constraint script that I added in (not part of the pre-existing scripts). I'm somewhat
suspicious of the results, but don't really have a way to check them right now.

Ultimately, if you're testing your implementation on a wide range of sizes on the AWS
machine, then everything is fine. Personally I would test on a wider variety of scales
than you have (your's look a bit too uniformly spaced, and are disconcertingly rounded),
but your testing approach looks fine.