alphametics: Update canonical data. Add a test case with many addends

[nikamirrr]

A test case update suggested to complement this PR:
exercism/python#1119

[rpottsoh]

Exercise description.

[nikamirrr]

Please, let me know if anything else needs to be fixed.
Not sure if there is any conflict with a description.

[rpottsoh]

Not sure if there is any conflict with a description

It is nice to cite the exercise decription as an aid to those who may not be familiar with the exercise; it makes it just a little bit easier for someone to go reference the description quickly.

[Insti]

I'm not sure adding this test is a good idea.
I'll have to have a think about it to come up with the reasoning why.

I suspect it is because optimising a solution to solve a huge problem like this is a different problem to "simply" solving the general alphametics problem.

But having a huge test available to those who want to try it is a good thing, so maybe it does belong but with a note that it is optional?

But most test generators just take everything from canonical-data.json and stick it in tests.

[nikamirrr]

In my opinion it is not very different from a 10-letter problem.
But having many repeating letters is stimulating for a more efficient way of handling them.
Other cases do not necessarily make this need obvious.
But I am OK with either decision.

[rpottsoh]

so maybe it does belong but with a note that it is optional?

If it were to be considered as optional, perhaps it would be better suited to be included in the description instead of the canonical data?

see ISBN-Verifier description

If it is decided to update the description, then a new PR should be created for the purpose of that task.

[Insti]

Reading through the comments on exercism/python#1119 makes me more inclined to want to add it.
Especially if the complexity of the solution is approximately the same as the existing 10 letter solution.

[nikamirrr]

I am OK with either decision, though I would have preferred making a contribution.
I mean, the alphametics is quite late in the python track. Permutations and string replacement may be too simple to implement and move on.
One does not even need to code a permutation, thanks to the itertools.

[Insti]

I don't want to make a call either way right now, I'd like to hear the opinion of some other people, so it might be worthwhile leaving this PR open for a week or so to give more time for people to give feedback.

The PR itself looks fine, but needs to bump the (middle) version number.

[ErikSchierboom]

Although I'm fine with the added case, it does appear to make an already very challenging exercise even more challenging. I understand that this extra cases prevents brute-forcing a solution, but is that so bad? Is the goal of the exercise to have the user come up with an efficient solution, or just to come up with a solution? IMHO I think we should not be forcing people to come up with an efficient solution but also allow the brute force option, as otherwise I fear that the number of people being able to finish this exercise will be even lower than what it already is.

Note that this may not be an issue with Exercism v2, as you could define the alphametics exercise as just being an optional exercise without any unlocks.

[nikamirrr]

Brute force runs all the cases (including the new big one) for 10s on my computer, vs 0.5s on the optimized case.
This makes me think that it will be very appropriate for push people to group the letters )
Otherwise, what is the challenge? Split the string and make a permutation call.

from itertools import permutations
from collections import defaultdict


def solve(an):
    lets = set(an) - set("=+ ")
    expr = defaultdict(int)
    nz = set()
    for s, sgn in zip(an.upper().split("=="), (1, -1)):
        for w in map(str.strip, s.split("+")):
            nz.add(w[0])
            for p, c in enumerate(reversed(w)):
                expr[c] += sgn * (10 ** p)
    lets = tuple(nz) + tuple(lets - nz)
    expr = tuple([expr[k] for k in lets])
    for digs in filter(lambda x: all(x[:len(nz)]),
                       permutations(range(10), len(lets))):
        if sum([d * m for d, m in zip(digs, expr)]) == 0:
            return dict(zip(lets, digs))
    return dict()

[petertseng]

I note that in https://github.com/exercism/docs/blob/master/you-can-help/improve-exercise-metadata.md#extracting-canonical-test-data, there is a sentence that says "Performance tests should not be included in the canonical test data."

This comment must not be taken as an affirmation or rejection of the statement "The newly added test is a performance test".

If it helps, as far as I can tell, the origin of this sentence is #230 (comment).

[petertseng]

If this PR is to be merged, since "update canonical data" could have many meanings, my request is that the commit message and PR title be edited to state in what way the canonical data is being updated (add a test case with many addends)

[nikamirrr]

Not sure what else I need to do being assigned to the PR, please, advise

[Stargator]

@nikamirrr Since you were working on this and it's your branch, I figured I'd assign it to you. I haven't reviewed it yet, so not sure if there's anything else that needs to be done.

[nikamirrr]

I don't see there is much I can do beside talking to people and closing the request.
I already provided all arguments. Someone in charge needs to make a decision.

[Stargator]

The PR itself looks fine, but needs to bump the (middle) version number.

Agreed.

Note that this may not be an issue with Exercism v2, as you could define the alphametics exercise as just being an optional exercise without any unlocks.

It's up to the tracks to determine that and the tracks can make this a core exercise or an exercise that is unlocked at some point and essentially not blocking progress of another exercise.

So I don't think we should worry about how tracks use it, but recognize even before v2, they have a number of ways to handle it.

[Stargator]

Since it seems this test case can be addressed via brute force, I'm fine with it. But I recognize that Exercism v2 may provide more flexibility to the user and not force them to do test cases when they may have already implemented workable code.

• Bump the middle version number

• Reword the commit message to contain the same message as the PR title: "alphametics: Update canonical data. Add a test case with many addends"

[rpottsoh]

@nikamirrr as suggested by @Stargator I think you only need to bump the version:

 "version": "1.1.0",

Then someone will likely merge the PR.

[nikamirrr]

OK. done

[Stargator]

Per the Pull Request Guidelines:

Put the name of the exercise in the subject line of the commit. E.g. hamming: Add test case for strands of unequal length

So I still recommend doing a rebase that rewords the commit message to fit this.

[nikamirrr]

Done, I guess

[Stargator]

This didn't need to be bumped.

[petertseng]

I don't understand the "41 addends" part of this description. I see 199 addends. Please clarify what "41 addends" means

[nikamirrr]

Sorry, made a new PR
#1045

[petertseng]

This is an interesting case. I hope all don't mind if I ask to brainstorm ways to do it fast. One thing to notice is that there are nine letters occupying the leftmost occupied column of some addend, so only one letter can be 0. So that can be assigned right away.

However, this case cripples columnwise solutions since they no longer have an advantage over brute force solutions. All nine remaining letters appear in rightmost column so columnwise solutions will evaluate 9! possibilities just like brute force solutions; so they might as well evaluate the entire problem instead of columnwise.

Unless we iterate from leftmost column first, since there are two letters in leftmost column, four in second-leftmost, and three in third-leftmost and then we are done. Sure, then we have to deal with potential values of carry-in, but it can be determined that the max carry-in is 7, 10, 15 for those three columns respectively, so we evaluate:

• 9 * 8 * 8 possibilities, of which only 35 are possible
• 35 * 7 * 6 * 5 * 4 * 11 possibilities, of which only 4488 are possible
• 4488 * 3 * 2 * 1 * 16 possibilities, of which only 3568 are possible and only 1 is possible if considering all further columns to the right

And this is faster than considering all 9! possibilities without any intermediate filtering.

So the fastest I was able to get this was 1.8 seconds. Can someone help make it faster, I couldn't figure out how to do 0.5 seconds like #1024 (comment) has it.

[nikamirrr]

I have another PR for the fast example solution
exercism/python#1119

[petertseng]

OK, then looks like the remaining speedup there is gained by immediately terminating when finding the first solution. As opposed to continuing to find all possible solutions (verify the problem for validity to make sure it has only one possible solution not multiple). OK, guess I have to keep finding all possible solutions to verify, but solver can shortcut.

Thank you for showing!

[nikamirrr]

This big problem has almost all characters in the lowest digit rank, so the initial permutation does not get reduced too much. But there are other problems where the digits are more spread through the ranks, they gain a lot of performance. 0.5s is the total time for all alphametics tests.

Another thought I had was to move the digits around the ranks.
For example, if a letter is spread among ranks 1 and 10 with multipliers of 4 and 3 for instance, move it all to rank 1 with a multiplier 44. That did not give much performance improvement.

Another idea was to spread the letters through ranks,
For example, a letter is at rand 1 with a multiplier 40, move it to rank 10 with a multiplier 4 and reduce the initial permutations.
But seems like a recursive call to check the next rank for one digit only is more expensive than to simply include it to permutations.

There are many ways to optimize this further, but I felt it is good enough for now.
Especially since this problem is almost brute-force due to many digits in the lowest ran,