Insensitivty to recontactPeriod Variable

[Eezzeldin]

Problem : Changing the variable of recontactPeriod In the range of 0 to 10000000000 does not show any differences on ADR-AFPR or VDR-AFPR curves at all.

Data Input: output of the unit "Score Model" in Azure ML Studio Step 4: Model Training and Evaluation [1].

Code : ExecuteRScript in [1].

Image1: ADR with the x-axis between 0 and 10.
Image2: ADR with the x-axis between 0 and 200.
Image3: VDR with the x-axis between 0 and 10.
Image4: VDR with the x-axis between 0 and 100.
Image5: VDR with the x-axis between 0 and 200.

Result: The very same curves appear on all images.

Request: Please fix this bug to make ADR,VDR,AFPR curves vary according to different recontactPeriods.Thank you.

ExecuteRScript.zip

[1] https://gallery.azure.ai/Experiment/Online-Fraud-Detection-Step-4-of-5-train-and-evaluate-model-2

[Eezzeldin]

@jeroenterheerdt Hello , May I please ask for a confirmation that this issue is being acknowledged and addressed?

[jeroenterheerdt]

@Eezzeldin - again - this is open source software - feel free to make changes so you do not have to wait for us to get around to it.

[Eezzeldin]

@jeroenterheerdt The sole purpose of this code base is to re-enforce the core difference between transactional and account fraud metrics which is summarized around recontactpreiod*. Minimum maintainability is required by Microsoft employees when it comes to the core functionality of their code base, It is not the purpose of open source. Your "again" here does not count, last time, I was wondering for an additional feature , which is the reason I accepted your suggestion to do it. I unfortunately would have to spend very long time to reverse engineer your code base and understand the source of this critical bug.

*A major difference between account-level metrics and transaction-level metrics is that, typically an account confirmed as a false positive (that is, fraudulent activity was predicted where it did not exist) will not be contacted again during a short period of time, to avoid inconveniencing the customer. [1]
*The metric used for assessing accuracy (performance) depends on how the original cases are processed. If each case is processed on a transaction by transaction basis, you can use a standard performance metric, such as transaction-based ROC curve or AUC. You can calculate and visualize both metrics using the Evaluate Model module.
https://gallery.azure.ai/Experiment/Online-Fraud-Detection-Step-4-of-5-train-and-evaluate-model-2

[jeroenterheerdt]

thanks for taking the time to explain the purpose of this code base and the your perspective on what minimum maintainability is. At this point in time I cannot help you fix this - and I invite you to help me bring it up to par.

[Eezzeldin]

I am saying with this bug here, the whole code is really pointless according to Microsoft own words as I copied it above.

[Eezzeldin]

• "I invite you to help me bring it up to par." sorry what is par?!
• Then how exactly can I get help with this?

[jeroenterheerdt]

I meant - improve the quality so it acceptable (https://idioms.thefreedictionary.com/up+to+par). You can fork the repo - make some changes and create a pull request. This has happened before and it would not only benefit you, but everyone else.

[Eezzeldin]

-I did not design this code, Microsoft did, obviously I would need to spend much more time to fix it.
-The code does not meet it's "only" functional requirement with this bug found.
-Is it the responsibility of Microsoft to make sure that the code satisfy it's very "only" functional requirement* or the open source community?!

*defined above by microsoft.