Unable to run score

[ritesh99rakesh]

I was trying to run my own algorithm on the NAB datasets as per option 1 on https://github.com/numenta/NAB/wiki/NAB-Entry-Points
Till optimize function, everything works fine but following error occurs while scoring

Running scoring step
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/home/ritesh/NUS/code/NAB/nab/scorer.py", line 165, in scoreDataSet
    return (detectorName, profileName, relativePath, threshold, bestRow.score,
AttributeError: 'NoneType' object has no attribute 'score'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "run.py", line 204, in <module>
    main(args)
  File "run.py", line 77, in main
    runner.score(args.detectors, detectorThresholds)
  File "/home/ritesh/NUS/code/NAB/nab/runner.py", line 208, in score
    scoreFlag))
  File "/home/ritesh/NUS/code/NAB/nab/scorer.py", line 99, in scoreCorpus
    results = pool.map_async(scoreDataSet, args).get(999999)
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
AttributeError: 'NoneType' object has no attribute 'score'

Please help ASAP

[liuyuqingNKCS]

I got the same problem, but still don't know how to fix it.
However, I think there must be something wrong in the scoreDataset function in sweeper.py, cause I found the logic below may return None for matchingRow. And then lead to the problem mentioned above.

    matchingRow = None
    prevRow = None

    for thresholdScore in scoresByThreshold:
      if thresholdScore.threshold == threshold:
        matchingRow = thresholdScore
        break
      elif thresholdScore.threshold < threshold:
        matchingRow = prevRow
        break
      
      prevRow = thresholdScore

[bhatiasiddharth]

@rhyolight @subutai: Can you please fix this or give a lead as to what is causing the problem mentioned here? Thanks.

[subutai]

@rhyolight @subutai: Can you please fix this or give a lead as to what is causing the problem mentioned here? Thanks.

We'll take a look - give us a couple of days. The function may make some assumptions about allowable values.

[lscheinkman]

@bhatiasiddharth Can you verify if the anomalyScore returned by your custom detector handleRecord method is a floating point number between 0 and 1?

[bhatiasiddharth]

No it is not. Does it need a score between 0 and 1 only?

[lscheinkman]

Yes, the raw anomaly score must be a number between 0 and 1.

[bhatiasiddharth]

Sure, thanks.

[lscheinkman]

Let me know if it works for you

[bhatiasiddharth]

Currently, the algorithm I am trying does not give scores between 0 and 1. I need to think how to do it. Once I try, I will let you know. Thanks.

[subutai]

Currently, the algorithm I am trying does not give scores between 0 and 1. I need to think how to do it. Once I try, I will let you know. Thanks.

Perhaps you can pass it through a sigmoid, or similar function to get the output in range? As long as it's a monotonic function, it shouldn't matter at all since we are just thresholding the score.

[bhatiasiddharth]

Thanks for letting me know. It worked.