-
Notifications
You must be signed in to change notification settings - Fork 26
BLN Inference Tutorial
In this tutorial we show how to make inferences in the university model we introduced in the BLN learning tutorial using the BLN Query Tool. Inference can not only be used for probabilistic reasoning about unknown facts in the model but also for checking whether the created model behaves in a predicted and sound way. Therefore it is recommended to do some kind of sanity checking after designing the model and learning its paramters from a training database. In this tutorial we will show some ways of checking the sanity of the university model we have defined in the learning tutorial and afterwards we will go into various inferences.
An important part of creating a model is to check whether the model behaves in a desired fashion. Therefore it is always recommended to execute some test inference to check the sanity of the model. In a BLN model these tests include for example:
- Checking the correctness of the Prolog formulas
- Checking the correctness of the formulas declared in the logic description
- Checking the correctness of special nodes in the network
We perform some sanity checking on the university model we have defined in the BLN learning tutorial.
The predicates teaches, advises, takes, similar and similarRST are declared in Prolog and the similarRST relation is defined as a reflexive, symmetric and transitive version of the similar predicate. We use a small evidence database to detect whether these predicates are really considered as closed world predicates and whether the similarRST predicate is defined correctly.
takes(John,Stat10) = True teaches(Jones,Phil80) = True similar(Stat20,Stat10) = True similar(CS106,Stat10) = True
The evidence database thus contains the student John, the professor Jones and the courses Stat10,Stat20,Phil80 and CS106. Infering the truth values of teaches, takes, advises and similar, we would expect these predicates to be evaluated to false for all instances not included in the evidence and similarRST to be evaluated to true for the appropriate combinations. For brevity reasons we only give the inference results for similarRST:
similarRST(Stat10,Phil80):
0,0000 True
1,0000 False
similarRST(Stat10,Stat10):
1,0000 True
0,0000 False
similarRST(Stat10,CS106):
1,0000 True
0,0000 False
similarRST(Stat10,Stat20):
1,0000 True
0,0000 False
similarRST(CS106,Phil80):
0,0000 True
1,0000 False
similarRST(CS106,Stat10):
1,0000 True
0,0000 False
similarRST(CS106,CS106):
1,0000 True
0,0000 False
similarRST(CS106,Stat20):
1,0000 True
0,0000 False
similarRST(Phil80,Phil80):
0,0000 True
1,0000 False
similarRST(Phil80,Stat10):
1,0000 True
0,0000 False
similarRST(Phil80,CS106):
1,0000 True
0,0000 False
similarRST(Phil80,Stat20):
1,0000 True
0,0000 False
similarRST(Phil80,Phil80):
1,0000 True
0,0000 False
similarRST(Phil80,Stat10):
0,0000 True
1,0000 False
similarRST(Phil80,CS106):
0,0000 True
1,0000 False
similarRST(Phil80,Stat20):
0,0000 True
1,0000 False
We see that the relation is true for all combinations of the courses Stat10, CS106 and Stat20. The course Phil80 is only similar to itself and not to any other course, so the similarRST is evaluated to true only for the reflexive case.
As defined in the learning tutorial the BLN logic description includes the formulas
likes(p,p). (EXIST c2 (similarRST(c,c2) ^ takes(p,c) ^ takes(p,c2) ^ !(c=c2)) <=> (takesSimilarCourse(p,c))).
To check these two rules we modify the evidence database to detect errors in these formulas. To make meaningful statements about the likes relation we introduce another member of the scientific staff to the model. The second formula can be tested by adding another student to the world and letting one of the students take part in two similar courses while the other student takes two non similar ones.
takes(John, Stat10) = True takes(John, CS106) = True takes(Mary, Phil80) = True takes(Mary, Stat20) = True teaches(Jones, Phil80) = True advises(Moriarty, Mary) = True similar(Stat20,Stat10) = True similar(CS106,Stat10) = True
We use this evidence database to infer the truth values of the query predicates likes and takesSimilarCourse.
likes(Moriarty,Moriarty):
1,0000 True
0,0000 False
likes(Moriarty,Jones):
0,3883 True
0,6117 False
likes(Jones,Moriarty):
0,3804 True
0,6196 False
likes(Jones,Jones):
1,0000 True
0,0000 False
Since we didn't state any evidence about likes in the evidence only reflexive statements are true due to the formula in the logic description. The other two combinations are sampled from the distribution obtained from the training database. In this our inference returns approximately the marginal distribution of likes which we can check against the values in the respective conditional probability table of the learnt version of the fragment network.
The instances of takesSimilarCourse are evaluated to true only for the combinations
takesSimilarCourse(John,Stat10):
1,0000 True
0,0000 False
takesSimilarCourse(John,CS106):
1,0000 True
0,0000 False
and thus as expected.
In the definition of the model we included the combination function
OR:teacherOfLikesAdvisorOf(c,s)|p1,p2
which is evaluated to true if the teacher of course c likes any of the students s advisors. In the evidence given above we did not state anything about the likes relation. There also is only one member of the scientific staff that advises a student. The only result we can expect is that teacherOfLikesAdvisorOf(Phil80,Mary) should be evaluated to true, as it is stated in the evidence that Professor Jones, the teacher of the course Phil80 likes Moriarty, the advisor of Mary.
teacherOfLikesAdvisorOf(Stat20,Mary):
0,0000 True
1,0000 False
teacherOfLikesAdvisorOf(Stat20,John):
0,0000 True
1,0000 False
teacherOfLikesAdvisorOf(Stat10,Mary):
0,0000 True
1,0000 False
teacherOfLikesAdvisorOf(Stat10,John):
0,0000 True
1,0000 False
teacherOfLikesAdvisorOf(CS106,Mary):
0,0000 True
1,0000 False
teacherOfLikesAdvisorOf(CS106,John):
0,0000 True
1,0000 False
teacherOfLikesAdvisorOf(Phil80,Mary):
1,0000 True
0,0000 False
teacherOfLikesAdvisorOf(Phil80,John):
0,0000 True
1,0000 False
As expected, the tacher of Phil80 likes Mary. All other instances of teacherOfLikesAdvisorOf are evaluated to false due to the fact that either there is no professor in the database that teaches the courses CS106, Stat10 and Stat20.
We should note here, that the model is not designed to work with evidence databases in which e.g. the teaches predicate is not given for all courses mentioned in the evidence as it is not possible in the real world that a course is not taught by a person. In the evidence database above we did not include any informations about the teachers of the courses Stat10, Stat20 and CS016. An important aspect of testing the model should be to always use the model in the intended way. As we did not state anything about the teachers of the courses above there is a chance of unwanted (i.e. unexpected) results. In the case above this did not lead to unexpected results as we defined the OR:teacherOfLikesAdvisorOf(c,s)|p1,p2 node in a way that it is true only when there exist professors and advisors which like each other and thus the evaluation of the courses without teachers can be interpreted as correct.
After checking the sanity of the university model we now perform some more inferences to test the model on some more interesting evidences.
Consider the following evidence database:
teaches(Smith, Stat20) = True teaches(Smith, Stat10) = True teaches(Moriarty, CS106) = True teaches(Jones, Phil80) = True advises(Moriarty, Mary) = True likes(Jones,Moriarty) = True likes(Smith,Moriarty) = False similar(Stat20,Stat10) = True similar(CS106,Stat10) = True
We use this training database to infer about the grades of the students participating in courses. As there is no information given about the students' intelligence level and the difficulty of the courses the grades will not depend on that information. The only deciding factors should be the values teacherOfLikesAdvisorOf and takesSimilarCourse. We see in the evidence, that Mary and Fred took similar courses, the only difference between both of them is that the teacher of Phil80 likes Moriarty, the advisor of Mary. The difference between John and Mary (or John and Fred respectively) is that John took Stat20 together with Fred and Mary but also took a similar course which Mary and John didn't do. First we compare the grades of Mary and Fred:
grade(Fred,Stat20):
0,1313 A ; 0,4245 B ; 0,2149 C ; 0,1336 D ; 0,0957 F ; 0,0000 None
grade(Fred,Stat10):
0,0000 A ; 0,0000 B ; 0,0000 C ; 0,0000 D ; 0,0000 F ; 1,0000 None
grade(Fred,CS106):
0,0000 A ; 0,0000 B ; 0,0000 C ; 0,0000 D ; 0,0000 F ; 1,0000 None
grade(Fred,Phil80):
0,1279 A ; 0,4248 B ; 0,2135 C ; 0,1350 D ; 0,0989 F ; 0,0000 None
grade(Mary,Stat20):
0,1310 A ; 0,4265 B ; 0,2128 C ; 0,1321 D ; 0,0978 F ; 0,0000 None
grade(Mary,Stat10):
0,0000 A ; 0,0000 B ; 0,0000 C ; 0,0000 D ; 0,0000 F ; 1,0000 None
grade(Mary,CS106):
0,0000 A ; 0,0000 B ; 0,0000 C ; 0,0000 D ; 0,0000 F ; 1,0000 None
grade(Mary,Phil80):
0,2095 A ; 0,3899 B ; 0,1736 C ; 0,1531 D ; 0,0739 F ; 0,0000 None
As we see, there is not much difference in the distribution of grades for the course Stat20. The grades for Phil80 however, are better for Mary (e.g. it is way more likely for her to get an A). As there are no other factors the only possible reason for this is that John likes Mary's advisor. Let's add in the grades of John:
grade(John,Stat20):
0,2016 A ; 0,3805 B ; 0,2283 C ; 0,1155 D ; 0,0740 F ; 0,0000 None
grade(John,Stat10):
0,0000 A ; 0,0000 B ; 0,0000 C ; 0,0000 D ; 0,0000 F ; 1,0000 None
grade(John,CS106):
0,2041 A ; 0,3773 B ; 0,2278 C ; 0,1161 D ; 0,0747 F ; 0,0000 None
grade(John,Phil80):
0,0000 A ; 0,0000 B ; 0,0000 C ; 0,0000 D ; 0,0000 F ; 1,0000 None
As John has taken two similar courses we expect him to have a higher average degree than Mary or Fred in Stat20. The infered distribution hints into that direction as he is way more likely to receive an A in this course than the other two students all other things being equal. (We stated that Professor Smith doesn't like the only advisor in the world so this doesn't interfere with the results).
In our last example we show a maybe familiar scenario:
takes(Mary, Phil80) = True takes(Fred, Phil80) = True
advises(Challenger, Fred) = True advises(Moriarty, Mary) = True intelligence(Fred) = Smart intelligence(Mary) = Weak grade(Mary, Phil80) = A grade(Fred, Phil80) = C
Both Mary and Fred take the course Phil 80, taught by Professor Jones. Although Fred's intelligence level is higher than Mary's, Mary receives a better grade than Fred in the course. Because there is no other data given in the evidence, we expect that the reason for this will be that Jones likes Mary's advisor more than Fred's. The results show that this is true:
likes(Jones,Moriarty):
0,5972 True
0,4028 False
likes(Jones,Challenger):
0,3137 True
0,6863 False
likes(Jones,Jones):
1,0000 True
0,0000 False
likes(Moriarty,Moriarty):
1,0000 True
0,0000 False
likes(Moriarty,Challenger):
0,3805 True
0,6195 False
likes(Moriarty,Jones):
0,3793 True
0,6207 False
likes(Challenger,Moriarty):
0,3797 True
0,6203 False
likes(Challenger,Challenger):
1,0000 True
0,0000 False
likes(Challenger,Jones):
0,3778 True
0,6222 False
Without further information the distribution for the combinations of (Moriarty,Challenger), (Moriarty,Jones), (Challenger,Moriarty), (Moriarty,Jones) is the marginal distribution of likes while the reflexive cases are all evaluated to be true with probability 1 because of the respective formula. However, we see that the probability of likes(Jones,Challenger) being true is below the marginal probability, while the probability of likes(Jones,Moriarty) is well above.