causality

Report for H. Jair Escalante

Download Fact sheet

 Performance as a function of the number of features

Figure 1: Performance as a function of the number of features. (Click to enlarge). Legend: red: REGED, green: SIDO, blue: CINA, black: MARTI. circle: version 1 (unmanipulated test set), full circle: version 2 (manipulated), star: version 3 (manipulated)

 Performance as a function of the number of features

Figure 2: Performance as a function of the number of features. (Click to enlarge). Legend: red: REGED, green: SIDO, blue: CINA, black: MARTI. circle: version 1 (unmanipulated test set), full circle: version 2 (manipulated), star: version 3 (manipulated)

 Performance as a function of the number of features

Figure 3: Performance as a function of the number of features. (Click to enlarge). Legend: red: REGED, green: SIDO, blue: CINA, black: MARTI. circle: version 1 (unmanipulated test set), full circle: version 2 (manipulated), star: version 3 (manipulated)

 Profile of features selected

Figure 4: Profile of features selected. (Click to enlarge). Legend: dcause=direct cause, deffect=direct effects, ocauses=other causes (indirect), oeffects=other effects (indirect), spouses=parent of a direct effect, orelatives=other relatives, unrelated=completely irrelevant

 Tscore performance of all entries

Figure 5: Tscore of all entries made as a function of entry order. (Click to enlarge). The dashed lines indicate the 25 and 50 percentiles.

 Fscore performance of all entries

Figure 6: Fscore of all entries made as a function of entry order. (Click to enlarge). The dashed lines indicate the 25 and 50 percentiles.

Performances:

Fnum=number of features (*=sorted list, **=table of results. Fnum given for best Tscore if table of results submitted), Fscore=MB AUC, Tscore=AUC of test predictions; Top Ts=best score among all valid last entries made by participants. Max Ts=best score reachable, including reference entries using the knowledge of true causal relationships not available to participants.
* Entry Method Fnum Fscore Tscore (Ts) Top Ts Max Ts <Tscore> Rank
REGED0 1485 DRF-LM-PSMS Final Run 2 32/999 ** 0.8778 0.99960.0010 1.0000 1.0000
REGED1 1485 DRF-LM-PSMS Final Run 2 128/999 ** 0.7996 0.94480.0039 0.9980 0.9980 0.8985 6
REGED2 1485 DRF-LM-PSMS Final Run 2 64/999 ** 0.7638 0.75120.0060 0.8600 0.9534
SIDO0 1485 DRF-LM-PSMS Final Run 2 1024/4932 ** 0.8442 0.93520.0075 0.9443 0.9467
SIDO1 1485 DRF-LM-PSMS Final Run 2 4932/4932 ** 0.4675 0.69130.0134 0.7532 0.7893 0.7474 8
SIDO2 1485 DRF-LM-PSMS Final Run 2 4932/4932 ** 0.4675 0.61570.0128 0.6684 0.7674
CINA0 1485 DRF-LM-PSMS Final Run 2 132/132 ** 0.9550 0.96700.0035 0.9788 0.9788
CINA1 1485 DRF-LM-PSMS Final Run 2 132/132 ** 0.4982 0.78730.0049 0.8977 0.8977 0.7675 9
CINA2 1485 DRF-LM-PSMS Final Run 2 128/132 ** 0.4982 0.54810.0044 0.8157 0.8910
MARTI0 1485 DRF-LM-PSMS Final Run 2 1024/1024 ** 0.5446 0.96730.0036 0.9996 0.9996
MARTI1 1485 DRF-LM-PSMS Final Run 2 512/1024 ** 0.4711 0.86360.0054 0.9470 0.9542 0.8691 5
MARTI2 1485 DRF-LM-PSMS Final Run 2 8/1024 ** 0.7055 0.77640.0061 0.7975 0.8273

Return to Causation and Predition Challenge Analysis