plot_critical_difference

plot_critical_difference(scores, labels, highlight=None, errors=False, cliques=None, clique_method='holm', alpha=0.05, width=6, textspace=1.5, reverse=True)

Draw critical difference diagram.

Step 1 & 2: Calculate average ranks from data Step 3: Use Friedman test to check whether the strategy significantly affects the classification performance Step 4: Compute critical differences using Nemenyi post-hoc test. (How much should the average rank of two strategies differ to be

statistically significant)

Step 5: Compute statistically similar cliques of strategies Step 6: Draw the diagram

See Janez Demsar, Statistical Comparisons of Classifiers over Multiple Data Sets, 7(Jan):1–30, 2006.

Parts of the code are copied and adapted from here: https://github.com/hfawaz/cd-diagram

Parameters:

scoresnp.array

scores (either accuracies or errors) of dataset x strategy

labelslist of estimators

list with names of the estimators. Order should be the same as scores

highlight: dict with labels and HTML colours to be used, default = None

dict with labels and HTML colours to be used for highlighting. Order should be the same as scores

errorsbool, default = False

indicates whether scores are passed as errors (default) or accuracies

cliqueslists of bit vectors, default = None

e.g. [[0,1,1,1,0,0], [0,0,0,0,1,1]] statistically similiar cliques of estimators If none, cliques will be computed depending on clique_method

clique_methodstring, default = “holm”

clique forming method, to include “nemenyi” and “holm”

alphafloat default = 0.05

Alpha level for statistical tests currently supported: 0.1, 0.05 or 0.01)

widthint, default = 6

width in inches

textspaceint

space on figure sides (in inches) for the method names (default: 1.5)

reversebool, default = True

if set to ‘True’, the lowest rank is on the right

Returns:

fig: matplotlib.figure

Figure created.