alpbench.evaluation.analysis.plot_functions¶

Classes

`BudgetPerformancePlot`(df, openml_id, ...[, ...])	This class plots the performance for a given learning algorithm and openmlid of different query strategies over the whole active learning procedure resulting in so-calles area under the budget curves (AUBC).
`HeatMapPlot`(df[, path_to_save, ...])	This class plots a heatmap of the performance of different active learning pipelines as well as win or lose- matrices for the specified learner comparing different query strategies.
`WinMatrixPlot`(df, learner_name[, ...])

class alpbench.evaluation.analysis.plot_functions.BudgetPerformancePlot(df, openml_id, learner_name, metric, path_to_save=None)[source]¶

Bases: object

This class plots the performance for a given learning algorithm and openmlid of different query strategies over the whole active learning procedure resulting in so-calles area under the budget curves (AUBC).

Parameters:

df (pd.DataFrame) – The dataframe.
openml_id (int) – The openml id.
learner_name (str) – The learner name.
metric (str) – The metric.
path_to_save (str) – The path to save the plot.

df¶

The dataframe.

Type:: pd.DataFrame

openml_id¶

The openml id.

Type:: int

learner_name¶

The learner name.

Type:: str

metric¶

The metric.

Type:: str

path_to_save¶

The path to save the plot.

Type:: str

plot_data¶

The data to plot.

Type:: dict

num_seeds¶

The number of seeds (used to determine the std error).

Type:: int

generate_plot_data()[source]¶: This function generates the data to plot.

show(show_fig=False)[source]¶: This function plots the performance of different query strategies over the budget and saves it as .pdf under the specified path.

class alpbench.evaluation.analysis.plot_functions.HeatMapPlot(df, path_to_save=None, statistical_significant=True, filter_ids='all')[source]¶

Bases: object

This class plots a heatmap of the performance of different active learning pipelines as well as win or lose- matrices for the specified learner comparing different query strategies.

Parameters:

data (dict) – The data to plot.
path_to_save (str) – The path to save the plot.
filter_ids (str) – The filter ids.
take_statistical_insignificant (bool) – Whether to take statistical insignificant values.

data¶

The data to plot.

Type:: dict

path_to_save¶

The path to save the plot.

Type:: str

filter_ids¶

The filter ids.

Type:: str

take_statistical_insignificant¶

Whether to take statistical insignificant values.

Type:: bool

generate_heatmap()[source]¶: This function generates heatmaps to compare performances of different query strategies combined with one fixed learning algorithm.

show(show_fig=False)[source]¶: This function plots heatmaps to compare performances of different active learning pipelines. The figures are saved under the specified path.

class alpbench.evaluation.analysis.plot_functions.WinMatrixPlot(df, learner_name, path_to_save=None, statistical_significant=True, filter_ids='all')[source]¶

Bases: object

generate_win_matrix()[source]¶: This function generates win-matrices to compare performances of different query strategies combined with one fixed learning algorithm.

show(show_fig=False)[source]¶: This function generates win-matrices to compare performances of different query strategies combined with one fixed learning algorithm.