alpbench.evaluation.experimenter.LogTableObserver

Classes

LogTableObserver(result_processor, X_test, ...)	This class observes the performance of the model and the labeling process and logs the results in the database.
SparseLogTableObserver(result_processor, ...)	This class observes the performance of the model and the labeling process and logs the results in the database.

class alpbench.evaluation.experimenter.LogTableObserver.LogTableObserver(result_processor, X_test, y_test)

Bases: StatisticalPerformanceObserver

This class observes the performance of the model and the labeling process and logs the results in the database.

Parameters:

• StatisticalPerformanceObserver (class) – The parent class of the LogTableObserver.

• result_processor (class) – The result processor class.

• X_test (array) – The test data.

• y_test (array) – The test labels.

model_performance_tbl

The name of the table where the model performance is logged.

Type:

str

labeling_log_tbl

The name of the table where the labeling process is logged.

Type:

str

observe_data(iteration, X_u_selected, y_u_selected, X_l_aug, y_l_aug, X_u_red, D_l_ind)

Computes labeling statistics and log the results in the database.

Parameters:

• iteration (int) – The current iteration.

• X_u_selected (array) – The selected unlabeled data.

• y_u_selected (array) – The selected unlabeled labels.

• X_l_aug (array) – The augmented labeled data.

• y_l_aug (array) – The augmented labeled labels.

• X_u_red (array) – The reduced unlabeled data.

• D_l_ind (np.array) – Indices of X_u selected to be labeled

Returns:

None

observe_model(iteration, model)

Computes model performances and log the results in the database.

Parameters:

• iteration (int) – The current iteration.

• model (class) – The model

Returns:

None

labeling_log_tbl = 'labeling_log'

model_performance_tbl = 'accuracy_log'

class alpbench.evaluation.experimenter.LogTableObserver.SparseLogTableObserver(result_processor, X_test, y_test)

Bases: StatisticalPerformanceObserver

This class observes the performance of the model and the labeling process and logs the results in the database. To reduce the number of logs, the results are only logged after a whole active learning procedure is finished.

Parameters:

• StatisticalPerformanceObserver (class) – The parent class of the LogTableObserver.

• result_processor (class) – The result processor class.

• X_test (array) – The test data.

• y_test (array) – The test labels.

model_performance_tbl

The name of the table where the model performance is logged.

Type:

str

labeling_log_tbl

The name of the table where the labeling process is logged.

Type:

str

log_data(dict)

Logs the labeling statistics in the database after the active learning procedure is finished.

Parameters:

dict (dict) – The evaluation scores.

Returns:

None

log_model(dict)

Logs the model performances in the database after the active learning procedure is finished.

Parameters:

dict (dict) – The evaluation scores.

Returns:

None

observe_data(iteration, X_u_selected, y_u_selected, X_l_aug, y_l_aug, X_u_red, D_l_ind)

Computes labeling statistics per iteration.

Parameters:

• iteration (int) – The current iteration.

• X_u_selected (array) – The selected unlabeled data.

• y_u_selected (array) – The selected unlabeled labels.

• X_l_aug (array) – The augmented labeled data.

• y_l_aug (array) – The augmented labels.

• X_u_red (array) – The reduced unlabeled data.

• D_l_ind (np.array) – Indices of X_u selected to be labeled

Returns:

The evaluation scores.

Return type:

eval_scores (dict)

observe_model(iteration, model)

Computes model performances per iteration.

Parameters:

• iteration (int) – The current iteration.

• model (class) – The model

Returns:

The evaluation scores.

Return type:

eval_scores (dict)

labeling_log_tbl = 'labeling_log'

model_performance_tbl = 'accuracy_log'