Skip to content

Xgboost Adapter

This module provides an adapter for the XGBoost pretrained model used for lecture type detection and text recognition.

Classes:

Name Description
XGBoostModelAdapter

Model adapter for the XGBoost pretrained model xgboost500.sav that classifies the lesson kind from images.

XGBoostModelAdapter

Model adapter for the XGBoost pretrained model from the misc/lecture type detection and text recognition folder.

Attributes:

Name Type Description
_labels dict

A dictionary mapping label indices to their corresponding names.

Methods:

Name Description
__init__

Initializes the model adapter with the given model path.
_extract_faces_info

Extracts face information from the current image.
_extract_features_from_image

Extracts features from the given image.
predict_probability

Predicts the probability distribution over classes for the given image.
predict_max_confidence

Predicts the class with the highest confidence for the given image.
get_label

Gets the label corresponding to the given prediction.
is_enough_slidish_like

Predicts if the image is likely to be a slide with a small margin of confidence.
Source code in apps/annotator/code/models/xgboost_adapter.py 
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
class XGBoostModelAdapter:
    '''
    Model adapter for the XGBoost pretrained model from the
    misc/lecture type detection and text recognition folder.

    Attributes
    ----------
    _labels : dict
        A dictionary mapping label indices to their corresponding names.

    Methods
    -------
    __init__(model_path: str) -> None
        Initializes the model adapter with the given model path.
    _extract_faces_info(detections: 'list[Detection] | None')
        Extracts face information from the current image.
    _extract_features_from_image(image: ImageClassifier, norm_minmax: bool = False)
        Extracts features from the given image.
    predict_probability(image: ImageClassifier)
        Predicts the probability distribution over classes for the given image.
    predict_max_confidence(image: ImageClassifier)
        Predicts the class with the highest confidence for the given image.
    get_label(prediction)
        Gets the label corresponding to the given prediction.
    is_enough_slidish_like(image: ImageClassifier)
        Predicts if the image is likely to be a slide with a small margin of confidence.
    '''

    _labels = {0: "Blackboard", 1: "Slide", 2: "Slide-and-Talk", 3: "Talk"}

    def __init__(self, model_path: str) -> None:
        '''
        Initializes the model adapter with the given model path.

        Parameters
        ----------
        model_path : str
            The path to the pretrained XGBoost model.
        '''
        try:
            self._model: XGBClassifier = load_model(open(model_path, 'rb'))
        except:
            raise FileExistsError("cannot find XGBoost model")

    def _extract_faces_info(self, detections: 'list[Detection] | None'):
        '''
        Extracts face information from the current image.

        Parameters
        ----------
        detections : list[Detection] or None
            A list of face detections or None if no faces are detected.

        Returns
        -------
        out_arr : ndarray
            An array with face information: [x_center, face_size, n_faces].
        '''
        out_arr = zeros((1, 3), dtype=float)
        if detections is None:
            return out_arr
        for detection in detections:
            bounding_box = detection.bounding_box
            xmin, width, height = bounding_box.origin_x, bounding_box.width, bounding_box.height
            face_size = width * height
            out_arr[0, 1] = max(out_arr[0, 1], face_size)
            if out_arr[0, 1] == face_size:
                out_arr[0, 0] = xmin + width / 2
            if out_arr[0, 2] < 2:
                out_arr[0, 2] += 1
        return out_arr

    def _extract_features_from_image(self, image: ImageClassifier, norm_minmax: bool = False):
        '''
        Extracts features from the given image.

        Parameters
        ----------
        image : ImageClassifier
            The image classifier object.
        norm_minmax : bool, optional
            Whether to normalize the features using min-max normalization (default is False).

        Returns
        -------
        out_arr : ndarray
            An array with extracted features.
        '''
        assert isinstance(image, ImageClassifier) and image.get_img().shape[2] == 3
        out_arr = empty((1, 19), dtype=float)
        out_arr[0, :16] = image.get_hists(normalize=norm_minmax, bins=16, grayscaled=True)
        if not norm_minmax:
            out_arr[0, :16] /= prod(image.get_img_shape()[:2])
        out_arr[0, 16:] = self._extract_faces_info(image.detect_faces())
        return out_arr

    def predict_probability(self, image: ImageClassifier):
        '''
        Predicts the probability distribution over classes for the given image.

        Parameters
        ----------
        image : ImageClassifier
            The image classifier object.

        Returns
        -------
        probs : ndarray
            The probability distribution over classes.
        '''
        return self._model.predict_proba(self._extract_features_from_image(image))

    def predict_max_confidence(self, image: ImageClassifier):
        '''
        Predicts the class with the highest confidence for the given image.

        Parameters
        ----------
        image : ImageClassifier
            The image classifier object.

        Returns
        -------
        prediction : int
            The class index with the highest confidence.
        '''
        return int(argmax(self.predict_probability(self._extract_features_from_image(image))))

    def get_label(self, prediction):
        '''
        Gets the label corresponding to the given prediction.

        Parameters
        ----------
        prediction : int or ndarray
            The prediction index or probability distribution.

        Returns
        -------
        label : str or dict
            The label corresponding to the prediction index or a dictionary of labels with their probabilities.
        '''
        if isinstance(prediction, int):
            assert 0 <= prediction <= 3
            return self._labels[prediction]
        elif isinstance(prediction, ndarray):
            assert prediction.shape == (1, 4)
            return {self._labels[i]: prediction[0, i] for i in range(4)}
        else:
            return None

    def is_enough_slidish_like(self, image: ImageClassifier):
        '''
        Predicts if the image is likely to be a slide with a small margin of confidence.

        Parameters
        ----------
        image : ImageClassifier
            The image classifier object.

        Returns
        -------
        is_slidish : bool
            True if the image is likely to be a slide, False otherwise.
        '''
        probs = self.predict_probability(image)
        best_slidish_prob = max(probs[0, (1, 2)])
        best_not_slidish_prob = max(probs[0, (0, 3)])
        return 1.15 * best_slidish_prob >= best_not_slidish_prob

__init__(model_path)

Initializes the model adapter with the given model path.

Parameters:

Name Type Description Default
model_path str

The path to the pretrained XGBoost model.

 required
Source code in apps/annotator/code/models/xgboost_adapter.py 
46
47
48
49
50
51
52
53
54
55
56
57
58
def __init__(self, model_path: str) -> None:
    '''
    Initializes the model adapter with the given model path.

    Parameters
    ----------
    model_path : str
        The path to the pretrained XGBoost model.
    '''
    try:
        self._model: XGBClassifier = load_model(open(model_path, 'rb'))
    except:
        raise FileExistsError("cannot find XGBoost model")

get_label(prediction)

Gets the label corresponding to the given prediction.

Parameters:

Name Type Description Default
prediction int or ndarray

The prediction index or probability distribution.

 required

Returns:

Name Type Description
label str or dict

The label corresponding to the prediction index or a dictionary of labels with their probabilities.
Source code in apps/annotator/code/models/xgboost_adapter.py 
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
def get_label(self, prediction):
    '''
    Gets the label corresponding to the given prediction.

    Parameters
    ----------
    prediction : int or ndarray
        The prediction index or probability distribution.

    Returns
    -------
    label : str or dict
        The label corresponding to the prediction index or a dictionary of labels with their probabilities.
    '''
    if isinstance(prediction, int):
        assert 0 <= prediction <= 3
        return self._labels[prediction]
    elif isinstance(prediction, ndarray):
        assert prediction.shape == (1, 4)
        return {self._labels[i]: prediction[0, i] for i in range(4)}
    else:
        return None

is_enough_slidish_like(image)

Predicts if the image is likely to be a slide with a small margin of confidence.

Parameters:

Name Type Description Default
image ImageClassifier

The image classifier object.

 required

Returns:

Name Type Description
is_slidish bool

True if the image is likely to be a slide, False otherwise.
Source code in apps/annotator/code/models/xgboost_adapter.py 
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
def is_enough_slidish_like(self, image: ImageClassifier):
    '''
    Predicts if the image is likely to be a slide with a small margin of confidence.

    Parameters
    ----------
    image : ImageClassifier
        The image classifier object.

    Returns
    -------
    is_slidish : bool
        True if the image is likely to be a slide, False otherwise.
    '''
    probs = self.predict_probability(image)
    best_slidish_prob = max(probs[0, (1, 2)])
    best_not_slidish_prob = max(probs[0, (0, 3)])
    return 1.15 * best_slidish_prob >= best_not_slidish_prob

predict_max_confidence(image)

Predicts the class with the highest confidence for the given image.

Parameters:

Name Type Description Default
image ImageClassifier

The image classifier object.

 required

Returns:

Name Type Description
prediction int

The class index with the highest confidence.
Source code in apps/annotator/code/models/xgboost_adapter.py 
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
def predict_max_confidence(self, image: ImageClassifier):
    '''
    Predicts the class with the highest confidence for the given image.

    Parameters
    ----------
    image : ImageClassifier
        The image classifier object.

    Returns
    -------
    prediction : int
        The class index with the highest confidence.
    '''
    return int(argmax(self.predict_probability(self._extract_features_from_image(image))))

predict_probability(image)

Predicts the probability distribution over classes for the given image.

Parameters:

Name Type Description Default
image ImageClassifier

The image classifier object.

 required

Returns:

Name Type Description
probs ndarray

The probability distribution over classes.
Source code in apps/annotator/code/models/xgboost_adapter.py 
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
def predict_probability(self, image: ImageClassifier):
    '''
    Predicts the probability distribution over classes for the given image.

    Parameters
    ----------
    image : ImageClassifier
        The image classifier object.

    Returns
    -------
    probs : ndarray
        The probability distribution over classes.
    '''
    return self._model.predict_proba(self._extract_features_from_image(image))