Clarification - Doubt regarding Soft Voting or Hard Voting Classifier in AdaBoost Ensemble ML Algorithm

Course Discussions Machine Learning
#1

As per the PPT in Ensemble Learning, Slide 79 (AdaBoost Ensemble ML Algorithm) conveys that —

In case if the training instances/observations/rows result in Underfitting, then the new Predictors/Independent Variables focus more and more on Hard Voting Classifier Cases. This is the functionality or rationale behind the execution of AdaBoost Ensemble ML Algorithm.

However, in slide 88, it conveys that—

Based on the Weights assigned to the variables (whether a Strong Learner or a Weak Learner), the Aggregation of these Weights define the Probability of an Instance/Row/Observation Selection…

Based on the aforesaid 2 statements, don’t they contradict each other???

NOTE: Here contradiction is in the sense of Voting Classifiers.

Now my doubt is —

Once Probability is attached (or calculated) pertaining to an Instance/Row/Observation selection by the concerned Software Tool - R/Python/SAS, doesn’t this become a Soft Voting Classifier???

Kindly correct me if my understanding is wrong. Kindly elaborate on them in case the grasping of the aforesaid concepts is faulty.

Looking forward to your inputs.

#2

What AdaBoost does is that it trains a set of classifiers, and based on their results it takes a subset of those classifiers. These are the classifiers that underfitted, their relative weights are then increased. A second set of classifiers are then trained based on these updated weights.

Also, when we say “The weights define the probability of an instance selection.”, what we mean is that once each classifier is trained, we update the probabilities of each of the training examples appearing in the training set for the next classifier.

For a better understanding, you can go through the link mentioned in the slides:

http://mccormickml.com/2013/12/13/adaboost-tutorial/

Also, there is another article from the author of AdaBoost, Robert Schapire:

http://rob.schapire.net/papers/explaining-adaboost.pdf

Hope this addresses the queries you have.

#3

These references regarding AdaBoost Ensemble ML Algorithm are indeed helpful. Appreciate Rajtilak forsharing these links…!!!