Model Evaluation & Selection Techniques

Course Discussions Machine Learning
#1

From what I have understood, I believe that there are the following Model Selection Techniques used in ML as follows:

a) Optimization Techniques - such as Gradient Descent (GD or Batch GD), SGD and Mini-Batch GD.
b) Probabilistic Techniques - AIC, BIC and so on (This is especially used in Logistic Regression where Probability values are generated)
c) Resampling Techniques - using k-fold method, Cross Validation, Bootstrap .

Following are my queries:

  1. Could you share some info regarding the Probabilistic techniques?

  2. Have seen its usage in R tool also. How are these values determined using Python?

  3. Also believe that such terminologies are used in Step-Wise Regression. Could you share some info regarding this technique?