We could use KNN, logistic regression, LDA, and decision trees for classification problems. Here we discuss a new set of methods: the maximal margin classifier, the support vector classifier, and the support vector machine, which are often loosely referred to as “support vector machines”. Read more
Recall that we use step functions, where the range of predictor $X$ is broken into several bins and in each bin the average of the observed response, certainly a constant, is used as the prediction. Step functions can be viewed as a special case of tree-based methods, the latter of which could be applied to multiple predictors. Read more
Linear models, including linear regression and models making improvements on it, can only offer a limited approximation of the real world. Therefore, moving beyond linearity is necessitated in various situations where more flexibility is required. Read more