m (space deleted) |
|||
| (2 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
'''Support Vector Machines''' | '''Support Vector Machines''' | ||
| − | + | # Requires solving a Quadratic Programming problem which can be computationally intensive | |
| − | + | # Finding the right kernel function, for linear classification of data required by SVMs, is non-trivial task. | |
| − | + | # Assuming a kernel function and optimizing the cost function are done as different steps (Neural Networks, where these are done simultaneously poses this as its advantage over SVMs) | |
| + | |||
'''Perceptron (with FLD)''' | '''Perceptron (with FLD)''' | ||
| − | + | # Requires the data to be linearly separable. If the classification accuracy of the perceptron method is bad, kernel methods (eg. SVMs) might be required. | |
| − | + | # If the required class means and covariances are not known, they can be estimated from the training set. Parameter estimation methods like maximum likelihood estimate or the maximum a posteriori estimate may be used | |
| − | + | # Regularization might be required (for finding the inverse) to avoid overfitting issues. | |
| − | |||
| − | + | '''KNN Classification''' | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | # This classification method gives very good results if huge training data is available. | |
| + | # For a given input data, with no apriori knowledge, choosing appropriate distance metric is very important. Distance metrics are used in density estimation methods (Parzen windows), clustering (k-means) and instance based classification methods (Nearest Neighbors) etc. Euclidean distance is used in most of the cases, but in cases where the relationship between data points is non-linear, selection of a distance metric is a challenge. Here is a reference addressing this issue: [http://www.citeulike.org/user/sdvillal/article/673356 citeulike] | ||
Latest revision as of 10:21, 7 April 2008
Support Vector Machines
- Requires solving a Quadratic Programming problem which can be computationally intensive
- Finding the right kernel function, for linear classification of data required by SVMs, is non-trivial task.
- Assuming a kernel function and optimizing the cost function are done as different steps (Neural Networks, where these are done simultaneously poses this as its advantage over SVMs)
Perceptron (with FLD)
- Requires the data to be linearly separable. If the classification accuracy of the perceptron method is bad, kernel methods (eg. SVMs) might be required.
- If the required class means and covariances are not known, they can be estimated from the training set. Parameter estimation methods like maximum likelihood estimate or the maximum a posteriori estimate may be used
- Regularization might be required (for finding the inverse) to avoid overfitting issues.
KNN Classification
- This classification method gives very good results if huge training data is available.
- For a given input data, with no apriori knowledge, choosing appropriate distance metric is very important. Distance metrics are used in density estimation methods (Parzen windows), clustering (k-means) and instance based classification methods (Nearest Neighbors) etc. Euclidean distance is used in most of the cases, but in cases where the relationship between data points is non-linear, selection of a distance metric is a challenge. Here is a reference addressing this issue: citeulike
