31 Support Vector Machine
Learn how to use Support Vector Machine (SVM), a powerful algorithm based on statistical learning theory.
Oracle Machine Learning for SQL implements SVM for classification, regression, and anomaly detection.
Related Topics
 Classification
 Regression
 Anomaly Detection
 DBMS_DATA_MINING  Model Settings
 DBMS_DATA_MINING — Algorithm Settings: Support Vector Machine
 Automatic Data Preparation
 Model Detail View for Support Vector Machine
 OML4SQL Examples
 OML4R Support Vector Machine Example
 OML4R GitHub Examples
Oracle Machine Learning for SQL
See Also:
Milenova, B.L., Yarmus, J.S., Campos, M.M., "Support Vector Machines in Oracle Database 10g: Removing the Barriers to Widespread Adoption of Support Vector Machines", Proceedings of the 31st VLDB Conference, Trondheim, Norway, 2005.
31.1 About Support Vector Machine
Support Vector Machine (SVM) is a powerful, stateoftheart algorithm with strong theoretical foundations based on the VapnikChervonenkis theory.
SVM has strong regularization properties. Regularization refers to the generalization of the model to new data.
31.1.1 Advantages of SVM
Support Vector Machine (SVM) implements solvers for scalability and handling large volumes of data.
Oracle Machine Learning for SQL SVM implementation includes two types of solvers, an Interior Point Method (IPM) solver and a SubGradient Descent (SGD) solver. The IPM solver provides stable and accurate solutions, however, it may not be able to handle data of high dimensionality. For highdimensional and/or large data, for example, text, ratings, and so on, the SGD solver is a better choice. Both solvers have highly scalable parallel implementations and can handle large volumes of data.
31.1.2 Advantages of SVM in Oracle Machine Learning for SQL
Oracle's SVM implementation provides scalability, usability, and enhanced performance.
Oracle Machine Learning has its own proprietary implementation of SVM, which exploits the many benefits of the algorithm while compensating for some of the limitations inherent in the SVM framework. SVM provides the scalability and usability that are needed in a production quality OML4SQL system.
31.1.2.1 Usability
Oracle's SVM minimizes data preparation and tuning, making it accessible for nonexperts.
Usability is a major enhancement, because SVM has often been viewed as a tool for experts. The algorithm typically requires data preparation, tuning, and optimization. Oracle Machine Learning minimizes these requirements. You do not need to be an expert to build a quality SVM model. For example:

Data preparation is not required in most cases.

Default tuning parameters are generally adequate.
Related Topics
31.1.2.2 Scalability
Oracle's SVM uses stratified sampling and incremental model building for large data sets.
When dealing with very large data sets, sampling is often required. However, sampling is not required with Oracle Machine Learning for SQL SVM, because the algorithm itself uses stratified sampling to reduce the size of the training data as needed.
Oracle's SVM is highly optimized. It builds a model incrementally by optimizing small working sets toward a global solution. The model is trained until convergence on the current working set, then the model adapts to the new data. The process continues iteratively until the convergence conditions are met. The Gaussian kernel uses caching techniques to manage the working sets.
Related Topics
31.1.3 KernelBased Learning
Learn about kernalbased functions to transform the input data for Support Vector Machine (SVM).
SVM is a kernelbased algorithm. A kernel is a function that transforms the input data to a highdimensional space where the problem is solved. Kernel functions can be linear or nonlinear.
Oracle Machine Learning for SQL supports linear and Gaussian (nonlinear) kernels.
In OML4SQL, the linear kernel function reduces to a linear equation on the original attributes in the training data. A linear kernel works well when there are many attributes in the training data.
The Gaussian kernel transforms each case in the training data to a point in an ndimensional space, where n is the number of cases. The algorithm attempts to separate the points into subsets with homogeneous target values. The Gaussian kernel uses nonlinear separators, but within the kernel space it constructs a linear equation.
Note:
Active Learning is not relevant in Oracle Database 12c Release 2 and later. A setting similar to Active Learning is ODMS_SAMPLING
.
Related Topics
31.2 Tuning an SVM Model
The Support Vector Machine (SVM) algorithm has builtin mechanisms that automatically choose appropriate settings based on the data.
You may need to override the systemdetermined settings for some domains.
Settings pertain to regression, classification, and anomaly detection unless otherwise specified.
See Also:
DBMS_DATA_MINING —Algorithm Settings: Support Vector Machine for a listing and explanation of the available model settings.Note:
The term hyperparameter is also interchangeably used for model setting.31.3 Data Preparation for SVM
Support Vector Machine (SVM) uses normalization and missing value treatment for data preparation.
The SVM algorithm operates natively on numeric attributes. SVM uses zscore normalization on numeric attributes. The normalization occurs only for twodimensional numeric columns (not nested). The algorithm automatically "explodes" categorical data into a set of binary attributes, typically one per category value. For example, a character column for marital status with values married
or single
is transformed to two numeric attributes: married
and single
. The new attributes can have the value 1
(true) or 0
(false).
When there are missing values in columns with simple data types (not nested), SVM interprets them as missing at random. The algorithm automatically replaces missing categorical values with the mode and missing numerical values with the mean.
When there are missing values in the nested columns, SVM interprets them as sparse. The algorithm automatically replaces sparse numerical data with zeros and sparse categorical data with zero vectors.
31.3.1 Normalization
Normalization ensures numeric attributes are on the same scale, preventing bias in the SVM model.
SVM require the normalization of numeric input. Normalization places the values of numeric attributes on the same scale and prevents attributes with a large original scale from biasing the solution. Normalization also minimizes the likelihood of overflows and underflows.
31.3.2 SVM and Automatic Data Preparation
You can prepare data by treating and transforming data manually or through Automatic Data Preparation (ADP) for Support Vector Machine (SVM).
The SVM algorithm automatically handles missing value treatment and the transformation of categorical data, but normalization and outlier detection must be handled by ADP or prepared manually. ADP performs minmax normalization for SVM.
Note:
Oracle recommends that you use ADP with SVM. The transformations performed by ADP are appropriate for most models.
Related Topics
31.4 SVM Classification
Support Vector Machine (SVM) classification is based on the concept of decision planes that define decision boundaries.
A decision plane is one that separates between a set of objects having different class memberships. SVM finds the vectors ("support vectors") that define the separators giving the widest separation of classes.
SVM classification supports both binary, multiclass, and multitarget classification. Multitarget alllows multiple class labels to be associated with a single row. The target type is a collection of type ORA_MINING_VARCHAR2_NT
.
Related Topics
31.4.1 Class Weights
Implement class weights in SVM to bias the model towards underrepresented classes.
In SVM classification, weights are a biasing mechanism for specifying the relative importance of target values (classes).
SVM models are automatically initialized to achieve the best average prediction across all classes. However, if the training data does not represent a realistic distribution, you can bias the model to compensate for class values that are underrepresented. If you increase the weight for a class, then the percent of correct predictions for that class must increase.
Related Topics
31.5 OneClass SVM
OneClass SVM detects anomalies by identifying cases that deviate from normal data patterns.
Oracle Machine Learning uses SVM as the oneclass classifier for anomaly detection. When SVM is used for anomaly detection, it has the classification machine learning technique but no target.
Oneclass SVM models, when applied, produce a prediction and a probability for each case in the scoring data. If the prediction is 1, the case is considered typical. If the prediction is 0, the case is considered anomalous. This behavior reflects the fact that the model is trained with normal data.
You can specify the percentage of the data that you expect to be anomalous
with the SVMS_OUTLIER_RATE
build setting. If you have some knowledge
that the number of "suspicious" cases is a certain percentage of your population, then
you can set the outlier rate to that percentage. The model approximately identifies that
many "rare" cases when applied to the general population.
31.6 SVM Regression
SVM regression uses epsiloninsensitivity loss function to achieve generalization and minimal error.
SVM uses an epsiloninsensitive loss function to solve regression problems.
SVM regression tries to find a continuous function such that the maximum number of data points lie within the epsilonwide insensitivity tube. Predictions falling within epsilon distance of the true target value are not interpreted as errors.
The epsilon factor is a regularization setting for SVM regression. It balances the margin of error with model robustness to achieve the best generalization to new data.
Related Topics