Exponential Smoothing

18 Exponential Smoothing

Learn about the Exponential Smoothing algorithm.

Related Topics

18.1 About Exponential Smoothing

Exponential smoothing is a forecasting method for time-series data. It is a moving average method where exponentially decreasing weights are assigned to past observations.

Exponential smoothing methods have been widely used in forecasting for over half a century. It has applications at the strategic, tactical, and operation level. For example, at a strategic level, forecasting is used for projecting return on investment, growth and the effect of innovations. At a tactical level, forecasting is used for projecting costs, inventory requirements, and customer satisfaction. At an operational level, forecasting is used for setting targets and predicting quality and conformance with standards.

In its simplest form, exponential smoothing is a moving average method with a single parameter which models an exponentially decreasing effect of past levels on future values. With a variety of extensions, exponential smoothing covers a broader class of models than competitors, such as the Box-Jenkins auto-regressive integrated moving average (ARIMA) approach. Oracle Machine Learning for SQL implements exponential smoothing using a state of the art state space method that incorporates a single source of error (SSOE) assumption which provides theoretical and performance advantages.

Exponential smoothing is extended to the following:

A matrix of models that mix and match error type (additive or multiplicative), trend (additive, multiplicative, or none), and seasonality (additive, multiplicative, or none)
Models with damped trends.
Models that directly handle irregular time series and time series with missing values.

Note:

For more information, see Ord, J.K., et al, Time Series Forecasting: The Case for the Single Source of Error State Space Approach, Working Paper, Department of Econometrics and Business Statistics, Monash University, VIC 3800, Australia, April 2, 2005.

18.1.1 Exponential Smoothing Models

Exponential Smoothing models are a broad class of forecasting models that are intuitive, flexible, and extensible.

Members of this class include simple, single parameter models that predict the future as a linear combination of a previous level and a current shock. Extensions can include parameters for linear or non-linear trend, trend damping, simple or complex seasonality, related series, various forms of non-linearity in the forecasting equations, and handling of irregular time series.

Exponential smoothing assumes that a series extends infinitely into the past, but that influence of past on future, decays smoothly and exponentially fast. The smooth rate of decay is expressed by one or more smoothing constants. The smoothing constants are parameters that the model estimates. The assumption is made practical for modeling real world data by using an equivalent recursive formulation that is only expressed in terms of an estimate of the current level based on prior history and a shock to that estimate dependent on current conditions only.The procedure requires an estimate for the time period just prior to the first observation, that encapsulates all prior history. This initial observation is an additional model parameter whose value is estimated by the modeling procedure.

Components of ESM such as trend and seasonality extensions, can have an additive or multiplicative form. The simpler additive models assume that shock, trend, and seasonality are linear effects within the recursive formulation.

18.1.2 Simple Exponential Smoothing

Simple exponential smoothing assumes the data fluctuates around a stationary mean, with no trend or seasonal pattern.

In a simple Exponential Smoothing model, each forecast (smoothed value) is computed as the weighted average of the previous observations, where the weights decrease exponentially depending on the value of smoothing constant α. Values of the smoothing constant, α, near one, put almost all weight on the most recent observations. Values of α near zero allows the distant past observations to have a large influence.

18.1.3 Models with Trend but No Seasonality

The preferred form of additive (linear) trend is sometimes called Holt’s method or double exponential smoothing.

Models with trend add a smoothing parameter γ and optionally a damping parameter φ. The damping parameter smoothly dampens the influence of past linear trend on future estimates of level, often improving accuracy.

18.1.4 Models with Seasonality but No Trend

When the time series average does not change over time (stationary), but is subject to seasonal fluctuations, the appropriate model has seasonal parameters but no trend.

Seasonal fluctuations are assumed to balance out over periods of length m, where m is the number of seasons, For example, m=4 might be used when the input data are aggregated quarterly. For models with additive errors, the seasonal parameters must sum to zero. For models with multiplicative errors, the product of seasonal parameters must be one.

18.1.5 Models with Trend and Seasonality

Holt and Winters introduced both trend and seasonality in an Exponential Smoothing model.

The original model, also known as Holt-Winters or triple exponential smoothing, considered an additive trend and multiplicative seasonality. Extensions include models with various combinations of additive and multiplicative trend, seasonality and error, with and without trend damping.

18.1.6 Prediction Intervals

To compute prediction intervals, an Exponential Smoothing (ESM) model is divided into three classes.

The simplest class is the class of linear models, which include, among others, simple ESM, Holt’s method, and additive Holt-Winters. Class 2 models (multiplicative error, additive components) make an approximate correction for violations of the Normality assumption. Class 3 modes use a simple simulation approach to calculate prediction intervals.

18.2 Data Preparation for Exponential Smoothing Models

Learn about preparing the data for an Exponential Smoothing (ESM) model.

To build an ESM model, you must supply the following :

Input data
An aggregation level and method, if the case id is a date type
Partitioning column, if the data are partitioned

In addition, for a greater control over the build process, the user may optionally specify model build parameters, all of which have defaults:

Model
Error type
Optimization criterion
Forecast Window
Confidence level for forecast bounds
Missing value handling
Whether the input series is evenly spaced

Related Topics

Oracle Machine Learning for SQL User’s Guide

18.2.1 Input Data

Time series analysis, requires ordered input data. Hence, each data row must consist of an [index, value] pair, where the index specifies the ordering.

When the CREATE_MODEL procedure is used to initiate an Exponential Smoothing (ESM) model build, the CASE_ID_COLUMN_NAME specifies the column used to compute the indices of the input and the TARGET_COLUMN_NAME specifies the column used to compute the observed time series values. The time column bears Oracle number, or Oracle date, timestamp, timestamp with time zone, or timestamp with local time zone. The input time series are sorted according to the values of CASE_ID (time label). The case id column cannot contain missing values. The value column can contain missing values indicated as NULL. ESM also supports partitioned models and in such cases, the input table contains an extra column specifying the partition. All [index, value] pairs with the same partition ID form one complete time series. The Exponential Smoothing algorithm constructs models for each partition independently, although all models use the same model settings.

Properties of the data can result in a warning message or settings are ignored. Settings are ignored when If the user specifies a model with either multiplicative trend, multiplicative seasonality or both and the data contains values Y_t<= 0, then the model type is set to the default. If the series contain fewer values than the number of user-specified seasons, then the seasonality specifications are ignored with a warning.

18.2.2 Accumulation

For the Exponential Smoothing algorithm, the accumulation procedure is applied when the column is a date type (date, datetime, timestamp, timestamp with timezone, or timestamp with local timezone).

The case id can be a NUMBER column whose sort index represents the position of the value in the time series sequence of values. The case id column can also be a date type. A date type is accumulated in accordance with a user specified accumulation window. Regardless of type, the case id is used to transform the column into an equally spaced time series. No accumulation is applied for a case id of type NUMBER. As an example, consider a time series about promotion events. The time column contains the date of each event, and the dates can be unequally spaced. The user must specify the spacing interval, which is the spacing of the accumulated or transformed equally spaced time series. In the example, if the user specifies the interval to be month, then an equally spaced time series with profit for each calendar month is generated from the original time series. Setting EXSM_INTERVAL is used to specify the spacing interval. The user must also specify a value for EXSM_ACCUMULATE, for example, EXSM_ACCU_MAX, in which case the equally spaced monthly series would contain the maximum profit over all events that month as the observed time series value.

18.2.3 Missing Value

Input time series can contain missing values. A NULL entry in the target column indicates a missing value. When the time column is of the type datetime, the accumulation procedure can also introduce missing values. The setting EXSM_SETMISSING can be used to specify how to handle missing values. The special value EXSM_MISS_AUTO indicates that, if the series contains missing values it is to be treated as an irregular time series.

Note:

Missing value handling setting must be compatible with model setting, otherwise an error is thrown.

18.2.4 Prediction

An Exponential Smoothing (ESM) model can be applied to make predictions by specifying the prediction window.

Setting EXSM_PREDICTION_STEP can be used to specify the prediction window. The prediction window is expressed in terms of number of intervals (setting EXSM_INTERVAL), when the time column is of the type datetime. If the time column is a number then the prediction window is the number of steps to forecast. Regardless of whether the time series is regular or irregular, EXSM_PREDICTION_STEP specifies the prediction window.

18.2.5 Parallellism by Partition

Oracle Machine Learning for SQL supports parallellism by partition.

For example, a user can choose PRODUCT_ID as one partition column and can generate forecasts for different products in a model build. Although a distinct smoothing model is built for each partition, all partitions share the same model settings. For example, if setting EXSM_MODEL is set to EXSM_SIMPLE, all partition models will be simple Exponential Smoothing models. Time series from different partitions can be distributed to different processes and processed in parallel. The model for each time series is built serially.

18.2.6 Initial Value Optimization

With long seasonal cycles, users can choose not to optimize the ESM model initial values beyond an initial estimate.

This is in contrast to standard ESM optimization, in which the initial values are adjusted during the optimization process to minimize error. Optimizing only the level, trend, and seasonality parameters rather than the initial values can result in significant performance improvements and faster optimization convergence. When domain knowledge indicates that long seasonal variation is a significant contributor to an accurate forecast, this approach is appropriate. Despite the performance benefits, Oracle does not recommend disabling the optimization of the initial values for typical short seasonal cycles because it may result in model overfitting and less reliable confidence bounds.

Related Topics

DBMS_DATA_MINING — Algorithm Settings:Exponential Smoothing

18.3 Multiple Time Series Models

Multiple time series is a convenience operation for constructing input to a time series regression. Multiple time series builds multiple time series models with a common time interval for use as input to a time series regression. One of the time series models is identified as the target time series of interest.

All of the time series output is produced for the target. The other time series are assumed to be correlated with the target.

This operation produces backcasts and forecasts on each time series and computes upper and lower confidence bounds for the identified target series. This operation can be used to forecast a wide variety of events, such as rainfall, sales, and customer satisfaction.

In the example of weather forecasting, the temperature and humidity attributes can be considered as the dependent or correlated time series and rainfall can be identified as the target time series.

Related Topics

Model Detail Views for Exponential Smoothing

18.3.1 Backcasts in Time Series

In the rainfall, temperature, and humidity multiple time series example,backcast are the estimate produced by the model for historical data.

For example, if rainfall is dependent on humidity, then it is useful to have a value of humidity for the period of interest. For periods that have already occurred and are being used to construct the model, such as last week, it is necessary to have the humidity from last week and not from last month.

18.3.2 How to Build Multiple Time Series Models

Oracle's exponential smoothing is enhanced to handle the building of multiple time series models with a single call to the model build method, in addition to single time series forecasting. Multiple time series is built by specifying a series list EXSM_SERIES_LIST. The rest of the parameters are the same as in ESM model.

In the weather forecast example, you can have a build data set and a score data set. The build data set contains the identified target series (rain), the dependent series - temperature and humidity. The DM$VP model detail view is used to display a forecast for the identified target series (rain), along with dependent series: temperate, and humidity. The DM$VR model detail view is used to display backcasts for target series (rain), humidity, and temperature. The backcasts and forecasts of the time series model can be fed into a regression technique like generalized linear model, neural network, or XGBoost for time series regression.

The sample code in the example uses Stock market data that you can download from https://github.com/oracle-samples/oracle-db-examples/blob/main/machine-learning/sql/23c/oml4sql-time-series-regression-dataset.sql and run it.

In the following example, the target attribute DAX is a Stock market index that is being forecast. The dependent attributes that are also popular stock market indexes - SMI, CAC, FTSE are passed as multiple series attributes. Exponential Smoothing settings are used to build a multiple time series model by specifying a series list (EXSM_SERIES_LIST) with multiple attributes.

Build a multiple time series model.

DECLARE
    v_setlst DBMS_DATA_MINING.SETTING_LIST;
BEGIN
    v_setlst('ALGO_NAME')            := 'ALGO_EXPONENTIAL_SMOOTHING';
    v_setlst('EXSM_INTERVAL')        := 'EXSM_INTERVAL_DAY';
    v_setlst('EXSM_MODEL')           := 'EXSM_ADDWINTERS_DAMPED';
    v_setlst('EXSM_SEASONALITY')     := '7';
    v_setlst('EXSM_PREDICTION_STEP') := '1';
    v_setlst('EXSM_SERIES_LIST')     := 'SMI,CAC,FTSE';
      
    dbms_data_mining.create_model2(
               MODEL_NAME          => 'MSDEMO_MODEL',
               MINING_FUNCTION     => 'TIME_SERIES',
               DATA_QUERY          => 'SELECT * FROM EUSTOCK',
               CASE_ID_COLUMN_NAME => 'DATES',
               TARGET_COLUMN_NAME  => 'DAX',
               SET_LIST            => v_setlst);
END;
/

Use the DM$VPMSDEMO_MODEL view to display the forecast.

SELECTCASE_ID, VALUE, PREDICTION, UPPER, LOWERFROMDM$VPMSDEMO_MODEL;

Use the DM$VRMSDEMO_MODEL view to display the backcasts.
```
SELECT * FROM DM$VRMSDEMO_MODEL
FETCH FIRST 10 ROWS ONLY;
```
The output of the this model is used in time series regression.

18.4 Time Series Regression

Time series regression is possible with the multi-series build. Time series regression expands the features that can be included in a time series model and possibly improves forecast accuracy. Some of the additional features can be other time series that are thought to be related or dependent to the "target" series.

Temperature and humidity are both dependent time series with rainfall, so by looking at historical data for these two attributes, we can make predictions about future rainfall. When the temperature is high and the humidity is high, there is a greater chance of rainfall.

A time series regression model will take into account the relationship between temperature and humidity, as well as other factors (for example, the location and elevation of the forecast location). The model then produces a prediction for the amount of rainfall (the target series), along with upper and lower bounds. For example, if the model predicts that there is a 90% chance of rain, and the upper bound for the amount of rainfall is 1 inch, then you might want to make sure that you have enough rain gear on hand.

Backcasts can be used to possibly improve the accuracy of forecasts for future time periods. The challenge with using regression to forecast is that the predictors' future values must be given. If, for example, temperature and humidity are the predictors, you need to know their future values on the same time scale as the rainfall series to make a forecast.

Related Topics

Model Detail Views for Exponential Smoothing

18.4.1 How to Build Time Series Regression Models

Oracle exponential smoothing solves the problem of knowing future values on the same time scale as the target series by forecasting the predictor time series using exponential smoothing.

To build a regression model that predicts a future period, the correlated series must have a value in that future period. Hence, all correlated series must be forecast. Backcasts are included for the correlated series as smoothed versions of the correlated series values that can be used as input to the regression model. Backcasts are also available for the target series, as these are part of the standard output of an Oracle machine learning time series model. Target series backcasts can also be included in the regression model.

You can also create build and score datasets. The build data set contains the target series (forecast series), for example, rain; the backcasted target series, for example, backcasted rain; and the backcasted dependent series, for example, backcasted temperature and humidity. The backcasts and forecasts of the time series models can both be used as input to the regression model. The series all use the same time periods, so that the values of the target and the predictors co-occur.

The score data set follows the same schema as the build data set but provides forecasts as required for future values. The score data set can be supplied to the apply procedure of the regression model. Backcasts can be smoother and more structurally consistent with forecasts. The incremental improvement of the regression model over the baseline model can be seen in the backcast of the target series.

Because of the database's versatility, different time series regression variations are possible. A user can add factors such as holidays and environmental changes to the build and score data sets that account for categorical variables. In multiple time series regression, flag variables can be used to account for events or conditions that may have a significant impact on the dependent variable. For example, you might use a flag variable to indicate whether a particular day is a public holiday, or whether a particular month is a winter month. The inclusion of such factors in the model can improve the accuracy of the forecast by accounting for the impact of categorical variables on the dependent variable.

Holidays can be expressed as a binary value column (0s and 1s). For example, a national_holiday column can be made that has a value of 1 for national holidays and a value of 0 at other times. In a demand forecast, a perceived change in the environment, like the introduction of a competitor's product, can also be shown as a binary value column, with 0 for times before the introduction and 1 for times after.

Furthermore, as a special case, if a user happens to know the future values of the dependent series, a user could replace the backcasts with the original values in the regression build procedure by creating a data set that joins to the original build table. This user-created data set replaces the build data set.

In the following example, a training, actual, and a test data sets are created using the stock market data. A special case of actual values are provided in the prediction data set to compare the accuracy of ESM and regression. The variable prod is a flag variable that accounts for categorical values. It indicates a change in the environment such as an introduction of a new product. The DM$VR<model_name> model detail view provides details of the time series regression build schema or the forecast of the target column.

Create a build/training data set.

BEGIN DROP TABLE tmesm_ms_train;
EXCEPTION WHEN OTHERS THEN NULL; END;
/
CREATE TABLE tmesm_ms_train as
SELECT CASE_ID, DAX, DM$DAX, DM$SMI, DM$CAC, DM$FTSE,
       CASE WHEN case_id < to_date('1998-02-03','YYYY-MM-DD')
       THEN 0 ELSE 1 END AS prod
FROM DM$VRMSDEMO_Model order by 1;

Create an actual data set (this is the special case scenario where the future values of dependent series is known).

BEGIN EXECUTE IMMEDIATE 'DROP TABLE tmesm_ms_actual';
EXCEPTION WHEN OTHERS THEN NULL;
END;
/
CREATE TABLE tmesm_ms_actual (case_id DATE, DAX binary_double);
INSERT INTO tmesm_ms_actual VALUES(DATE '1998-02-04', 4633.008);
commit;

Query the table to see the output:
```
select * from tmesm_ms_actual;
```
CASE_ID and DAX value 4633.00 is displayed.

Create a test data set.

BEGIN EXECUTE IMMEDIATE 'DROP TABLE tmesm_ms_test';
EXCEPTION WHEN OTHERS THEN NULL; END;
CREATE TABLE tmesm_ms_test as
SELECT a.case_id, b.DAX, DM$DAX, DM$SMI, DM$CAC, DM$FTSE, 1 prod
FROM   DM$VTMSDEMO_model a, tmesm_ms_actual b
WHERE  a.case_id=b.case_id;

The output displays that the procedure completed successfully and a tmesm_ms_test table is created.

Create a GLM time series regression model using the training data set.

SET echo OFF;
BEGIN DBMS_DATA_MINING.DROP_MODEL('MS_GLM_MODEL');
EXCEPTION WHEN OTHERS THEN NULL; END;
/
SET echo ON;
DECLARE
    v_setlst DBMS_DATA_MINING.SETTING_LIST;
BEGIN
    v_setlst('ALGO_NAME')          := 'ALGO_GENERALIZED_LINEAR_MODEL';
     
    DBMS_DATA_MINING.CREATE_MODEL2(
               MODEL_NAME          => 'MS_GLM_MODEL',
               MINING_FUNCTION     => dbms_data_mining.regression,
                DATA_QUERY         => 'SELECT * FROM tmesm_ms_train',
               CASE_ID_COLUMN_NAME => 'CASE_ID',
               TARGET_COLUMN_NAME  => 'DAX',
               SET_LIST            => v_setlst);   
END;
/

Analyze your model by viewing model detail views.

SELECT VIEW_NAME, VIEW_TYPE
FROM   USER_MINING_MODEL_VIEWS
WHERE  MODEL_NAME='MSDEMO_MODEL'
ORDER BY VIEW_NAME;

View the backcasts.

SELECT *
FROM   DM$VRMSDEMO_MODEL
ORDER BY CASE_ID
FETCH FIRST 10 ROWS ONLY;

View the forecast.
```
SELECT *
FROM DM$VTMSDEMO_MODEL
ORDER BY CASE_ID
FETCH FIRST 10 ROWS ONLY;
```
Further, you may compare the baseline (ESM) forecast with that of the regression forecast.

You can view the complete example by accessing oml4sql-time-series-regression.sql from https://github.com/oracle-samples/oracle-db-examples/tree/main/machine-learning/sql/23c.

Related Topics

Model Detail Views for Exponential Smoothing