Understanding Time Series Structural Changes

Author:Murphy | View: 21097 | Time: 2025-03-23 11:34:01

In this article, we:

Define what time series structural changes are and what distinguishes them from outliers.
Overview different types of structural changes.
Explore change point detection methods, such as CUSUM, using the kats and ruptures packages.

Introduction

Stationarity is a central concept in time series analysis and Forecasting. Under stationarity, the properties of time series, such as the mean value, remain the same over time, apart from spurious fluctuations.

Yet, stationary is rarely observed in real-world datasets. Time series are amenable to structural breaks or changes. These introduce non-stationary variations into a time series, altering its distribution. The timestep that marks the onset of a change is referred to as a change point.

Detecting structural changes is valuable in Time Series Analysis and forecasting. The emerging distribution often renders past data obsolete, and consequently, the models fit therein. This requires you to update your models using recent data or other appropriate strategy. If change points occur in historical data, you can deal with them with feature engineering approaches such as intervention analysis.

Note that structural changes are not the same thing as outliers. An outlier is an observation that is significantly different from normal behavior, where normal behavior is defined by the underlying time series structure or distribution. Change points alter the structure of the time series altogether to a different distribution.

Let's learn more about what kind of structural changes exist.

Types of structural changes

There are different types of change. Understanding the nature of a change is important to know how to tackle them.

Level changes

Changes can occur in the mean or the variance of the series. Changes in the mean are described by shifts in the average value, which can be either permanent or transitory.

Permanent level shifts occur if the series' average changes permanently to a different value. For example, suppose a retail business opens a new store. Then, the sales time series will shift to a new level reflecting the increased customer base.

Example of a permanent level shift. Image by author.

Level shifts are transitory when the average value returns to the original level after some period. Suppose a machine in a manufacturing plant is shut down for maintenance. The production time series will experience a lower value during the maintenance period. But, we expect the production to go back to normal once that machine goes back into operation.

Example of a transitory level shift. Image by author.

Variance changes

If variance changes, this leads to an increased or decreased dispersion. Time series with a non-constant variance are also referred to as heteroskedastic. You can check a previous article to learn how to deal with this issue.

Changes in the variance can also be permanent or transitory.

Example of a permanent change in variance. Image by author.

Recurrent changes in variance are a symptom of volatility clustering. Volatility clustering occurs when a time series exhibits periods of low variance followed by periods of high variance. In other words, there are periods where large (small) changes are followed by more large (small) changes.

Here's an example of a time series representing hourly wind speed changes:

Example of a time series with volatility clustering. Image by author.

Volatility clustering is common in domains like climate (e.g. wind speed) or finance (e.g. asset prices).

Hands-on: Change point detection using Python

Change point detection methods aim to identify the point at which the distribution of time series has changed.

In the rest of this article, we'll explore different change point detection methods.

Let's start by loading a time series from datasetsforecast:

from datasetsforecast.m4 import M4

dataset, *_ = M4.load('./data', 'Monthly')

series = dataset.query(f'unique_id=="M1430"').reset_index(drop=True)
series['time'] = pd.date_range(end='2023-12-01', periods=series.shape[0], freq='M')

This dataset (source in reference [4]) was also used above for the permanent level shift example.

Using kats

kats is a popular time series library developed by Meta. It contains several change detection methods, such as CUSUM, or Bayesian online changepoint detection.

kats requires a specific data structure that you can set as follows:

from kats.consts import TimeSeriesData

ts = TimeSeriesData(df=series[['time', 'y']])

Now, we're ready to do change detection.

One of the most popular methods for detecting changes in the mean value of time series is CUSUM (short for cumulative sum). CUSUM works by computing the cumulative sum of deviations relative to the expected value. If there is no change in the mean of the series, the cumulative sum will exhibit random variations centered around zero. A positive (or negative) change in the cumulative sum indicates an upward (or downward) shift in the mean.

Here's how to use the CUSUM method available in kats:

from kats.tests.detectors.test_cusum_detection import CUSUMDetector

model = CUSUMDetector(ts)
change_points = model.detector(direction=['decrease', 'increase'])
model.plot(change_points)

The detector method takes a direction argument that controls the direction of the change. In this case, we want to monitor the series for both upward and downward shifts in the mean.

CUSUM does an accurate detection of the change point:

Detecting changes using CUSUM from kats. Image by author.

A similar approach to CUSUM is to compute z-scores in a moving average. This is done, for example, by the RobustStatDetector from kats:

from kats.tests.detectors.test_robust_stat_detection import RobustStatDetector

model = RobustStatDetector(ts)
change_points = model.detector(p_value_cutoff=0.001, comparison_window=12)
model.plot(change_points)

In this method, we set the p-value threshold to 0.001. We also set the window size to 12 – this amounts to one year of data since the series is collected monthly.

Detecting changes using RobustStatDetector from kats. Image by author.

RobustStatDetector detects several change points close to each other.

Using ruptures

The ruptures library also contains the implementation of several change point detection methods. These include PELT (Pruned Exact Linear Time), kernel-based, or dynamic programming algorithms.

Here's an example of how to use PELT to detect changes in the mean:

import ruptures as rpt

# converting the time series into a numpy array
signal = series['y'].values

# fitting the PELT model
model = rpt.Pelt(model="rbf").fit(signal)

# getting the change points
result = model.predict(pen=10)

PELT works by segmenting a time series into different blocks that exhibit different statistical properties (e.g. mean). It does so by minimizing a user-defined cost function that depends on what change you're trying to detect. In this case, it uses a model based on a radial basis function (model="rbf").

Here's the result of the model:

Detecting changes using PELT from ruptures. Image by author

How do you deal with changes?

So, how can you deal with change points?

We explored in a previous article how to handle changes in variance.

Differencing the time series can be a simple remedy to handle changes in the mean. This operation helps stabilize the average level by computing the difference between consecutive observations.

Here are the first differences of the example series:

Differenced time series. Image by author.

You can also try to add a dummy binary variable that takes the value of 0 before change, and 1 afterward.

Adding a dummy variable to account for the change point. Image by author.

Transitory changes can be modeled using regime-switching models – models that take different parameters depending on the current structure or regime. An instance of a regime-switching model is threshold auto-regression (also known as TAR). TAR models have different parameters for different time series value ranges.

Key Takeaways

Time series are amenable to structural changes that alter their distribution
Changes can occur in the mean or variance. They can also be permanent or transitory
You can detect changes using several methods, such as CUSUM or PELT. These are available in packages like kats or ruptures
Data preprocessing is often used to stabilize changes in variance. Differencing or feature engineering can be used to deal with changes in the mean level

Thanks for reading and see you in the next story!

References

[1] Kats library documentation

[2] ruptures library documentation

[3] Bai, Jushan, and Pierre Perron. 2003. "Computation and Analysis of Multiple Structural Change Models." Journal of Applied Econometrics 18 (1): 1–22.

[4] M4 Dataset retrieved from datasetsforecast (MIT license)

Tags: Artificial Intelligence Data Science Forecasting Time Series Analysis Time Series Forecasting