Storing data to use for aeon estimators

aeon includes time series forecasting and machine learning. These two communities have different conventions on how to store data and what to call data structures. Some of the differences are

1. Forecasters almost always stores data in pandas data structures, whereas machine learners use numpy arrays almost exclusively.

2. n forecasting a 2 dimensional data is almost always shape (n_timepoints, n_timeseries) whereas in machine learning we would tend to store data in a (n_timeseries, n_timepoints) array.

3. In forecasting, a variable y refers to a time series for which we are attempting to make a forecast, hence y is assumed to be ordered. In machine learning, y is a list of either class labels (for classification) or observations of a response vairable (for regression). The ordering of values in y is determined by the ordering of the X input.

Because of these sources of confusion, we recommend that you store data in pandas data structures for forecasting and numpy arrays for machine learning.

Forecasting data

aeon forecasting uses pd.Series, pd.DataFrame and pd.Multiindex to store data. It has some built in forecasting datasets and tools for downloading commonly used benchmarks, loading_data.ipynb forecasting section. For details of the forecasting functionality, see the numerous forecasting notebooks.

pd.Series are used to store a univariate time series with entries corresponding to different time points.

# Forecasting data in a pandas.Series
import numpy as np
import pandas as pd

from aeon.forecasting.trend import TrendForecaster

y = pd.Series([20.0, 40.0, 60.0, 80.0, 100.0])
forecaster = TrendForecaster()
forecaster.fit(y)  # fit the forecaster
forecaster.predict(fh=[1, 2, 3])  # forecast the next 3 values

pd.Series are used to store a univariate time series with entries corresponding to different time points.

pd.DataFrame are used to store multiple time series, where each column is a time series, and each row corresponds to a different, distinct time point. The index is the time point and should be monotonic. This creates two series called Sales and Temperature, and stores observations for time points 0,1,2,3,4,5.

ice_creams = {
    "Sales": [111, 100, 90, 80, 65, 89],
    "Temperature": [26, 21, 19, 14, 12, 22],
}
# Create DataFrame
ice_creams = pd.DataFrame(ice_creams)
print(ice_creams)
from aeon.forecasting.arima import ARIMA

forecaster = ARIMA()
forecaster.fit(ice_creams)
forecaster.predict(fh=[1, 2, 3])

   Sales  Temperature
0    111           26
1    100           21
2     90           19
3     80           14
4     65           12
5     89           22

	Sales	Temperature
6	90.419446	20.412514
7	91.108616	19.907821
8	91.443222	19.747369

You can add a date-time index, and this is required by some forecasters (e.g. Prophet).

ice_creams["datetime"] = pd.to_datetime(
    [
        "01-06-2018 23:15:00",  # Creating data
        "02-09-2019 01:48:00",
        "08-06-2020 13:20:00",
        "07-03-2021 14:50:00",
        "07-06-2022 11:50:00",
        "03-05-2023 16:50:00",
    ]
)
ice_creams = ice_creams.set_index("datetime")
print(ice_creams)

                     Sales  Temperature
datetime
2018-01-06 23:15:00    111           26
2019-02-09 01:48:00    100           21
2020-08-06 13:20:00     90           19
2021-07-03 14:50:00     80           14
2022-07-06 11:50:00     65           12
2023-03-05 16:50:00     89           22

pd.DataFrame also have the capability to store multiple indexes, which can be used to represent whats called Panel data in forecasting hierarchical data. A Panel is a collection of (possibly) multivariate data.

from aeon.utils._testing.hierarchical import _make_hierarchical

y = _make_hierarchical()
y.head()

			c0
h0	h1	time
h0_0	h1_0	2000-01-01	4.865249
		2000-01-02	5.238293
		2000-01-03	3.543190
		2000-01-04	4.600117
		2000-01-05	3.682226

forecaster.fit(y, fh=[1, 2]).predict()

			c0
h0	h1	time
h0_0	h1_0	2000-01-13	4.516463
		2000-01-14	4.306045
	h1_1	2000-01-13	4.485677
		2000-01-14	5.390378
	h1_2	2000-01-13	4.018436
		2000-01-14	3.979373
	h1_3	2000-01-13	4.437017
		2000-01-14	4.132266
h0_1	h1_0	2000-01-13	4.777346
		2000-01-14	4.632433
	h1_1	2000-01-13	3.594355
		2000-01-14	4.510663
	h1_2	2000-01-13	4.367914
		2000-01-14	4.317008
	h1_3	2000-01-13	4.536718
		2000-01-14	4.520627

np.ndarray can be used with the forecasters in aeon, although we recommend using pandas. One dimensional np.ndarray are treated as a single time series. 2D numpy array are treated as multiple series of shape (n_timeseries, n_timepoints). Forecasters fit independently on each series.

y = np.array([20.0, 40.0, 60.0, 80.0, 100.0])
forecaster = TrendForecaster()
forecaster.fit(y)  # fit the forecaster
forecaster.predict(fh=[1, 2, 3])  # forecast the next 3 values

array([[120.],
       [140.],
       [160.]])

y = np.array([[20.0, 40.0, 60.0, 80.0, 100.0], [100.0, 90.0, 80.0, 70.0, 60.0]])
y = y.transpose()
forecaster = TrendForecaster()
forecaster.fit(y)  # fit the forecaster
forecaster.predict(fh=[1, 2, 3])  # forecast the next 3 values

array([[120.,  50.],
       [140.,  40.],
       [160.,  30.]])

Machine learning data

Machine learning algorithms generally use collections of instances or cases stored as numpy arrays. Like scikit-learn, pytorch and keras, we primarily use numpy arrays. A collection contains a number of time series cases (or just cases) which we refer to in code as n_cases. Each case contains a number of time series observations, which we denote n_timepoints.

X = np.array(
    [
        [[20.0, 40.0, 60.0, 80.0, 100.0]],  # Univariate series as 3D array
        [[100.0, 90.0, 80.0, 70.0, 60.0]],
    ]
)  # n_cases = 2, n_channels =1, n_timepoints = 5
print("X shape = ", X.shape, " First series =", X[0], "second series = ", X[1])

X shape =  (2, 1, 5)  First series = [[ 20.  40.  60.  80. 100.]] second series =  [[100.  90.  80.  70.  60.]]

X = np.array(
    [
        [[20, 40, 600, 55], [10, 11, 12, 11], [-4, 1, 6.6, 2]],
        [[10, 90, 80, 100], [14, 70, 60, 22], [49, 49, 66, 9]],
        [[14, 6, 10, -401], [44, 70, 60, 22], [49, 52, 33, 49]],
        [[22, 93, 18, 100], [34, 170, 0, 87], [49, 49, 33, 49]],
    ]
)
# n_cases = 4, n_channels =3, n_timepoints = 4
print("X shape = ", X.shape, "\n First series =\n", X[0], "\nsecond series = \n", X[1])
from aeon.clustering.k_means import TimeSeriesKMeans

kmeans = TimeSeriesKMeans(metric="euclidean", n_clusters=2)
kmeans.fit(X)
kmeans.predict(X)

X shape =  (4, 3, 4)
 First series =
 [[ 20.   40.  600.   55. ]
 [ 10.   11.   12.   11. ]
 [ -4.    1.    6.6   2. ]]
second series =
 [[ 10.  90.  80. 100.]
 [ 14.  70.  60.  22.]
 [ 49.  49.  66.   9.]]

array([0, 1, 1, 1], dtype=int64)

The target variable for classification should be stored as a np.ndarray of integers or strings

y = np.array([1, 1, 0, 0])
y2 = np.array(["pass", "pass", "fail", "fail"])
from aeon.classification.distance_based import KNeighborsTimeSeriesClassifier

knn = KNeighborsTimeSeriesClassifier(distance="dtw")
knn.fit(X, y)
knn.fit(X, y2)
knn.predict(X)

array(['pass', 'pass', 'fail', 'fail'], dtype='<U4')

For regression, the target variable should be of type float

y = np.array([1.5, 4.3, -2.0, 10])
from aeon.regression.distance_based import KNeighborsTimeSeriesRegressor

knn_r = KNeighborsTimeSeriesRegressor(distance="dtw")
knn_r.fit(X, y)
knn_r.predict(X)

array([ 1.5,  4.3, -2. , 10. ])

If the time series are not all equal length, they should be stored as a list of 2D numpy arrays. Some estimators can deal with unequal length series. Those that can’t will raise an exception if passed unequal length series. Note we assume that channels are all the same length for any given series.

x0 = np.array([[20, 40, 60, 55, 66], [10, 11, 12, 11, 66], [-4, 15, 6.6, 12, 44]])
x1 = np.array([[10, 90, 80], [70, 60, 22], [49, 66, 9]])
x2 = np.array([[22, 93, 18, 100], [34, 170, 0, 87], [49, 49, 33, 49]])
X_uneq = []
X_uneq.append(x0)
X_uneq.append(x1)
X_uneq.append(x2)
y = np.array([0, 0, 1])
knn.fit(X_uneq, y)
knn.predict(X_uneq)

array([0, 0, 1])

aeon has several standard problems baked in, and facilities for loading data from external sources. Please see the data loading notebook

---

Generated using nbsphinx. The Jupyter notebook can be found here.