Time Series Similarity Search with aeon¶

Similarity search answers a simple question: “given this time series, which other series – or pieces of series – look most like it?” It is the building block behind finding repeated patterns (motifs), spotting unusual segments (anomalies), and nearest-neighbour classifiers. For example, given a single heartbeat you could search a long recording for every beat that resembles it, or search a library of recordings for the patients whose signal looks most like it.
The similarity_search module provides these searches through a familiar, scikit-learn-style fit / predict interface:
``fit(X)`` indexes a 3D collection of series, of shape
(n_cases, n_channels, n_timepoints).``predict(q, k=…)`` takes a query and returns the
kclosest matches together with their distances.
There are two flavours of search, and picking between them is your first decision:
Search type |
What it looks for |
Query length |
|---|---|---|
``subsequence`` |
the best-matching window anywhere inside the indexed series |
shorter than the series |
``whole_series`` |
the indexed series that, as a whole, look most like the query |
same length as the series |
This notebook introduces both, starting from the simplest exact method and building up to a fast approximate index, and ends with a cheat-sheet of which estimator to reach for.
Other similarity search notebooks¶
This notebook is an overview of the module and its estimators. Three companion notebooks go deeper:
The theory and math behind the similarity search estimators in aeon
The mechanics of SimHash and its use as an approximate index
Analysis of the performance and speed-ups of the estimators – including how to tune the approximate index for accuracy vs query speed.
1. Setup and Data¶
First, let’s import the estimators and load some example data, then define a couple of plotting helpers used throughout the notebook.
[1]:
# Imports
from aeon.datasets import load_arrow_head
from aeon.similarity_search.subsequence import MASS, NaiveSubsequenceSearch
from aeon.similarity_search.whole_series import NaiveSeriesSearch
X, _ = load_arrow_head()
print("Collection shape (fit input):", X.shape)
# Create a query series: 1 channel, 50 timepoints
q = X[0, :, 100:150] # Extract a short query from first series
print("Query shape (predict input):", q.shape)
Collection shape (fit input): (211, 1, 251)
Query shape (predict input): (1, 50)
The next cell just defines two helper functions for plotting search results. You can run it and move on without reading it – it is not part of the module’s API, only of this notebook.
[2]:
import matplotlib.pyplot as plt
import numpy as np
def plot_subsequence_search_results(
X, query, matches, distances, title="Subsequence Search Results"
):
"""
Plot the results of a subsequence similarity search.
Parameters
----------
X : np.ndarray, shape (n_cases, n_channels, n_timepoints)
The fitted collection of time series.
query : np.ndarray, shape (n_channels, query_length)
The query subsequence.
matches : np.ndarray, shape (n_matches, 2)
Match indices as (case_idx, timestamp) pairs.
distances : np.ndarray, shape (n_matches,)
Distances to each match.
title : str
Title for the plot.
"""
n_matches = len(matches)
query_length = query.shape[1]
fig, axes = plt.subplots(n_matches + 1, 1, figsize=(12, 2.5 * (n_matches + 1)))
if n_matches == 0:
return
# Plot query
axes[0].plot(query[0], "b-", linewidth=2, label="Query")
axes[0].set_title(f"{title} - Query (length={query_length})")
axes[0].set_xlabel("Time")
axes[0].legend()
axes[0].grid(True, alpha=0.3)
# Plot each match
colors = plt.cm.tab10(np.linspace(0, 1, n_matches))
for i, ((case_idx, timestamp), dist) in enumerate(zip(matches, distances)):
ax = axes[i + 1]
series = X[case_idx, 0]
# Plot full series in gray
ax.plot(series, "gray", alpha=0.5, linewidth=1, label="Full series")
# Highlight the matched subsequence
match_end = timestamp + query_length
ax.plot(
range(timestamp, match_end),
series[timestamp:match_end],
color=colors[i],
linewidth=2.5,
label=f"Match (dist={dist:.4f})",
)
# Mark start and end points
ax.axvline(x=timestamp, color=colors[i], linestyle="--", alpha=0.7)
ax.axvline(x=match_end - 1, color=colors[i], linestyle="--", alpha=0.7)
ax.set_title(f"Match {i+1}: Case {case_idx}, Timestamp {timestamp}")
ax.set_xlabel("Time")
ax.legend(loc="upper right")
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
def plot_whole_series_search_results(
X, query, matches, distances, title="Whole Series Search Results"
):
"""
Plot the results of a whole series similarity search.
Parameters
----------
X : np.ndarray, shape (n_cases, n_channels, n_timepoints)
The fitted collection of time series.
query : np.ndarray, shape (n_channels, n_timepoints)
The query series.
matches : np.ndarray, shape (n_matches,)
Indices of matched cases.
distances : np.ndarray, shape (n_matches,)
Distances to each match.
title : str
Title for the plot.
"""
n_matches = len(matches)
fig, axes = plt.subplots(1, n_matches + 1, figsize=(4 * (n_matches + 1), 3))
if n_matches == 0:
return
# Plot query
axes[0].plot(query[0], "b-", linewidth=2)
axes[0].set_title("Query")
axes[0].set_xlabel("Time")
axes[0].grid(True, alpha=0.3)
# Plot each match
colors = plt.cm.tab10(np.linspace(0, 1, n_matches))
for i, (case_idx, dist) in enumerate(zip(matches, distances)):
ax = axes[i + 1]
ax.plot(X[case_idx, 0], color=colors[i], linewidth=2)
ax.set_title(f"Match {i+1}: Case {case_idx}\nDist={dist:.4f}")
ax.set_xlabel("Time")
ax.grid(True, alpha=0.3)
fig.suptitle(title, fontsize=12, fontweight="bold")
plt.tight_layout()
plt.show()
2. Subsequence Search¶
Subsequence search slides the query across every series in the collection and returns the windows that match it best – useful when the pattern you care about is shorter than, and located somewhere inside, your series. predict returns a 2D array of shape (n_matches, 2), where each row is a (case_index, timestamp_index) pair locating a match, together with the matching distances.
2.1 Naive search¶
NaiveSubsequenceSearch is the simplest method: it computes the distance between the query and every subsequence in the collection, then keeps the k closest. By default the distance is the squared Euclidean distance (no square root is taken), so the values reported below are squared Euclidean distances. The distance parameter accepts any distance from aeon.distances (by name or as a callable), with extra arguments passed through distance_params, e.g.
distance="dtw", distance_params={"window": 0.2}. It is exact and easy to reason about, but it scans everything.
``normalize=True`` matches shape rather than absolute level: each window is z-normalized (zero mean, unit variance) before the distance is computed, so a pattern still matches even if it is shifted up/down or rescaled. Use
normalize=Falseto match raw amplitudes. In the cell below the query is a slice taken from the collection itself, so its best match is that exact window – which is why the top distance is0.
[3]:
# Fit NaiveSubsequenceSearch, use the length of q (50)
subseq_naive = NaiveSubsequenceSearch(length=50, normalize=True)
subseq_naive.fit(X) # Fit on 3D collection
# Predict with k=3 to find top 3 closest subsequences
# Returns (indices, distances) where indices has shape (n_matches, 2)
# with (case_idx, timestamp)
matches, distances = subseq_naive.predict(q, k=3)
print("NaiveSubsequenceSearch matches shape:", matches.shape)
print("Matches (case_idx, timestamp):")
print(matches)
print("\nDistances:")
print(distances)
NaiveSubsequenceSearch matches shape: (3, 2)
Matches (case_idx, timestamp):
[[ 0 100]
[ 30 97]
[ 48 97]]
Distances:
[0. 7.53827418 9.23849797]
[4]:
# Visualize NaiveSubsequenceSearch results
plot_subsequence_search_results(
X, q, matches, distances, title="Naive Subsequence Search"
)
2.2 MASS (FFT-based)¶
MASS computes the same exact distances as ``NaiveSubsequenceSearch`` with its default squared Euclidean distance – so it returns the identical matches you just saw – but far faster: it evaluates the distance to all subsequences at once using a Fast Fourier Transform, instead of one window at a time. The speed-up grows with the series and query length (the performance notebook quantifies it). The trade-off is that the FFT trick is specific to the Euclidean
distance, so MASS has no distance parameter.
Rule of thumb: for exact subsequence search with the (squared) Euclidean distance, use MASS by default, and reach for NaiveSubsequenceSearch as a reference, for very small inputs, or when you need a different distance. The two cells below run the same search and return the same result.
[5]:
# MASS is an FFT-based algorithm for fast subsequence search
# length=50 matches the query length
mass = MASS(length=50, normalize=True)
mass.fit(X) # Fit on 3D collection
# Predict with k=3 to find top 3 closest subsequences
# Returns (indices, distances) where indices has shape (n_matches, 2)
# with (case_idx, timestamp)
matches, distances = mass.predict(q, k=3)
print("MASS matches shape:", matches.shape)
print("Matches (case_idx, timestamp):")
print(matches)
print("\nDistances:")
print(distances)
MASS matches shape: (3, 2)
Matches (case_idx, timestamp):
[[ 0 100]
[ 30 97]
[ 48 97]]
Distances:
[0. 7.53827418 9.23849797]
[6]:
# Visualize MASS subsequence search results
plot_subsequence_search_results(
X, q, matches, distances, title="MASS Subsequence Search"
)
3. Whole Series Search¶
Whole series search estimators find the most similar complete time series within a collection. Unlike subsequence search, the query must have the same length as the series in the collection. The predict method returns two 1D arrays of shape (n_matches,): indexes, the indices of the most similar cases, and distances, the corresponding distances (i.e. matches_whole, distances_whole = naive_whole.predict(...)).
3.1 Naive search (Exact)¶
NaiveSeriesSearch computes exact nearest neighbors, optionally on z-normalized series via normalize=True. Like NaiveSubsequenceSearch, it uses the (squared) Euclidean distance by default, and any distance from aeon.distances (or a callable) can be selected through the distance and distance_params parameters. As in the subsequence case the default distances are squared (no square root is taken), so the values reported below are squared Euclidean distances.
[7]:
# Query is a single 2D series (same length as collection series)
q_whole = X[3] # Use one series from collection as query
print("Query shape:", q_whole.shape)
# Fit and predict with NaiveSeriesSearch whole series search
naive_whole = NaiveSeriesSearch()
naive_whole.fit(X)
matches_whole, distances_whole = naive_whole.predict(q_whole, k=3)
print("\nWhole series matches (case indices):", matches_whole)
print("Distances:", distances_whole)
Query shape: (1, 251)
Whole series matches (case indices): [ 3 24 102]
Distances: [0. 5.8934335 5.99519587]
[8]:
# Visualize whole series NaiveSeriesSearch results
plot_whole_series_search_results(
X, q_whole, matches_whole, distances_whole, title="Naive Whole Series Search"
)
3.2 SimHashIndexANN (Approximate)¶
The exact NaiveSeriesSearch above compares the query against every series, which gets slow on large collections. SimHashIndexANN trades a little accuracy for a lot of speed using Locality-Sensitive Hashing (LSH).
The intuition: give every series a short fingerprint such that similar series tend to get the same fingerprint. At query time you then only compare against the series sharing the query’s fingerprint, instead of the whole collection.
Concretely, the index uses SimHash – the sign of a few random projections – to hash each series into n_tables independent hash tables of n_bits_per_table bits each. To answer a query it:
looks up the query’s bucket in every table and gathers the series found there (the candidates);
ranks candidates by their collision count – the number of tables in which they share the query’s bucket (more shared buckets ⇒ more likely close);
returns the top
k, reporting the proxy distance1 / collision_count(smaller = collided in more tables).
Because no exact distance is ever computed and only a handful of candidates are touched, queries are fast – but results are approximate: a true neighbour is missed if it never lands in the query’s bucket, and ties in the collision count are broken arbitrarily. The matches below therefore need not equal the exact ones above. (The SimHash deep-dive notebook explains the mechanics in detail, and the performance notebook shows how n_tables
and n_bits_per_table trade recall against query speed.)
[9]:
from aeon.similarity_search.whole_series import SimHashIndexANN
# Same fit/predict interface as NaiveSeriesSearch, but the returned neighbors are
# APPROXIMATE and the reported distance is the proxy 1 / collision_count
# (explained above), so they need not match the exact matches.
simhash_index = SimHashIndexANN(random_state=0)
simhash_index.fit(X)
matches_sh, distances_sh = simhash_index.predict(q_whole, k=3)
print("approximate matches (case indices):", matches_sh)
print("proxy distance (1 / collision count):", distances_sh)
approximate matches (case indices): [ 3 24 40]
proxy distance (1 / collision count): [0.05 0.0625 0.06666667]
[10]:
# Visualize approximate (SimHash) search results
plot_whole_series_search_results(
X, q_whole, matches_sh, distances_sh, title="Approximate Whole Series Search"
)
4. Cheat-sheet: which estimator to reach for¶
Estimator |
Search type |
Exact / approximate |
Key parameters |
Reach for it when |
|---|---|---|---|---|
|
subsequence |
exact |
|
Default for exact subsequence search: same results as |
|
subsequence |
exact |
|
A reference / sanity check, very small inputs where MASS’s setup is not worth it, or when you
need a non-Euclidean |
|
whole series |
exact |
|
Whole-series nearest neighbours when the collection is small enough to scan exhaustively and you need exact distances. |
|
whole series |
approximate |
|
Large collections where exact |
The exact estimators use the (squared) Euclidean distance by default, optionally on z-normalized series via normalize=True. The two naive estimators accept any distance from aeon.distances through their distance and distance_params parameters, while MASS is tied to the Euclidean case by its FFT-based algorithm. SimHashIndexANN never computes an exact distance – it ranks candidates by collision count and reports the proxy distance 1 / collision_count.
Where to next?¶
Distance profiles – the theory and math behind these estimators.
SimHash deep dive – the mechanics of the approximate
SimHashIndexANNindex, from random projections to hash buckets.Performance & speed-ups – the internal optimizations, plus how to tune
SimHashIndexANN(n_tables,n_bits_per_table) and the float32/float64 trade-off between accuracy and query speed.
Generated using nbsphinx. The Jupyter notebook can be found here.