Where Convergent Cross-Mapping Fails

Where Convergent Cross-Mapping Fails#

October 13, 2024

Convergent Cross-mapping (CCM) is a technique for causal inference in dynamical systems. CCM claims to infer cause and effect directionality given two time series \(X\) and \(Y\). The 2012 paper of Sugihara et al introduced CCM and they showed that for a deterministic “coupled” chaotic system where \(X\) influences \(Y\) and vice versa by some degree, CCM is able to infer the strength of effect. (For a more in depth background about CCM and how it works, see Chapter 6 of our online Time Series Handbook.)

Hundreds of papers have since used CCM to infer causal direction in ecosystems and other natural systems. In our 2022 paper, we used CCM to infer possible causal relations among factors in a water dam system.

However, because CCM is built with the assumption of a determinstic coupled chaotic dynamical system, it’s natural to ask, “how applicable is CCM for time series data that come from systems that violate the assumption of CCM?”

In this article we investigate the ability of CCM to infer directionality between two observed time series variables on different classes of functions:

Deterministic Chaotic Systems - we investigate the effect of the presence of hidden confounding variables
Chaotic Systems with Random Noise - here, the system is no longer fully deterministic.
Additive Noise Models (ANMs) - standard model for Structural Causal Models of the form \(Y = f(X) + N_X\) where \(N_X\) is an independent random noise. We use non-linear Gaussian models. Linear Gaussian models have been shown to be unidentifiable (we cannot infer directionality of causation) unless the variances of the noise are all equal. (See the works of Scholkopf, Peters, Janzing, Pearl).

We summarize our findings below.

	variable relations	causal graph	\(X \rightarrow Y\)	\(Y \rightarrow X\)	\(Z \rightarrow X\)	\(Z \rightarrow Y\)	\(X \rightarrow Z\)	\(Y \rightarrow Z\)
1	chaotic (deterministic)	\(X \perp Y, Z_t \rightarrow X_{t+1}, Z_t \rightarrow Y_{t+2}\)	✔	✔	✔	✔	✔	✔
2	chaotic (deterministic)	\(X_t \rightarrow Y_{t+1}\), \(Y_t \rightarrow X_{t+1}, X_t \rightarrow X_{t+1}\), \(Y_t \rightarrow Y_{t+1}\)	✔	✔	NA	NA	NA	NA
3	chaotic (deterministic)	\(X_t \rightarrow Y_{t+1}\), \(Y_t \rightarrow X_{t+1}\)	✔	✔	NA	NA	NA	NA
4	chaotic (w/ noise)	\(X \perp Y, Z_t \rightarrow X_{t+1}, Z_t \rightarrow Y_{t+2}\)	✘	✔	✔	✔	✔	✘
5	chaotic (w/ noise)	\(X_t \rightarrow Y_{t+1}\), \(Y_t \rightarrow X_{t+1}, X_t \rightarrow X_{t+1}\), \(Y_t \rightarrow Y_{t+1}\)	✔	✔	NA	NA	NA	NA
6	chaotic (w/ noise)	\(X_t \rightarrow Y_{t+1}\), \(Y_t \rightarrow X_{t+1}\)	✔	✔	NA	NA	NA	NA
7	additive noise model	\(X \perp Y, Z_t \rightarrow X_{t+1}, Z_t \rightarrow Y_{t+2}\)	✔	✔	✘	✘	✘	✘
8	additive noise model	\(X_t \rightarrow Y_{t+1}\), \(Y_t \rightarrow X_{t+1}, X_t \rightarrow X_{t+1}\), \(Y_t \rightarrow Y_{t+1}\)	✘	✘	NA	NA	NA	NA
9	additive noise model	\(X_t \rightarrow Y_{t+1}\), \(Y_t \rightarrow X_{t+1}\)	✔	✔	NA	NA	NA	NA

We find that in the presence of a confounding variable in a chaotic system, CCM was able to infer the right causal directions.

However, once small Gaussian noise (standard deviation = 0.015) is added to the variables at each timestep, CCM fails to infer the right relationships between some variables (item 4 in the table)

Similarly, CCM fails to consistently infer the right causal relations if they are in the form of Additive Noise Models (ANMs).

These results tell us that we should be careful when using CCM especially when the relations among variables do not follow a fully deterministic chaotic system. (We have yet to check for more kinds of causal graphs to investigate if CCM fails for some causal graphs even if chaotic.)

Chaotic Models#

First we define our chaotic system builder. The chaotic function follows this form:

\(A = A (r - r A - \beta B)\)

Where \(\beta\) is the strength of effect of \(B\) on \(A\)

def chaotic_func(A, B, r, beta):
    return A * (r - r * A - beta * B)

Example 1#

We define a coupled chaotic function among three time series \(X\), \(Y\), and \(Z\). But only \(X\) and \(Y\) are observed. We visualize below the relationships among the variables. In this case, \(Z_t \rightarrow X_{t+1}\) and \(Z_t \rightarrow Y_{t+2}\) but there is no causal relation between \(X\) and \(Y\).

../_images/c5eaa6e58c14524f68d0a518e3bc43b515073967ce8bae9ed84d3023ebd139c9.png

We define a determinsitic coupled chaotic system as follows:

# Based on Supplementary Materials https://www.science.org/doi/10.1126/science.1227079
# params
r_x = 3.4
r_y = 3.6
r_z = 3.8
B_xy = 0.0 # effect on x given y (effect of y on x)
B_yx = 0.0 # effect on y given x (effect of x on y)
B_yz = 0.2 # effect of z on y
B_xz = 0.2 # effect of z on x

X0 = 0.3 # initial val following Sugihara et al
Y0 = 0.3 # initial val following Sugihara et al
Z0 = 0.3
t = 2000 # time steps

X = [X0, X0]
Y = [Y0, Y0]
Z = [Z0, Z0]
for i in range(t):
    Z_ = chaotic_func(Z[-1], 0, r_z, 0)    
    X_ = chaotic_func(X[-1], Z[-1], r_x, B_xz)
    Y_ = chaotic_func(Y[-1], Z[-2], r_y, B_yz)
    X.append(X_)
    Y.append(Y_)
    Z.append(Z_)

../_images/86a5a1dff7d0ad22560b5d4ab11f4c403f1782d84be390b7424ac4b7e0fe416c.png

Here, we see clearly that there is no convergence of cross-mapping correlations. In this case, CCM correctly inferred that there is no causal link between \(X\) and \(Y\).

../_images/e4e333ebae4f690a35ee7e474e6940c357c303236c6619ac2a74cafc52dbb648.png

Can CCM infer that \(Z \rightarrow X\)? We can see below that yes, it can.

../_images/f5924ac3c8dc07e6d569411a1fe35132107c8a36f0c18c9fd29c11b4b71a73a0.png

Similarly, CCM can infer that \(Z \rightarrow Y\)

../_images/b5fe3c00ec4d8e947fe641bf372a07e4fee7ec8bdbdf947ac5aa31fa94cb4b3b.png

Example 2#

In this case, \(X_t \rightarrow Y_{t+1}\) and \(Y_t \rightarrow X_{t+1}\) but also \(X_t \rightarrow X_{t+1}\) and \(Y_t \rightarrow Y_{t+1}\). We make the effect of \(X\) on \(Y\) stronger than \(Y\) on \(X\).

../_images/adae8b78290dba5960714b41a11454648bd712cbe1350f07e708c70ec3149587.png

# params
r_x = 3.4
r_y = 3.6
B_xy = 0.1 # effect on x given y (effect of y on x)
B_yx = 0.4 # effect on y given x (effect of x on y)

X0 = 0.3 # initial val following Sugihara et al
Y0 = 0.3 # initial val following Sugihara et al
t = 2000 # time steps

X = [X0]
Y = [Y0]
for i in range(t):
    X_ = chaotic_func(X[-1], Y[-1], r_x, B_xy)
    Y_ = chaotic_func(Y[-1], X[-1], r_y, B_yx)
    X.append(X_)
    Y.append(Y_)

../_images/c1f2cde7020e6e60010dc455838ef705359605f3ac8275b2863481a62c471046.png

Here we find that CCM was able to correctly infer the feedback loop and also the order of strength of effect, where \(X\) has stronger effect on \(Y\)

../_images/48dfa6cadf35115f75ec3ac3cee51d62cf3bda4b7eecd9f66e539580716e56aa.png

Example 3#

Here, we keep \(X_t \rightarrow Y_{t+1}\) and \(Y_t \rightarrow X_{t+1}\) but we remove \(X_t \rightarrow X_{t+1}\) and \(Y_t \rightarrow Y_{t+1}\). We make the effect of \(X\) on \(Y\) stronger than \(Y\) on \(X\).

../_images/4c377dc27f971a4d06421487a095e4680738c5e8abe14a4fbacc47159f77824b.png

# params
r_x = 3.4
r_y = 3.6
B_xy = 0.1 # effect on x given y (effect of y on x)
B_yx = 0.4 # effect on y given x (effect of x on y)

X0 = 0.3 # initial val following Sugihara et al
Y0 = 0.3 # initial val following Sugihara et al
t = 2000 # time steps

X = [X0]
Y = [Y0]
for i in range(t):
    X_ = chaotic_func(Y[-1], Y[-1], r_x, B_xy)
    Y_ = chaotic_func(X[-1], X[-1], r_y, B_yx)
    X.append(X_)
    Y.append(Y_)

../_images/ed72507b9e06a622615ffd2f5d6c7f5bd2d50a622d55e859ccdabdb9373ccd29.png

CCM was able to infer the feedback loop between the two, but was unable to correctly order the causal relation by strength.

../_images/93b86ac5de917c0cdd2011b3b7047491afd9a93b97dedbe1c6bac07f2cf43462.png

Chaotic Models + Noise#

We conduct the same experiments as above but we add independent random Gaussian noise on each variable.

Example 1#

Again, we define a coupled chaotic function among three time series \(X\), \(Y\), and \(Z\). But only \(X\) and \(Y\) are observed. We visualize below the relationships among the variables. In this case, \(Z_t \rightarrow X_{t+1}\) and \(Z_t \rightarrow Y_{t+2}\) but there is no causal relation between \(X\) and \(Y\).

On top of this, we add random Gaussian noise per variable.

We use the same determinsitic coupled chaotic system as above, but we add a small Gaussian noise.

# params
r_x = 3.4
r_y = 3.6
r_z = 3.8
B_xy = 0.0 # effect on x given y (effect of y on x)
B_yx = 0.0 # effect on y given x (effect of x on y)
B_yz = 0.2 # effect of z on y
B_xz = 0.2 # effect of z on x

np.random.seed(42)
X_std = 0.015
Y_std = 0.015
Z_std = 0.015

X0 = 0.3 # initial val following Sugihara et al
Y0 = 0.3 # initial val following Sugihara et al
Z0 = 0.3
t = 2000 # time steps

X = [X0, X0]
Y = [Y0, Y0]
Z = [Z0, Z0]
for i in range(t):
    Z_ = chaotic_func(Z[-1], 0, r_z, 0) + np.random.normal(0, Z_std)   
    X_ = chaotic_func(X[-1], Z[-1], r_x, B_xz) + np.random.normal(0, X_std)   
    Y_ = chaotic_func(Y[-1], Z[-2], r_y, B_yz) + np.random.normal(0, Y_std)   
    X.append(X_)
    Y.append(Y_)
    Z.append(Z_)

../_images/f53e0a582a81fa506fff769bee122efcd88206b8196423677bd168a9365caec2.png

Once we added noise, CCM incorrectly infers that \(X \rightarrow Y\). So now we see that when the dynamical system is not perfectly determinsitic, CCM fails. (Even when the Gaussian noise has a small standard deviation = 0.015). This may be a problem since we expect real world dynamical systems to have some randomness.

../_images/708a20a9519d4ef7bd730927eb09e08907582fa810731e0bb56c10dfe82b737a.png

Can CCM infer that \(Z \rightarrow X\)? We can see below that yes, it still can.

../_images/a51b389d965db44d4645aa96e209b654e3b04f675b5ba88426ce7653b63a43d1.png

Similarly, CCM can still infer that \(Z \rightarrow Y\). But it seems that CCM infers a small \(Y \rightarrow Z\) which is incorrect.

../_images/cc90c4ab85639658569d638e559f9c261738d05bd05eaa33983b6dd124685653.png

Example 2#

# params
r_x = 3.4
r_y = 3.6
B_xy = 0.1 # effect on x given y (effect of y on x)
B_yx = 0.4 # effect on y given x (effect of x on y)

np.random.seed(42)
X_std = 0.015
Y_std = 0.015

X0 = 0.3 # initial val following Sugihara et al
Y0 = 0.3 # initial val following Sugihara et al
t = 2000 # time steps

X = [X0]
Y = [Y0]
for i in range(t):
    X_ = chaotic_func(X[-1], Y[-1], r_x, B_xy) + np.random.normal(0, X_std)
    Y_ = chaotic_func(Y[-1], X[-1], r_y, B_yx) + np.random.normal(0, Y_std)
    X.append(X_)
    Y.append(Y_)

../_images/c79068361101dc72c0c517bcb3a49323818fb9b3b2cb241976e79ae0ec240645.png

Here we find that CCM was able to correctly infer \(X \rightarrow Y\) and some weaker \(Y \rightarrow X\)

../_images/b2a4f881cdb48f0748a3e11c3fb632f98c178475dbbc13b615cbf88c3d7d8b0f.png

Example 3#

# params
r_x = 3.4
r_y = 3.6
B_xy = 0.1 # effect on x given y (effect of y on x)
B_yx = 0.4 # effect on y given x (effect of x on y)

X_std = 0.015
Y_std = 0.015

X0 = 0.3 # initial val following Sugihara et al
Y0 = 0.3 # initial val following Sugihara et al
t = 2000 # time steps

X = [X0]
Y = [Y0]
for i in range(t):
    X_ = chaotic_func(Y[-1], Y[-1], r_x, B_xy) + np.random.normal(0, X_std)
    Y_ = chaotic_func(X[-1], X[-1], r_y, B_yx) + np.random.normal(0, Y_std)
    X.append(X_)
    Y.append(Y_)

../_images/1ced5bc2672f0f9b1eb72071455546fa9368f58e6d527a9b3828bc64ae4368c6.png

CCM was able to infer the feedback loop between the two, but was unable to correctly order the causal relation by strength.

../_images/0f888716fb78744614afcf59a6c34906d58c3a9d5c7327036d3976f3fd336ad9.png

Additive Noise Models#

Additive Noise Models (ANMs) follow this general mathematical form: \(Y = f(X) + N_X\) where \(N_X\) is independent noise.

Example 1#

We still follow the same causal graph as Example 1 in the chaotic function class above. However, instead of a chaotic relationship between variables, we use ANM formulation. Based on the insights above on chaotic models with noise (essentially it no longer becomes chaotic technically since chaotic systems are deterministic), we may hypothesize that CCM will fail on ANMs as well.

# create a time series SCM
np.random.seed(42)
L = 2000
alpha = 0.5
beta = 0.2
X_std = 0.1
Y_std = 0.1
Z_std = 0.1
X = [np.random.normal(0, X_std), np.random.normal(0, X_std)]
Y = [np.random.normal(0, Y_std), np.random.normal(0, Y_std)]
Z = [np.random.normal(0, Z_std), np.random.normal(0, Z_std)]
for t in range(L):
    X += [alpha * X[-1] + beta * Z[-1] + np.random.normal(0, X_std)]
    Y += [alpha * Y[-1] + beta * Z[-2] + np.random.normal(0, Y_std)]
    Z += [alpha * Z[-1] + np.random.normal(0, Z_std)]

../_images/7f8fa6f516756b4e814ce2318d8da5e0cc61650b21441554af8f6b37b3b7f6a0.png

Interestingly, CCM correctly infers that there is not causal link between \(X\) and \(Y\)

../_images/424f337a4b411552eae9cb1d3a2c13846e75b6d3239385807daef05fd3389971.png

However, CCM incorrectly infers a weak causal link \(X \rightarrow Z\) which is the opposite of the actual relation for embedding dimension, E=4. For E=2 and E=3, CCM does not infer any causal effect.

../_images/8c9d7cba8b28ffb2082ccbc912bb7934b449dad22640890af4b2d973e8d0d159.png

Similarly, CCM incorrectly infers a weak causal link \(Y \rightarrow Z\) for E=4, which is the opposite of the actual relation.

../_images/42633301d436326e9a7f046d3e177a4dc4f2f1b23cbb2f2622b7fa6e30bb14c0.png

Example 2#

Again this is the same causal graph as Example 2 above, but we use ANMs as the functional relations among variables instead of a chaotic function.

# create a time series SCM
np.random.seed(42)
L = 2000
alpha = 0.5
beta = 0.2
X_std = 0.1
Y_std = 0.1
X = [np.random.normal(0, X_std), np.random.normal(0, X_std)]
Y = [np.random.normal(0, Y_std), np.random.normal(0, Y_std)]
for t in range(L):
    X += [alpha * X[-1] + beta * Y[-1] + np.random.normal(0, X_std)]
    Y += [alpha * Y[-1] + beta * X[-1] + np.random.normal(0, Y_std)]

../_images/226cd3d2cf489f0b28f1826fac9c02c1bedcb1bf95de6700050e85c40856c08a.png

Unfortunately CCM does not infer any causal relation, which is incorrect.

../_images/ad67423c65349a335e9766fabe3740ee893eb16d28b35b19d3beb51676ac431b.png

Example 3#

Here, we make the effect of \(X\) on \(Y\) stronger than \(Y\) on \(X\).

# create a time series SCM
np.random.seed(42)
L = 2000
alpha = 0.5
beta = 0.1
X_std = 0.1
Y_std = 0.1
X = [np.random.normal(0, X_std), np.random.normal(0, X_std)]
Y = [np.random.normal(0, Y_std), np.random.normal(0, Y_std)]
for t in range(L):
    X += [beta * Y[-1] + np.random.normal(0, X_std)]
    Y += [alpha * X[-1] + np.random.normal(0, Y_std)]

../_images/f72b1eca7bb4dcfbbff186ba47a48dfc08db47758776f37a8e0f3580172d3838.png

CCM correctly infers the bidirectional causal relation with a stronger \(X \rightarrow Y\) than \(Y \rightarrow X\)

../_images/76c145f06ff2c728f4ae90e3bc0e5601157638815d14244066d65ba929fa045f.png

Chaotic System in the Original CCM Paper (Sugihara, et al, 2012)#

# FROM Supplementary Materials https://www.science.org/doi/10.1126/science.1227079
# params
r_x = 3.8
r_y = 3.5
B_xy = 0.02 # effect on x given y (effect of y on x)
B_yx = 0.1 # effect on y given x (effect of x on y)

X0 = 0.4 # initial val following Sugihara et al
Y0 = 0.2 # initial val following Sugihara et al
t = 3000 # time steps

X = [X0]
Y = [Y0]
for i in range(t):
    X_ = chaotic_func(X[-1], Y[-1], r_x, B_xy)
    Y_ = chaotic_func(Y[-1], X[-1], r_y, B_yx)    
    X.append(X_)
    Y.append(Y_)

../_images/d20db0e1664327abeec0b9dc4a616df03d9f22e59ce00ea75856a22708bc4919.png

Where Convergent Cross-Mapping Fails

Contents

Where Convergent Cross-Mapping Fails#

Chaotic Models#

Example 1#

Example 2#

Example 3#

Chaotic Models + Noise#

Example 1#

Example 2#

Example 3#

Additive Noise Models#

Example 1#

Example 2#

Example 3#

Chaotic System in the Original CCM Paper (Sugihara, et al, 2012)#