---
title: "Getting Started with NNS: Correlation and Dependence"
author: "Fred Viole"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Getting Started with NNS: Correlation and Dependence}
  %\VignetteEngine{knitr::rmarkdown}
  \usepackage[utf8]{inputenc}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

```{r setup2,message=FALSE,warning = FALSE}
library(NNS)
library(data.table)
require(knitr)
require(rgl)
require(meboot)
```

# Correlation and Dependence
The limitations of linear correlation are well known.  Often one uses correlation, when dependence is the intended measure for defining the relationship between variables.  NNS dependence **`NNS.dep`** is a signal:noise measure robust to nonlinear signals.

Below are some examples comparing NNS correlation **`NNS.cor`** and **`NNS.dep`** with the standard Pearson's correlation coefficient `cor`.

## Linear Equivalence
Note the fact that all observations occupy the co-partial moment quadrants.
```{r linear,fig.width=5,fig.height=3,fig.align = "center"}
x = seq(0, 3, .01) ; y = 2 * x
```

```{r linear1,fig.width=5,fig.height=3,fig.align = "center", results='hide', echo=FALSE}
NNS.part(x, y, Voronoi = TRUE, order = 3)
```

```{r res1}
cor(x, y)
NNS.dep(x, y)
```

## Nonlinear Relationship
Note the fact that all observations occupy the co-partial moment quadrants.
```{r nonlinear,fig.width=5,fig.height=3,fig.align = "center", results='hide'}
x = seq(0, 3, .01) ; y = x ^ 10
```

```{r nonlinear1,fig.width=5,fig.height=3,fig.align = "center", results='hide', echo=FALSE}
NNS.part(x, y, Voronoi = TRUE, order = 3)
```

```{r res2a}
cor(x, y)
NNS.dep(x, y)
```


## Cyclic Relationship
Even the difficult inflection points, which span both the co- and divergent partial moment quadrants, are properly compensated for in **`NNS.dep`**.
```{r nonlinear_sin,fig.width=5,fig.height=3,fig.align = "center", results='hide'}
x = seq(0, 12*pi, pi/100) ; y = sin(x)
```

```{r nonlinear1_sin,fig.width=5,fig.height=3,fig.align = "center", results='hide', echo=FALSE}
NNS.part(x, y, Voronoi = TRUE, order = 3, obs.req = 0)
```

```{r res2_sin}
cor(x, y)
NNS.dep(x, y)
```

## Dependence
Note the fact that all observations occupy only co- or divergent partial moment quadrants for a given subquadrant.
```{r dependence,fig.width=5,fig.height=3,fig.align = "center"}
set.seed(123)
df <- data.frame(x = runif(10000, -1, 1), y = runif(10000, -1, 1))
df <- subset(df, (x ^ 2 + y ^ 2 <= 1 & x ^ 2 + y ^ 2 >= 0.95))
```

```{r circle1,fig.width=5,fig.height=3,fig.align = "center", results='hide', echo=FALSE}
NNS.part(df$x, df$y, Voronoi = TRUE, order = 3, obs.req = 0)
```

```{r res3}
NNS.dep(df$x, df$y)
```




# p-values for `NNS.dep()`
p-values and confidence intervals can be obtained from sampling random permutations of $y \rightarrow y_p$ and running **`NNS.dep(x,$y_p$)`** to compare against a null hypothesis of 0 correlation, or independence between $(x, y)$.

Simply set **`NNS.dep(..., p.value = TRUE, print.map = TRUE)`** to run 100 permutations and plot the results.

```{r permutations}
## p-values for [NNS.dep]
x <- seq(-5, 5, .1); y <- x^2 + rnorm(length(x))
```

```{r perm1,fig.width=5,fig.height=3,fig.align = "center", results='hide', echo=FALSE}
NNS.part(x, y, Voronoi = TRUE, order = 3)
```

```{r permutattions_res,fig.width=5,fig.height=3,fig.align = "center"}
NNS.dep(x, y, p.value = TRUE, print.map = TRUE)
```

# Multivariate Dependence `NNS.copula()`
These partial moment insights permit us to extend the analysis to multivariate
instances and deliver a dependence measure $(D)$ such that $D \in [0,1]$. This level of analysis is simply impossible with Pearson or other rank
based correlation methods, which are restricted to bivariate cases.

```{r multi, warning=FALSE}
set.seed(123)
x <- rnorm(1000); y <- rnorm(1000); z <- rnorm(1000)
NNS.copula(cbind(x, y, z), plot = TRUE, independence.overlay = TRUE)
```

# Simulating a Multivariate Dependence Structure
Analogous to an empirical copula transformation, we can generate `new data` from the dependence structure of our `original data` via the following steps:

* **Determine the dependence structure:**

This is accomplished using **`LPM.ratio(1, x, x)`** for continuous variables, and **`LPM.ratio(0, x, x)`** for discrete variables, which are the empirical CDFs of the marginal variables.

* **Generate or supply `new data`:**

`new data` does not have to be of the same distribution or dimension as the `original data`, nor does each dimension of `new data` have to share a distribution type.

* **Apply dependence structure to `new data`:**

We then utilize **`LPM.VaR(...)`** to ascertain `new data` values corresponding to `original data` position mappings, and return a matrix of these transformed values with the same dimensions as `new.data`.  


```{r multisim}
# Add variable x to original data to avoid total independence (example only)
original.data <- cbind(x, y, z, x)

# Determine dependence structure
dep.structure <- apply(original.data, 2, function(x) LPM.ratio(1, x, x))
  
# Generate new data with different mean, sd and length (or distribution type)
new.data <- sapply(1:ncol(original.data), function(x) rnorm(dim(original.data)[1]*2, mean = 10, sd = 20))

# Apply dependence structure to new data
new.dep.data <- sapply(1:ncol(original.data), function(x) LPM.VaR(dep.structure[,x], 1, new.data[,x]))
```

## Compare Multivariate Dependence Structures
```{r comparison, warning=FALSE}
NNS.copula(original.data)
NNS.copula(new.dep.data)
```


# References
If the user is so motivated, detailed arguments and proofs are provided within the following:

* [Nonlinear Nonparametric Statistics: Using Partial Moments](https://www.amazon.com/dp/1490523995/ref=cm_sw_su_dp)

* [Nonlinear Correlation and Dependence Using NNS](https://www.ssrn.com/abstract=3010414)

* [Deriving Nonlinear Correlation Coefficients from Partial Moments](https://www.ssrn.com/abstract=2148522)

* [Beyond Correlation: Using the Elements of Variance for Conditional Means and Probabilities](https://www.ssrn.com/abstract=2745308)





