---
title: "Load and prepare ILSA files"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Load and prepare ILSA files}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE
)
```

```{r, echo=FALSE}
library(ILSAmerge)
```

## Check what can be loaded

Using `ILSAfile.info()` we can identify different populations within an ILSA.

Moreover, we can also use this information for loading an specific population
into the memory. Using the 'REDS' 2021 example data provided by this package:
```{r}

ILSAfile.info(inputdir = system.file("extdata/reds", package = "ILSAmerge"))

```

## Load all information

We can decide to load a population, like `BCGV1`, using `justload()`:
```{r}
loaded <- justload(inputdir = system.file("extdata/reds", package = "ILSAmerge"),
         population = "BCGV1")
```

This will result in a list of elements loaded by `haven`, in our example, a list
of 11 elements:
```{r}
class(loaded)

length(loaded)

loaded[[1]]
```

## Load only column information

For some purposes, we might need to load only the column information, most 
probably to check if the attributes are correct. We can do this by setting
`justattributes = TRUE`:
```{r}
loaded <- justload(inputdir = system.file("extdata/reds", package = "ILSAmerge"),
         population = "BCGV1", justattributes = TRUE)
```

This will load all 11 datasets with 0 rows each:
```{r}
length(loaded)

loaded[[1]]
```

## Read a single file

We can also read any SPSS file or any file produced by `ILSAmerge()` by using `readILSA()`.
For example, to read the student questionnaire of Burkina Faso in REDS 2021:

```{r}

dirdat <- system.file("extdata/reds", package = "ILSAmerge")

bfa <- readILSA(file.path(dirdat,"bsgbfav1.sav"))

bfa

```

Meanwhile, for reading a merged ILSA file, we run also `readILSA()` but on the merged file:

```{r}
# merge data
ILSAmerge(inputdir = dirdat,outputdir = tempdir(),filetype = "sav")
# rename merged files
ILSArename(inputdir = tempdir())

stu <- readILSA(file.path(tempdir(),"REDS_2021_student.sav"))

stu

```

## Remove tibble attributes

As we can see, `bfa`, and `stu` are both tibbles produced by package `haven`. Nonetheless, this type of objects can produce some errors while using other packages for analysis. Therefore, we should always create another object that is a simple data frame so we can treat directly with the numeric data, always trying also to remove all labeled missing values. For this, we use `untibble()`:

```{r}

bfa2 <- untibble(tibble = bfa, mistoNAs = TRUE)

stu2 <- untibble(tibble = stu, mistoNAs = TRUE)

class(bfa)
class(bfa2)

class(stu)
class(stu2)

```