```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE, cache = FALSE)
```

## R 'hadley' workshop | Day 2 - advanced

## Modern R

[David Robinson](http://varianceexplained.org/) summarized the goal on his laptop

```{r, echo=FALSE, out.width='55%'}
knitr::include_graphics("http://lsru.github.io/r_workshop/img/drob_r_pipeline_600.jpg")
```

see also what Karl Broman is recommanding for [people who learnt R a while ago](http://kbroman.org/hipsteR/)


## Modern pipeline


```{r, echo=FALSE, out.width='90%'}
knitr::include_graphics("http://varianceexplained.org/images/broom_slides/broom_slides.001.jpg")
```

source: [David Robinson](http://varianceexplained.org/r/broom-slides/)
check out David's [broom presentation](http://varianceexplained.org/files/broom_presentation.pdf)

## reading data

Hadley Wickham and Wes McKinney recently released `feather`, a new python / R project.

It rapidly stores dataframes as binary files and preserves column types.
<!-- Column structure and binary storage. Column types are preserved -->

```{r, echo=FALSE, out.width='60%'}
knitr::include_graphics("http://lsru.github.io/r_workshop/img/feather_perf_20.png")
```

- [Rstudio blog](https://blog.rstudio.org/2016/03/29/feather/)
- [performance benchmark](https://blog.dominodatalab.com/the-r-data-i-o-shootout/)
- [github repo](https://github.com/wesm/feather)

## Managing multiple models

Tutorial based on the great conference by [Hadley Wickham][1]

![](http://lsru.github.io/r_workshop/img/hadley_purrr.png)

[1]:https://www.youtube.com/watch?v=rz3_FDVt9eg

## purrr::map / dplyr::do

progress bar will be [added](https://github.com/hadley/purrr/issues/149)

```{r, echo=FALSE, out.width='60%'}
knitr::include_graphics("http://lsru.github.io/r_workshop/img/purrr_do.png")
```

## Functional programming and nested data_frames {.codefont}

Using `purrr` and `tidyr`. Hadley is focusing on every part of `R` to _clean_ it up.

`purrr` revisits the `apply` family in a consistent way.
`tidyr::nest` nests list in `tibble::data_frame` to keep related things together.

## For loops emphasise on objects and not actions {.codefont}

compare (notice `seq_along` instead of `1:length(mtcars)`)
```{r}
means <- vector("double", ncol(mtcars))
for (i in seq_along(mtcars)) {
  means[i] <- mean(mtcars[[i]])
}
means
```

and
```{r}
library("purrr")
map_dbl(mtcars, mean)
```

## Nested map

```{r}
library("purrr")
library("dplyr", warn.conflicts = FALSE)
funs <- list(mean = mean, median = median, sd = sd)
funs %>%
  map(~ mtcars %>% map_dbl(.x))
```


## Gapminder

Global vs individual trend
```{r, out.width='70%'}
library("gapminder")
library("ggplot2")
theme_set(theme_bw(14))
gapminder %>%
  ggplot(aes(x = year, y = lifeExp, group = country))+
  geom_line()
```


## Keep related things together

Linear model _per_ country
```{r}
library("tidyr")
by_country_lm <- gapminder %>%
  mutate(year1950 = year - 1950) %>%
  group_by(continent, country) %>%
  nest() %>%
  mutate(model = map(data, ~ lm(lifeExp ~ year1950, data = .x)))
by_country_lm
```

## broom cleanup

```{r, echo=FALSE, out.width='90%'}
knitr::include_graphics("http://lsru.github.io/r_workshop/img/broom_cleanup.png")
```

## Tidying model coefficients

Use broom to extract, as neat data frames out of `lm()`:

- coefficients estimates: **slope** and **intercept**
- $r^2$ 
- residuals

```{r}
library("broom")
models <- by_country_lm %>%
  mutate(glance  = map(model, glance),
         rsq     = glance %>% map_dbl("r.squared"),
         tidy    = map(model, tidy),
         augment = map(model, augment))
models
```

# Exploratory plots

## Does linear models fit all countries?

```{r, out.width='60%'}
models %>%
  ggplot(aes(x = rsq, y = reorder(country, rsq)))+
  geom_point(aes(colour = continent))+
  theme(axis.text.y = element_text(size = 6))
```

## focus on non-linear trends

```{r, cache=FALSE}
models %>%
  filter(rsq < 0.55) %>%
  unnest(data) %>%
  ggplot(aes(x = year, y = lifeExp))+
  geom_line(aes(colour = continent))+
  facet_wrap(~ country)
```

## shiny - code

```{r, cache=FALSE, eval=FALSE}
library("shiny")
inputPanel(
  selectInput("country", "Select Country", levels(models$country))
)
output$rsq <- renderPlot({
  models %>%
  filter(country == input$country) %>%
  unnest(data) %>%
  ggplot(aes(x = year, y = lifeExp))+
  geom_line(aes(colour = continent))
})
renderUI({
 plotOutput("rsq", height = "400", width = "600")
})
```

## shiny

```{r, echo=FALSE, cache=FALSE}
library("shiny")
inputPanel(
  selectInput("country", "Select Country", levels(models$country))
)
renderPlot({
  models %>%
  filter(country == input$country) %>%
  unnest(data) %>%
  ggplot(aes(x = year, y = lifeExp))+
  geom_line(aes(colour = continent))
})

```


## shiny - rsquare

```{r, cache=FALSE, echo=FALSE}
library("shiny")
inputPanel(
  sliderInput("rsq", "Select rsquared", min = 0, max = 1, 
              value = c(0, 0.2), dragRange = TRUE)
)
renderPlot({
  models %>%
  filter(rsq >= input$rsq[1], rsq <= input$rsq[2]) %>%
  unnest(data) %>%
  ggplot(aes(x = year, y = lifeExp))+
  geom_line(aes(colour = continent))+
  facet_wrap(~ country)
})

```

## All in all

```{r,out.width='70%'}
models %>%
  unnest(tidy) %>%
  select(continent, country, rsq, term, estimate) %>%
  #filter(continent != "Africa") %>%
  spread(term, estimate) %>%
  ggplot(aes(x = `(Intercept)`, y = year1950))+
  geom_point(aes(colour = continent, size = rsq))+
  geom_smooth(se = FALSE)+
  scale_size_area()+
  labs(x = "Life expectancy (1950)",
       y = "Yearly improvement")
```

## animation made easy

[gganimate](https://github.com/dgrtwo/gganimate) by David Robinson

```{r, eval=FALSE}
library("gganimate")
library("gapminder")
p <- ggplot(gapminder, aes(gdpPercap, lifeExp, size = pop, 
                           color = continent, frame = year))+
  geom_point()+scale_x_log10()
gg_animate(p, 'img/gapminder.gif')
```
```{r, echo=FALSE, out.width='50%'}
knitr::include_graphics("img/gapminder.gif")
```


## Error handling

`purrr` proposes `safely()` and `possibly()` to enable error-handling.

safely() is a type-stable version of try. It always returns a list of two elements, the result and the error, and one will always be NULL.
```{r}
safely(log)(10)
safely(log)("a")
```

to be investigated

## Recommended reading

- [purrr applied](http://ijlyttle.github.io/isugg_purrr/presentation.html#%281%29) by Ian Lyttle
- [R for data science](http://r4ds.had.co.nz/intro.html) by Hadley
- [iterations - purrr](http://r4ds.had.co.nz/iteration.html) by Hadley
- [purrr 0.1](https://blog.rstudio.org/2015/09/29/purrr-0-1-0/) by Hadley
- [purrr 0.2](https://blog.rstudio.org/2016/01/06/purrr-0-2-0/) by Hadley

## Acknowledgments

* Hadley Wickham
* Robert Rudis
* Ian Lyttle
* David Robinson
* Eric Koncina
