class: center, middle, inverse, title-slide .title[ # Tidy work in Tidyverse ] .subtitle[ ## R Foundation for Life Scientists ] .author[ ### Marcin Kierczak ] --- exclude: true count: false <link href="https://fonts.googleapis.com/css?family=Roboto|Source+Sans+Pro:300,400,600|Ubuntu+Mono&subset=latin-ext" rel="stylesheet"> <link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.3.1/css/all.css" integrity="sha384-mzrmE5qonljUremFsqc01SB46JvROS7bZs3IO2EmfFsd15uHvIt+Y8vEf7N7fWAU" crossorigin="anonymous"> <!-- ----------------- Only edit title & author above this ----------------- --> --- name: learning_outcomes # Learning Outcomes <br> Upon completing this module, you will: * know what `tidyverse` is and a bit about its history * be aware of useful packages within `tidyverse` * be able to use basic pipes (including native R pipe) * know whether the data you are working with are tidy * be able to do basic tidying of your data * be able to transform, group and summarise data with `dplyr` * be able to join tables --- name: tidyverse_overview # Tidyverse -- What is it all About? * [tidyverse](http://www.tidyverse.org) is a collection of 📦 📦 * created by [Hadley Wickham](http://hadley.nz) * has become a *de facto* standard in data analyses * a philosophy of programming or a **programming paradigm**: everything is about the 🌊 flow of 🧹 tidy data .center[ <img src="data/slide_tidyverse/hex-tidyverse.png", style="height:200px;"> <img src="data/slide_tidyverse/Hadley-wickham2016-02-04.jpeg", style="height:200px;"> <img src="data/slide_tidyverse/RforDataScience.jpeg", style="height:200px;"> ] .vsmall[sources of images: www.tidyverse.org, Wikipedia, www.tidyverse.org] --- name: tidyverse_curse # ?(Tidyverse OR !Tidyverse) > Navigating the balance between base R and the tidyverse is a challenge to learn.<br>[-Robert A. Muenchen](http://r4stats.com/articles/why-r-is-hard-to-learn/) -- .center[<img src="data/slide_tidyverse/tidyverse.webp", style="height:400px;">] --- name: intro_to_pipes # Pipes or Let my Data Flow 🌊 .pull-left-50[ .center[<img src="data/slide_tidyverse/pipe_magritte.jpg", style="width:300px;">] .vsmall[René Magritte, *La trahison des images*, [Wikimedia Commons](https://en.wikipedia.org/wiki/The_Treachery_of_Images#/media/File:MagrittePipe.jpg)] .center[<img src="data/slide_tidyverse/magrittr.png", style="width:150px;">] ] -- .pull-right-50[ * Let the data flow. * *Ceci n'est pas une pipe* -- `magrittr` * The `%>%` pipe: + `x %>% f` `\(\equiv\)` `f(x)` + `x %>% f(y)` `\(\equiv\)` `f(x, y)` + `x %>% f %>% g %>% h` `\(\equiv\)` `h(g(f(x)))` ] -- .pull-right-50[ instead of writing this: ``` r data <- iris data <- head(data, n=3) ``` ] -- .pull-right-50[ write this: ``` r iris %>% head(n=3) ``` ``` ## Sepal.Length Sepal.Width Petal.Length Petal.Width Species ## 1 5.1 3.5 1.4 0.2 setosa ## 2 4.9 3.0 1.4 0.2 setosa ## 3 4.7 3.2 1.3 0.2 setosa ``` ] --- name: native_r_pipe # Native R Pipe From R 4.1.0 there is a native pipe `|>`. It is simpler than `%>%`: * `x |> f(y)` is rewritten to `f(x, y)` at parse time, * the placeholder is `_` and it must be used with a **named** argument, * no `.` placeholder, no `%T>%`, no `%$%` (those come from `magrittr`), * anonymous function: `x |> (\(d) d[1:3, ])()`. ``` r c(1:5) |> mean() mtcars |> lm(mpg ~ hp, data = _) |> coef() ``` ``` ## [1] 3 ## (Intercept) hp ## 30.09886054 -0.06822828 ``` ``` r c(1:5) %>% mean() mtcars %>% lm(mpg ~ hp, data = .) %>% coef() ``` ``` ## [1] 3 ## (Intercept) hp ## 30.09886054 -0.06822828 ``` > In this course we use `%>%`, because the lab needs `%T>%` and `%$%`. In new code `|>` is perfectly fine for the verbs below. --- name: tibble_intro # Tibbles .pull-left-50[ .center[<img src="data/slide_tidyverse/tibble_tweet.jpg">] ] .pull-right-50[ * `tibble` is one of the unifying features of tidyverse, * it is a *better* `data.frame` realization, * objects `data.frame` can be coerced to `tibble` using `as_tibble()` ] --- name: convert_to_tibble # Convert `data.frame` to `tibble` ``` r as_tibble(iris) ``` ``` ## # A tibble: 150 × 5 ## Sepal.Length Sepal.Width Petal.Length Petal.Width Species ## <dbl> <dbl> <dbl> <dbl> <fct> ## 1 5.1 3.5 1.4 0.2 setosa ## 2 4.9 3 1.4 0.2 setosa ## 3 4.7 3.2 1.3 0.2 setosa ## 4 4.6 3.1 1.5 0.2 setosa ## 5 5 3.6 1.4 0.2 setosa ## 6 5.4 3.9 1.7 0.4 setosa ## 7 4.6 3.4 1.4 0.3 setosa ## 8 5 3.4 1.5 0.2 setosa ## 9 4.4 2.9 1.4 0.2 setosa ## 10 4.9 3.1 1.5 0.1 setosa ## 11 5.4 3.7 1.5 0.2 setosa ## 12 4.8 3.4 1.6 0.2 setosa ## 13 4.8 3 1.4 0.1 setosa ## 14 4.3 3 1.1 0.1 setosa ## 15 5.8 4 1.2 0.2 setosa ## # ℹ 135 more rows ``` --- name: tibble_from_scratch # Tibbles from scratch with `tibble()` ``` r tibble( x = 1, # recycling y = runif(4), z = x + y^2, outcome = rnorm(4) ) ``` -- ``` ## # A tibble: 4 × 4 ## x y z outcome ## <dbl> <dbl> <dbl> <dbl> ## 1 1 0.904 1.82 -1.39 ## 2 1 0.139 1.02 -0.279 ## 3 1 0.989 1.98 -0.133 ## 4 1 0.947 1.90 0.636 ``` --- name: more_on_tibbles # More on Tibbles * When you print a `tibble`: + all columns that fit the screen are shown, + first 10 rows are shown, + data type for each column is shown. ``` r as_tibble(cars) ``` ``` ## # A tibble: 50 × 2 ## speed dist ## <dbl> <dbl> ## 1 4 2 ## 2 4 10 ## 3 7 4 ## 4 7 22 ## 5 8 16 ## 6 9 10 ## 7 10 18 ## 8 10 26 ## 9 10 34 ## 10 11 17 ## 11 11 28 ## 12 12 14 ## 13 12 20 ## 14 12 24 ## 15 12 28 ## # ℹ 35 more rows ``` --- name: tibble_printing_options # Tibble Printing Options * `my_tibble %>% print(n = 50, width = Inf)`, * `options(tibble.print_min = 15, tibble.print_max = 25)`, * `options(dplyr.print_min = Inf)`, * `options(tibble.width = Inf)` --- name: subsetting_tibbles # Subsetting Tibbles ``` r vehicles <- as_tibble(cars[1:5,]) vehicles %>% print(n = 5) ``` ``` ## # A tibble: 5 × 2 ## speed dist ## <dbl> <dbl> ## 1 4 2 ## 2 4 10 ## 3 7 4 ## 4 7 22 ## 5 8 16 ``` -- We can subset tibbles in a number of ways: ``` r vehicles[['speed']] # try also vehicles['speed'] vehicles[[1]] vehicles$speed ``` ``` ## [1] 4 4 7 7 8 ## [1] 4 4 7 7 8 ## [1] 4 4 7 7 8 ``` -- > **Note!** Not all old R functions work with tibbles, then you have to use `as.data.frame(my_tibble)`. --- name: tibbles_partial_matching # Tibbles are Stricter than `data.frames` ``` r cars5$spe # partial matching ``` ``` ## [1] 4 4 7 7 8 ``` ``` r vehicles$spe # no partial matching ``` ``` ## Warning: Unknown or uninitialised column: `spe`. ``` ``` ## NULL ``` ``` r cars5$gear ``` ``` ## NULL ``` ``` r vehicles$gear ``` ``` ## Warning: Unknown or uninitialised column: `gear`. ``` ``` ## NULL ``` --- name: loading_data # Loading Data In `tidyverse` you import data using the `readr` package: * `read_csv()` -- comma-delimited, `read_csv2()` -- semicolon-delimited, * `read_tsv()` -- tab-delimited, `read_delim()` -- generic (any delimiter), * `read_fwf()` -- fixed-width files. -- Frequently used arguments: * `skip` -- number of rows to skip, * `col_names` -- a vector of column names (or `FALSE`), * `col_types` -- force column types, e.g. `cols(id = col_character())`, * `comment` -- character that designates a comment, * `na` -- how missing values are represented. `readr` tells you the column types it guessed. Check them! Excel files: `readxl::read_excel()`. --- name: readr_writing # Writing to a File The `readr` package also provides functions useful for writing tibbled data into a file: * `write_csv()` * `write_tsv()` * `write_excel_csv()` They **always** save: * text in UTF-8, * dates in ISO8601 But saving in csv (or tsv) does mean you lose information about the type of data in particular columns. You can avoid this by using: * `write_rds()` and `read_rds()` to read/write objects in R binary rds format, * `arrow::write_parquet()` and `arrow::read_parquet()` for a fast binary format that other programming languages can access. --- name: basic_data_transformations # Basic Data Transformations with `dplyr` Let us create a tibble: ``` r set.seed(42) jewels <- as_tibble(diamonds) %>% slice_sample(n = 200) jewels %>% head(n = 5) ``` ``` ## # A tibble: 5 × 10 ## carat cut color clarity depth table price x y z ## <dbl> <ord> <ord> <ord> <dbl> <dbl> <int> <dbl> <dbl> <dbl> ## 1 0.39 Ideal I VVS2 60.8 56 849 4.74 4.76 2.89 ## 2 1.12 Very Good G SI2 63.3 58 4478 6.7 6.63 4.22 ## 3 0.51 Very Good G VVS2 62.9 57 1750 5.06 5.12 3.2 ## 4 0.52 Very Good D VS1 62.5 57 1829 5.11 5.16 3.21 ## 5 0.28 Very Good E VVS2 61.4 55 612 4.22 4.25 2.6 ``` .center[ <img src="data/slide_tidyverse/diamonds.png", style="height:200px"> ] --- name: filter # Picking Observations using `filter()` ``` r jewels %>% filter(cut == 'Ideal' | cut == 'Premium', carat >= 0.23) %>% head(n = 4) ``` ``` ## # A tibble: 4 × 10 ## carat cut color clarity depth table price x y z ## <dbl> <ord> <ord> <ord> <dbl> <dbl> <int> <dbl> <dbl> <dbl> ## 1 0.39 Ideal I VVS2 60.8 56 849 4.74 4.76 2.89 ## 2 0.9 Ideal D SI1 62.1 57 4523 6.18 6.25 3.86 ## 3 0.33 Ideal G VVS1 62 55 838 4.45 4.49 2.77 ## 4 0.71 Premium G VS2 62.1 62 2623 5.71 5.65 3.53 ``` -- >⛵ Be careful with floating point comparisons! <br> 🏴☠️ Also, rows with comparison resulting in `NA` are skipped by default! ``` r jewels %>% filter(near(0.23, carat) | is.na(carat)) %>% head(n = 4) ``` --- name: arrange # Rearranging Observations using `arrange()` ``` r jewels %>% arrange(cut, carat, desc(price)) ``` -- ``` ## # A tibble: 200 × 10 ## carat cut color clarity depth table price x y z ## <dbl> <ord> <ord> <ord> <dbl> <dbl> <int> <dbl> <dbl> <dbl> ## 1 0.3 Fair F VVS2 65.5 61 743 4.19 4.15 2.73 ## 2 0.96 Fair F I1 62.9 66 1917 6.29 6.07 3.91 ## 3 1 Fair H VS2 64.8 62 4861 6.22 6.13 4 ## 4 1 Fair J VS1 66.6 57 3080 6.07 6.03 4.03 ## 5 1.01 Fair F SI1 67.2 60 4276 6.06 6 4.05 ## 6 1.26 Fair I SI2 64.8 57 4551 6.73 6.69 4.35 ## 7 1.5 Fair D SI2 68.8 57 7469 6.9 6.86 4.73 ## 8 1.5 Fair E SI2 65.8 57 7459 7.17 7.11 4.7 ## 9 0.38 Good H SI2 63.4 57 547 4.58 4.63 2.92 ## 10 0.41 Good E VVS1 63.5 57 1002 4.73 4.75 3.01 ## 11 0.59 Good F VS1 60.5 61 1861 5.43 5.5 3.31 ## 12 0.7 Good F VS2 62.6 61 2591 5.56 5.62 3.5 ## 13 0.7 Good F SI1 64.3 63 2059 5.58 5.55 3.58 ## 14 0.9 Good F SI1 63.3 60 3950 6.09 6.11 3.86 ## 15 0.92 Good I SI1 57.4 65 3122 6.42 6.48 3.7 ## # ℹ 185 more rows ``` -- >The `NA`s always end up at the end of the rearranged `tibble`! --- name: select # Selecting Variables with `select()` Simple `select` with a range: ``` r jewels %>% select(color, clarity, x:z) %>% head(n = 4) ``` ``` ## # A tibble: 4 × 5 ## color clarity x y z ## <ord> <ord> <dbl> <dbl> <dbl> ## 1 I VVS2 4.74 4.76 2.89 ## 2 G SI2 6.7 6.63 4.22 ## 3 G VVS2 5.06 5.12 3.2 ## 4 D VS1 5.11 5.16 3.21 ``` -- Exclusive `select`: ``` r jewels %>% select(-(x:z)) %>% head(n = 4) ``` ``` ## # A tibble: 4 × 7 ## carat cut color clarity depth table price ## <dbl> <ord> <ord> <ord> <dbl> <dbl> <int> ## 1 0.39 Ideal I VVS2 60.8 56 849 ## 2 1.12 Very Good G SI2 63.3 58 4478 ## 3 0.51 Very Good G VVS2 62.9 57 1750 ## 4 0.52 Very Good D VS1 62.5 57 1829 ``` --- name: rename # Renaming Variables >`rename()` changes column names and keeps all columns; `select(new = old)` renames but keeps only the selected columns. ``` r jewels %>% rename(var_x = x) %>% head(n = 5) ``` -- ``` ## # A tibble: 5 × 10 ## carat cut color clarity depth table price var_x y z ## <dbl> <ord> <ord> <ord> <dbl> <dbl> <int> <dbl> <dbl> <dbl> ## 1 0.39 Ideal I VVS2 60.8 56 849 4.74 4.76 2.89 ## 2 1.12 Very Good G SI2 63.3 58 4478 6.7 6.63 4.22 ## 3 0.51 Very Good G VVS2 62.9 57 1750 5.06 5.12 3.2 ## 4 0.52 Very Good D VS1 62.5 57 1829 5.11 5.16 3.21 ## 5 0.28 Very Good E VVS2 61.4 55 612 4.22 4.25 2.6 ``` --- name: bring_to_front # Bring columns to front >use `relocate()` (or `everything()` in `select`) to bring some columns to the front: ``` r jewels %>% relocate(x:z) %>% head(n = 4) # same as select(x:z, everything()) ``` -- ``` ## # A tibble: 4 × 10 ## x y z carat cut color clarity depth table price ## <dbl> <dbl> <dbl> <dbl> <ord> <ord> <ord> <dbl> <dbl> <int> ## 1 4.74 4.76 2.89 0.39 Ideal I VVS2 60.8 56 849 ## 2 6.7 6.63 4.22 1.12 Very Good G SI2 63.3 58 4478 ## 3 5.06 5.12 3.2 0.51 Very Good G VVS2 62.9 57 1750 ## 4 5.11 5.16 3.21 0.52 Very Good D VS1 62.5 57 1829 ``` --- name: mutate # Create/alter new Variables with `mutate` ``` r jewels %>% mutate(p = x + z, q = p + y) %>% select(-(depth:price)) %>% head(n = 5) ``` -- ``` ## # A tibble: 5 × 9 ## carat cut color clarity x y z p q ## <dbl> <ord> <ord> <ord> <dbl> <dbl> <dbl> <dbl> <dbl> ## 1 0.39 Ideal I VVS2 4.74 4.76 2.89 7.63 12.4 ## 2 1.12 Very Good G SI2 6.7 6.63 4.22 10.9 17.6 ## 3 0.51 Very Good G VVS2 5.06 5.12 3.2 8.26 13.4 ## 4 0.52 Very Good D VS1 5.11 5.16 3.21 8.32 13.5 ## 5 0.28 Very Good E VVS2 4.22 4.25 2.6 6.82 11.1 ``` --- name: transmute # Create/alter new Variables with `transmute` 🧙 >Only the transformed variables will be retained. ``` r jewels %>% transmute(carat, cut, sum = x + y + z) %>% head(n = 5) ``` ``` ## # A tibble: 5 × 3 ## carat cut sum ## <dbl> <ord> <dbl> ## 1 0.39 Ideal 12.4 ## 2 1.12 Very Good 17.6 ## 3 0.51 Very Good 13.4 ## 4 0.52 Very Good 13.5 ## 5 0.28 Very Good 11.1 ``` --- name: grouped_summaries # Group and Summarize ``` r jewels %>% group_by(cut) %>% summarize(max_price = max(price), mean_price = mean(price), min_price = min(price)) ``` ``` ## # A tibble: 5 × 4 ## cut max_price mean_price min_price ## <ord> <int> <dbl> <int> ## 1 Fair 7469 4294. 743 ## 2 Good 18077 5410. 547 ## 3 Very Good 17751 4045. 450 ## 4 Premium 18493 5586. 345 ## 5 Ideal 15497 3348. 397 ``` -- ``` r jewels %>% group_by(cut, color) %>% summarize(max_price = max(price), mean_price = mean(price), min_price = min(price)) %>% head(n = 4) ``` ``` ## # A tibble: 4 × 5 ## # Groups: cut [1] ## cut color max_price mean_price min_price ## <ord> <ord> <int> <dbl> <int> ## 1 Fair D 7469 7469 7469 ## 2 Fair E 7459 7459 7459 ## 3 Fair F 4276 2312 743 ## 4 Fair H 4861 4861 4861 ``` --- name: other_data_manipulations # Counting, Unique Rows and Slicing ``` r jewels %>% count(cut, sort = TRUE) jewels %>% distinct(cut, color) %>% head(n = 3) ``` ``` ## # A tibble: 5 × 2 ## cut n ## <ord> <int> ## 1 Ideal 81 ## 2 Premium 49 ## 3 Very Good 46 ## 4 Good 16 ## 5 Fair 8 ## # A tibble: 3 × 2 ## cut color ## <ord> <ord> ## 1 Ideal I ## 2 Very Good G ## 3 Very Good D ``` -- `slice_*()` replace the older `top_n()` and `sample_n()`: ``` r jewels %>% slice_max(price, n = 2) jewels %>% group_by(cut) %>% slice_sample(n = 1) ``` ``` ## # A tibble: 2 × 10 ## carat cut color clarity depth table price x y z ## <dbl> <ord> <ord> <ord> <dbl> <dbl> <int> <dbl> <dbl> <dbl> ## 1 2 Premium F SI1 62.6 59 18493 8.1 8.04 5.05 ## 2 2.01 Good G SI1 60.8 61 18077 7.99 8.09 4.89 ## # A tibble: 5 × 10 ## # Groups: cut [5] ## carat cut color clarity depth table price x y z ## <dbl> <ord> <ord> <ord> <dbl> <dbl> <int> <dbl> <dbl> <dbl> ## 1 1 Fair H VS2 64.8 62 4861 6.22 6.13 4 ## 2 0.9 Good F SI1 63.3 60 3950 6.09 6.11 3.86 ## 3 1.24 Very Good J I1 61.9 55 2940 6.85 6.92 4.26 ## 4 0.91 Premium G SI1 62.6 58 3975 6.17 6.14 3.85 ## 5 0.9 Ideal D SI1 62.1 57 4523 6.18 6.25 3.86 ``` --- name: grouping_notes # Grouping Notes * `summarise()` drops the last grouping level (the message about `.groups`). * Use `ungroup()` when you need to regroup within the same pipe. * Since `dplyr` 1.1 you can group for a single operation with `.by`: ``` r jewels %>% summarise(mean_price = mean(price), .by = cut) ``` ``` ## # A tibble: 5 × 2 ## cut mean_price ## <ord> <dbl> ## 1 Ideal 3348. ## 2 Very Good 4045. ## 3 Fair 4294. ## 4 Premium 5586. ## 5 Good 5410. ``` --- name: across_case_when # `across()` and `case_when()` Apply the same function to many columns: ``` r jewels %>% summarise(across(c(carat, price), list(mean = mean, sd = sd))) ``` ``` ## # A tibble: 1 × 4 ## carat_mean carat_sd price_mean price_sd ## <dbl> <dbl> <dbl> <dbl> ## 1 0.847 0.505 4260. 4280. ``` -- Create a variable from several conditions: ``` r jewels %>% mutate(size = case_when(carat < 0.5 ~ "small", carat < 1 ~ "medium", .default = "large")) %>% count(size) ``` ``` ## # A tibble: 3 × 2 ## size n ## <chr> <int> ## 1 large 76 ## 2 medium 61 ## 3 small 63 ``` --- name: concept_of_tidy_data # The Concept of Tidy Data Data are tidy *sensu Wickham* if: * each and every observation is represented as exactly one row, * each and every variable is represented by exactly one column, * thus each data table cell contains only one value. Usually data are untidy in only one way. However, if you are unlucky, they are really untidy and thus a pain to work with... --- name: tidy_data # Tidy Data <img src="data/slide_tidyverse/tidy_data.png" style="height:100px"> -- .center[**Are these data tidy?**] .pull-left-70[ <table class="table table-striped table-hover table-responsive table-condensed" style="width: auto !important; "> <thead> <tr> <th style="text-align:center;"> Sepal.Length </th> <th style="text-align:center;"> Sepal.Width </th> <th style="text-align:center;"> Petal.Length </th> <th style="text-align:center;"> Petal.Width </th> <th style="text-align:center;"> Species </th> </tr> </thead> <tbody> <tr> <td style="text-align:center;"> 5.1 </td> <td style="text-align:center;"> 3.5 </td> <td style="text-align:center;"> 1.4 </td> <td style="text-align:center;"> 0.2 </td> <td style="text-align:center;"> setosa </td> </tr> <tr> <td style="text-align:center;"> 4.9 </td> <td style="text-align:center;"> 3.0 </td> <td style="text-align:center;"> 1.4 </td> <td style="text-align:center;"> 0.2 </td> <td style="text-align:center;"> setosa </td> </tr> <tr> <td style="text-align:center;"> 4.7 </td> <td style="text-align:center;"> 3.2 </td> <td style="text-align:center;"> 1.3 </td> <td style="text-align:center;"> 0.2 </td> <td style="text-align:center;"> setosa </td> </tr> </tbody> </table> ] -- .pull-right-30[ <table class="table table-striped table-hover table-responsive table-condensed" style="width: auto !important; "> <thead> <tr> <th style="text-align:center;"> Species </th> <th style="text-align:center;"> variable </th> <th style="text-align:center;"> value </th> </tr> </thead> <tbody> <tr> <td style="text-align:center;"> setosa </td> <td style="text-align:center;"> Sepal.Length </td> <td style="text-align:center;"> 5.1 </td> </tr> <tr> <td style="text-align:center;"> setosa </td> <td style="text-align:center;"> Sepal.Width </td> <td style="text-align:center;"> 3.5 </td> </tr> <tr> <td style="text-align:center;"> setosa </td> <td style="text-align:center;"> Petal.Length </td> <td style="text-align:center;"> 1.4 </td> </tr> </tbody> </table> ] <br> <hr><br> -- .pull-left-50[ <table class="table table-striped table-hover table-responsive table-condensed" style="width: auto !important; "> <thead> <tr> <th style="text-align:center;"> Sepal.L.W </th> <th style="text-align:center;"> Petal.L.W </th> <th style="text-align:center;"> Species </th> </tr> </thead> <tbody> <tr> <td style="text-align:center;"> 5.1/3.5 </td> <td style="text-align:center;"> 1.4/0.2 </td> <td style="text-align:center;"> setosa </td> </tr> <tr> <td style="text-align:center;"> 4.9/3 </td> <td style="text-align:center;"> 1.4/0.2 </td> <td style="text-align:center;"> setosa </td> </tr> <tr> <td style="text-align:center;"> 4.7/3.2 </td> <td style="text-align:center;"> 1.3/0.2 </td> <td style="text-align:center;"> setosa </td> </tr> </tbody> </table> ] -- .pull-right-50[ <table class="table table-striped table-hover table-responsive table-condensed" style="width: auto !important; "> <tbody> <tr> <td style="text-align:left;"> Sepal.Length </td> <td style="text-align:center;"> 5.1 </td> <td style="text-align:center;"> 4.9 </td> <td style="text-align:center;"> 4.7 </td> <td style="text-align:center;"> 4.6 </td> </tr> <tr> <td style="text-align:left;"> Sepal.Width </td> <td style="text-align:center;"> 3.5 </td> <td style="text-align:center;"> 3.0 </td> <td style="text-align:center;"> 3.2 </td> <td style="text-align:center;"> 3.1 </td> </tr> <tr> <td style="text-align:left;"> Petal.Length </td> <td style="text-align:center;"> 1.4 </td> <td style="text-align:center;"> 1.4 </td> <td style="text-align:center;"> 1.3 </td> <td style="text-align:center;"> 1.5 </td> </tr> <tr> <td style="text-align:left;"> Petal.Width </td> <td style="text-align:center;"> 0.2 </td> <td style="text-align:center;"> 0.2 </td> <td style="text-align:center;"> 0.2 </td> <td style="text-align:center;"> 0.2 </td> </tr> <tr> <td style="text-align:left;"> Species </td> <td style="text-align:center;"> setosa </td> <td style="text-align:center;"> setosa </td> <td style="text-align:center;"> setosa </td> <td style="text-align:center;"> setosa </td> </tr> </tbody> </table> ] --- name: tidying_data_pivot_longer # Tidying Data with `tidyr::pivot_longer` If some of your column names are actually values of a variable, use `pivot_longer` (replaces `gather`): ``` r jewels2 %>% head(n = 5) ``` ``` ## # A tibble: 5 × 3 ## cut `2008` `2009` ## <ord> <int> <dbl> ## 1 Ideal 849 891. ## 2 Very Good 4478 4520. ## 3 Very Good 1750 1792. ## 4 Very Good 1829 1871. ## 5 Very Good 612 654. ``` ``` r jewels2 %>% pivot_longer(c(`2008`, `2009`), names_to = 'year', values_to = 'price') %>% head(n = 5) ``` ``` ## # A tibble: 5 × 3 ## cut year price ## <ord> <chr> <dbl> ## 1 Ideal 2008 849 ## 2 Ideal 2009 891. ## 3 Very Good 2008 4478 ## 4 Very Good 2009 4520. ## 5 Very Good 2008 1750 ``` --- name: tidying_data_pivot_wider # Tidying Data with `tidyr::pivot_wider` If some of your observations are scattered across many rows, use `pivot_wider` (replaces `spread`): ``` r jewels3 ``` ``` ## # A tibble: 9 × 5 ## cut price clarity dimension measurement ## <ord> <int> <ord> <chr> <dbl> ## 1 Ideal 849 VVS2 x 4.74 ## 2 Ideal 849 VVS2 y 4.76 ## 3 Ideal 849 VVS2 z 2.89 ## 4 Very Good 4478 SI2 x 6.7 ## 5 Very Good 4478 SI2 y 6.63 ## 6 Very Good 4478 SI2 z 4.22 ## 7 Very Good 1750 VVS2 x 5.06 ## 8 Very Good 1750 VVS2 y 5.12 ## 9 Very Good 1750 VVS2 z 3.2 ``` ``` r jewels3 %>% pivot_wider(names_from=dimension, values_from=measurement) %>% head(n = 4) ``` ``` ## # A tibble: 3 × 6 ## cut price clarity x y z ## <ord> <int> <ord> <dbl> <dbl> <dbl> ## 1 Ideal 849 VVS2 4.74 4.76 2.89 ## 2 Very Good 4478 SI2 6.7 6.63 4.22 ## 3 Very Good 1750 VVS2 5.06 5.12 3.2 ``` --- name: tidying_data_separate # Tidying Data with `separate` If some of your columns contain more than one value, use `separate`: ``` r jewels4 ``` ``` ## # A tibble: 5 × 4 ## cut price clarity dim ## <ord> <int> <ord> <chr> ## 1 Ideal 849 VVS2 4.74/4.76/2.89 ## 2 Very Good 4478 SI2 6.7/6.63/4.22 ## 3 Very Good 1750 VVS2 5.06/5.12/3.2 ## 4 Very Good 1829 VS1 5.11/5.16/3.21 ## 5 Very Good 612 VVS2 4.22/4.25/2.6 ``` ``` r jewels4 %>% separate(dim, into = c("x", "y", "z"), sep = "/", convert = T) ``` ``` ## # A tibble: 5 × 6 ## cut price clarity x y z ## <ord> <int> <ord> <dbl> <dbl> <dbl> ## 1 Ideal 849 VVS2 4.74 4.76 2.89 ## 2 Very Good 4478 SI2 6.7 6.63 4.22 ## 3 Very Good 1750 VVS2 5.06 5.12 3.2 ## 4 Very Good 1829 VS1 5.11 5.16 3.21 ## 5 Very Good 612 VVS2 4.22 4.25 2.6 ``` --- name: tidying_data_unite # Tidying Data with `unite` If one value is split across several columns, use `unite`: ``` r jewels5 ``` ``` ## # A tibble: 5 × 7 ## cut price clarity_prefix clarity_suffix x y z ## <ord> <int> <chr> <chr> <dbl> <dbl> <dbl> ## 1 Ideal 849 VV S2 4.74 4.76 2.89 ## 2 Very Good 4478 SI 2 6.7 6.63 4.22 ## 3 Very Good 1750 VV S2 5.06 5.12 3.2 ## 4 Very Good 1829 VS 1 5.11 5.16 3.21 ## 5 Very Good 612 VV S2 4.22 4.25 2.6 ``` ``` r jewels5 %>% unite(clarity, clarity_prefix, clarity_suffix, sep='') ``` ``` ## # A tibble: 5 × 6 ## cut price clarity x y z ## <ord> <int> <chr> <dbl> <dbl> <dbl> ## 1 Ideal 849 VVS2 4.74 4.76 2.89 ## 2 Very Good 4478 SI2 6.7 6.63 4.22 ## 3 Very Good 1750 VVS2 5.06 5.12 3.2 ## 4 Very Good 1829 VS1 5.11 5.16 3.21 ## 5 Very Good 612 VVS2 4.22 4.25 2.6 ``` --- name: missing_complete # Completing Missing Values Using `complete` ``` r jewels %>% head(n = 10) %>% select(cut, clarity, price) %>% mutate(continent = sample(c('AusOce', 'Eur'), size = 10, replace = T)) -> missing_stones ``` ``` r missing_stones %>% complete(cut, continent) ``` ``` ## # A tibble: 14 × 4 ## cut continent clarity price ## <ord> <chr> <ord> <int> ## 1 Fair AusOce SI1 4276 ## 2 Fair Eur <NA> NA ## 3 Good AusOce <NA> NA ## 4 Good Eur <NA> NA ## 5 Very Good AusOce VVS2 1750 ## 6 Very Good AusOce VS1 954 ## 7 Very Good Eur SI2 4478 ## 8 Very Good Eur VS1 1829 ## 9 Very Good Eur VVS2 612 ## 10 Premium AusOce VS2 2623 ## 11 Premium Eur <NA> NA ## 12 Ideal AusOce VVS2 849 ## 13 Ideal Eur SI1 4523 ## 14 Ideal Eur VVS1 838 ``` --- name: joins # Joining Data with `_join` .pull-left-50[ ``` ## # A tibble: 5 × 2 ## key value1 ## <dbl> <chr> ## 1 1 a ## 2 2 b ## 3 3 c ## 4 4 d ## 5 5 e ``` ] .pull-right-50[ ``` ## # A tibble: 5 × 2 ## key value2 ## <dbl> <chr> ## 1 1 A ## 2 2 B ## 3 3 C ## 4 6 F ## 5 7 G ``` ] ``` r inner_join(tibble1, tibble2, by = 'key') ``` ``` ## # A tibble: 3 × 3 ## key value1 value2 ## <dbl> <chr> <chr> ## 1 1 a A ## 2 2 b B ## 3 3 c C ``` --- name: joins2 # Mutating and Filtering Joins * `inner_join()` -- rows with a match in both tables, * `left_join()` / `right_join()` -- keep all rows of the left / right table (`NA` when no match), * `full_join()` -- keep all rows of both, * `anti_join()` -- rows of `x` with **no** match in `y`; `semi_join()` -- rows **with** a match. ``` r left_join(tibble1, tibble2, by = 'key') anti_join(tibble1, tibble2, by = 'key') ``` ``` ## # A tibble: 5 × 3 ## key value1 value2 ## <dbl> <chr> <chr> ## 1 1 a A ## 2 2 b B ## 3 3 c C ## 4 4 d <NA> ## 5 5 e <NA> ## # A tibble: 2 × 2 ## key value1 ## <dbl> <chr> ## 1 4 d ## 2 5 e ``` Different key names: `by = c("id" = "key")` or `by = join_by(id == key)`. --- name: more_tidyverse # Some Other Friends * `stringr` for string manipulation and regular expressions, * `forcats` for working with factors, * `lubridate` for working with dates, * `purrr` for functional programming (`map()` and friends), * `ggplot2` for graphics -- the next module! --- name: end-slide class: end-slide # Thank you. Questions? [More?](https://r4ds.hadley.nz/) .end-text[ <p class="smaller"> <span class="small" style="line-height: 1.2;">Graphics from </span><img src="./assets/freepik.jpg" style="max-height:20px; vertical-align:middle;"><br> Created: 12-Nov-2026 • <a href="https://www.scilifelab.se/">SciLifeLab</a> • <a href="https://nbis.se/">NBIS</a> </p> ]