Build reproducible and shareable data analyses using R packages

Build reproducible and shareable data analyses using R packages
---
name: build-reproducible-and-shareable-data-analyses-using-r-packages
class: center middle inverse title-slide

]

# "Build reproducible and shareable data analyses using R packages

## _Conférence AFH 2022_

### Sébastien Rochette, Florence Mounier

.prez_github[
Material of this course is on Github: <a href="https://github.com/statnmap/teach-package-dev-rmdfirst" target='_blank'>statnmap/teach-package-dev-rmdfirst</a>
]

---

One step package building
---
class: center, middle, inverse, title-slide
name: package-flash-build

## One step package building
### From Rmd to package
---
class: slide

### Learning goals

- Understand the methodology proposed by the "Rmd First" method
- Be able to refactor a code into correctly formatted functions
- Understand the structure of a package
- Be able to build a documented and tested R package

---
class: slide

### Building a package in a few steps

- Start with a Rmd
- Build your functions inside
- Document your functions
- Inflate as a Package

.pull-left[
<img src="stagiaire_complet_files/C00-package-objectives.en_files/images/squirrels_rmd_html_snapshot.png" width="90%" />

]
.pull-right[
<img src="stagiaire_complet_files/C00-package-objectives.en_files/images/squirrels_inflated.png" width="90%" />
]

---
class: slide

### How this tutorial will be held

- E-learning platform: https://bakacode.io
- Instructors speak on slides


- Quizz where every attendees will be able to participate
- Direct questions where attendees are asked to participate
- Hands-on parts, in breakout rooms, where attendees are asked to share their screens for instructors to help
- A 5 minutes break every hour
- We stop 15 minutes before the end to say goodbye

---
class: slide

### Code of Conduct

All conference participants agree to:

- Be considerate in language and actions, and respect the boundaries of fellow participants.
- Refrain from demeaning, discriminatory, or harassing behaviour and language. 
- Alert a member of the ThinkR team if you notice someone in distress, or observe violations of this code of conduct, even if they seem inconsequential.

<!--class: inverse, contact-slide

# Contact

## ThinkR - <a href="http://thinkr.fr">http://thinkr.fr</a>-->

---

What is Bakacode?
---
class: center, middle, inverse, title-slide
name: what-is-bakacode-en

## What is Bakacode?
### The ThinkR e-learning platform
---
class: slide

### Take a tour!

**BakACode** is the home-made [ThinkR](https://rtask.thinkr.fr) e-learning platform

- Everything you need during the course is available in one place
- The server has all dependencies required for your tutorial
- You can communicate directly with your instructors

<img src="stagiaire_complet_files/C05-comment_utiliser_bakacode.en_files/images/bakacode_long_3.png" width="55%" style="display: block; margin: auto;" />
 
**Please take this tour to get used to the platform**

---
class: slide

### Take the tour and use it to answer the quiz at the end of the chapter

- The instructor creates a file named "hello.Rmd" in the "shared/" directory
  + You will need it for the quiz

Now we let you 3 minutes to read the following slides and answer the quiz at the end of this chapter

---
class: slide
### Connection

.pull-left[
<img src="stagiaire_complet_files/C05-comment_utiliser_bakacode.en_files/images/baka_login.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[
#### Connection

- Use your credentials to connect with username and password
]

Connect to <https://bakacode.io>

---
class: slide
### Sessions

.pull-left[
<img src="stagiaire_complet_files/C05-comment_utiliser_bakacode.en_files/images/baka_sessions.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[
#### Sessions overview

- View your current and past training sessions

]

---
class: slide
### Home page

.pull-left[
<img src="stagiaire_complet_files/C05-comment_utiliser_bakacode.en_files/images/baka_courses.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[
#### Learning objectives

- Learning objectives of the tutorial

]

---
class: slide
### Home page

- Learning objectives of the tutorial

#### Sections

- A box for each section
- Date and description of the section
- Click on the box to show chapters presented

]

---
class: slide
### Home page - launch

.pull-left[
<img src="stagiaire_complet_files/C05-comment_utiliser_bakacode.en_files/images/baka_courses_launch.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[
#### Launch tutorial

- Click on the Launch ("Lancer") button to start the tutorial

]

---
class: slide

### Pratice - presentation

.pull-left[
<img src="stagiaire_complet_files/C05-comment_utiliser_bakacode.en_files/images/baka_rstudio.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[
#### Vertical separator

- Maintain left click on the vertical bar and use the mouse to change left/right windows ratio

]

---
class: slide

### Pratice - presentation

- Maintain left click on the vertical bar and use the mouse to change left/right windows ratio

#### Slides

- Use arrows of your keyboard to go up and down in the slides
- You can copy-paste code parts

]

---
class: slide

### Pratice - presentation

- Maintain left click on the vertical bar and use the mouse to change left/right windows ratio

#### Slides

- Use arrows of your keyboard to go up and down
- You can copy-paste code parts

#### RStudio server

- RStudio project opened with your data and Rmd files for exercises
- Be sure to work in a project!
]

---
class: slide

### Pratice - search

.pull-left[
<img src="stagiaire_complet_files/C05-comment_utiliser_bakacode.en_files/images/baka_toc.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[
#### Search chapter

- Click on the slides
- Click on the black box in the top right corner to open the table of content
- Choose your chapter
- Click on the black box to quit the table of content and navigate

]

---
class: slide

### Pratice - search

.pull-left[
<img src="stagiaire_complet_files/C05-comment_utiliser_bakacode.en_files/images/baka_search.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[
#### Search words

- Click on the slides
- Use CTRL + F to open the slides search bar or click on the magnifier icon
- Write your word
- Press Enter to find the next occurence of your word
- Use ESC / ECH to quit the search bar and navigate

]

---
class: slide

### Pratice - export

.pull-left[
<img src="stagiaire_complet_files/C05-comment_utiliser_bakacode.en_files/images/baka_export.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[
#### Download your files

- Download files on your computer
- Select one or more files in the file Pane using checkboxes
- Open the 'More' menu
- Choose 'Export'
- 'Download'
]

> We recommend that you export your full project at the end of the course

---
class: slide

### Top menu

.pull-left[
<img src="stagiaire_complet_files/C05-comment_utiliser_bakacode.en_files/images/baka_rstudio.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[
#### Home

- Go back to home page "Accueil"

]
---
class: slide

### Top menu

.pull-left[
<img src="stagiaire_complet_files/C05-comment_utiliser_bakacode.en_files/images/baka_rstudio.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[
#### Courses

- Navigate through the list of courses sections available
]

---
class: slide

### Top menu

.pull-left[
<img src="stagiaire_complet_files/C05-comment_utiliser_bakacode.en_files/images/baka_rstudio.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[
#### RStudio

- Opens your current RStudio session as full screen in a new tab
- Note that this will close the RStudio session in the "Practice" interface

]

---
class: slide

### Top menu

.pull-left[
<img src="stagiaire_complet_files/C05-comment_utiliser_bakacode.en_files/images/baka_rstudio.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[
#### Resources

- Opens a new tab with R cheatsheets as PDF
- You can open or download them

]

---
class: slide

### Top menu

.pull-left[
<img src="stagiaire_complet_files/C05-comment_utiliser_bakacode.en_files/images/baka_rstudio.png" width="100%" style="display: block; margin: auto;" />
]
.pull-right[

#### Disconnect

- Disconnect from BakACode

]

---
class: slide

## Quiz

- Can you download the pdf of the courses?
  + It is stored in the "pdf/" folder, in the Home directory 
  + If you loose the connection, at least you have the slides
  
- Did you find the "shared/hello.Rmd" file ?
  + You can open it and write your name in it.
  + Save it quickly as it is shared with other attendees!
  
- Can you use the table of content or the search bar to retrieve my email somewhere at the end of the slides?
  + Write it in the chat in private, and keep it for yourself, just in case

<!--class: inverse, contact-slide

# Contact

## ThinkR - <a href="http://thinkr.fr">http://thinkr.fr</a>-->

---

Start with the documentation
---
class: center, middle, inverse, title-slide
name: package-preambule-en

## Start with the documentation
### What are vignettes?
---
class: slide

### Do you know package {attachment}?

Let's say you don't!

#### How to discover what does a package?

---
class: slide

### How to discover what does a package?
#### My questions as a user

- What does it do?
- How to install it with its dependencies?
- What are its function?
- How to fill parameters of this function?
- Can I have an example on how to use this function?
- Can I have an example on how to use the package as a whole?
- Will it work with the last version of R and dependencies?

---
class: slide

### How to discover what does a package?
#### What does it do?

- CRAN page: https://cran.r-project.org/web/packages/attachment/index.html

![](stagiaire_complet_files/C00-package_preambule.en_files/images/pkg_attachment_cran.png)

---
class: slide

### How to discover what does a package?
#### How to install it with its dependencies?

- `install.packages('attachment')`

???

You don't have to think about dependencies...  
The developers prepared everything for you

---
class: slide

### How to discover what does a package?
#### What are its functions?

- Index of the package

---
class: slide

### How to discover what does a package?
#### How to fill parameters of this function?

- `?att_amend_desc`

---
class: slide

### How to discover what does a package?
#### Can I have an example on how to use this function?

- `?att_amend_desc` => Examples

---
class: slide

### How to discover what does a package?
#### Can I have an overview on how to use the package as a whole?

- Vignettes
- GitHub: https://thinkr-open.github.io/attachment/articles/fill-pkg-description.html

---
class: slide

### How to discover what does a package?
#### Will it work with the last version of R and dependencies?

- README Check, https://github.com/ThinkR-open/attachment
- Unit tests, code coverage, Continuous Integration

???

Unit tests and CI are set up by developers to ensure reproducibility and maintainability

---
class: slide

### How to discover what does a package?
#### My answers as a user

|Questions |Answers |
|:------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------|
|What does it do? |<a href='https://cran.r-project.org/web/packages/attachment/index.html'>CRAN page</a> |
|How to install it with its dependencies? |`install.packages('attachment')` |
|What are its functions? |`?attachment` => Index |
|How to fill parameters of this function? |`?att_amend_desc` |
|Can I have an example on how to use this function? |`?att_amend_desc` => Examples |
|Can I have an overview on how to use the package as a whole? |<a href='https://thinkr-open.github.io/attachment/articles/fill-pkg-description.html'>Vignettes, GitHub</a> |
|Will it work with the last version of R and dependencies? |<a href='https://github.com/ThinkR-open/attachment'>README Check</a> |

_There is a dedicated website that gathers all these answers: https://thinkr-open.github.io/attachment/_

???

We will explore this website later

---
class: slide

### How to discover what does a package?
#### Developers point of view

- All these sources of information are adressed by developers

#### If you start with the documentation, from the vignette, you will not have to think too much about all of these

???

You will build all the doc along the way

---
class: slide

### Let's talk again about vignettes

+ HTML pages telling the stories of the package
+ List vignettes using: `vignette(package = "thepackage")`.

---
class: slide

### Let's talk again about vignettes
 
#### Content of a vignette

<pre class='rcode-input'>vignette(topic = "colwise", package = "dplyr")</pre>

<img src="stagiaire_complet_files/C00-package_preambule.en_files/images/vignette2.png" width="60%" />
 
???

The story of the package as a whole
 
---
class: slide

### Let's talk again about vignettes

#### Vignette = Rmarkdown (usually)

- What is the story of the package
- What is the function for
- How to use it with or without parameters
- Reproducible examples
- Different outputs depending on chosen parameters

#### Let's start building our package from its story ! 
 
---
class: slide

### Quizz: What is a vignette?

- A: A website somewhere on Internet
- B: A Sticker you can add on your laptop
- C: A R script with the help of function accessible in the installed package
- D: A html page built from a Rmd file, accessible in the installed package

---
class: slide

### Quizz: Have you already built a package with all these?

- A: Yes, everything. Functions, examples, tests, vignettes
- B: Only part of documentation. Functions, examples, maybe vignettes
- C: Only functions in a R/ directory
- D: No. I never built a package from scratch

<!--class: inverse, contact-slide

# Contact

## ThinkR - <a href="http://thinkr.fr">http://thinkr.fr</a>-->

---

Discover the structure of a package with {fusen}
---
class: center, middle, inverse, title-slide
name: package-fusen-discover-en

## Discover the structure of a package with {fusen}
### Where does all of this come from?
---
class: slide

### {fusen} : adopt "Rmd-first" method

- Start with documentation
- Develop everything in a familiar place: the RMarkdown
- {fusen} inflates the package for you

> What if there was a package that could take an Rmd file, kind of like a sheet of paper, and if you follow the right folding, you can blow it up like a package?

---
class: slide 
### [Short version] - Preamble

- What are the principal components of a package?
- Where do I have to write what?
- How does {fusen} make my life easier?

We suggest you to :

+ Watch your trainers create a package {mytools} by following these short steps, without practicing yourself

+ Redo this package {mytools} on your own in a new project

We won't explain everything here, but you will see the components of a package and where they fit. For the details, that's the goal of your complete training!

---
class: slide 
### [Short] 1: Create a new {fusen} project

In Rstudio :

- File > New project > New directory > Package using {fusen} 
- Choose the name of the package (explicit, in lower case) 
  - Name of the package: "mytools"
> _no capital letters, underscores, spaces or special characters_
- Choose the {fusen} template in the dropdown menu: "teaching"

- Choose the directory where to save the project
- Create the project

???

- You are about to build a package. This is a set of tools for testing package structure.   
Thus, {mytools}

- The directory is the "Home" in our platform using `~`

---
class: slide 
### [Short] 2: Open the {fusen} Rmd flat template

- The project opens up on a flat template file: "flat_teaching.Rmd"

- Here are the main components of a package, in a unique Rmd file
  + Note the name of the chunks that are required
  + The present file is pre-filled with examples

.pull-left[
<img src="stagiaire_complet_files/C01a-package_fusen_discover.en_files/images/fusen_skeleton_desc.png" width="75%" />
]

.pull-right[
<img src="stagiaire_complet_files/C01a-package_fusen_discover.en_files/images/fusen_skeleton_fun.png" width="75%" />
]

???

You can see that {fusen} opens up the "flat_teaching.Rmd" file in RStudio.  
There are a few additional files that we will explore later.  
This Rmd file is the "teaching" template with different chunks, which you will not modify this time.

Let's quickly explore the content of this Rmd. A description, some functions along with examples and tests.
And a final 'development' chunk asking to inflate

---
class: slide 
### [Short] 3 : Description

Describe your future package:

- Change the name with yours in the `description` chunk
- Execute the complete content of the `description` chunk
  + "CTRL + SHIFT + ENTER" should be good
  + There are two functions to execute here
- A "DESCRIPTION" file appears with the same information

#### DESCRIPTION is the first source of documentation for your package

???

Yeah, you started with documentation !  
There are some more fields in the DESCRIPTION file, but we'll see them later

---
class: slide 
### [Short] 4 : Inflate the package

- Go down the Rmd file and inflate

**You built a package!**

---
class: slide 
### [Short] 5 : Does it work ?

- The 'Build' tab should already appear in RStudio
  + Otherwise, restart your RStudio session

- Install the package
  + Panel Build > Install and Restart
  
- Test the package directly in the console
  + `mytools::add_one(value = 56)`

- Test the knit of the *vignette*
  + The vignette is opened "get-started.Rmd"
  + Hit the "Knit" button
  
- Check that the help for your function appears
  + `?add_one`
  + Run the reproducible example from help

---
class: slide 
### [Short] 5 : Does it work ?

If you verified everything listed above, your RStudio should look like this

---
class: slide
### Your turn

#### Go back to section "[Short] 1" and execute the steps yourself until this slide

<!--class: inverse, contact-slide

# Contact

## ThinkR - <a href="http://thinkr.fr">http://thinkr.fr</a>-->

---

Package express with {fusen}
---
class: center, middle, inverse, title-slide
name: package-express-fusen-en

## Package express with {fusen}
### An Rmd is a package
---
class: slide

### Preparation: Tools
 
To create a package, we will use:

+ RStudio.
+ {fusen} package
+ the packages {pkgbuild}, {devtools}, {usethis} and {attachment} to save time.
+ the {roxygen2} package to generate the documentation.
+ the {testthat} package to generate unit tests.
+ Rtools.exe (optional and under windows only).

<pre class='rcode-input'>install.packages(c("fusen", "devtools", "usethis", "pkgbuild", "roxygen2", "attachment", "testthat"))</pre>

Rtools is available here:   [https://cran.r-project.org/bin/windows/Rtools/](https://cran.r-project.org/bin/windows/Rtools/)  
Once (properly) installed `pkgbuild::has_rtools()` should return `TRUE`. 
Rtools installs everything needed to compile c++ etc.

???

All of these should be installed on the server already

---
class: slide

### {fusen}: adopt "Rmd-first" method

- Start with documentation
- Develop everything in a familiar place: the RMarkdown
- {fusen} inflates the package for you

> What if there was a package that could take an Rmd file, kind of like a sheet of paper, and if you follow the right folding, you can blow it up like a package?

---
class: slide

### How {fusen} works?

- {fusen} copy-pastes in the right place

---
class: slide 
### Longer version - Preamble

This procedure contains **10 steps**, to be done in order.
Some points are not explained in detail to allow you to obtain a functional R package quickly.

We suggest you to:

+ Watch your trainers create a package by following these steps, without practicing yourself
--

+ Do this package {hello} on your own in a new project

> Procedure is split in two parts to let you practice. You will thus have 2 times: "Watch - Do"

---
class: slide 
### Step 1: Create a new {fusen} project

In Rstudio:

- File > New project > New directory > Package using {fusen} 
- Choose the name of the package (explicit, in lower case) 
  - Name of the package: "hello"
> _no capital letters, underscores, spaces or special characters_
- Choose the {fusen} template in the dropdown menu: "minimal"

- Choose the directory where to save the project
- Create the project

> You could also directly run: `fusen::create_fusen(path = "~/hello", template = "minimal")`

???

You are about to build a package. This is a set of tools to be polite with other people, starting by saying hello.   
Thus, {hello}

- The directory is the "Home" in our platform using `~`

---
class: slide

### Step 2: Open the {fusen} Rmd templates

This time the template is divided into two flat Rmd files

- "0-dev_history.Rmd" with the general development commands. In particular the `description` part.
- "flat_minimal.Rmd" with the empty skeleton for creating a function like in the previous "teaching" template

???

You can see that {fusen} opens up the "0-dev_history.Rmd" file in RStudio.  
There are a few additional files that we will explore later.  
This Rmd file is the "minimal" template with different chunks, empty or not empty, that we will fill together in the next steps

---
class: slide 
### Step 3: Description

The description of the package takes place in the first `description` chunk of the "dev_history.Rmd" file  
  
The first fields to fill in:

- Title: "Quick Description of the Goal of your Package" (Title Case, no dot at the end)
- Description: "Long description. Sentence case with a dot at the end of the sentences."
- `Authors@R`: vector of one or more `person()` 
  + `person("Sébastien", "Rochette", email = "sebastien@thinkr.fr", role = c("aut", "cre"))`
- License: The choice of the license

#### Let's run the content of chunk `description`

> Observe the content of file "DESCRIPTION" created

???

You are about to build a package, you need to inform the user about its aim. Here, the aim is a set of tools to be polite with other people, starting by saying hello.     
You also need to say who you are, so that users know who to call in case of problems.  
Finally, the license allows you to say how you want your package to be used and shared. Without license, no one is supposed to use your package.

Observe the content of DESCRIPTION. Note that what you wrote in the Rmd is now copied in this file, but you did not have to move from your Rmd. So we stay in the Rmd, and we continue the development.

---
class: slide

### Step 4: Documentation of the package

A package is created to automate some operations.
Starting with the documentation forces you to think about the structure of the package and the logical sequence of operations.

- Say what you do in the Rmd text part
- Do what you said in the `function` chunk

#### Open the "flat_minimal.Rmd" template

#### Let's run the content of chunk `development`

---
class: slide

### Step 4: Documentation of the package

#### Let's see the process of writing a function

You can say hello to someone in particular using `say_hello()`.

```{r development}
library(glue) # On top with others

message("Hello someone")

someone <- "Seb"
message(glue("Hello {someone}"))
```
````
]

.pull-right[
1. Describe in words the first operation that the package will have to solve
2. Define the input data, the possible modifiable parameters, the output result
3. Write the R code to perform these operations in a `development` chunk
4. Call necessary packages in a `development` chunk only, at the very beginning, with other `library()` calls
]

???

Here we take a simple example with a code to say hello. This will be the first tool of our package to be polite with people around us.  
Forget that you are about to build a package. Now we develop in a Rmd as usual.  
First, we write what we are about to do.  
Then, we write some code to say hello. But, I want it to be able to say hello to someone else than me. 
So I add a parameter.

---
class: slide

### Step 5: Embed in a function

- Move code in the `function` chunk as soon as you transformed it as a function
- Add examples of use in the `examples` chunk

````markdown
## Say hello to someone

You can say hello to someone in particular using `say_hello()`.

```{r function}
say_hello <- function(someone) {
 message(glue("Hello {someone}"))
}
```

```{r examples}
say_hello(someone = "Seb")
```
````

- Each chunk has a specific task
- Run the chunk `function` to make it available
- Run the function in the `examples` chunk of the Rmd to try it

???

- Use the code written previously.  
- Embed in the `function()` function.  
- Use the parameter as a parameter of the function.

- In the examples chunk, we have a reproducible example.
- We will use it as much as we can
- We use it to illustrate the use of the function.
- You'll see that we will also use it for unit tests. Later.

---
class: slide

### Step 5: Embed in a function

The created function can now be documented

- Add the doc in {roxygen2} format
  + `@param` to present the content of inputs
  + `@export` for the function to be accessible to the users
  + `@importFrom package function` for functions coming from other packages
  + `@return` to describe the object that comes out of the function

- Use RStudio menu: Code > Insert Roxygen Skeleton

> You must declare the dependencies for the package with `@importFrom`.
The calls to `library()` are only there for development, like a classic Rmd, but are not used by the package when inflated

???

As you just wrote the code of your function, you know exactly

- what the aim is,
- what are the parameters for,
- what is the output 
- what are the dependencies needed

Write it now. In an hour, you will have forgotten it!

Note that:

- because we are in an Rmd, you won't be able to get the auto-completion for roxygen
- because we are in an RStudio Server, the keyboard shortcut for roxygen may be in competition with your browser shortcuts

Let's see how it looks on the next slide

---
class: slide

### Step 5: Embed in a function

- Your "flat_minimal.Rmd" file should look like this:

````markdown
## Say hello to someone

You can say hello to someone in particular using `say_hello()`.

```{r function}
#' Show a message in the console to say Hello to someone
#' 
#' @param someone Character. Name of the person to say hello to
#' @importFrom glue glue
#' @return Used for side effect. Outputs a message in the console
#' @examples
#' @export

say_hello <- function(someone) {
 message(glue("Hello {someone}"))
}
```

```{r examples}
say_hello(someone = "Seb")
```
````

???

Here you can see the minimal roxygen content.

- Title
- Use of param
- importFrom external dependencies
- return an output. This one's special. There is no re-usable output.
- example is in the following chunk. We will let {fusen} deal with this. This is specific to {fusen} to not fill the example here. We'll see this later.
- export the function to let it available to the user when they will call library(your package)
- There may still be a `development` chunk. That's ok.

---
class: slide

### Your turn!

- We are in the half way point of the process
- Got back to Step 1 of this Longer version
- Follow the 5 steps until this slide

???

Tell us in the Chatroom when you are done

---
class: slide

### Step 6: Write a unit test

- What do you look for when you run your example?
- What makes you say "it works"?

#### Write your thoughts into code

````markdown
```{r tests}
test_that("say_hello works", {
  expect_message(say_hello(someone = "Seb"), "Hello Seb")
})
```
````

???

- We use the same exact example to write the unit test directly.  
- I won't detail the syntax here, and I won't detail all possible test functions.
- For now, I only want you to have the reflex to do it right after your example, because you already have the test in your mind

---
class: slide

### Step 7: Inflate the package

- Let {fusen} inflate the Rmd into a documented and tested package

<pre class='rcode-input'>fusen::inflate(flat_file = "dev/flat_minimal.Rmd", vignette_name = "Say Hello!")</pre>

> If there are any errors or warnings, read them carrefuly, address them in the "flat_minimal.Rmd" and inflate the package again.

???

- Let {fusen} inflate the Rmd into a package
- There are multiple messages, we do not read all of them. 
- We try to address errors at least and re-inflate if needed

---
class: slide

### Step 8: Explore created folders and files

- "DESCRIPTION" with dependencies
- "R/" with the function
- "man/" with LateX documentation
- "tests/testthat/" with unit tests
- "vignettes/" with documentation

---
class: slide

### Step 8: Explore created folders and files

- "DESCRIPTION" with dependencies
- "R/" with the function
- "man/" with LateX documentation
- "tests/testthat/" with unit tests
- "vignettes/" with documentation

---
class: slide

### Quiz: Find good definitions

Link files to their description:

1. DESCRIPTION  
  
2. dev_history / flat_minimal  
  
3. vignettes  
  
4. script with roxygen

5. testthat

**Possible answers:**

- a: ACBDE

- c: EABCD
]
.pull-right[
*Documentation*  
  
A. Development process for developers
  
B. Present all the functions of the package and its story  
   
C. How to use each function (for the user)

D. Testing the functions for the developers
  
E. Content and objectives of the package for all

- b: EDCBA

- d: BAECD
]

???
- 1E and dependencies for installation
- 2A reusable from one package to another
- 3B with a story
- 4C and build info (@importFrom, @export)
- 5D unit tests

---
class: slide

### Step 9: Verify again the package

Generate documentation

- `attachment::att_amend_desc()`

Check that the package follows the packages rules

- `devtools::check()`  
  
- Solve potential problems in the "flat_minimal.Rmd"
- Re-inflate the package if necessary

- Reach **0 Error, 0 Warnings, 0 Notes**

> Store this commands in the "0-dev_history.Rmd" file

> Note that `fusen::inflate()` already launches this two commands, but who knows!

???

It is always good to know these commands, although `inflate()` already does them. You may need them if you go back to a classical way of maintaining your package

---
class: slide

### Step 10: Install and use your package

- The 'Build' tab should already appear in RStudio
  + Otherwise, restart your RStudio session
  
- Note that you can "check" in the 'Build' panel
  + Panel "Build" > "Check"
  
- Install the package
  + Panel "Build" > "Install and Restart
  
- Test the package directly in the console
  + `hello::say_hello("Toto")`

- Test the knit of the vignette
  
- Check that the help for your function appears
  + `?say_hello`
  + Run the reproducible example from help

---
class: slide

### Your turn

- Go back to step 6 of this Longer version
- Continue development and inflate your package

> Edit "flat_minimal.Rmd" and re-execute `inflate()` as many times as necessary until everything runs smoothly.

> Bonus: If you are motivated, you can start again from the beginning of the 10 steps procedure with a new package name {hello2}

---
class: slide

### How to add new functionnalities?

This would require to start over from 'Step 4' to:

- Upgrade your existing function

- Add a new function in the current flat template
  + New entitled section with `function`, `examples`, `tests`
  + Inflate
  + Install
  
- There is a RStudio _Addin_ to "Add {fusen} chunks"

- Or create a new flat template using `fusen::add_flat_template("add")` for a new family of functions, thus new vignette

- There is a RStudio _Addin_ to "Add {fusen} flat template"

- And `inflate()` this new "flat_additional.Rmd"

---
class: slide

### And the classical way without {fusen}?

- You can use the 'Rmd first ' approach writing your code in a Rmarkdown file
  + this can directly be your vignette
- Then, you'll need to fill / copy the different files yourself
  - "R/"
  - "tests/"
  - "vignettes/"

---
class: slide

### And the classical way without {fusen}?

- File > New Project > New directory > Package with devtools
- Fill DESCRIPTION file
- Run function for the desired license
  + `usethis::use_*_license()`
- Develop in a Rmd
  + Either in the sub-directory "dev/" + `usethis::use_build_ignore("dev/")`
  + Or, develop in a vignette directly
- Copy/Cut in the correct place
  + functions + examples => "R/"
      + `usethis::use_r("ma_fonction")`
  + tests => "tests/testthat/"
      + `usethis::use_testthat()`
      + `usethis::use_test("ma_fonction")`
  + vignettes => "vignettes/"
      + `usethis::use_vignette("Le titre de ma vignette")`
- Generate documentation
  + Either `attachment::att_amend_desc()`
  + Ou `roxygen2::roxygenise()` + Fill in DESCRIPTION (Suggests, Imports)
- Check the package
  + `devtools::check()` => `0 errors, 0 warnings, 0 notes`

---
class: slide

### And the classical way without {fusen}?

- You need to fill the different files yourself
  - "R/"
  - "tests/"
  - "vignettes/"

- While developing you could
  - Run an example of your function in "R/" directly with Ctrl + Enter
  - Run the unit test by clicking on "Run test"
  - Run the vignette if your package is installed

- Note that you can still do these actions using {fusen} after `inflate()`

> Be careful, when using {fusen}, if you want to modify some code, go back to the "flat_*.Rmd" and do `inflate()` again

???

Show that we can do it with fusen too as it is a real classical package

---
class: slide

### Exercise: Take time to finish your drawings

Where were moved the pieces of code from chunks:

- `description`?
- `function`?
- `example`?
- `tests`?
- `development`?

> Verify and update your previous drawings.
> See each thing that {fusen} does for you!

???

- `description`: in DESCRIPTION
- `function`: in the independent .R file with the name of the function
- `example`: in the independent .R file
  + in the independent .R file with the function name
  + in the thumbnail
- tests`: in the independent tests/testthat folder with the function name
- development`: nowhere. it stays in the development tracking file "dev_history.Rmd

<!--class: inverse, contact-slide

# Contact

## ThinkR - <a href="http://thinkr.fr">http://thinkr.fr</a>-->

---

Include datasets in your package
---
class: center, middle, inverse, title-slide
name: encapsuler-donnees-en

## Include datasets in your package
### What about data?
---
class: slide

### Include datasets

In a package, it can be useful to include data.

+ To demonstrate the use of a function with a relevant example
+ To disseminate information

There are three types of datasets to include, thus three ways to include them into a package.

1. A raw data file (xlsx, csv or other), **NOT** available to the end user, but accessible to the developers only, stored in `data-raw/` folder

2. An example dataset in `rda` format, to be loaded as is, available to the user, stored in the `data/` folder (such as `iris` or `mtcars` for example)

3. A data file (xlsx, csv or other) not transformed into `rda`, available to the end user, stored in the `inst/` folder

???

Let's see how and why

---
class: slide

### 1. Include datasets in `data-raw/`
 
#### Internal dataset, accessible to developers only
 
- Developers store raw datasets to keep the source of examples close to them
- Users can not access them, there are not installed

*Steps:*

- Create folder `data-raw/` at the root of the package using: `usethis::use_data_raw()`
- Insert raw datasets inside
- Use R/Rmd scripts to prepare a cleaner / smaller dataset for future examples

*Information:*

- `data-raw` is not installed with the package
 + It is not accessible to the user
- Example: <https://github.com/ThinkR-open/prenoms/tree/master/data-raw>

???

- Prepare data that you will need for your reproducible examples
- Keep the original source of your datasets in a safe place
- Note that with {fusen}, you can realise the preparation steps in a `development` chunk.

---
class: slide

### 2. Include datasets in `data/`

#### Exported dataset, accessible to the user

- Developers includes a dataset in the package using: `usethis::use_data(my_dataset)` 
- Users load the dataset named `my_dataset` using: `data(my_dataset)`

*Steps:*

- Create `data-raw/` to prepare your dataset using: `usethis::use_data_raw("my_dataset")`
- Prepare your dataset as needed for reproducible examples
- Store `my_dataset` as internal data

<pre class='rcode-input'># Read some raw data 
# my_data_to_clean &lt;- readr::read_csv("my_raw_data.csv")

# Or use existing dataset like `diamonds`
my_dataset &lt;- dplyr::slice_sample(diamonds, prop = 0.2)
usethis::use_data(my_dataset, overwrite = TRUE)</pre>

- `overwrite = TRUE` overwrite the dataset if already exists, as for an update
- Have a look at the content of newly created folder `data/`

???

- We use the directory `data-raw` previously created to prepare dataset
- `use_data()` stores the reproducible example at the right place in the right format

---
class: slide

### 2. Include datasets in `data/`

#### Exported dataset, accessible to the user

- Developers includes a dataset in the package using: `usethis::use_data(my_dataset)` 
- Users load the dataset named `my_dataset` using: `data(my_dataset)`

*Information:*

- `my_dataset` is accessible to the user after package installation
<pre class='rcode-input'>library(mypackage)
data(my_dataset)</pre>
- `my_dataset` is stored as `.rda` file, only readable by R, similar to `.RData` files

???

Package needs to be installed to access the dataset

---
class: slide

### 2. Include datasets in `data/`

#### Exported dataset, accessible to the user

- Exported datasets need to be documented, like functions
- by convention the documentation will be written in a file of the form "R/doc_my_dataset.R".
--

- The following script will automatically build the documentation to be completed:

<pre class='rcode-input'>my_dataset &lt;- dplyr::slice_sample(diamonds, prop = 0.2)
usethis::use_data(my_dataset, overwrite = TRUE)

cat(sinew::makeOxygen("my_dataset"),
 file = "R/doc_my_dataset.R")
rstudioapi::navigateToFile("R/doc_my_dataset.R")</pre>

_à rajouter dans "data-raw/my_dataset.R"_

???

Documentation is always the key for a correct package. 
Check will remind you of missing data documentation.

---
class: slide

### 2. Include datasets in `data/`

#### Exported dataset, accessible to the user

- Exported datasets need to be documented, like functions

*Information:*

2 main tags:

- `@format` (required) a summary of the data.
  - if `@format` is not specified, then roxygen will edit a standard one.

- `@source` : the origin of the data, the source.

We do not `@export` a dataset.

---
class: slide

### 3. Include datasets in `inst/`

#### Raw dataset, accessible to the user

- Developers includes a dataset in the `inst/` folder as is directly
- Users find the dataset using `system.file("my_dataset.csv", package = "mypackage")`

*Steps*

- Create `inst/` folder using: `dir.create(here::here("inst"))`
- Add a dataset inside
- Install the package
- Users can access the path of the dataset and read the file as usual

<pre class='rcode-input'># Store "my_dataset.csv" in "inst/" folder
the_data_path &lt;- system.file("my_dataset.csv", package = "mypackage")
the_data &lt;- readr::read_csv(the_data_path)</pre>

---
class: slide

### 3. Include datasets in `inst/`

#### Raw dataset, accessible to the user

- Developers includes a dataset in the `inst/` folder as is directly
- Users find the dataset using `system.file("my_dataset.csv", package = "mypackage")`

*Information*

- Any type of data is allowed
- Use the path for your reproducible examples
- Organise the content of `inst/` as your want

???

- system.file() does not read the dataset, it returns the path to the dataset
- The dataset is only available after installation of the package

---
class: slide

### Deal with datasets during development

- Datasets in `data/` or `inst/` are only available after package installation
- Simulate installation using `pkgload::load_all()` during development
  + `system.file()` temporarily returns development path (not installed path)

*Steps*

- Check your reproducible examples in the Rmd file with package data

<pre class='rcode-input'># For development only
pkgload::load_all()

# Same code to add in your `examples` or `tests`
# Can be tested directly during development
the_data_path &lt;- system.file("my_dataset.csv", package = "mypackage")
the_data &lt;- readr::read_csv(the_data_path)</pre>

> Note that `pkgload::load_all()` also loads functions in development, as it simulates a real installation

???

---
class: slide

### Quizz

If I want to provide a dataset to the user in any format I want, where do I store it?

- A: `extdata/`
- B: `inst/`
- C: `data/`
- D: `data-raw/`

---
class: slide

### Your turn!

1. Look at the instructor demonstration with *Steps*
2. Do it yourself

*Steps*

- Create a new {fusen} project with `full` template
- Run the content of the `description` chunk in the "dev_history.Rmd"
- Uncomment `development` chunk in "Read data" section in the "flat_full.Rmd"
- Play with data in `inst/` during development

*Bonus*

- Create function `check_data_integrity()` that reads a dataset like `nyc_squirrels` and check its integrity
  + For instance, `primary_fur_color` columns should only contains a unique color, there should not be any `+` sign inside this column
  + Stop the function if the integrity is not good
  + Return a message if everything is ok
- Create a reproducible example using the dataset saved in `inst/` with `system.file()`
- Create unit tests with reproducible examples where function should fail

???

https://github.com/statnmap/squirrels.fusen/blob/main/dev/dev_history.Rmd

<pre class='rcode-input'>#' Check data integrity
#'
#' @param x dataframe with at least columns "lat", "long" and "primary_fur_color"
#'
#' @return Original dataframe if all tests are good. Otherwise stops.
#' @export
check_data_integrity &lt;- function(x) {
 # Verify points are in New York around Central Park
 all_coords_ok &lt;- all(
 c(
 min(x[["lat"]]) &gt; 40.76400,
 max(x[["lat"]]) &lt; 40.80100,
 min(x[["long"]]) &gt; -73.98300,
 max(x[["long"]]) &lt; -73.94735
 )
 )
 if (!all_coords_ok) {stop("Not all data are in Central Park")}

# Verify there is only one color in primary_fur_color.
 # A `+` in the column is a sign of multiple colours
 if (any(grepl("+", x[["primary_fur_color"]], fixed = TRUE))) {
 stop("There are multiple colors in some 'primary_fur_color'")
 }

message("All tests are good !")
}</pre>

<pre class='rcode-input'># A working example
my_data_example &lt;- data.frame(
 lat = c(40.77, 40.78),
 long = c(-73.95, -73.96),
 primary_fur_color = c("grey", "black")
)
check_data_integrity(my_data_example)</pre>

<!--class: inverse, contact-slide

# Contact

## ThinkR - <a href="http://thinkr.fr">http://thinkr.fr</a>-->

---

Versioning a {fusen} package with git
---
class: center, middle, inverse, title-slide
name: versionner-fusen-git-en

## Versioning a {fusen} package with git
### And the particular case of {fusen}?
---
class: slide

## Version a {fusen} package

2 possibilities:

- Your {fusen} package has not been created yet, and you want to set up the versioning before starting the developments

- Your {fusen} package has already been created, and you now want to version it

---
class: slide

## Versioning a {fusen} package

#### Case of a new {fusen} package

- Initialize the empty project on GitLab _(remembering to use the name of your future package as project name)_

- Get the `https` link to your project by clicking on the `clone` button

- In RStudio, create the new project File > New Project > Version Control > git and link to the Repository URL

Initiate your new {fusen} package:

<pre class='rcode-input'>fusen::create_fusen(path = ".",
 template = "minimal",
 overwrite = TRUE)</pre>

---
class: slide

## Versioning a {fusen} package

#### Case of a new {fusen} package

In the  *Terminal*,

+ Switch in a new _main_ branch:
 
<pre class='rcode-input'>
git switch -c main
</pre>

+ Select modified files in your project:

+ Write a commit (with an explicit message):

<pre class='rcode-input'>
git commit -m "Init fusen package"
</pre>

+ Push your changes back to the remote:

<pre class='rcode-input'>
git push -u origin main
</pre>

You can now begin your developments.

---
class: slide

## Versioning a {fusen} package

#### Case of an existing {fusen} package

> This procedure can be used for any type of R project, and therefore for any type of package (not only {fusen} packages)

- Initialize the empty project on GitLab _(remembering to use the name of your package as project name)_

- Retrieve the `https` link to your project by clicking on the `clone` button

- Open the RStudio project and run the command `usethis::use_git()` in the console _(the {usethis} package must be installed)_

- Answer yes to all the questions asked in the console, if relevant. A first "Initial Commit" is done for you.

- RStudio restarts and git is operational locally

???

ATTENTION
Si vous modifiez cette slide, modifiez aussi M14S01C06 pour git1jour

---
class: slide

## Versioning a {fusen} package

#### Case of an existing {fusen} package

Now you have to link this project to your **`remote`**:

<pre class='rcode-input'>usethis::use_git_remote("origin",
 url = "https://gitlab.com/my_name/mypackage.git",
 overwrite = TRUE)</pre>

In the *Terminal*, type:

```
git push -u origin main
```

You just added a **`remote`** and made a first **`push`** on the **`main`** branch.

???

ATTENTION
Si vous modifiez cette slide, modifiez aussi M14S01C06 pour git1jour

---
class: slide

## Exercise

Set up the _git_ versioning for your `{hello}` package

<!--class: inverse, contact-slide

# Contact

## ThinkR - <a href="http://thinkr.fr">http://thinkr.fr</a>-->

---

What about data analyses in a package?
---
class: center, middle, inverse, title-slide
name: data-analyses-packages

## What about data analyses in a package?
### From Rmd to package to Rmd
---
class: slide

## Create shareable reports through packages

### Create a package with your functions

Then, two possible ways to build your analysis reports:

- Classical package: Use external projects to use your package functions in Rmd reports
- Compendium-like: Include the Rmd reports inside the package development project

---
class: slide

## A Compendium is like a package

The compendium logic is to separate code from the report output:

- One place to store your R scripts in "R/"
- One place to store the Rmarkdown report in "analyses/"

This is similar to package logic:

- One place to store R scripts in "R/"
- One place to store Rmarkdown examples of use in "vignettes/"

### Let's combine both worlds to benefit from robustness of packages

---
class: slide

## Combine {fusen} and Compendium structure

- Create a {fusen} project, versioned with git

<pre class='rcode-input'># install.packages("fusen")
fusen::create_fusen(template = "full", with_git = TRUE)</pre>

- Add a "reports/" directory that will not interfer with the package

<pre class='rcode-input'>dir.create("reports")
usethis::use_build_ignore("reports")</pre>

_Note: you can add the Compendium structure in the "reports/" sub-directory, with package {rcompendium}_
 
<pre class='rcode-input'># install.packages("rcompendium")
rcompendium::add_compendium("reports")</pre>

---
class: slide

## Combine {fusen} and Compendium structure

- Add a reduced dataset for the package in "inst/" or "data/"
  + Use it for examples, unit tests, documentation
  
  
- Build your functions in a "flat" file, like any {fusen} package
  + Write the content of the vignette as a reduced version of your future report
  + Create examples, unit tests, documentation as usual
  + Commit regularly
  
  
- Use the "reports/" directory to store your reports and outputs, while using the functions of your package
  + Create a Rmarkdown report using your functions loaded with `library(my.package)`
  + Do not forget to install your package prior to building your report

---
class: slide

## Why would I include my analysis inside my package?

**Because of:**

- {rim}: manage R versions
- {renv}: manage R packages versions
- and Docker: manage OS and system dependencies

---
class: slide

## Why would I include my analysis inside my package?

- Develop your project in a controled environment, with fixed versions of packages
  - Functions, tests and examples are valid for a specific set of packages versions with a specific R version
  - Use your functions for your analysis reports with the same working environment
  - Use `attachment::create_renv_for_dev()` (>=0.2.5) to build your "renv.lock" file
  
- Allow other users to create their analyses in the same environment, by cloning your repository
  + Share your work with the world, without bothering about their R installation
  
- You can build a Docker container to also fix system dependencies when using your package. {dockerfiler} can help you to set up the container.

---
class: slide

## Share your work

- Build and publish your {pkgdown} website to present how to use your package

<pre class='rcode-input'># Have a proper Readme - Fill and knit
usethis::use_readme_rmd()

# Allow {pkgdown}
usethis::use_pkgdown()

# Try it locally
pkgdown::build_site()

# GitHub
# Add your credentials for GitHub
gitcreds::gitcreds_set()
# Send your project to a new GitHub project
usethis::use_github()

# Build and publish with GitHub Actions
usethis::use_github_action("pkgdown")

# Build and publish on GitLab
gitlabr::use_gitlab_ci()</pre>

<!--class: inverse, contact-slide

# Contact

## ThinkR - <a href="http://thinkr.fr">http://thinkr.fr</a>-->

---
name: thanks-for-joining-this-tutorial

layout:true
.remark-slide-number-left[]

---
class: slide

## Thanks for joining this tutorial!

### Sébastien Rochette, Florence Mounier
### sebastien@thinkr.fr, florence@thinkr.fr

Material of this course is on Github (with answers): <a href="https://github.com/statnmap/teach-package-dev-rmdfirst" target='_blank'>statnmap/teach-package-dev-rmdfirst</a>

<!--class: inverse, contact-slide

# Contact

## ThinkR - <a href="http://thinkr.fr">http://thinkr.fr</a>-->