Scientific workflows using R and git

class: center, middle, inverse, title-slide

.title[
# Scientific workflows using R and git
]
.author[
### Sara Mortara & Andrea Sánchez-Tapia
]
.institute[
### re.green | ¡liibre!
]
.date[
### 2022-07-07
]

---

## today:

+ scientific workflows

+ good practices for writing scripts

+ use our .Rproj structure to run a script

+ generate outputs

+ `commit` of the day

---

## scientific workflows

+ reproducibility - for you, collegues, and community

+ script based tools (`R`, `python`)

+ version control (`git`)

+ share methods, and protocols

+ peer review

---

## our project structure

__names__ and __paths__ are essential to a reproducible workflow

```r
project/
*   ├── .gitignore
    ├── data/
    ├── docs/
    ├── figs/
    ├── R/
*       └── 02_importing_data.R
    ├── output/
    ├── README.md
*   └── .Rproj
```

---
background-image: url(figs/jenny_bryan.jpeg)
background-size: 220px 240px
background-position: 85% 50%

## Rstudio projects

forget `setwd()` and meet [Jenny Bryan](https://www.tidyverse.org/blog/2017/12/workflow-vs-script/)

![](figs/jenny_bryan_wf.png)

---
# .Rproj defines the wd

![](figs/rstudio_proj.png)

---
## .Rproj + git = <3

---
## .Rproj + git = <3

---
## .Rproj + git = <3

---
## .Rproj + git = <3

---
class: inverse, middle, center

## workkflow: scripts

---

## writing scripts

+ Never create a single script with all the analysis
    + `01_read_and_format_data.R`
    + `02_diversity_analysis.R`
    + `03_pca_analysis.R`
    + `04_simulations.R`
    `...`

+ Ideally, each script statrs __reading__ a particular input/data and ends __writing__ results

+ the next script can __read raw data__ or __results from previous scriptss__.

---
## example

<small>
.pull-left[
+ `R/01_data_clean.R`
]
.pull-right[
+ reads __`data/data_raw.csv`__
+ writes __`data/data_processed.csv`__
]

.pull-left[

+ `R/02_diversity_analysis.R`
]
.pull-right[
+ reads __`data/data_processed.csv`__
+ writes __`results/02_diversity.csv`__   __`figs/02_diversity.png`__
]

</small>

---
## example
<small>
.pull-left[
+ `R/03_pca_analysis.R`
]
.pull-right[
+ reads __`data/data_processed.csv`__
+ writes __`figs/03_pca.png`__
]

.pull-left[
+ `R/04_simulations.R`
]
.pull-right[
+ reds __`data/dados_processed.csv`__
+ saves __`results/04_simulations.rda`__   __`figs/04_simulations.png`__
]

</small>

---
## example

if an object is too large, or it takes too much time to process, it can be saved as an R object (__`.rda`__)  
exemple:  
__`save(object, "./results/04_simulations.rda")`__

+ following scripts can start loading these objects:

example: in the script `05_analysing_simulations.R`

__`load("results/04_simulacoes.rda")`__

_but never save the workspace!_

---
class: middle, center

## organizing each script

.footnote[Prints from swcarpentry.github.io/r-novice-inflammation/06-best-practices-R]

---
## each script

+ a header containing who, how, when, where, and why __METADATA__

![](./figs/01meta.png)
--

+ a part loading all needed packages from the begining with `library()`*

![](./figs/02library.png)

---
## each script

+  reads needed data (__empty workspace__)

![](./figs/01read.png)

+ Coding a variable that will not change

+ Commenting every step
  
--

+ Writing in the HD the result from each step

---
## each script

+ the script must be able to be run in sequence from start to finish.

+ No repetitions,
  + No lines out of order
  + No parentheses or non-closing calls (`png` ---> `dev.off()`)

+ You should be able to erase the _workspace_ mid-session and rebuild

+ Do not define functions inside the script. Put the functions in a separate script and folder __`/fct/edit.R`__ and call via __`source()`__.

---
## additional tips

+ use concise and informative names
    + __`a <- `__ NO

+ do not use names already taken: `cor <-` (color) `cor()`
`c <-`

+ __If you copy and paste more than three times it's time to write a loop or a function__

---
## more tips

+ https://owi.usgs.gov/blog/intro-best-practices/
+ https://swcarpentry.github.io/r-novice-inflammation/06-best-practices-R/
+ https://www.r-bloggers.com/r-code-best-practices/
+ https://www.tidyverse.org/articles/2017/12/workflow-vs-script/

---
## Getting back to the project

.tiny[

```r
.
├── 2022_scientific_computing_intro.Rproj
├── data
│   └── raw
*│       ├── cestes
*│       │   ├── comm.csv
*│       │   ├── coord.csv
*│       │   ├── envir.csv
*│       │   ├── README.md
*│       │   ├── splist.csv
*│       │   └── traits.csv
│       └── portal_data_joined.csv
├── docs
*│   └── scientific_workflows.Rmd
├── figs
├── output
├── R
*│   ├── 01_intro.R
*│   └── 02_importing_data.R
└── README.md
```

]

---
## Getting back to git

CRLF vs LF

`git config --global core.autocrlf false`

More on this topic [here](https://www.aleksandrhovhannisyan.com/blog/crlf-vs-lf-normalizing-line-endings-in-git/#crlf-vs-lf-what-are-line-endings-anyway)

---
## Getting back to git

`git pull origin main`

`git add .gitignore`

`git add .`

`git commit -m "adding project's first strucure"`

---
## Running a script and generating outputs

`git pull origin main`

`git add output/02_envir_summary.csv`

`git add figs/02_species_abundance.png`

`git commit -m "a very informative message about the scripts you're adding"`

`git push origin main`

---

## Creating a report

- Rmarkdown basic structure

run `docs/scientific_workflow.Rmd`

---
class: center, middle
# ¡Thanks!

<center>
<svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#A70000;height:1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M476 3.2L12.5 270.6c-18.1 10.4-15.8 35.6 2.2 43.2L121 358.4l287.3-253.2c5.5-4.9 13.3 2.6 8.6 8.3L176 407v80.5c0 23.6 28.5 32.9 42.5 15.8L282 426l124.6 52.2c14.2 6 30.4-2.9 33-18.2l72-432C515 7.8 493.3-6.8 476 3.2z"></path></svg> [saramortara@gmail.com](mailto:saramortara@gmail.com) | [andreasancheztapia@gmail.com](mailto:andreasancheztapia@gmail.com)