Reproducibility

Reproducible Research
​Better methods can’t make up for mediocre theory​
​https://www.nature.com/articles/d41586-019-03350-5​
A push for reproducibility in biomedical research
​http://www.emoryhealthsciblog.com/a-push-for-reproducibility-in-biomedical-research/?utm_source=feedburner&utm_medium=twitter&utm_campaign=Feed%3A+EmoryHealthNowBlog+(Lab+Land)​
Statcheck
Controversial software is proving surprisingly accurate at spotting errors in psychology papers
​http://www.sciencemag.org/news/2017/11/controversial-software-proving-surprisingly-accurate-spotting-errors-psychology-papers​
​http://statcheck.io/index.php​
We need a similar program for #pathology articles. Though most pathology articles do not report #statistics in APA style. #statcheck
​http://statcheck.io/​
Stat-checking software stirs up psychology
​http://www.nature.com/news/stat-checking-software-stirs-up-psychology-1.21049​
Coursera: Reproducible Templates for Analysis and Dissemination
​https://www.coursera.org/learn/reproducible-templates-analysis/supplement/Pw4r9/articles-resources-and-file-organization-examples​
Reproducibility Articles
​“Why Most Published Research Findings Are False” by John P. A. Ioannidis
​“Public Availability of Published Research Data in High-Impact Journals” by Alawi A. Alsheikh-Ali , Waqas Qureshi, Mouaz H. Al-Mallah, John P. A. Ioannidis.
​“Gene name errors are widespread in the scientific literature” by Mark Ziemann, Yotam Eren, and Assam El-Osta.
​“Deriving chemosensitivity from cell lines: Forensic bioinformatics and reproducible research in high-throughput biology” by Keith A. Baggerly and Kevin R. Coombes.
​“Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff”​
by Thomas Herndon, Michael Ash, Robert Pollin.
An example of an article given the highest designation for a fully reproducible article:
​“Air pollution and health in Scotland: a multicity study”​
published in 2009 by Duncan Lee, Claire Ferguson, and Richard Mitchell. To see the article's marking, download the PDF and look for the marking letter in a bold box at the top right.
​“Ten Simple Rules for Reproducible Computational Research”​
by Geir Kjetil Sandve, Anton Nekrutenko, James Taylor, Eivind Hovig.
​“Open-Source Genomic Analysis of Shiga-Toxin–Producing E. coli O104:H4” (multiple authors). The associated Dataset GigaScience and GitHub Wiki .
Document Conversion
​Pandoc​
A universal document converter
Other Resources
There is an R package called “rrrpkg,” which was created to “facilitate reproducible research.” Their focus is on creating a Research Compendium. Here are some illustrations of
​directory structure and file organization​
they use that may be helpful to you.
​Center for Open Science​
​Gitbook​
​Fivethirtyeight.com​
Reproducible Research
A push for reproducibility in biomedical research
​http://www.emoryhealthsciblog.com/a-push-for-reproducibility-in-biomedical-research/?utm_source=feedburner&utm_medium=twitter&utm_campaign=Feed%3A+EmoryHealthNowBlog+(Lab+Land)​
Statcheck
Controversial software is proving surprisingly accurate at spotting errors in psychology papers
​http://www.sciencemag.org/news/2017/11/controversial-software-proving-surprisingly-accurate-spotting-errors-psychology-papers​
​http://statcheck.io/index.php​
We need a similar program for #pathology articles. Though most pathology articles do not report #statistics in APA style. #statcheck
​http://statcheck.io/​
Stat-checking software stirs up psychology
​http://www.nature.com/news/stat-checking-software-stirs-up-psychology-1.21049​
Coursera: Reproducible Templates for Analysis and Dissemination
​https://www.coursera.org/learn/reproducible-templates-analysis/supplement/Pw4r9/articles-resources-and-file-organization-examples​
Reproducibility Articles
​“Why Most Published Research Findings Are False” by John P. A. Ioannidis
​“Public Availability of Published Research Data in High-Impact Journals” by Alawi A. Alsheikh-Ali , Waqas Qureshi, Mouaz H. Al-Mallah, John P. A. Ioannidis.
​“Gene name errors are widespread in the scientific literature” by Mark Ziemann, Yotam Eren, and Assam El-Osta.
​“Deriving chemosensitivity from cell lines: Forensic bioinformatics and reproducible research in high-throughput biology” by Keith A. Baggerly and Kevin R. Coombes.
​“Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff”​
by Thomas Herndon, Michael Ash, Robert Pollin.
An example of an article given the highest designation for a fully reproducible article:
​“Air pollution and health in Scotland: a multicity study”​
published in 2009 by Duncan Lee, Claire Ferguson, and Richard Mitchell. To see the article's marking, download the PDF and look for the marking letter in a bold box at the top right.
​“Ten Simple Rules for Reproducible Computational Research”​
by Geir Kjetil Sandve, Anton Nekrutenko, James Taylor, Eivind Hovig.
​“Open-Source Genomic Analysis of Shiga-Toxin–Producing E. coli O104:H4” (multiple authors). The associated Dataset GigaScience and GitHub Wiki .
Document Conversion
​Pandoc​
A universal document converter
Other Resources
There is an R package called “rrrpkg,” which was created to “facilitate reproducible research.” Their focus is on creating a Research Compendium. Here are some illustrations of
​directory structure and file organization​
they use that may be helpful to you.
​Center for Open Science​
​Gitbook​
​Fivethirtyeight.com​
Reproducibility
Reproducible Research
A push for reproducibility in biomedical research
​http://www.emoryhealthsciblog.com/a-push-for-reproducibility-in-biomedical-research/?utm_source=feedburner&utm_medium=twitter&utm_campaign=Feed%3A+EmoryHealthNowBlog+(Lab+Land)​
Statcheck
Controversial software is proving surprisingly accurate at spotting errors in psychology papers
​http://www.sciencemag.org/news/2017/11/controversial-software-proving-surprisingly-accurate-spotting-errors-psychology-papers​
​http://statcheck.io/index.php​
We need a similar program for #pathology articles. Though most pathology articles do not report #statistics in APA style. #statcheck
​http://statcheck.io/​
Stat-checking software stirs up psychology
​http://www.nature.com/news/stat-checking-software-stirs-up-psychology-1.21049​
Coursera: Reproducible Templates for Analysis and Dissemination
​https://www.coursera.org/learn/reproducible-templates-analysis/supplement/Pw4r9/articles-resources-and-file-organization-examples​
Reproducibility Articles
​“Why Most Published Research Findings Are False” by John P. A. Ioannidis
​“Public Availability of Published Research Data in High-Impact Journals” by Alawi A. Alsheikh-Ali , Waqas Qureshi, Mouaz H. Al-Mallah, John P. A. Ioannidis.
​“Gene name errors are widespread in the scientific literature” by Mark Ziemann, Yotam Eren, and Assam El-Osta.
​“Deriving chemosensitivity from cell lines: Forensic bioinformatics and reproducible research in high-throughput biology” by Keith A. Baggerly and Kevin R. Coombes.
​“Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff”​
by Thomas Herndon, Michael Ash, Robert Pollin.
An example of an article given the highest designation for a fully reproducible article:
​“Air pollution and health in Scotland: a multicity study”​
published in 2009 by Duncan Lee, Claire Ferguson, and Richard Mitchell. To see the article's marking, download the PDF and look for the marking letter in a bold box at the top right.
​“Ten Simple Rules for Reproducible Computational Research”​
by Geir Kjetil Sandve, Anton Nekrutenko, James Taylor, Eivind Hovig.
​“Open-Source Genomic Analysis of Shiga-Toxin–Producing E. coli O104:H4” (multiple authors). The associated Dataset GigaScience and GitHub Wiki .
Document Conversion
​Pandoc​
A universal document converter
Other Resources
There is an R package called “rrrpkg,” which was created to “facilitate reproducible research.” Their focus is on creating a Research Compendium. Here are some illustrations of
​directory structure and file organization​
they use that may be helpful to you.
​Center for Open Science​
​Gitbook​
​Fivethirtyeight.com​
Şöyle birşey düşünün, Pankreas patolojisi ile ilgileniyorsunuz. "Bizim pankreas serisi ne durumda" diye merak ettiniz. Yaptığınız şey birkaç düğmeye basmak, ve o zamana kadar bölümünüzde rapor edilen pankreas vakalarının yaş, cinsiyet, tümör çapı, tümör tipi, evre, derece, lenf nodu durumu vesair bilgileri sağ kalım grafikleri ile word dökümanı olarak oluşturuluveriyor. Bu hayal değil. Yapılabilir. Makul bir bilgi işlem çalışanı, CAP ve AJCC'ye uygun doldurulması zorunlu yapılandırılmış patoloji raporları, ana veri tablosuna erişim, biraz SQL, biraz R, biraz da R Markdown kullanarak bunu yapmak işten bile değil.
​https://www.serdarbalci.com/2018/05/tekrarlanabilir-ve-otomatik-raporlar.html​
1
# https://github.com/spgarbet/tangram
2
# http://htmlpreview.github.io/?https://github.com/spgarbet/tg/blob/master/vignettes/example.html
3
​
4
library(tangram)
5
library(Hmisc)
6
getHdata(pbc)
7
# View(pbc)
8
table <- tangram(drug ~ bili + albumin + stage + protime + sex + age + spiders, data = pbc)
9
​
10
table
11
html5(table)
12
latex(table)
13
index(table)
14
​
15
write(
16
html5(tangram("drug ~ bili[2] + albumin + stage::Categorical + protime + sex + age + spiders", pbc, msd=TRUE, quant=seq(0, 1, 0.25)),
17
      fragment=TRUE, inline="hmisc.css", caption = "HTML5 Table Hmisc Style", id="tbl2"),
18
"tangram1.html")
19
​
20
write(
21
html5(tangram("drug ~ bili[2] + albumin + stage::Categorical + protime + sex + age + spiders", pbc),
22
      fragment=TRUE, inline="nejm.css", caption = "HTML5 Table NEJM Style", id="tbl3"),
23
"tangram_nejm.html")
24
​
25
​
26
tbl <- tangram("drug ~ bili[2] + albumin + stage::Categorical[1] + protime + sex[1] + age + spiders[1]", 
27
               data=pbc,
28
               pformat = 5)
29
write(html5(tbl,
30
      fragment=TRUE,
31
      inline="lancet.css",
32
      caption = "HTML5 Table Lancet Style", id="tbl4"
33
),
34
"tangram_lancet.html")
35
​
36
index(tangram("drug ~ bili + albumin + stage::Categorical + protime + sex + age + spiders", pbc))[1:20,]
37
​
38
​
39
library(readxl)
40
MDL307_Data <- read_excel("MDL307 - Data.xlsx")
41
​
42
MDL307_Data <- as.data.frame(MDL307_Data)
43
​
44
names(MDL307_Data)
45
​
46
View(MDL307_Data)
47
​
48
MDL307_Data$biyokimyasalrekurrens <- as.factor(MDL307_Data$biyokimyasalrekurrens)
49
levels(MDL307_Data$biyokimyasalrekurrens)[1] <- "yok"
50
levels(MDL307_Data$biyokimyasalrekurrens)[2] <- "var"
51
​
52
collist <- c("gleasonskor",
53
                 "tersiyer",
54
                 "kribriform",
55
                 "cerrahisinir",
56
                 "ekstaprostatik",
57
                 "lenfnodu",
58
                 "seminalvezikul"
59
                 )
60
​
61
​
62
MDL307_Data[collist] <- lapply(MDL307_Data[collist], as.factor)
63
​
64
​
65
table <- tangram(biyokimyasalrekurrens ~ yas +
66
                 gleasonskor +
67
                 tersiyer +
68
                 kribriform +
69
                 kribriformyuzde +
70
                 cerrahisinir +
71
                 ekstaprostatik +
72
                 lenfnodu +
73
                 seminalvezikul +
74
                 biyokimyasalrekurrens,
75
                 data = MDL307_Data)
76
table
Copied!
Export R output to a file
​https://www.r-bloggers.com/export-r-output-to-a-file/​
1
out <- capture.output(summary(my_very_time_consuming_regression))
2
​
3
cat("My title", out, file="summary_of_my_very_time_consuming_regression.txt", sep="n", append=TRUE)
Copied!
Şöyle birşey düşünün, Pankreas patolojisi ile ilgileniyorsunuz. "Bizim pankreas serisi ne durumda" diye merak ettiniz. Yaptığınız şey birkaç düğmeye basmak, ve o zamana kadar bölümünüzde rapor edilen pankreas vakalarının yaş, cinsiyet, tümör çapı, tümör tipi, evre, derece, lenf nodu durumu vesair bilgileri sağ kalım grafikleri ile word dökümanı olarak oluşturuluveriyor. Bu hayal değil. Yapılabilir. Makul bir bilgi işlem çalışanı, CAP ve AJCC'ye uygun doldurulması zorunlu yapılandırılmış patoloji raporları, ana veri tablosuna erişim, biraz SQL, biraz R, biraz da R Markdown kullanarak bunu yapmak işten bile değil.
​https://www.serdarbalci.com/2018/05/tekrarlanabilir-ve-otomatik-raporlar.html​
1
# https://github.com/spgarbet/tangram
2
# http://htmlpreview.github.io/?https://github.com/spgarbet/tg/blob/master/vignettes/example.html
3
​
4
library(tangram)
5
library(Hmisc)
6
getHdata(pbc)
7
# View(pbc)
8
table <- tangram(drug ~ bili + albumin + stage + protime + sex + age + spiders, data = pbc)
9
​
10
table
11
html5(table)
12
latex(table)
13
index(table)
14
​
15
write(
16
html5(tangram("drug ~ bili[2] + albumin + stage::Categorical + protime + sex + age + spiders", pbc, msd=TRUE, quant=seq(0, 1, 0.25)),
17
      fragment=TRUE, inline="hmisc.css", caption = "HTML5 Table Hmisc Style", id="tbl2"),
18
"tangram1.html")
19
​
20
write(
21
html5(tangram("drug ~ bili[2] + albumin + stage::Categorical + protime + sex + age + spiders", pbc),
22
      fragment=TRUE, inline="nejm.css", caption = "HTML5 Table NEJM Style", id="tbl3"),
23
"tangram_nejm.html")
24
​
25
​
26
tbl <- tangram("drug ~ bili[2] + albumin + stage::Categorical[1] + protime + sex[1] + age + spiders[1]", 
27
               data=pbc,
28
               pformat = 5)
29
write(html5(tbl,
30
      fragment=TRUE,
31
      inline="lancet.css",
32
      caption = "HTML5 Table Lancet Style", id="tbl4"
33
),
34
"tangram_lancet.html")
35
​
36
index(tangram("drug ~ bili + albumin + stage::Categorical + protime + sex + age + spiders", pbc))[1:20,]
37
​
38
​
39
library(readxl)
40
MDL307_Data <- read_excel("MDL307 - Data.xlsx")
41
​
42
MDL307_Data <- as.data.frame(MDL307_Data)
43
​
44
names(MDL307_Data)
45
​
46
View(MDL307_Data)
47
​
48
MDL307_Data$biyokimyasalrekurrens <- as.factor(MDL307_Data$biyokimyasalrekurrens)
49
levels(MDL307_Data$biyokimyasalrekurrens)[1] <- "yok"
50
levels(MDL307_Data$biyokimyasalrekurrens)[2] <- "var"
51
​
52
collist <- c("gleasonskor",
53
                 "tersiyer",
54
                 "kribriform",
55
                 "cerrahisinir",
56
                 "ekstaprostatik",
57
                 "lenfnodu",
58
                 "seminalvezikul"
59
                 )
60
​
61
​
62
MDL307_Data[collist] <- lapply(MDL307_Data[collist], as.factor)
63
​
64
​
65
table <- tangram(biyokimyasalrekurrens ~ yas +
66
                 gleasonskor +
67
                 tersiyer +
68
                 kribriform +
69
                 kribriformyuzde +
70
                 cerrahisinir +
71
                 ekstaprostatik +
72
                 lenfnodu +
73
                 seminalvezikul +
74
                 biyokimyasalrekurrens,
75
                 data = MDL307_Data)
76
table
Copied!
Export R output to a file
​https://www.r-bloggers.com/export-r-output-to-a-file/​
1
out <- capture.output(summary(my_very_time_consuming_regression))
2
​
3
cat("My title", out, file="summary_of_my_very_time_consuming_regression.txt", sep="n", append=TRUE)
Copied!
Reproducible Reports
Şöyle birşey düşünün, Pankreas patolojisi ile ilgileniyorsunuz. "Bizim pankreas serisi ne durumda" diye merak ettiniz. Yaptığınız şey birkaç düğmeye basmak, ve o zamana kadar bölümünüzde rapor edilen pankreas vakalarının yaş, cinsiyet, tümör çapı, tümör tipi, evre, derece, lenf nodu durumu vesair bilgileri sağ kalım grafikleri ile word dökümanı olarak oluşturuluveriyor. Bu hayal değil. Yapılabilir. Makul bir bilgi işlem çalışanı, CAP ve AJCC'ye uygun doldurulması zorunlu yapılandırılmış patoloji raporları, ana veri tablosuna erişim, biraz SQL, biraz R, biraz da R Markdown kullanarak bunu yapmak işten bile değil.
​https://www.serdarbalci.com/2018/05/tekrarlanabilir-ve-otomatik-raporlar.html​
1
# https://github.com/spgarbet/tangram
2
# http://htmlpreview.github.io/?https://github.com/spgarbet/tg/blob/master/vignettes/example.html
3
​
4
library(tangram)
5
library(Hmisc)
6
getHdata(pbc)
7
# View(pbc)
8
table <- tangram(drug ~ bili + albumin + stage + protime + sex + age + spiders, data = pbc)
9
​
10
table
11
html5(table)
12
latex(table)
13
index(table)
14
​
15
write(
16
html5(tangram("drug ~ bili[2] + albumin + stage::Categorical + protime + sex + age + spiders", pbc, msd=TRUE, quant=seq(0, 1, 0.25)),
17
      fragment=TRUE, inline="hmisc.css", caption = "HTML5 Table Hmisc Style", id="tbl2"),
18
"tangram1.html")
19
​
20
write(
21
html5(tangram("drug ~ bili[2] + albumin + stage::Categorical + protime + sex + age + spiders", pbc),
22
      fragment=TRUE, inline="nejm.css", caption = "HTML5 Table NEJM Style", id="tbl3"),
23
"tangram_nejm.html")
24
​
25
​
26
tbl <- tangram("drug ~ bili[2] + albumin + stage::Categorical[1] + protime + sex[1] + age + spiders[1]", 
27
               data=pbc,
28
               pformat = 5)
29
write(html5(tbl,
30
      fragment=TRUE,
31
      inline="lancet.css",
32
      caption = "HTML5 Table Lancet Style", id="tbl4"
33
),
34
"tangram_lancet.html")
35
​
36
index(tangram("drug ~ bili + albumin + stage::Categorical + protime + sex + age + spiders", pbc))[1:20,]
37
​
38
​
39
library(readxl)
40
MDL307_Data <- read_excel("MDL307 - Data.xlsx")
41
​
42
MDL307_Data <- as.data.frame(MDL307_Data)
43
​
44
names(MDL307_Data)
45
​
46
View(MDL307_Data)
47
​
48
MDL307_Data$biyokimyasalrekurrens <- as.factor(MDL307_Data$biyokimyasalrekurrens)
49
levels(MDL307_Data$biyokimyasalrekurrens)[1] <- "yok"
50
levels(MDL307_Data$biyokimyasalrekurrens)[2] <- "var"
51
​
52
collist <- c("gleasonskor",
53
                 "tersiyer",
54
                 "kribriform",
55
                 "cerrahisinir",
56
                 "ekstaprostatik",
57
                 "lenfnodu",
58
                 "seminalvezikul"
59
                 )
60
​
61
​
62
MDL307_Data[collist] <- lapply(MDL307_Data[collist], as.factor)
63
​
64
​
65
table <- tangram(biyokimyasalrekurrens ~ yas +
66
                 gleasonskor +
67
                 tersiyer +
68
                 kribriform +
69
                 kribriformyuzde +
70
                 cerrahisinir +
71
                 ekstaprostatik +
72
                 lenfnodu +
73
                 seminalvezikul +
74
                 biyokimyasalrekurrens,
75
                 data = MDL307_Data)
76
table
Copied!
Export R output to a file
​https://www.r-bloggers.com/export-r-output-to-a-file/​
1
out <- capture.output(summary(my_very_time_consuming_regression))
2
​
3
cat("My title", out, file="summary_of_my_very_time_consuming_regression.txt", sep="n", append=TRUE)
Copied!

Writing Journal Articles - Previous

Review

Next - Writing Journal Articles

Open Access

Last modified 2yr ago

Reproducible Research

​Better methods can’t make up for mediocre theory​

A push for reproducibility in biomedical research

Statcheck

Coursera: Reproducible Templates for Analysis and Dissemination

Reproducibility Articles

Document Conversion

Other Resources

Reproducible Research

Statcheck

Coursera: Reproducible Templates for Analysis and Dissemination

Reproducibility Articles

Document Conversion

Other Resources

Reproducibility

Reproducible Research

Statcheck

Coursera: Reproducible Templates for Analysis and Dissemination

Reproducibility Articles

Document Conversion

Other Resources

Reproducible Reports

Better methods can’t make up for mediocre theory