Output

## 0.1 Installation and setup

Installation for this session can be achieved by following two alternatives.

1. **Using “renv” package by building an environment from the “.lock” file using

Navigate to the “session_meta/” directory of the workshop and run following code chunk.

R
## make sure that the renv.lock file for the session exists and do following.
install.packages("renv")
renv::restore(lockfile = "./Meta_tutorial/renv_r_4_1_0.lock", prompt = TRUE) 
1. Using the docker image
bash
########### Rstudio image for meta analyses ###########
# Your user is 'omics' (without the quotes)
# Replace <yourpassword> with your desired password
docker run -d --rm -p 8787:8787 -e PASSWORD=<yourpassword> ash706/omicsint_r

# 1 Introduction

Meta analyses are commonly used in clinical studies to assess the effect of a treatment or a genetic locus on a phenotype. With the advent of GWAS (Genome wide association studies), various studies report summary statistics on associated effects of genetic loci on phenotype. Meta analyses can be useful in determining whether the effect size is consistent across the body of data

The goal of a synthesis is to understand the results of any study in the context of all the other studies. First, we need to know whether or not the effect size is consistent across the body of data. If it is consistent, then we want to estimate the effect size as accurately as possible and to report that it is robust across the kinds of studies included in the synthesis. On the other hand, if it varies substantially from study to study, we want to quantify the extent of the variance and consider the implications. Meta-analysis is able to address these issues whereas the narrative review is not. We start with an example to show how meta-analysis and narrative review would approach the same question, and then use this example to highlight the key differences between the two.

# 2 Meta analyses Methods

1. p-value
1. Fisher: Sum of minus log-transformed P-values where larger Fisher score reflects stronger aggregated differential expression evidence.
2. Stouffer: Sum of inverse normal transformed P-values where larger Stouffer score to reflect stronger aggregated statistical evidence.
3. adaptively weighted Fisher(AW), original publication : assigns different weights to each individual study  and it searches through all possible weights to find the best adaptive weight with the smallest derived p-value. One significant advantage of this method is its ability to indicate which studies contribute to the evidence aggregation and elucidates heterogeneity in the meta-analysis.
4. minimum p-value (minP): The minP method takes the minimum p-value among the K studies as the test statistic
5. maximum p-value (maxP): The maxP method takes maximum p-value as the test statistic
6. rth ordered p-value (rOP): The rOP method takes the r-th order statistic among sorted p-values of Kcombined studies

Note: The assumption of uniformly distributed P-values under the null hypothesis or can be done non-parametrically by permutation-based analysis

1. Effect Size based
1. fixed effects model (FEM):FEM combines the effect size across K studies by assuming a simple linear model with an underlying true effect size plus a random error in each study
2. random effects model (REM): REM extends FEM by allowing random effects for the inter-study heterogeneity in the model.
1. Rank based
1. rank product (rankProd)RankProd and RankSum are based on the common biological belief that if a gene is repeatedly at the top of the lists ordered by up- or down-regulation fold change in replicate experiments, the gene is more likely a DE gene.
2. naive sum of ranks and naive product of ranks: These two methods apply a naïve product or sum of the DE evidence ranks across studies.

## 2.1 Statistical considerations

In addition to statistical methods mentioned above, a number of factors need to be considered for the choice of method. Different test statistics maybe used depending on the type of outcome variable (e.g. t-statistic or moderated t-statistic for binary outcome, F-statistic for multi-class outcome, regression or correlation coefficient for continuous outcome and log-rank statistic for survival outcome).

Here we will be using MetaDE package that has implemented above mentioned methods.

A summary of the methods and their implementations for respective outcome variables are listed in the table below as presented in the original publication.

For a detailed review of the applied methods for meta analyses, related benchmarks and

## 2.2 Required packages

R
library(magrittr)
library(plyr)
library(preproc)
library(MetaQC)
library(MetaDE)
library(MetaPath)
library(MetaDCN)

# 3 Summary of the session

I this session we will be working with the “prostate8.rda” data set that is located in the data directory of the “session_meta” directory of the github repository for the course.

Some of the R packages packages we are using here are not available through conda repositories and therefore need to be install from the github repositories for the respective packages

The session comprises of following main steps

1. Explore the data and the packages
2. Filter the data based on QC
3. Perform differential expression analyses and meta analyses
4. Perfrom pathway analyses for meta analyses results
5. Perfrom pathway meta analyses
6. Network meta analyses (optional)
7. Exercise: compare different meta analyses methods including AW-Fisher, REM, FEM ## Perform QC

# 4 Data for the session

We will using prostate dataset

Load the protaste data set

R
load(file = "./data/prostate8.rda")

The prostate data is comprised of 8 microarray studies from different microarray platforms. Here it is provided in a list format where data matrices and corresponding labels are provided.

Let us take a quick look at the data.

R
names(prostate8)
## [1] "data"      "dataLabel"

We can have a look at the structure of each data sets and evaluate what sort of filtering steps we need to take.

R
library(magrittr)
lapply(prostate8$data, dim) %>% as.data.frame(row.names = c("genes", "samples")) ## Welsh Yu Lapointe Varambally Singh Wallace Nanni Tomlins ## genes 8798 8799 13579 19738 8799 12689 12688 9703 ## samples 34 146 103 13 102 89 30 57 As we can see, the studies have different number of genes measured and contain different number of samples. In a meta analyses, studies can have different samples sizes but genes should match between different studies. To be able to merge data sets, all data matrices should have the same annotations Let us look at the row names which are already set to gene names. R lapply(prostate8$data, row.names) %>% lapply(.,head, 5) %>% as.data.frame()
##   Welsh    Yu Lapointe Varambally Singh Wallace  Nanni Tomlins
## 1  ACTB MAPK3    ITGB2       DDR1 MAPK3    DDR1   DDR1     ZFX
## 2 GLRA1  TIE1     HMBS       PAX8  TIE1    RFC2   RFC2    PIGS
## 3 KCNB2 CXCR5    GATA6       THRA CXCR5    PAX8  HSPA6    ZPBP
## 4 MGAT5 DUSP1    ICAM5       CCL5 DUSP1   ESRRA   PAX8  NOTCH3
## 5  BMP3 MMP10     RIN2      ESRRA MMP10    GAS6 PTPN21  IGSF11

It is clear that the gene names are set to gene symbols for all data sets and we can merge theses data set.

The second element of the lists corresponds to data labels where each value corresponds to disease status of the individual where the sample came from.

R
prostate8$dataLabel %>% lapply(. ,table) %>% as.data.frame() %>% t ## [,1] [,2] ## Welsh.Var1 "0" "1" ## Welsh.Freq " 9" "25" ## Yu.Var1 "0" "1" ## Yu.Freq "81" "65" ## Lapointe.Var1 "0" "1" ## Lapointe.Freq "41" "62" ## Varambally.Var1 "0" "1" ## Varambally.Freq "6" "7" ## Singh.Var1 "0" "1" ## Singh.Freq "50" "52" ## Wallace.Var1 "0" "1" ## Wallace.Freq "20" "69" ## Nanni.Var1 "0" "1" ## Nanni.Freq " 7" "23" ## Tomlins.Var1 "0" "1" ## Tomlins.Freq "27" "30" Data labels indicate “0” as control and “1” as disease groups. We can set these values to character format for convenience. If everything looks good, we are good to go to the next step. Take a moment to explore the data in your own ways and fimilarise yourself wit it. # 5 Data preprocessing ## 5.1 Gene matching Usually different microarray platforms use their own probe IDs or experiments from different omics platforms can have annotations for transcripts of proteins. To perform meta-analysis, one needs to match probe/transcript IDs from different platforms to the unique official gene ID, such as ENTREZ ID or gene symbol. Options for situations for microarray data include • take the average value of expression values across multiple probe IDs to represent the corresponded gene symbol • select the probe ID with the largest interquartile range (IQR) of expression • Or some version of summary at the gene level depending on the omics platform. We do not cover the pre-processing steps for different omics technologies at the moment but the workshop participants are encouraged to apply the domain specific knowledge when setting up a meta analyses study. R MetaQC::metaOverlap(prostate8$data) %>%
lapply(dim) %>% as.data.frame(row.names = c("genes", "samples"))
##         Welsh   Yu Lapointe Varambally Singh Wallace Nanni Tomlins
## genes    4241 4241     4241       4241  4241    4241  4241    4241
## samples    34  146      103         13   102      89    30      57

As you can see here, only 4241 genes are present in all studies and if we are to filter out this way, we lose a lot of data.

Biologically, it is likely that most genes are either un-expressed or un-informative. In gene expression analysis to find DE genes, these genes contribute to the false discoveries, so it is desirable to filter out these genes prior to analysis. After genes are matched across studies, the unique gene symbols are available across all studies. Two sequential steps of gene filtering can be performed. In the first step, we filter out genes with very low gene expression that are identified with small average expression values across majority of studies. And then we can remove genes that are not variable in your data sets using variance estimates as they are not useful in comparisons.

You can take a look at preproc package for some of the functions available for filtering the data and the intuition behind the methods. Here Annotate(), Impute(), Filter() and Merge() maybe useful for pre-processing steps of the data analyses.

R
data2 <- prostate8$data data2 <- preproc::Merge(data2) data.type = rep("microarray", length(data2)) # a character vector for each study type data2_filt <- preproc::Filter(data2, del.perc = c(0.1, 0.1),data.type = data.type ) # Here we specify the percentage of genes to be filtere, and specify the type of data for each study summary_preproc <- rbind(as.data.frame(lapply(prostate8$data, dim))[1,],
as.data.frame(lapply(data2, dim))[1,],
as.data.frame(lapply(data2_filt, dim))
)

rownames(summary_preproc) <- c("orginal", "merged", "mergeFiltered", "samples")

summary_preproc
##               Welsh   Yu Lapointe Varambally Singh Wallace Nanni Tomlins
## orginal        8798 8799    13579      19738  8799   12689 12688    9703
## merged         4241 4241     4241       4241  4241    4241  4241    4241
## mergeFiltered  3434 3434     3434       3434  3434    3434  3434    3434
## samples          34  146      103         13   102      89    30      57

Note that the multiple gene expression data sets may not be very well aligned by genes, and the number of genes in each study maybe different. When we combine a large number of studies, the number of common genes may be very small, so we need to allow for genes appearing in most studies and missing in few studies etc.

## 5.2 Load pathway database

Let us perform quality control of the data for this meta analyses. We are using the “MetaQC” that identifies ways to objectively perform quality control for the microarray studies.

R
load(file = "./data/pathways.rda")

DList=prostate8$data colLabel=prostate8$dataLabel
#GList=pathway[[1]]
#GList=pathwayDatabase

filterGenes=TRUE
cutRatioByMean=0.3 #
cutRatioByVar=0.3
#tic() not run
QCresult=MetaQC(DList, colLabel, GList=c(Hallmark.genesets, KEGG.genesets, Immunologic.genesets),filterGenes,cutRatioByMean,cutRatioByVar ) ## This will take some time depending on the number of studies and the type of data
#toc()

If you are unable to run the above steps, please load the “QC.rda” from the results directory of the session.

Output
R
QCresult$scoreTable ## IQC EQC AQCg AQCp CQCg CQCp ## Welsh 4.6297212 5.0993305 28.269084964 0.000000 1.671669e+02 1.434671e+02 ## Yu 9.4947631 9.4876742 21.725636675 0.000000 1.594930e+02 1.383082e+02 ## Lapointe 3.5445936 3.9678407 24.326120224 0.000000 9.159887e+01 2.045790e+01 ## Varambally 4.2482870 3.7655707 4.272765763 0.000000 1.858428e+01 3.795110e+01 ## Singh 0.8946880 2.0302734 14.740646685 6.945533 4.447641e+01 5.839619e+01 ## Wallace 8.1666052 8.9729647 0.003464049 0.000000 4.453816e-05 3.183405e-04 ## Nanni 0.8134973 0.6480432 0.000000000 0.000000 3.324084e-01 2.562951e-04 ## Tomlins 0.9366496 0.4514895 0.984994108 0.000000 7.892800e+00 1.767292e+01 IQC, EQC, AQCg, AQCp, CQCg, CQCp Internal quality control index:small IQC indicated that the study had heterogeneous coexpression structure with other studies and was considered a candidate problematic study that should be excluded from meta-analysis the external quality control (EQC): small EQC indicated that the study had low association with pathway in terms of gene pairwise correlation structure and maybe considered a candidate problematic study. accuracy quality control (AQC) and a consistency quality control (CQC). Large AQCg measure for a given study indicate that DE genes produced by study were reproducible compared to DE genes detected by meta-analysis excluding study Having a large CQCg measure for a given study indicated that DE evidence produced by study was consistent with DE evidence generated by meta-analysis excluding study. Please read the original MetQC publication for a better understanding of the measures. We can now look at the two dimensional biplot based on PCA analyses and get an idea of any studies that may not be of great quality. R MetaQC::plotMetaQC(QCresult$scoreTable)

Although the plotMetaQC() function is a useful tool for visualization, it does not necessarily allow automatic inclusion/exclusion criteria.

Note that our visualization and summarization tools were not meant for an automated recommendation for inclusion/exclusion decision. In the examples we explored, there were roughly three categories in the QC results: definite exclusion cases with poor quality, definite inclusion cases with good quality and borderline cases.

### 5.2.1 Update based on the QC

Here Nanni and Tomlins are the two studies that maybe of low quality and should perhaps be excluded from the analyses.

R
filterGenes=TRUE
cutRatioByMean=0.3 #
cutRatioByVar=0.3
to_remove <- c("Nanni", "Tomlins")
prostate6 <- list(data = within(prostate8$data, rm(Nanni, Tomlins)), dataLabel=within(prostate8$dataLabel, rm(Nanni, Tomlins)))
prostate_fil <- list(data = MetaQC::metaOverlap(prostate6$data), dataLabel=prostate6$data)
prostate6$data <-MetaQC::metaFilterData(prostate_fil$data,cutRatioByVar = cutRatioByVar, cutRatioByMean = cutRatioByMean)
lapply(prostate6, names)

as.data.frame(lapply(prostate6$data, dim)) ##$data
## [1] "Welsh"      "Yu"         "Lapointe"   "Varambally" "Singh"
## [6] "Wallace"
##
## $dataLabel ## [1] "Welsh" "Yu" "Lapointe" "Varambally" "Singh" ## [6] "Wallace" ## ## Welsh Yu Lapointe Varambally Singh Wallace ## 1 3399 3399 3399 3399 3399 3399 ## 2 34 146 103 13 102 89 R DList=prostate6$data
colLabel=prostate6$dataLabel #GList=pathway[[1]] #GList=pathwayDatabase filterGenes=TRUE cutRatioByMean=0.3 # cutRatioByVar=0.3 QCresult2=MetaQC(DList, colLabel, GList=c(Hallmark.genesets, KEGG.genesets, Immunologic.genesets), filterGenes, cutRatioByMean, cutRatioByVar) AGAIN, if you are unable to run the code, please load R load("results/QC_step2.rda") R QCresult2$scoreTable
##                  IQC       EQC        AQCg AQCp         CQCg         CQCp
## Welsh      4.3956311 3.9486143 38.42833030    0 1.913178e+02 4.028780e-02
## Yu         7.0164906 6.8265364 42.25733812    0 1.964917e+02 1.920686e-14
## Lapointe   1.5414898 1.7423903 16.70625993    0 9.884958e+01 0.000000e+00
## Varambally 3.5800781 2.9649972  7.18472473    0 2.391464e+01 0.000000e+00
## Singh      0.4912204 0.3180309 12.36863835    0 4.555982e+01 0.000000e+00
## Wallace    6.4044351 6.7765190  0.02200338    0 2.785632e-03 0.000000e+00
R
plotMetaQC(scoreTable = QCresult2$scoreTable) # 6 Meta Differential Expression analyses We have listed the methods for differential expression meta analyses in the introduction section that are implemented in the MetaDE package. Here we will try the best performing methods including AW Fischer, REM and rOP. A detailed comparison of relative performances of each of the statistical methods is descibed Lun-Ching etal. R data <- prostate6$data # Extract the expression matrices

## 7.4 Step 3

R
data(hashtb)
#data("./data/hashtb.rda")
#xx <- data(hashtb)
set.seed(1)
MAPE.Clustering.results <- MAPE.Clustering(summary=CPI_result$summary,Num_Clusters = 6, kappa.result = CPI.kappa_result$kappa,sil_cut=0.01,
Num_of_gene_lists=CPI_result$Num_of_gene_lists,genelist =CPI_result$genelist,
pathway=CPI_result$pathway, enrichment=CPI_result$enrichment,
method=CPI.kappa_result$method,software=CPI_result$method,                                n.text.permute = 1000, output_dir = "./results/MetaPEResults/"  )

You can take a look at the “Clustering_Summary.csv” to gen an idea of reproducibility of pathways and the amount of evidence provided by each study.

Of the heat

# 8 Network Meta analyses using MetaDCN

Co-expression analysis and network analysis of microarray data are used to investigate potential transcriptional co-regulation and gene interactions. Network analyses typically work with the gene–gene co-expression matrix, which represents the correlation between each pair of genes in the study. A crucial assumption is that the magnitude of the co-expression between any pair of genes is associated with a greater likelihood that the two genes interact. Thus, networks of interactions between genes are inferred from the co-expression matrix.

Here we will use MetaDCN to extract network modules in disease vs control conditions.

## 8.1 Correction and adjacency matrices

R
#data("./data/pathwayDatabase.rda")
#data(example)
# Generate correction and adjacency matrices for data and permutation
# GeneNet returns a list of information which will be used for SearchBM and MetaDCN function, and several RData files stored in folder path.
# AdjacencyMatrices.RData is a list of adjacency matrices for case and control in each study in the order of case studies and control studies.
#
# CorrelationMatrices.RData is a list of correlation matrices for case and control in each study.
#
# AdjacencyMatricesPermutationP.RData is a list of correlation matrices for case and control in each study in permutation P.
GeneNetRes.2  <- GeneNet(data , clin.data, caseName="Cancer", controlName="Control", meanFilter=0.8, SDFilter=0.8, edgeCutoff=0.1, permutationTimes=4, CPUNumbers=4,  pathwayDatabase=c(Biocarta.genesets,GOBP.genesets,GOCC.genesets,GOMF.genesets,
KEGG.genesets,Reactome.genesets), silent=FALSE, folder = "./results/MetaDCNResults/")

## 8.2 Correlations/Co-expressions

R
# This function will search for basic modules differentially co-expressed between case and control
# SearchBM will return a list and several Rdata, csv and png files saved in the folder path specified in GeneNet inputs.List of basic module information:

# w1
# w1 weight with the most basic modules detected
#
# BMInCase
# data matrix listing the information of basic modules higher correlated in case
#
# BMInControl
# data matrix listing the information of basic modules higher correlated in control
#
# permutation_energy_direction_p.Rdata is a list of energies of basic modules from permutation p.
#
# basic_modules_summary_direction_weight_w.csv is a summary of basic modules detected using weight w in forward/backward search.
#
# threshold_direction.csv is a table listing number of basic modules detected under different FDRs in forward/backward search.
#
# figure_basic_module_c_repeat_r_direction_weight_w.png is a plot of basic module from component c repeat r using weight w in forward/backward search.

SearchBMRes <- SearchBM(GeneNetRes.2, MCSteps=500, jaccardCutoff=0.7, repeatTimes=5, outputFigure=TRUE, silent=FALSE )

## 8.3 Find and assemble basic modules

R
#This function will assemble basic modules detected from SearchBM into supermodules.
# w1:w1 used
#
# BMInCaseSig: Summary of basic modules higher correlated in Case controling FDR
#
# BMInControlSig
# Summary of basic modules higher correlated in Control controling FDR
#
# Supermodule: Summary of supermodules
#A number chosen from (100, 200, ..., 700) to specify the weight1 used in objective function (optional). If not specified, w1 from SearchBM function will be used (recommended).

MetaDNCRes <- MetaDCN(GeneNetRes.2, SearchBMRes, FDRCutoff=0.05, w1=NULL, silent=FALSE)

You get the modules, their pathway annotations and cytoscape files in the results directory.

For the final assembled modules have a look at the “module_assembly_summary_weight_500.csv” and find out ho many “supe modules you are able to identity on q value cut off of 0.05”

# 9 Session info

Output
## R version 4.1.0 (2021-05-18)
## Platform: x86_64-apple-darwin17.0 (64-bit)
## Running under: macOS Big Sur 10.16
##
## Matrix products: default
## BLAS:   /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRblas.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
##
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
##
## attached base packages:
##  [1] grid      parallel  stats4    stats     graphics  grDevices datasets
##  [8] utils     methods   base
##
## other attached packages:
##  [1] MetaDCN_0.0-1               igraph_1.2.6
##  [3] snowfall_1.84-6.1           snow_0.4-3
##  [5] MetaPath_2.0                shape_1.4.6
##  [7] Rgraphviz_2.36.0            ggplot2_3.3.5
##  [9] gplots_3.1.1                cluster_2.1.2
## [11] irr_0.84.1                  lpSolve_5.6.15
## [13] ConsensusClusterPlus_1.56.0 impute_1.66.0
## [15] genefilter_1.74.0           GSEABase_1.54.0
## [17] graph_1.70.0                annotate_1.70.0
## [19] XML_3.99-0.6                MetaDE_2.2.3
## [21] MetaQC_1.0                  multtest_2.48.0
## [23] GSA_1.03.1                  preproc_1.2-4
## [25] DMwR_0.4.1                  lattice_0.20-44
## [27] AnnotationDbi_1.54.1        IRanges_2.26.0
## [29] S4Vectors_0.30.0            Biobase_2.52.0
## [31] BiocGenerics_0.38.0         plyr_1.8.6
## [33] magrittr_2.0.1              captioner_2.2.3
## [35] bookdown_0.22               knitr_1.33
##
## loaded via a namespace (and not attached):
##  [1] colorspace_2.0-2            ellipsis_0.3.2
##  [3] class_7.3-19                XVector_0.32.0
##  [5] GenomicRanges_1.44.0        fs_1.5.0
##  [7] bit64_4.0.5                 fansi_0.5.0
##  [9] splines_4.1.0               cachem_1.0.5
## [11] geneplotter_1.70.0          jsonlite_1.7.2
## [13] png_0.1-7                   shiny_1.6.0
## [15] compiler_4.1.0              httr_1.4.2
## [17] Matrix_1.3-4                fastmap_1.1.0
## [19] limma_3.48.1                later_1.2.0
## [21] htmltools_0.5.1.1           tools_4.1.0
## [23] gtable_0.3.0                glue_1.4.2
## [25] GenomeInfoDbData_1.2.6      Rcpp_1.0.7
## [27] jquerylib_0.1.4             vctrs_0.3.8
## [29] Biostrings_2.60.1           xfun_0.24
## [31] stringr_1.4.0               openxlsx_4.2.4
## [33] mime_0.11                   lifecycle_1.0.0
## [35] renv_0.13.2                 gtools_3.9.2
## [37] edgeR_3.34.0                zlibbioc_1.38.0
## [39] MASS_7.3-54                 zoo_1.8-9
## [41] scales_1.1.1                promises_1.2.0.1
## [43] MatrixGenerics_1.4.0        SummarizedExperiment_1.22.0
## [45] RColorBrewer_1.1-2          yaml_2.2.1
## [47] quantmod_0.4.18             curl_4.3.2
## [49] memoise_2.0.0               sass_0.4.0
## [51] rpart_4.1-15                stringi_1.7.3
## [53] RSQLite_2.2.7               highr_0.9
## [55] TTR_0.24.2                  caTools_1.18.2
## [57] zip_2.2.0                   BiocParallel_1.26.1
## [59] GenomeInfoDb_1.28.1         rlang_0.4.11
## [61] pkgconfig_2.0.3             matrixStats_0.59.0
## [63] bitops_1.0-7                evaluate_0.14
## [65] ROCR_1.0-11                 bit_4.0.4
## [67] samr_3.0                    DESeq2_1.32.0
## [69] R6_2.5.0                    combinat_0.0-8
## [71] DelayedArray_0.18.0         DBI_1.1.1
## [73] withr_2.4.2                 pillar_1.6.1
## [75] xts_0.12.1                  survival_3.2-11
## [77] KEGGREST_1.32.0             abind_1.4-5
## [79] RCurl_1.98-1.3              tibble_3.1.2
## [81] crayon_1.4.1                shinyFiles_0.9.0
## [83] KernSmooth_2.23-20          utf8_1.2.1
## [85] rmarkdown_2.9               locfit_1.5-9.4
## [87] blob_1.2.1                  digest_0.6.27
## [89] xtable_1.8-4                httpuv_1.6.1
## [91] munsell_0.5.0               bslib_0.2.5.1

End of document