Skip to main content

Engineering Production-Grade Shiny Apps

Introduction

Welcome to “Engineering Production-Grade Shiny Apps” by Colin Fay, Sébastien Rochette, Vincent Guyader and Cervan Girard.

This book is available in print, published in the R Series by Chapman and Hall/CRC.

You can grab a copy on routledge.com.

The online version of this book is free to read here (thanks to Chapman & Hall/CRC), and licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Motivation

This book will not get you started with shiny, nor talk how to work with shiny once it is sent to production. What we will be discussing in this book is the process of building an application that will later be sent to production.

Why this topic? Lots of blog posts and books talk about getting started with shiny (Chang et al. 2022) or about what to do once your application is ready to be sent to production. Very few (if any) talk about this area in-between: how to confidently work with shiny once you know the basics, and before you send it to production.

This is what this book is going to talk about: building robust shiny applications that are ready to be sent to production. We’ll focus on the process, the workflow, and the necessary tools for building production-grade shiny applications.

Audience for this book

If you are starting to read this book, we assume you have a working knowledge of how to build a small application using shiny, and want to know how to go one step further. In other words, you already have some knowledge about how shiny works, are able to build a small working application, and want to know how to build a serious, production-grade application that can be sent to production.

The content of this book oscillates between conceptual discussions (e.g., what is complexity?), exploration of project-management questions (e.g., how do we organize work when building production shiny applications?), and technical topics (e.g. what are shiny modules, or how do I optimize shiny?), with a large portion of the book being dedicated to technical questions.

This book will then be of interest to two groups:

  • Team managers who want to help to organize work, and shiny developers who want to learn about project management. This group will find relevant content in the first 1/3 of this book (roughly until Chapter 5).

  • Developers who want to cover medium to advanced shiny topics that will be relevant to production. This group will be more interested in the rest of the book, which roughly spans from Chapter 6 to the end. Note that the technical elements covered in this book range from intermediate to advanced topics, and we hope that you will find relevant topics for your current goals, whatever your level is.

In practice, we’ve come to realize that a lot of shiny projects are built by small teams with a large number of these “teams” being composed of only one developer. In this context, we have realized that project management and development are often assigned to the very same person,1 making these two skills deeply integrated when carrying production projects.

That’s why this book tries to reconcile both these worlds, as building production-grade software needs a reliable and operational code-base which is made possible by having solid technical skills, but also reliable and operational team work and project management.

What is “production”?

All throughout this book, we will be using the term “production” to refer to the way we build and deploy our application. But what is “production”?

There have been a lot of definitions of what production is, and even today if you ask around you will get a wide range of different answers. We like to think that a piece of software is in production when it combines the three following properties, not only for the users, but also the engineers working on it:

  • It is used, even if only by one person.

  • It is relied upon by its user(s).

  • It has real life impact if something goes wrong.

These three properties impact two specific groups: users and the developers.

Indeed, the users rely on the app to work so that they can do their job, and expect it to deliver meaningful results that they can count on. From the engineering point of view, a production-grade software can be relied upon in the sense that developers count on it to run as expected, and they need to rely on the software to be resilient to change, i.e to be modular, documented, and strongly tested so that changes can be integrated with confidence.

A production software also has real-life impact if something goes wrong: users will make wrong decisions, they might be unable to do their day-to-day work, and there are all the things that can happen when the software you use on a daily basis fails to run. From the engineering point of view, a production-grade software has real impact when something goes wrong: someone has to fix the bug, the company selling the software might lose money, data can be lost, and so on.

Given these two properties, you can understand why being in production doesn’t necessarily mean being served to gazillions of users,2 and serving trillions of gigabytes of data: even software that is used by one person who relies on this application to do their job is a production software.

This is what this book is about: building shiny applications that can be used, on which you and your users can rely, and including all the tools that will help you prevent things from going wrong, and when they eventually do, making sure you are equipped to quickly fix the bugs.

Book structure

  • Part 1, “Building Successful shiny Apps” gives a general context for what we mean by “production-grade” and “successful” shiny applications, and what challenges arise when you are dealing with a large-scale application designed for production. In this part, we will define what we mean by “Successful”, stress the importance of project management, develop how to structure your project for production, and introduce the golem (Fay, Guyader, Rochette, et al. 2023) package. We will finally, briefly introduce to our development workflow: a workflow that will be explored in Parts 2 to 6.

  • Part 2 through 6 explore the workflow for building successful applications. Part 2: Design (Step 1) underlines the centrality of the user experience when engineering an application, and emphasizes the importance of designing before coding. Part 3: Prototype (Step 2) stresses the importance of prototyping, explores the setting of a golem-based application, and presents shinipsum, fakir, and the “Rmd First” development methodology. Part 4: Build (Step 3) explores the building step of the application, i.e the core engineering of the application once the prototyping phase is finished. Part 5: Strengthen (Step 4) explores in-depth testing, continuous integration, and version control. Part 6: Deploy (Step 5) develops the various possibilities for preparing your application to be deployed.

  • Part 7, “Optimizing” tackles the question of optimization, first by introducing the general concepts about optimization, then by presenting some common caveats of shiny apps, and then showing how to optimize R code, and how to use JavaScript to lighten R work. This part ends with a gentle introduction to CSS (Cascading Style Sheets).

About the authors

Colin Fay

Colin has written the vast majority of this book. He’s responsible for its general structure, and for the original design of the workflow described. Most of the time (if not every time) “we” actually refers to him. He is the lead developer of the golem framework, and creator of many tools described in this book.

Colin works at ThinkR, a French agency focused on everything R related. During the day, he helps companies to take full advantage of the power of R, by building tools (packages, web apps, etc.) and setting up infrastructure. His main areas of expertise are data and software engineering, infrastructure, web applications (front-end and back-end), and R in production.

During the night, Colin is also a hyperactive open-source developer and an open-data advocate. You can find a lot of his work on his GitHub account (https://github.com/ColinFay) and on ThinkR’s account (https://github.com/thinkr-open).

He is also active in the R and Data community, and is an international speaker.

Sébastien Rochette

Sébastien has been instrumental in the review of most of this book’s chapters. He has also written the section about prototyping in RMarkdown, a concept he initiated.

Sébastien is a data scientist at ThinkR, where he teaches anything R related from beginner to expert level, guides R developers towards implementation of best practices, and creates tailor-made R solutions for the needs of his customers.

Vincent Guyader

Vincent is the founder of ThinkR. He created the first proof-of-concept framework for shiny applications inside packages; an idea which has led to the creation of golem. If you feel like a GitHub archaeologist, this very first version is still available with a little bit of exploration!

With more than ten years of experience with R, and a scientific and technical background, Vincent is an R enthusiast. He still has his hands in the code, whether to develop applications, analyze data, or build packages. When he’s not coding, he plays with Docker and manages servers. He strongly believes that meeting highly technical challenges is not incompatible with pedagogy: he passionately trains very diverse learner profiles in R.

Cervan Girard

Cervan has worked on some of the example applications that are used inside this book, namely {shinipsumdemo}, {databasedemo}, {grayscale}, {bs4dashdemo}, and {shinyfuture}.

Cervan is a Data Scientist at ThinkR. He is enthusiastic and motivated when it comes to rolling up his sleeves for new challenges, even if it means venturing dangerously into the depths of R, learning new languages, and experimenting outside your comfort zone.

Whatever the challenge, he remains reliable, constructive, and efficient when it comes to using his skills to train or develop. He also enjoys training learners of all levels in the R language.

Disclaimer

Open source is moving (very) fast, and some of the tools described in this book are still under active development. But good news! A large portion of this book is about the methodology, and not purely the technology, so even if some of the packages and code sections used in this book can (and will) become obsolete, a significant part of what is described in this book will still be relevant.

When reading this book, remember that they are a “snapshot” of various technologies and packages, which might someday become out of date, have breaking changes, or even disappear. The last revisions of the pages you’re reading now have been done on March 12, 2024. We will try to update the online version whenever changes are made to the packages used in this book, so feel free to browse the online version3 for up-to-date information.

Software information and conventions

This book was built with knitr (Xie 2023b) and bookdown (Xie 2023a).

Package names are in curly brackets in code format (e.g., rmarkdown), and inline code and file names are formatted in a typewriter font (e.g., knitr::knit('doc.Rmd')). Function names are formatted in a typewriter font and followed by parentheses (e.g., render_book()). Larger code blocks are formatted in a typewriter font and have a gray backgroud, e.g.:

When we describe a package that can be installed from GitHub, we use the install_github() function from the remotes (Csárdi et al. 2023) package. In other words, the following code:

remotes::install_github("ColinFay/dockerstats")

means that the package will be installed from GitHub, and that you can use the remotes package, which has to be installed on your machine if not already there.

Want to help?

Any feedback on the book is very welcome. Feel free to open an issue, or to make a Pull Request if you spot a typo.

Other resources

One single book could not cover everything there is to know about shiny. Here are some resources you can use in addition to this book. You can also find more resources in the bibliography.

Acknowledgments

Special thanks

We want to thank ThinkR for allowing us time to write this book, and for always being supportive during the project.

Colin wants to personally thank Christophe Dervieux for all his help with this book and any other projects, and for being such an awesome person. He also wants to thank Eric Nantz for taking the time to write the Foreword, for being one of the first golem adopters, and for always being enthusiastic about all things gravitating around the golemverse. A big thanks also to David Granjon for his precious feedback on this book.

ThinkR Website