Skip to main content

Engineering Production-Grade Shiny Apps

5 The Workflow

Building a robust, production-ready web application will be made easier by following a given workflow. The one we are advocating is divided in five steps:

  • Design
  • Prototype
  • Build
  • Strengthen
  • Deploy

In this chapter, we will give an brief overview of the different steps: the rest of the book will cover each of these steps in more depth.

Of course, as with any workflow, this one is not a one-size-fits-all solution: all projects are unique, with technical requirements, specific planning and team of coder(s). But we think that following this workflow will help you get good habits when it comes to structuring your application project, even more if you know from day one that the application you are going to work on is a large application, whether in terms of codebase, complexity, or time.

Note that the ideas behind this workflow, and its process, could be used outside of a shiny project: it can be applied to any coding project, even outside of the R world. Of course, the tools presented in this book are R and shiny specific, but the general ideas can be bootstrapped to be used outside of this context.

5.1 Step 1: Design

The first part of the workflow is the design part.

This very first step is the one that happens before starting to code: it is the one where you are thinking about the general implementation and features of the application, and where you build the general roadmap for the coding process. During the process of designing, you will define how the application will be built: somewhere between users’ dreams, what is technically possible, and the time you have to build the application.

This first step is not shiny or R specific, it is something software engineers do for any software or web application: discuss with the clients,25 the end users, and the developers who will work on the project. The idea with this first step is to get a clear idea of what everybody involved in the project wants/is able to do:

  • From the client/end user’s point of view, this step involves working on getting a clear idea of what they want the application to do, and to confront this view with the developers to evaluate what is possible to do, how much time it will take to implement desired features, etc.

  • From the developer team point of view, this step also involves getting a clear idea of what the client is asking, in other words it involves translating the requirements to technical specifications. For example, the client might write something like “Save the plot inside a database so that we can search for them later on”: from an application user point of view, this is a clear feature, from a developer point of view, this requirement can be translated in many ways.

This first step actually implies a lot of thinking before coding. The main goal of this step is to spend time thinking about the application while you still do not have anything implemented, so that you do not discover blocking elements once it is too late, or at least once you already have written a lot of code. We have all been in a situation during a project where we tell ourselves: “I wish I had known this sooner”: working on designing the application before building it helps lowering the chances for this kind of bad surprise.

This first part of the workflow will span three chapters:

  • “UX Matters”, Chapter 6, is a chapter where we introduce the concepts of “User eXperience” (UX), and why it is a crucial concept when you are building your application. This chapter will cover the importance of simplicity when creating web applications, the danger of trying to implement too many components (aka “feature creep”), and finally we will introduce some general rules about web accessibility. These topics are vast topics, and a lot of literature and online resources exist for all these subjects: further readings and resources are linked inside each section.

  • Chapter 7, “Don’t rush into coding” underlines why “coding first” might not be the best strategy when it comes to building a production application. We will also quickly introduce concept maps, and list some of the common questions you might want to ask the people involved in the project.

  • Finally, this first part of the workflow covers a gentle introduction to CSS, which might be a crucial skill to master when it comes to sending an application to production: either your clients already have a CSS template that they want to include in the application, or they want their application to have the color and design that match the one from the company. Also, when building a professional application, chances are that you will want your app to stand out from the crowd: hence a little bit of CSS. This part is included in the design part because it is something that you might want to think about from the very beginning: for example, some companies have pre-existing shiny templates, they might want to include specific fonts, logo, icons, etc. These are things better known before starting to code: it is easier to start working inside a shiny template than migrating an existing code to a template.

5.2 Step 2: Prototype

The prototype part is the one during which you will build the front-end and the back-end, but separately.

As you may know, a shiny application is an interface (the front-end, or “UI”) used to communicate information to the end users that are computed on the server side (the back-end, or “server”).

To start on solid ground, you need to build the two (front and back) separately.

On one hand, work on the general appearance, without working on any actual algorithmic implementation: position of the inputs and outputs, general design, interactions, etc.; everything that does not rely on computation on the back-end. This “UI first” approach will be made possible for shiny with notably one package, shinipsum (Fay and Rochette 2023b).

On the other hand, you (or someone from your team), will be working on building the back-end logic, which comprises the actual outputs that are going to be displayed, the algorithm that will compute results, and all the elements that do not need an interactive runtime to work. For this point, you can use what we call a “Rmd-first” approach, by combining R functions with the writing of vignettes that describe the internals of the application. This part of the workflow will be developed in two chapters:

  • Chapter (8), “Setting up for success with golem”, will cover the basics of getting started with the golem package so that you can start your prototyped application with solid foundation.

  • Chapter (9), “Building an “ipsum-app”” will cover the importance of prototyping when it comes to building applications, then present shinipsum and fakir, and finally will introduce how you can use the “Rmd First” methodology to prototype your application back-end.

5.3 Step 3: Build

The build part is the one where you will combine the business (or back-end) logic with the front-end. In this third part, you will work on the core engine of the application, making the business logic work inside the interactive logic of your application.

This step of the workflow is cover in Building app with golem (10), a chapter that presents the various functions you can use to build your application, i.e the one you will be using to combine your back-end and front-end.

In this step, we will cover:

  • How to handle dependencies in your project, i.e how to use external libraries inside your project
  • How to organize modules and functions inside your project
  • How to add tests for the back-end of your application (testing will be covered in more depth in Step 4)
  • How to document your application and its codebase, and how to add code coverage and continuous integration
  • How to leverage the internal dev functions from golem to modify the behavior of specific functions based on an option()

5.4 Step 4: Strengthen

The strengthen part covers how to ensure your application is immortal, in the sense that we defined in Chapter 1 of this book.

In this part, we will go through unit tests, reproducible development environments, version control, and continuous integration in the context of shiny applications. Building a solid testing suite is crucial to the success of a project, as it allows a project to be stable in the long run, be it when you will want to add new feature or refactor existing code:

Refactoring requires we be able to confidently ensure that behavior remains identical at every iteration. We can increase our confidence that nothing has changed by writing a suite of tests (unit, integration, end-to-end), and we should not seriously consider moving forward with any refactoring effort until we’ve established sufficient test coverage.

Refactoring at Scale (Lemaire 2020)

This step of the workflow will span over chapters.

  • The first one, “Build yourself a safety net” (Chapter 11), details how to build a testing environment for your shiny application, be it for testing the back-end or the front-end. In this chapter, you will be introduced to testthat for testing your application back-end, tools that are more linked to testing the front-end like NodeJS puppeteer module, shinytest and crrry for testing interactive logic, shinyloadtest and dockerstats for testing your application load. This chapter will also cover renv and Docker, two essential tools for developing in a reproducible environment.

  • In Chapter 12, “Version Control”, you will be introduced to git and to automated testing using continuous integration (CI) platforms like Travis CI or GitHub Actions.

5.5 Step 5: Deploy

To deploy is to send your application into production once it is built.

Being exhaustive here would be an impossible task: there are countless ways to make your application accessible to its targeted users, but we will try to cover some basics in this part. And of course, where and how you will be deploying your application depends on a lot of parameters. For example, who are the end users, and how do they want to use your application? If the end users are familiar with R and use it on a daily basis, they might be looking for an application that runs with library(app), i.e they need the application to be available as an R package they can install on their machine. If the end users are not coders, they might need the application to be available only as a web application, so they just have to open a browser and navigate to a URL. Both these cases raise other questions: how can you make the package available on a repository so that R users can get it with install.packages()? If the application is to be made available on a URL, how will it be deployed? What deployment server is available to you, or to the company ordering the application? These questions (and more) will be covered in the deploy part of this book.

In this part, we will present a series of methods to prepare your application to be deployed on various environments, notably:

  • Sharing your application as a package so that it can be installed manually, through GitHub, or shared on a package repository like the CRAN or BioConductor
  • Sending it to an RStudio platform
  • Building a Docker image to serve your app on a cloud provider

This step of the workflow is covered in the Chapter 13, “Deploy your application”.

ThinkR Website