Chapter 5 The workflow

Building a robust, production-ready web application will be made easier by following a given workflow. The one we are advocating for is divided in five steps:

  • Design
  • Prototype
  • Build
  • Strengthen
  • Deploy

In this chapter, we will give an brief overview of the different steps of this workflow: the rest of the book will cover in more depth each of these steps.

Of course, as with any workflow, this one is not a one-size-fits-all solution: all projects are unique, with technical requirements, specific planning and team of coder(s). But we think that following this workflow will help you get good habits when it comes to structuring your application project, even more if you know from day one that the application you are going to work on is a large application, be it in term of code base, complexity, or time.

Note that the ideas behind this workflow, and its process, could be used outside of a Shiny project: it can be applied to any coding project, even outside of the R world. Of course, the tools presented in this book are R and Shiny specific, but the general ideas can be bootstrapped to be used outside of this context.

5.1 Part 1: Design

The first part of the workflow is the design part.

This very first step is the one that happens before starting to code: it is the one where you are thinking about the general implementation and features of the application, and where you build the general roadmap for the coding process. During the process of designing, you will define how the application will be build: somewhere between users’ dreams, what is technically possible, and the time you have to build the application.

This first step is not Shiny, nor R specific, it is something software engineers do for any software or web application: discuss with the clients20 , the end-users, and the developers who will work on the project. The idea with this first step is to get a clear idea of what everybody involved in the project wants/is able to do:

  • From the client/end-users point of view, this steps involves getting a clear idea of what they want the application to do, and to confront this view with the developers to evaluate what is possible to do, how many time it will take to implement desired features, etc.

  • From the developer team point of view, this step also involves getting a clear idea of what the client is asking, in other words it involves translating the requirements to technical specifications. For example, the client might write something like “Save the plot inside a database so that we can search for them later on”: from an application user point of view, this is a clear feature, from a developer point of view, this requirement can be translated in many ways.

This first step actually implies a lot of thinking before coding. The main goal of this step is to spend time thinking about the application while you still do not have anything implemented, so that you do not discover blocking elements once it is too late, or at least once you already have written a lot of code. We have all been in a situation during a project where we tell ourselves: “I wish I had known this sooner”: working on designing the application before building it helps lowering the chances for this kind of bad surprises.

This first part of the workflow will span over three chapters:

  • UX Matters, a chapter where we introduce to the concepts of User Experience, and why it is a crucial concept when you are building your application. This chapter will cover the importance of simplicity when creating web applications, the danger of trying to implement too much components (aka “feature creep”), and finally we will introduce some general rules about Web Accessibility. These topics are vast topics, and a lot of literature and online resources exists for all these subjects: further readings and resources are linked inside each section.

  • Don’t rush into coding underlines why “coding first” might not be the best strategy when it comes to building a production application. We will also quickly introduce concept maps, and list some of the common questions you might want to ask the people involved in the project.

  • Finally, this first part of the workflow covers a gentle introduction to CSS, which might be a crucial skill to master when it comes to send an application to production: either your clients already have a CSS that they want to include in the application, or they want their application to have the color and design that match the one from the company. Also, when building a professional application, chances are that you will want your app to stand out from the crowd: hence a little bit of CSS. This part is included in the design part because it is something that you might want to think about from the very beginning: for example, some company have pre-existing {shiny} templates, they might want to include specific fonts, logo, icons, etc. These are things better known before starting to code: it is easier to start working inside a {shiny} template than migrating an existing code to a template.

5.2 Part 2: Prototype

The Prototype part is the one during which you will build the front-end and the back-end, but separately.

As you may know, a Shiny application is an interface (the front-end, or “UI”) used to communicate information to the end-users that are computed on the server side (the back-end, or “server”).

To start on solid ground, you need to build the two (front and back) separately:

On one hand, work on the general appearance, without working on any actual algorithmic implementation: position of the inputs and outputs, general design, interactions, etc. everything that does not rely on computation on the back end. This “UI first” approach will be made possible for Shiny with notably one package, {shinipsum} (Fay and Rochette 2020b).

On the other hand, you (or someone from your team), will be working on building the back-end logic, which are the actual outputs that are going to be displayed, the algorithm that will compute results, and all the elements that do not need an interactive runtime to work. For this point, you can use what we call a “Rmd-first” approach, by combining R functions with the writing of Vignettes that describes the internal of the application. This part of the workflow will be developed in two chapters:

  • The Setting up for success with {golem} chapter will cover the basics of getting started with the {golem} package so that you can start your prototyped application with solid foundation.

  • The Building an “ipsum-app” chapter will cover the importance of prototyping when it comes to building applications, then present {shinipsum} and {fakir}, and finally will introduce how you can use the “Rmd First” methodology to prototype your application back-end.

5.3 Part 3: Build

The Build part is the one where you will combine the business (or back-end) logic with the front-end. In this third part, you will work on the core engine of the application, making the business logic work inside the interactive logic of your application.

This step of the workflow is cover in the Building app with {golem}, a chapter that presents the various functions you can use to build your application, i.e the one you will be using to combine your back-end and front-end.

In this step, we will cover:

  • How to handle dependencies in your project, i.e how to use leverage external libraries inside your project
  • How to organize modules and functions inside your project
  • How to add tests for the back-end of your application (testing will be covered in more depth in Part 4)
  • How to document your application and its codebase, and how to add code coverage and continuous integration
  • How to leverage the internal dev functions from {golem} to modify the behavior of specific functions based on an option()

5.4 Part 4: Strengthen

Strengthening your app is ensuring your application is immortal, in the sense that we defined in the first chapter of this book.

In this part, we will go through unit tests, reproducible development environment, version control and continuous integration in the context of Shiny applications. Building a solid testing suite is crucial to the success of a project, as it allows a project to be stable on the long run, be it when you will want to add new feature, or refactor existing code:

Refactoring requires we be able to confidently ensure that behavior remains identical at every iteration. We can increase our confidence that nothing has changed by writing a suite of tests (unit, integration, end-to-end), and we should not seriously consider moving forward with any refactoring effort until we’ve established sufficient test coverage.

This step of the workflow will span over two chapters.

  • The first one, Build yourself a safety net, details how to build a testing environment for your Shiny Application, be it for testing the back-end or the front-end. In this chapter, you will be introduced to {testthat} for testing your application back-end, tools that are more linked to testing the front-end like NodeJS puppeteer module, {shinytest} and {crrry} for testing interactive logic, {shinyloadtest} and {dockerstats} for testing your application load. This chapter will also cover {renv} and Docker, two essential tools for developing in a reproducible environment.

  • In the Version Control chapter, you will be introduced to git and to automated test using Continuous Integration platforms like Travis CI or GitHub Actions.

5.5 Part 5: Deploy

To Deploy is to send your application into production once it is built.

Being exhaustive here would be an impossible task: there are countless ways to make your application accessible to its targeted users, but we will try to cover some basics in this part. And of course, where and how you will be deploying your application depends of a lot of parameters. For example, who are the end users, and how do they want to use your application? If the end users are familiar with R and use it on a daily basis, they might be looking for an application that runs with library(app), i.e they need the application to be available as an R package they can install on their machine. If the end users are not coders, they might need the application to be available only as a web application, so they just have to open a browser and navigate to an URL. Both these cases raise other questions: how to make the package available on a repository so that R users can get it with install.packages()? If the application is to be made available on an URL, how will it be deployed? What deployment server is available to you, or to the company ordering the application? These questions (and more) will be covered in the Deploy part of this book.

In this part, we will present a series of methods to prepare your application to be deployed on various environments, notably:

  • Sharing your application as a package so that it can be installed manually, through GitHub, or shared on a package repository like the CRAN or BioConductor
  • Sending it to an RStudio platform
  • Building a Docker image to serve your app on a cloud provider.

This step of the workflow is covered in the Deploy your application chapter.

References

Fay, Colin, and Sebastien Rochette. 2020b. Shinipsum: Lorem-Ipsum-Like Helpers for Fast Shiny Prototyping. https://github.com/Thinkr-open/shinipsum.


  1. We use the term “client” in a lose sense, meaning the person(s) who is/are ordering the application.↩︎


ThinkR Website