Use case - Building an App, from Start to Finish | Engineering Production-Grade Shiny Apps

About the application

In this appendix, we will build a “minify” application, an application that takes a CSS, JavaScript, HTML or JSON input, and output a minified version of this file. We will rely on the {minifyr} package to build this application

Here is for example what the specifications for this app could look like

Hello! 

We want to build a small application that can 
minify CSS, JavaScript, HTML and JSON. 

In this app, user will be able either to paste 
the content or to upload a file. 

Once the content is pasted/upladed, they select 
the type, which is pre-selected based on the 
file extension. Then they click on a button, 
and the content is minified. 

They can then copy the output, or download it as a file. 

Cheers!

Step 1: Design

Deciphering the Specifications

General Observations

As this app is pretty straightforward, it would be better to handle everything in the same page, i.e everything should happen on the same page (no tab)
It would be a plus to have the “before minification” / “after minification” gain, so that the users have a better idea of the purpose of the application

User Experience Considerations

We should provide a link to an explanation of what is minification
User might get different result based on what minifying algorithm they use, which can be surprising at first. The application should alert about this.
For long printed outputs, if we use verbatimTextOuput, we should be careful about the page width, as these elements will natively overflow on the x-axis of the page. This should be doable with the following CSS: pre{ white-space: pre-wrap; word-break: keep-all; }
We should be careful about using semantic html for the inputs and output

Technical Points

As {minifyr} wraps a NodeJS module, we will need to install NodeJS when deploying
To be sure that the process works, we should check the validity of the file extension from the UI and from the server side

Building a Concept Map

Here is the concept map for this application, using Xmind.

Asking Questions

About the end users

Who are the end users of your app?

This application is mainly useful for web-developers.

Are they tech-literate?

Yes.

In which context will they be using your app?

Notably at work, or while building pet projects.

On what machines?

Laptop/personal computer. Small chance of using this on a smartphone.

What browser version will they be using?

Hard to say, but given that we aim for a public of developers, probably modern browsers.

Will they be using the app in their office, on their phone while driving a tractor, in a plant, while wearing lab coats?

Nothing of the like: they should be using this application while developing, so chances are they are using it at a desk.

Building personas

Let’s pick two random names for our personas, and two fake companies where they might be working at.

nms <- withr::with_seed(
  608, {
    charlatan::ch_name(2)
    
  }
)
nms

[1] "Delina Stanton" "Theo Torphy"

company <- withr::with_seed(
  608, {
    charlatan::ch_company(2)
  }
)
company

[1] "Witting-Witting" "Stehr-Stehr"

Delina Stanton - `{shiny}` developer at Witting-Witting

Delina Stanton is a {shiny} developer at Witting-Witting. She’s been learning R at grade-school during her Master’s Degree in statistics. When she started at Witting-Witting, she was mainly doing data analysis in Rmd, but has gradually switched to building {shiny} applications full-time. She discovered minification while reading the “Engineering Production-Grade Shiny App” book, and now want to add this to her {shiny} application.

Theo Torphy - web developer and trainer at Stehr-Stehr

Theo Torphy is a web developer at Stehr-Stehr. He studied web development at the university, where he learned about minification. He is now also in charge of training new recruits for the company he works at, and also gives some lecture to the university he went to. Most of the minification he does is automated, but he is looking for a tool he can use during trainings and classes to explain how minification works.

This step is available at github.com/ColinFay, folder minifying/step-1-design.

Step 2: Prototyping

In this step, we will be building the back-end of the application on one side, and the UI on the other side. Once we have the back-end settled and the UI defined, we will be working on making the two work with each other.

Backend in Rmd

Our back-end will be composed of two functions:

guess_minifier, which will take a function, and return the available algorithms for that file: for example, if you have a JavaScript file, you’ll be able to use the minifyr_js_babel()`()`,minifyr_js_gcc()(), minifyr_js_terser()`()`,minifyr_js_uglify()(), and `minifyr_js_yui()()functions. If the type is not guessed based on the extension, the function should fail gracefully, and not make{shiny}` crash. We’ll chose to return an empty string if this extension is not guessed.

library(minifyr)
guess_minifier <- function(file){
  # We'll start by getting the file extension
  ext <- tools::file_ext(file)
  # Check that the extension is correct, if not, return early
  # It's important to do this kind of check also 
  # on the server side as HTML manual tempering 
  # would allow to also send other type of files
  if (
    ! ext %in% c("js", "css", "html", "json")
  ){
    # Return early
    return(list())
  }
  # We'll then retrieve the available 
  # pattern based on the extension
  patt <- switch(
    ext, 
    js = "minifyr_js_.+",
    html = "minifyr_html_.+",
    css = "minifyr_css_.+",
    json = "minifyr_json_.+"
  )
  # List all the available functions to minify the file
  list(
    file = file,
    ext = ext,
    # We return this pattern so that 
    # it will be used to update the selectInput that 
    # is used to select an algo
    pattern = patt, 
    functions = grep(
      patt, 
      names(
        loadNamespace("minifyr")
      ), 
      value = TRUE
    )
  )
  
}

# minifyr comes with a series of examples, 
# so we can use them as tests
guess_minifier(
  minifyr_example("css")
)[2:4]
guess_minifier(
  minifyr_example("js")
)[2:4]
guess_minifier(
  minifyr_example("html")
)[2:4]
guess_minifier(
  minifyr_example("json")
)[2:4]
# Try with a non valid extension
guess_minifier(
  "path/to/text.docx"
)

A compress() function, which takes three parameters: the file as input, the algo, outputed from our last function, nd the selection, which is the one selected by the user. The compressed file will be outputed to a tempfile.

compress <- function(algo, selection){
  # Creating a tempfile using our algo object
  tps <- tempfile(fileext = sprintf(".%s", algo$ext))
  # Getting the function with the selection
  converter <- get(
    grep(selection, algo$functions, value = TRUE)
  )
  # Do the conversion
  converter(algo$file, tps)
  # Return the temp file
  return(tps)
}

algo <- guess_minifier(
  minifyr_example("js")
)

compress(
  algo = algo, 
  selection = "babel"
)

Finally, a compare() function, that can compare the size of two files, so that we can measure the minification gain. This function will take two file paths.

compare <- function(original, minified){
  # Get the file size of both
  original <- fs::file_info(original)$size
  minified <- fs::file_info(minified)$size
  return(original - minified)
}

So, for the complete process:

algo <- guess_minifier(
  minifyr_example("js")
)

compressed <- compress(
  algo = algo, 
  selection = "babel"
)
compare(
  minifyr_example("js"), 
  compressed
)

Now, time to move this into a Vignette!

UI prototyping

Let’s start by drawing a small mock-up of our front-end using Excalidraw.

We would love this application to be “full screen”, and to do that, we’ll take inspiration from the split-screen layout available at W3Schools. To mock the UI, we will also use the {shinipsum} package.

In this first step, we will start generating the module skeleton for the application. Here, we will have a left module for the left part of the app, and right for the right. Each will receive their corresponding class, based on the CSS from W3. Now that these two spots are available, we’ll add the two modules, with some fake output to simulate our application behavior. The left side will be functional, in the sense that uploading a file will randomly add algorithms to the selectInput(), and clicking on the Launch the minification will regenerate a fake text.

Now, let’s pick a soft palette of colors, using coolors.co, and a font family from fonts.google.com. We went for:

One of the monochrome palette from coolors.co.
The Sora font fonts.google.com/specimen/Sora. There are not that much text displayed on the screen, so this font should work well.

We then used CSS to arrange our page: size, padding, alignment, colors… If you want to know more about this file, it’s located in the inst/app/www folder of the package.

This step is available at github.com/ColinFay/minifying, folder step-2-prototype.

Step 3: Build

Now we’ve got the backend in an Rmd, the front end working with {shinipsum}, now is the time to make the two work together!

Here is the logic we will be adding to the application:

When a file is uploaded, we also check the format from the server-side: UI only restriction using accept will not be enough to stop users who really want to upload something else.
If the file comes with the right extension, we update the algorithm selection, and read it inside the “Original content” block.
Once the user clicks on “Launch the minification”, we create a temp file, and minify the original file inside this temp file.
When the file is minified, we update the gain output to reflect how many bytes have been gained from the minification, and add the result of this minification to the “Minified content” block.
Finally, when the “Download the output” button is clicked, the minified file is downloaded.

During this process, we will migrate the functions from the Rmd to their own files, use external dependencies, and document our business logic functions. You can refer to the dev/02_dev.R file if you want to read the exact steps taken here.

This step is available at github.com/ColinFay/minifying, folder step-3-build.

Step 4: Strengthen

As of now, we have a working application. Time to strengthen it!

Here are the few steps we will be working on:

Turning our business logic into an R6 Class, so that we can build tests around it, and prevent the data structure from taking part into the interactivity mechanic. This R6 class will generate an object at the very start of our app, and it will be passed into the modules.
As the minification process takes a couple of seconds, we will add a small progress bar so that the user know something is happening.
As we will use R6, we will need to manually set the reactive context invalidation. To do so, we will use triggers from {gargoyle}.
Chances are that the users will be testing several algorithms when using the application, and we don’t want the minification process to happen another time when it is called on the same file and with the same algorithm, even more as the process involves calling an external Node process. To prevent that, we will be caching the function that does the computation.
Create an unseen input that will upload data, so that we can build an interactivity test using {crrry}. This input will look like this on the server:

observeEvent( input$testingtrigger , {
  
  if (golem::app_dev()){
    file$original_file <- minifyr::minifyr_example(
      ext = input$testingtext
    )
    file$guess_minifier()
    file$type <- input$upload$type
    file$minified_file <- NULL
    file$original_name <- input$upload$name
    gargoyle::trigger("uploaded")
  }
  
})

We use this pattern so that we can combine it with a testing suite with {crrry}, using the following pattern:

test <- crrry::CrrryProc$new(
  chrome_bin = pagedown::find_chrome(),
  # Process to launch locally
  fun = "golem::document_and_reload();run_app()",
  # Note that you will need httpuv >= 1.5.2 for randomPort
  chrome_port = httpuv::randomPort(), 
  headless = FALSE
)

test$wait_for_shiny_ready()
ext <- c("css", "js", "json", "html")
for (i in 1:length(ext)){
  # Set the extension value
  test$shiny_set_input("left_ui_1-testingtext", ext[i])
  # Trigger the file to be read
  test$shiny_set_input("left_ui_1-testingtrigger", i)
  # Launch the minification
  test$shiny_set_input("left_ui_1-launch", i)
}

test$stop()

It’s safer to wrap these tests between if(interactive()), as running the checks outside of your current session might not launch the app correctly, and launching external processes (the one running the app with Chrome) might fail when run non-interactively. And on top of that, running these inside you CI might cause some pain, and of course if will not work on CRAN checks.

We’ll also be building “standard” function checks, which you can find in the test/ folder.

This step is available at github.com/ColinFay/minifying, folder step-4-strengthen.

Step 5: Deploy

As an example, we will deploy this app through three medium: as a package, on RStudio Connect, and with Docker.

Deploy as a tar.gz

To share an application as a tar.gz, you can call devtools::build(), which will compile a tar.gz file inside the parent folder of your current application. You can then share this archive, and install it with remotes::install_local("path/to/tar.gz"). Note that this can also be done with base R, but {remotes} offers a smarter way when it comes to manage dependencies of your archived package.

This tar.gz can also be send to a package repository; be it the CRAN or any other package manager you might have in your company.

Deploy on RStudio Connect

Once we are sure that the server running connect has NodeJS installed, and that we have installed the minify module with minifyr::minifyr_npm_install(), we can create the app.R using golem::add_rstudioconnect_file(), and then push to the Connect server.

Deploy with Docker

To create the Dockerfile, we’ll start by launching golem::add_dockerfile(). This function will compute the system requirements⁷⁴., and create a generic Dockerfile for your application. Once this is done, we will create/update the .dockerignore file at the root of the package, so that unwanted files are not bundled with our docker image.

Inside our Dockerfile, we will also change the default repo to use “https://packagemanager.rstudio.com/all/latest”, which proposes precompiled packages for our system, making the installation faster. We will also add an installation of NodeJS, which is needed by our application

Then, we can go to our terminal, and compile the image!

docker build -t minifying .

Now we’ve got a working image! We can try it with:

docker run -p 2811:80 minifying

This step is available at github.com/ColinFay/minifying, folder step-5-deploy.

Use case - Building an App, from Start to Finish