Chapter 2 Planning Ahead

2.1 Working with a “long-term” mindset

“Rome ne fut pas faite toute en un jour”

French proverb

2.1.1 Prepare for success

Whatever your ambitions for your Shiny application are, you should take time today to set robust foundations that will save a lot of time in the future.

A common thing you will hear about Shiny is that it is a good prototyping tool. This can not be denied. Building a Proof of Concept (PoC) for an app is relatively straightforward if you compare to what it needs to build applications in other languages. With Shiny, you can build a “it works on my machine” web application in a couple of hours, and show it to your team, your boss, your investors… Thanks to the way Shiny was designed, you do not have to care about websockets, ports, HTML, JavaScript libraries, and all the things that are elegantly bundled into Shiny.

Hence, you can have a quick, hacky application that will work on your machine, and that, very rapidly. But that is not the way you should start. Indeed, starting with hacky foundations will lead to two possibilities:

  • You will have to rewrite everything from scratch to have a robust application for production.
  • If you do not want to rewrite all the code, you will get stuck with a legacy codebase for the application, built on top of hacky functions, and sent to production using hacky solutions.

Either way, that is a heavy technical debt.

Shiny is a good tool for prototyping, but there is no harm in starting your application on solid ground, even for a prototype: the sooner you start with a robust framework the better, and the longer you wait the harder it gets to convert your application to a production-ready one. The larger the codebase, the harder it is to untangle everything and make it work.

In this book, we will present a framework called {golem}, which is a toolbox for building production-grade Shiny applications. Even if {golem} is focused on production, there is no reason not to use it for your proof of concepts: starting a new {golem} project is relatively straightforward, and even if you do not use the advanced features, you can use it for very small apps. The benefit of starting straight inside a {golem} application really outweigh the cost. We hear a lot the question “When should I switch to {golem}?”, and the answer is simple: do not switch to {golem} , start with it. That way, you are getting ready for complexity, and if, one day, you need to turn this small app into a production app, the foundations are there.

2.1.2 Develop with the KISS principle

The KISS principle states that most systems work best if they are kept simple rather than made complicated; therefore, simplicity should be a key goal in design, and unnecessary complexity should be avoided.

The KISS principle, as “Keep It Simple, Stupid”, should drive the implementation of features in the application to allow anyone in the future, including original developers, to take over on the development.

The story behind this principle is supposed to be that Kelly Johnson, lead engineer at the Lockheed Skunk Works, gave his workers a set of very common tools and said that every airplane should be repairable with these tools, and these tools only, so that repairing an aircraft should be possible for any average engineer.

This should be a principle to keep in mind when building application. Indeed, large scale Shiny projects can lead to many people working on the code base, for a long period of time. A large team means a variety of skills, with some common ground in Shiny development, but potentially various levels when it comes to R, web development, or production engineering. So when choosing how and what to implement, try to make a rule to go for the simplest solution,12 i.e. the one that any common Shiny Developer would be able to understand and maintain. If you go for an exotic solution or a complex technology, be sure that you are doing it for a good reason: unknown or hard to grasp technology reduces the chance of finding someone that will be able to maintain that piece of code in the future, and lower the smoothness of collaboration, as “Code you can easily comprehend elevates absolutely everyone on your team, no matter their tenure or experience level(Lemaire 2020).

2.2 Working as a team: Tools & Structure

Working as a team, whatever the coding project, requires adequate tools, discipline and organization. Complex Shiny Apps usually imply that several people will work on the application. For example, at ThinkR, 3 to 4 people usually work in parallel on the same application, but there might be more people involved on larger projects. The choice of tools and how the team is structured is crucial for a successful application.

2.2.1 From the tools point of view

2.2.1.1 Version Control & Test all the things

To get informed about a code break during development, you will need to write tests for your app, and use continuous integration (CI) so that you are sure this is automatically detected.13 When you are working on a complex application, chances are that you will be working on it for a significant period of time, meaning that you will write code, modify it, use it, go back to it after a few weeks, change some other things, and probably break things. Breaking things is a natural process of software engineering, notably when working on a piece of code during a long period. Remember the last chapter where we defined that complex applications are too large to be understood fully? Adding code that breaks the codebase will happen with complex app, so the sooner you take measure to solve these changes, the better.

As you can not prevent code to break, you should at least get the tooling to:

  • Be informed that the codebase is broken: this is the role of tests combined with CI
  • Be able to identify changes between versions, and potentially, get back in time to a previous code base: this is the role of version control

We will go deeper into testing and version control in chapter 14.

2.2.1.2 Small is beautiful

Building an application with multiple small and independent pieces will lighten your development process and your mental load. Previous chapter introduced the notion of complexity in size, where the app grows so large that it is very hard to have a good grasp of it. A large code base implies that the safe way to work is to split the app into pieces.
Splitting a Shiny project is made possible by following two methods:

  • Split your app into Shiny Modules, so that your app can be though of as a tree, making it possible for every developer to concentrate on one node, and only one, instead of having to think about the global infrastructure when implementing features
Representation of a Shiny application with its modules and sub-modules.

FIGURE 2.1: Representation of a Shiny application with its modules and sub-modules.

  • Extract your core “non-reactive” functions, that we will also call the “business logic”, and include them in external files, so that you can work on these outside of the app. Working on independent functions to implement features will prevent you to relaunch the whole application every time you need to add something new.

We will get back to Shiny modules and how to organize your project in the next chapter.

2.2.2 From the Team Point of View

We recommend to define two kind of developers: a unique person (or maybe two) to be in charge of supervising the whole project and developers that will work on specific features.

2.2.2.1 A person in charge

The person in charge of the development will have an complete view of the entire project and manage the team so that all developers implement features that fit together.

With complex applications, it can be hard to have the complete understanding of what the entire app is doing. Most of the time, it is not necessary for all developers to have this complete picture. By defining one person in charge, this “manager” will have to get the whole picture: what each part of the software is doing, how to make everything work together, avoid development conflicts, and of course check that, at the end of the day, the results returned by the built application are the correct ones.

The project manager will be the one that kicks off the project, and write the first draft of the application. If you follow this book’s workflow, this person will first create a {golem} project, fill the information, and define the application structure by providing the main modules, and potentially work on the prototyped UI of the app.

Once the skeleton of the app is created, this person in charge will list all the things that have to be done. We strongly suggest to use git with a graphical interface (GitLab, GitHub, Bitbucket, …), as the graphical interface of these services will help you manage the project. These tasks are defined as issues, and will be closed during development. These interfaces can also be used to set continuous integration.

If the team follows a git flow (described in chapter @(ref: secure)), the manager will also be in charge of reviewing and accepting the pull/merge requests to the main dev branch if they solve the associated issues.

Do not worry if this sounds like a foreign language to you, we will get back to this method later in this book (chapter @(ref: secure)).

2.2.2.2 Developers

Developers will focus on small features. If the person in charge has correctly separated the work between developers of the team, they will be focusing on one or more parts of the application, but do not need to know every single bit of what the application is doing. In a perfect world, the application is split in various Shiny Modules, one module equals one file and each member of the team will be assigned to the development of one or more modules.

It is simpler to work in this context where one developer is assigned to one module, although we know that in reality it may be a little more complex, and several members of the team might go back and forth working on a common module. But, the person in charge will be there to help make all the pieces to fit together.

References

Lemaire, Maude. 2020. Refactoring at Scale. Henry Holt. https://learning.oreilly.com/library/view/refactoring-at-scale/9781492075523/.


  1. Which might not be the most “elegant” solution, but production code requires pragmatism.↩︎

  2. Relying on automatic tooling for monitoring code base is way safer than relying on developers to do manual checks every time they commit code.↩︎


ThinkR Website