2 Planning Ahead
2.1 Working with a “long-term” mindset
Rome ne fut pas faite toute en un jour.
French proverb
2.1.1 Prepare for success
Whatever your ambitions for your shiny application, you should take time today to set robust foundations that will save a lot of time in the future.
A common thing you will hear about shiny is that it is a good prototyping tool. This cannot be denied. Building a Proof of Concept (PoC) for an app is relatively straightforward if you compare to what is needed to build applications in other languages. With shiny, you can build an “it works on my machine” web application in a couple of hours, and show it to your team, your boss, your investors. Thanks to the way shiny was designed, you do not have to care about websockets, ports, HTML (HyperText Markup Language), JavaScript libraries, and all the things that are elegantly bundled into shiny.
Hence, you can have a quick, hacky application that will work on your machine, and very rapidly. But that is not the way you should start. Indeed, starting with hacky foundations will lead to two possibilities:
- You will have to rewrite everything from scratch to have a robust application for production.
- If you do not want to rewrite all the code, you will get stuck with a legacy codebase for the application, built on top of hacky functions, and sent to production using hacky solutions.
Either way, that is a heavy technical debt.
shiny is a good tool for prototyping, but there is no harm in starting your application on solid ground, even for a prototype: the sooner you start with a robust framework the better, and the longer you wait the harder it gets to convert your application to a production-ready one. The larger the codebase, the harder it is to untangle everything and make it work.
In this book, we will present a framework called golem, which is a toolbox for building production-grade shiny applications. Even if golem is focused on production, there is no reason not to use it for your proof of concepts: starting a new golem project is relatively straightforward, and even if you do not use the advanced features, you can use it for very small apps. The benefit of starting straight inside a golem application really outweighs the cost. We hear a lot the question “When should I switch to golem?” The answer is simple: do not switch to golem, start with it. That way, you are getting ready for complexity, and if, one day, you need to turn this small app into a production app, the foundations are there.
2.1.2 Develop with the KISS principle
The KISS principle states that most systems work best if they are kept simple rather than made complicated; therefore, simplicity should be a key goal in design, and unnecessary complexity should be avoided.
KISS principle, Wikipedia article (https://en.wikipedia.org/wiki/KISS_principle)
The KISS principle, as “Keep It Simple, Stupid”, should drive the implementation of features in the application to allow anyone in the future, including original developers, to take over on the development.
The story behind this principle is supposed to be that Kelly Johnson, lead engineer at the Lockheed Skunk Works, gave his workers a set of very common tools and said that every airplane should be repairable with these tools, and these tools only, so that repairing an aircraft should be possible for any average engineer.
This should be a principle to keep in mind when building applications. Indeed, large-scale shiny projects can lead to many people working on the codebase, for a long period of time. A large team means a variety of skills, with some common ground in shiny development, but potentially various levels when it comes to R, web development, or production engineering. When choosing how and what to implement, try to make a rule to go for the simplest solution,16 i.e. the one that any common shiny developer would be able to understand and maintain. If you go for an exotic solution or a complex technology, be sure that you are doing it for a good reason: unknown or hard-to-grasp technology reduces the chance of finding someone that will be able to maintain that piece of code in the future, and reduce the smoothness of collaboration, as “Code you can easily comprehend elevates absolutely everyone on your team, no matter their tenure or experience level” (Lemaire 2020).
2.2 Working as a team: Tools and structure
Working as a team, whatever the coding project, requires adequate tools, discipline and organization. Complex shiny apps usually imply that several people will work on the application. For example, at ThinkR, 3 to 4 people usually work in parallel on the same application, but there might be more people involved on larger projects. The choice of tools and how the team is structured is crucial for a successful application.
2.2.1 From the tools point of view
A. Version control and test all things
To get informed about a code break during development, you will need to write tests for your app, and use continuous integration (CI) so that you are sure this is automatically detected.17 When you are working on a complex application, chances are that you will be working on it for a significant period of time, meaning that you will write code, modify it, use it, go back to it after a few weeks, change some other things, and probably break things. Breaking things is a natural process of software engineering, notably when working on a piece of code during a long period. Remember the last chapter where we explained that complex applications are too large to be understood fully? Adding code that breaks the codebase will happen with complex apps, so the sooner you take measures to solve these changes, the better.
As you cannot prevent code from breaking, you should at least get the tooling to:
- Be informed that the codebase is broken: this is the role of tests combined with CI.
- Be able to identify changes between versions, and potentially, get back in time to a previous codebase: this is the role of version control.
We will go deeper into testing and version control in chapter 14.
B. Small is beautiful
Building an application with multiple small and independent pieces will lighten your development process and your mental load. The previous chapter introduced the notion of complexity in size, where the app grows so large that it is very hard to have a good grasp of it. A large codebase implies that the safe way to work is to split the app into pieces.
Splitting a shiny project is made possible by following two methods:
Extract your core “non-reactive” functions, which we will also call the “business logic”, and include them in external files, so that you can work on these outside of the app. Working on independent functions to implement features will prevent you from relaunching the whole application every time you need to add something new.
Split your app into shiny modules, so that your app can be though of as a tree, making it possible for every developer to concentrate on one node, and only one, instead of having to think about the global infrastructure when implementing features.
Figure 2.1 is, for example, a representation of a shiny application with modules and sub-modules. You will not be able to decipher the text inside the node, but the idea is to give you a sense of how a shiny application with modules can be organized and split into smaller pieces that are all related to each other in a tree form.
FIGURE 2.1: Representation of a shiny application with its modules and sub-modules.
We will get back to shiny modules and how to organize your project in the next chapter.
2.2.2 From the team point of view
We recommend that you define two kinds of developers: a unique person (or maybe two) to be in charge of supervising the whole project and developers that will work on specific features. Note that this is how a project should be organized in a perfect world: in practice, a lot of shiny projects are managed by one developer who is in charge of managing the project, interacting with the client, and building the whole codebase.
A. A person in charge
The person in charge of the development will have a complete view of the entire project and manage the team so that all developers implement features that fit together.
With complex applications, it can be hard to have the complete understanding of what the entire app is doing. Most of the time, it is not necessary for all developers to have this complete picture. By defining one person in charge, this “manager” will have to get the whole picture: what each part of the software is doing, how to make everything work together, avoid development conflicts, and of course check that, at the end of the day, the results returned by the built application are the correct ones.
The project manager will be the one that kicks off the project, and writes the first draft of the application. If you follow this book’s workflow, this person will first create a golem project, fill in the information, and define the application structure by providing the main modules, and potentially work on the prototyped UI of the app.
Once the skeleton of the app is created, this person in charge will list all the things that have to be done.
We strongly suggest that you use Git with a graphical interface (GitLab, GitHub, Bitbucket, etc.) as the graphical interface to help you manage the project.
These tasks are defined as issues, and will be closed during development.
These interfaces can also be used to set continuous integration.
If the team follows a git flow (described in Chapter 12), the manager will also be in charge of reviewing and accepting the pull/merge requests to the main dev branch if they solve the associated issues.
Do not worry if this sounds like a foreign language to you, we will get back to this method later in this book (Chapter 12).
B. Developers
Developers will focus on small features. If the person in charge has correctly separated the work between developers of the team, they will be focusing on one or more parts of the application, but do not need to know every single bit of what the application is doing. In a perfect world, the application is split in various shiny modules, one module equals one file, and each member of the team will be assigned to the development of one or more modules.
It is simpler to work in this context where one developer is assigned to one module, although we know that in reality it may be a little more complex, and several members of the team might go back and forth working on a common module. But the person in charge will be there to help make all the pieces fit together.