Cloud-based development has the potential to become the new paradigm for software engineering.
Cloud development or cloud-based development has been around for quite a few years. However, it has never really become mainstream. So far. I believe there are shifts in the software market that could lead to a renaissance or even a final breakthrough of this development approach. Before I start with these, let’s take a look at what I mean with “cloud development”.
What is Cloud Development?
There are many definitions of cloud development or cloud-based development (I will use these terms interchangeably). For the purpose of this article, I will use this definition: Cloud development is a software development approach that uses a cloud environment (instead of a local computer) to execute the unfinished software during the actual development phase. That means that your software is running in the cloud while you are still developing it and it thus usually never runs on your local computer. This approach is mainly useful if you are developing software that is meant to run in the cloud, so it will be pushed to staging, testing and production also in cloud environments.
Some others define cloud development as using an IDE that is browser-based and online. While a browser-based editor is usually linked to a cloud environment to execute the software, it is also possible to use local editors and execute the software in the cloud (e.g. using Visual Studio's remote environment features), which is why I will use the broader definition above.
Another more recent term coming from the Kubernetes and CNCF community is “cloud-native development”, which is also related to my definition of cloud development, but is more a general concept referring “to application development that is container-based, dynamically orchestrated, and leverages microservices architectures”. It is thus rather focused on what is developed and not so much on how.
Cloud Development Has Never Become Mainstream
The idea of cloud development is not very new. You can find articles about this topic that are more than 8 years old. Cloud development experienced its first surge in popularity with the emergence of online IDEs such as Cloud9 and Eclipse Che. This may be the reason why some people directly associate these tools with the term cloud development. These browser-based IDEs provide you with tools and editors that are running in the cloud and can be directly accessed via a normal browser such as Google Chrome.
Since the source code is already written in the cloud, it is usually also executed in a connected cloud environment, which makes it “actual” cloud development according to my definition. There were additional tools to provide these environments and the necessary cloud computing resources for the execution of the software, such as Codenvy or Koding. Overall, these tools promise their users to program software from any computer no matter how much computing power it has and without having anything to set up.
Some of the products mentioned above are not active anymore or have been acquired, so their tools are now integrated into other offerings, e.g. Cloud9 is now part of AWS and Codenvy was acquired by RedHat. However, this setup for developers has never reached mass adoption, mainly because it forced users to switch to an online IDE that was less powerful and simply different than the ones developers were used to from local development. Additionally, it might have just been too early for this approach of development as even cloud computing itself was not as common as it is today.
So, what has changed now?
Why Cloud Development Could Now Have A Breakthrough
The business environment has changed
In the past years, a lot of things have changed in the software world that makes development in the cloud much more sensible and easier:
- Using the cloud to run software has become normal. Nowadays, using the cloud for production workloads is the standard for many companies. This transition is related to the emergence of the Software-as-a-Service model (SaaS) to sell software and is a necessary first step towards cloud development as only if the production loads are in the cloud it makes sense to also move the development runtime into the cloud. The wide adoption of cloud computing thus also increases the potential userbase for cloud development.
- Software has become more complex. With the rise of Artificial Intelligence (AI), Machine Learning (ML) and microservices, the complexity of software and thus also the computing resource needs to run this software have significantly increased. Since local computers have limited computing power by their nature, they are not capable anymore to run every software that their users want to develop. This might even make the use of the cloud during development inevitable in some cases.
- Software has become independent of the runtime environment. Due to container technologies such as Docker and Kubernetes, software is now often packaged in containers that can run in any environment, cloud and local, as long as the base technologies are available. That means, if you are already using Kubernetes for your production loads and you work with containers, the switch from local to cloud-based development will be pretty straightforward.
Some Obstacles Have Been Cleared
In the past, there also were some challenges that are now (at least partly) solved, which makes cloud development more feasible:
- Latency/Internet Connection. If you want to develop in the cloud, you always need an internet connection to communicate with the cloud. Fortunately, the average internet connection has become faster and faster in the past years and WIFI access is ubiquitous. Since for development purposes often just small source code files are changed, not much data has to be transferred during development, so latency is nowadays often insignificant.
- Deployment Process Time. If you are not using an online IDE, your code needs to be transferred to the cloud somehow and your application needs to be updated. Especially for container technologies, new solutions such as DevSpace can automatically transfer your changes to the cloud and will update your application without having to restart the containers. This reduces the time for deployment as you do not need to run a whole deployment pipeline for every small change and cloud development feels like local development.
- Migration Effort To The Cloud. Adapting a legacy application so it will smoothly run in a cloud environment can take quite some time. While it remains questionable if there ever will be a (perfect) tool that can transform every software application into a perfect cloud-native app, moving to the cloud seems to be necessary for more and more companies anyway, given the macro trends towards cloud-computing and SaaS.
- Effort To Manage Cloud Access. If every developer interacts with the cloud, they need access to the cloud in some way. Centrally managing and controlling this access can be a huge challenge, especially for larger teams. However, due to public cloud solutions, it is quite easy to create new cloud-based development instances. It is even possible to just use a single instance and then share the access among developers. If you are using such a multi-tenancy setup with Kubernetes, every developer could, for example, get an isolated namespace that can be created on demand by the developer when you use a platform solution such as loft.
- Security Issues. Running your code in the cloud already during development means that essentially all your source code is in the cloud from the beginning. This is not necessarily a problem as most companies use code repositories in the cloud already anyway. Still, some people might fear that someone might get unauthorized access to your development secrets. For them, private clouds could be a good solution.
- Cloud Cost. If you are using a public cloud, you have to pay for the resources you use. If all developers in your team need their own cloud environment, the cost for the computing resources can become pretty high very quickly. This is a general problem of cloud development that cannot be completely solved, but there are ways to reduce the cost impact.
With container technologies, you even have the opportunity to “replicate” the cloud environment locally. For this, you can use Docker Desktop and Docker Compose (or for Kubernetes: minikube and DevSpace) to work in a local environment that feels like a cloud environment. Since it is running locally, you are limited to the resources of your computer but do not have to pay anything extra.
Benefits of Cloud Development
So, overall the conditions have become much better and using cloud development is easier than ever. The question is now why would you want to do it.
Indefinite Computing Power
While you are limited to the resources your computer has for local development, using the cloud gives you virtually endless computing power. You can also use specific hardware that you could not use otherwise. Here, especially GPUs are relevant as they are needed for many AI and ML applications. Also for microservice applications, you may need significant power to get all services up and running, which sometimes is simply impossible to do locally. Companies in such a situation just have to move to the cloud, also for the development, if they want to continue to improve their applications efficiently.
Less Setup Effort
Getting to run an application can be quite a lot of work, irrespective of the runtime environment. However, if you use cloud development, it is possible that one person in your team sets up and configures everything and all other team members can directly start. In the Kubernetes world, this can be done with open-source tools such as Helm and DevSpace, where you can configure the whole environment that then just has to be deployed in a cloud environment. This replicability is a major strength of the cloud as there are no differences in hardware or operating systems (unless you want to have different configurations). It is also extremely flexible, so you can adjust it specifically to your individual needs. Additionally, the public cloud providers offer a range of tools and building blocks, just take a look at the services of Google Cloud or AWS. Together with the huge variety of open source projects for cloud applications and environments, that can be illustrated with the CNCF landscape, you can thus get started very fast.
New Collaboration Possibilities & Standardization
Due to standardization, e.g. expressed in shared configuration files for your cloud environment, it is easily possible to replicate bugs and to support each other in a team. It is even possible to give a colleague direct access to your cloud environment to fix something or to share your work results. This can result in more teamwork and a new form of collaboration in your team in which everybody can contribute their strengths.
Access From Anywhere
Since your application is running in the cloud already during development, you do not have to always use the same computer that has a very specific setup. You can rather switch your local hardware as you like, which makes it easier if your computer breaks and needs to be replaced and which also supports the modern work culture, such as working from home or on the go.
Living The DevOps Culture
Developing software that is made for the cloud directly in the cloud makes a lot of sense as you will always use a very similar environment throughout the whole lifecycle of your application. This can reduce the number of bugs and problems that can otherwise occur after deploying the app to production. For this, cloud-based development will foster the DevOps culture in your team.
Cloud development is a concept that has been around for quite a few years but failed to really become mainstream in the past. However, due to a combination of macro trends towards cloud and Software-as-a-Service with technical advancements such as the container technologies Docker and Kubernetes, cloud development now has the chance to finally become the new standard to develop cloud-based applications. Even if probably not everybody will leave the local environment in the (near) future, it could reach a wide adoption in companies that have adopted DevOps practices and want to develop truly cloud-native applications.