In the last post of this series, I shared a link to a podcast that described CI/CD. This can be broadly described as a ‘software engineering pipeline where changes are made to software in a structured and organised way which are then made available for use’. I should add that this is my own definition!
The abbreviation CI/CD is sometimes used with the term DevOps, which is a combination of two words: development and operations. In the olden days of software engineering, there used to be two teams: a development team, and an operations team. One team build the software; the other team rolled it out and supported its delivery. To all intents and purposes, this division is artificial, and also unhelpful. The idea of DevOps is to combine the two together.
Looking at all these terms more broadly, DevOps can be thought as a philosophical ideal about how work is done and organised, whereas CI/CD release to specific practices. Put more simply, CI/CD makes DevOps possible.
A broader question is: how do we make CI/CD possible? The answer lies in the ability to run process and to tie together bits of infrastructure together. By infrastructure, we might mean servers. When we have cloud computing, we have choices about what servers and services we use.
All this takes us to the next podcast.
Infrastructure as Code (IaC)
In SE Radio 482: Luke Hoban on Infrastructure as Code Hoban is asked a simple question: What is IaC and why does it matter (2:00)? The paraphrased answer is that IaC can describe “a desired state of the [software] environment”, where that environment is created using cloud infrastructure. An important point in the podcast is “when you move to the cloud, there is additional complexity that you need to deal with”. This software environment (or infrastructure) can also be an entire software architecture which comprises of different components that do different things (and help to satisfy your different requirements). IaC matters, since creating an infrastructure by hand introduces risk, since engineers may forget to carry out certain steps. A checklist, in some senses, becomes code.
When the challenge becomes “how to compose the connections between … thousands of elements, infrastructure becomes a software problem”. Different approaches to solve this software are mentioned. There is a declarative approach (you declare stuff in code), and an imperative approach (you specify a set of instruction in code). There are also user interfaces as well as textual languages. Taking a declarative approach, there are models that make use of formalisms (or notations) such as JSON or YAML. A scripting approach, through the use of application programming interfaces may make use of familiar programming languages, such as Python, which allow you to apply existing software engineering practices. When you start to use code to describe your infrastructure, you can then begin to use software engineering principles and tools on that code, such as version control.
Towards the end of the podcast, a question was asking about testing (52:30), which is a topic that will be discussed in a later blog. Engineers may create unit tests to check to see what elements have been created and to validate characteristics of a deployed infrastructure. Integration testing may be carried out using a ‘production like’ staging environment before everything is deployed.
Reflections
A computing lecturer once gave a short talk during a tutorial that changed how I looked at things. He said that one of the most fundamental principles in computing and computer science is the principle of abstraction. Put in other words, if a problem becomes too difficult to solve in one go, break the problem down into the essential parts that make up that problem, and work with those bits instead.
A colleague (who works in the school) once expressed the same idea in another way, which was “if you get into trouble, abstract up a level”. In the context of the discussion “getting into trouble” means everything becoming too complicated to control. The phrase “abstract up a level” means breaking the problem down into bits, and putting them into procedures or function, which you can then start to manage more easily.
Infrastructure as Code is a really good example of “abstracting up” to solve a problem that started to become too complicated.
IaC has facilitated the development of a CI/CD pipeline. Interestingly, a CI/CD pipeline can also facilitate the development of IaC.