Code re-use with Terraform Modules

Module Basics

At its most basic, A Terraform module is nothing more than a directory of Terraform scripts. Any directory containing Terraform scripts can be referenced from another Terraform directory as a module. Everything in that directory will be included in an apply command in the calling directory. The top level directory for your Terraform scripts should be considered to be the root module, and itself could be called from another Terraform script.

It is this approach that allows any subdirectories to be called in a Terraform “apply. To include a subdirectory in a deployment, it must be named as a module in the parent directory. For example this code will execute all the Terraform configurations in the “keygen” subdirectory:

module "keygen" {
  source = "keygen"
  display_name = "keygen"
  subnet_domain_name = "hello-subnet"
}

However, modules are most useful as a re-use structure that allows the instantiation of a single deployment pattern across multiple environments and applications. The local of the directory of scripts can be anywhere, and can use relative or absolute paths. Here’s an example from our cloud foundation library where we call a module held outside the current path:

module "keygen" {
  source = "../../../cloud-foundation/modules/cloud-foundation-library/keygen"
  display_name = "keygen"
  subnet_domain_name = "hello-subnet"
}

Let’s extend the example in the previous section - the deployment of a web server cluster with a variable number of servers. This is a very common pattern across most organizations - in many applications. Terraform modules allow that deployment pattern to be wrapped up and shared as a re-usable deployment.

The fundamental re-use principle here is that the module can be considered to be a “black-box” in that the user (the person deploying the module) can use the pattern without understanding how it works. As we mentioned in the previous section, building complex, flexible Terraform deployments is a really a high-value, expert skill, and so if we can capture all that expertise in a re-usable module, then the savings on time and cost can be substantial

A second key advantage of modules is that they also encapsulate best practice. Even a simple web-server cluster may encapsulate multiple best-practice elements, for example, resilience, security, and configuration. Ensuring a high level of standardization to a known best practice reduces failures, minimizes maintenance cost, and tightens security.

A final advantage is in the efficiency of managing change. Let us extend our web server cluster example and suggest introducing a new, improved load-balancer service. Implementing that change across all deployments would only require the module to be changed, not every configuration. Of course, the roll-out of that change needs managing, but the amount of code change is significantly reduced.

We will return to module version management later.

Writing Modules

In the previous section Terraform in Complex Environments, we covered the basics of making Terraform non-specific to an environment by removing hard-coded references. We also added flexibility with the use of iterators, for-loops, and conditionals. These approaches are also the basics for writing a Terraform module.

As already stated, a Terraform module is nothing more than a set of Terraform files in a folder. The one significant difference in a module is defining an expected set of input variables required to call the module. These are defined in the variables.tf file in the module directory.

When a Terraform script needs to reference a module, it includes a “module” section, and within that section, it specifies the location of the module and the required parameter values. While it is tempting to think of a module as being “called”, a more accurate view would be as an include in that, conceptually, the subdirectory content specifies some part of the graph data model that is added into the root Terraform model for the deployment.

If you are new to Terraform and trying to read a Terraform script, there is a crucial difference from programming languages that you should remember.

module "webserver" {
  source = "./webserver-cluster"
}

When you see this construct, it is not the definition of a module. It is the call to a module. In the example above, the webserver-cluster subdirectory is included in the apply at plan or apply.

Module references may refer to anywhere on the same local server via a relative or absolute file name. They may also reference remotely stored modules via a URL. The remote store may be one of a number of different storage types - e.g., object storage, GitHub, etc. Modules may also be stored in a dedicated Terraform registry - either Terraform Enterprise or an open-source implementation. Please see the Terraform documentation for more details.

Module Variables

Modules may also have defined inputs and outputs. These are defined in two files, Here’s an example of a variables.tf file that defined the input variables required for a module:

variable "region" {
  type = string
}
variable "tenancy_ocid" {}
variable "compartment_ocid" {}

… and here’s a short example of an outputs.tf file :

output "JDK_Version" {
  value = local.jdk_version
}

Finally, values for variables can be provided in a number of ways - e.g. via shell variables, but the most common is through the use of a terraform.tfvars file.

region="eu-frankfurt-1"
compartment_ocid="ocid1.compartment.oc1..aLongStringofAlphanumerics"

Module Versions

When a module is used across multiple deployments and applications, the efficiencies and improvements can be very significant indeed. However, there’s also a downside - commonly known a the “blast radius”.

A mistake in a simple, single deployment Terraform script may damage a single environment; a mistake, bug or error in a widely-used Terraform module could (if naively implemented) take down a substantial proportion of your business. In a later section, we will cover how good governance and security can reduce the “blast-radius,” but here, we will look at module versioning.

We recommend that strong versioning of Terraform modules is put in place before any widespread re-use of modules. When used, a Terraform module forms a critical part of the description of your IT estate and and changes must be well managed across the estate. Two steps are required to enforce this :

Terraform has a variable constraint capability that allows a calling script to specify a specific version or a range of acceptable versions. A specific version should be specified for critical infrastructure that will require active change management. For infrastructure that you wish to automatically update if a module is changed, then a range should be used. A commonly used approach is to limit automatically accepted changes to low-risk changes that maintain forward compatibility. Use the minor version numbers (e.g. 1.2 -> 1.3) to define forward-compatible changes, and use a major version number change (e.g. 2.6 -> 3.0) for changes that may impact on previous deployments.

However, these version controls are only available if you are using a Terraform module repository. In any other case, you will need to embed the version in the

The use of third-party hosted module repositories is a tricky subject. While they are often recommended, their use introduces an additional external dependency to your cloud deployment with all the security implications that might incur. Our recommendation at this time would be for any company using a third-party module repository to take copies of the module at initial introduction to your organisation, and to, in effect, create an internal branch of that module. You will not get automatic updates, but it is the more secure approach.

More information on options to store Terrafrom modules can be found In the Terraform documentstion : Module Sources