Things to Watch Out for When Deploying AWS Infrastructure with Terraform

Introduction:

Terraform is a tool for Infrastructure as Code (IaC) made by HashiCorp, which helps in making process of creating and managing cloud resources more simple, including AWS infrastructure. When using Terraform configurations, you can automate and keep versions of your infrastructure. Here is a list of some tips and things to watch out for when using Terraform to make AWS resources

  1. Utilize Terraform Workspaces for Keeping Environments Separate

When making AWS infrastructure for different environments (for example, development, staging, and production), it is very important to keep separation and not have conflicts. Terraform workspaces give an easy way to manage multiple environments with same configuration files:

  • Make new workspace for each environment, keeping their states separate.
  • Use variables and backend configurations specific to workspace to customize resource settings for each environment.
  1. Manage State Files with Security and Collaboration

Terraform’s state files store current state of your infrastructure and are very important for tracking changes. Making sure security and integrity of state files is very important, especially when working in a team:

  • Use remote backends like Amazon S3 to store state files with security and allow team to work together.
  • Enable state file versioning and use AWS KMS for encryption to make security better.
  • Use Terraform locking mechanisms, like DynamoDB for the S3 backend, to stop concurrent modifications and not damage state files.
  1. Organize Your Terraform Code with Use of Modules

When your Terraform configurations become big, it is important to organize your code for reusability, being easy to maintain, and having consistency. Modules are effective way for this:

  • Create reusable modules for common infrastructure parts, like VPCs, subnets, and security groups.
  • Use version control and central repository for managing your modules.
  1. Plan with Care and Use Lifecycle Rules for Stopping Accidental Resource Deletion

Terraform’s ability to manage resources can also have risk of deleting important infrastructure by mistake. To not have unintended results, follow these practices:

  • Always run terraform plan before terraform apply to see changes that will be made to your infrastructure.
  • Use lifecycle configuration block with prevent_destroy attribute set to true for important resources, making sure Terraform doesn’t delete them by mistake.
  • Take advantage of resource import function to bring existing infrastructure under Terraform management without accidental deletions.
  1. Keep Secrets Safe with AWS Secrets Manager and SSM Parameter Store

Handling secret data, like API keys and credentials, is important part of deploying AWS infrastructure. To keep secrets safe, use AWS Secrets Manager or SSM Parameter Store:

  • Store secret data in AWS Secrets Manager or SSM Parameter Store, keeping them out of your Terraform configuration files.
  • Use Terraform resources (aws_secretsmanager_secret_version, aws_ssm_parameter) or data sources (aws_secretsmanager_secret, aws_ssm_parameter) that are related to get and use these secrets in your configurations.

Conclusion:

Use terraform!!, use cloudformation, use IaC tool!!!

GOLANG CHEATSHEET

1.1 GOLANG CONVERSION

Name Comment
Convert string to int i, _ := strconv.ParseInt(“12345”, 10, 64)
Convert string to int i, err := strconv.Atoi(“-42”)
Convert string to list L := strings.Split(“hi,golang”, “”)
Convert string to []byte []byte("abcXX")
Convert string to byte byte(str1[])
Convert byte to string string(byte('a'))=
Convert string to float32 f, _ := strconv.ParseFloat(“3.1415”, 32)
Convert int to float32 0.5*float32(age)+7>= float32(age2)
Convert int to string s := strconv.Itoa(-42). Notice: not string(-42)
Convert rune to string string(rune1)
Convert string list to string strings.Join(list, ", ")
Convert int list to string fmt.Sprintf("%s", l)
Convert list to byte byteI := byte(65)
Convert byte to int int(byte('a'))
Convert byte to string string(byte('a'))
Convert bytes to string string([]byte("abcXX"))
Convert int32 to int32 Pointer func int32Ptr(i int32) *int32 { return &i }
Convert string[] to string strings.Join([]string{"a", "b"}, ",")
Format string fmt.Sprintf("%.3f", float64(v)/1000)
Format string fmt.Sprintf("At %v, %s", e.When, e.What)
Format string fmt.Printf("int: %d, float: %f, bool: %t\n", 123, 78.9, true)

1.2 DEEP DIVE INTO GOLANG

  • Go’s original target was networked system infrastructure, what we now call cloud software
Name Comment
Golang goroutine
How Golang implement defer Execute a piece of code before a function returns with FILO mechanism
How Golang implement timer
Golang for vs loop
Garbage Colection
Golang CSP vs Erlang Actor Model Each actor model, which actor has its own queue
How Golang channel feature is implement? YouTube: GopherCon 2017: Understanding Channels, Link: Golang Channel
Golang return a tuple func dfs(root *TreeNode, max *float64) (sum int, cnt int), LeetCode: Maximum Average Subtree
Use strings.Builder, instead of string LeetCode: Unique Email Addresses
Variable Conversion float64(x_int/y_int) != float64(x_int)/float64(y_int), LeetCode: Maximum Average Subtree
For a list of objects, pass by value or reference f(l []*TreeNode) vs f(l *[]*TreeNode), LeetCode: Lowest Common Ancestor of a Binary Tree

1.3 GOLANG QUALITY IMPROVEMENT TOOLS

Name Comment
gosec Golang security checker, gosec ./...
golangci-lint lint check, golint
errcheck errcheck checks that you checked errors, errcheck ./...
delve Delve is a debugger for the Go programming language.
ginkgo BDD Testing Framework for Go
mock GoMock is a mocking framework for Golang
envtest provides libraries for integration testing by starting a local control plane
go-junit-report Convert go test output to junit xml
gocover-cobertura go tool cover to XML (Cobertura) export tool
gocovmerge Merge coverprofile results from multiple go cover runs

1.4 GOLANG DEPENDENCIES

Name Comment
goimports updates your Go import lines, adding missing ones and removing unreferenced ones
dep Go deps, now recommend go modules
Install go dep go get -u github.com/golang/dep/cmd/dep
Go modules export GO111MODULE=on, go mod download
Initialize new module in current directory go mod init XXX
Download modules to local cache go mod download
Add missing and remove unused modules go mod tidy
Make vendored copy of dependencies go mod vendor
Reference Link: Using Go Modules

1.5 GOLANG ERRORS

Name Comment
… does not support indexing *variable[0] -> (*variable)[0]

1.6 GOLANG COMMON

Name Comment
Upgrade golang to 1.12 in mac brew upgrade go, go version
Reference Link: The Go Programming Language Specification

1.7 GOLANG CODE STRUCTURE & COMMON ALGORITHMS

Name Comment
Online Go Playgroud https://play.golang.org/
One line if statement if a >= 1 { fmt.Print(“yes”) }
Declare variables with initializers var ischecked, v, str = false, 2, “yes!”
goroutine Define functions to run as distince subprocesses
switch code/example-switch.go
queue LeetCode: Number of Recent Calls
bfs code/tree-bfs.go
trie tree code/tree-trie.go

1.8 SYNTAX SUGAR: FROM PYTHON TO GOLANG

Name Python Golang
sum slice sum([1, 2, 3]) sum := 0; for i := range nums { sum += nums[i] }
Get last item nums[-1] nums[len(nums)-1]
For for i in range(10): for i := 0; i < 10; i++
Loop list for num in [1, 2] for num := range[]int{1, 2} { fmt.Print(num) }
Loop string for ch in str: for _, ch := range str { fmt.Print(ch) }
Iterator for num in nums: for _, num := range nums {fmt.Print(num)}
While while isOK: for isOK
Check ch range ord(ch) in range(ord('a'), ord('z')+1) ch >=’a’ && ch <=’z’
Get min min(2, 6, 5)
Check is nil root is None root == nil
Reverse list nums[::-1] Need to create your own function. Weird!

1.9 SURPRISES IN GOLANG

Name Comment
Modulus returns negative numbers In golang, -3 % 2 == -1

1.10 GOLANG ARRAY/LIST/SLICE

Name Comment
Make a array var a [2]string; a[0]=”hello”; a[1]=”world”
Create array with given values l := [6]int{2, 3, 7, 5, 11, 13}
Create array with given values l := []string{“a”, “c”, “b”, “d”}
Create dynamically-sized arrays a := make([]int, 5)
Create dynamically-sized arrays a := make([]int, 1, 5) // 5 is capacity
Sort string array sort.Strings(l); fmt.Print(l)
Sort int array sort.Ints(l) //in-place change
Golang sort one array by another array LeetCode: Largest Values From Labels
Sort in descending order sort.Sort(sort.Reverse(sort.IntSlice(keys)))
Append item l = append(l, “e”)
Append items l = append(l, “e”, “b”, “c”)
Append item to head/prepend l = append([]string{"a"}, l...)
Remove last item l = l[:len(l)-1]
Remove item by index l = append(l[0:1], l[2:]...)
Slices of a array var l2 = l[1:3] // Notice: it’s a reference
Copy a list b := make([]int, len(a)); copy(b, a)
Join two lists l1 = append(l1, l2...)
Use pointer of array list code/pointer-array.go

1.11 GOLANG STRING

Name Comment
Format string fmt.Sprintf("At %v, %s", e.When, e.What)
Format string fmt.Printf("int: %d, float: %f, bool: %t\n", 123, 78.9, true)
Padding zero fmt.Printf("%02d:%02d", 2, 10)
Split string var L = strings.Split("hi,golang", ",")
Replace string var str2 = strings.Replace("hi,all", ",", ";", -1)
Replace string strings.Replace("aaaa", "a", "b", 2) //bbaa
Split string by separator strings.Split(path, " ")
Count characters strings.Count("test", "t")
Substring strings.Index("test", "e")
Join string strings.Join([]string{"a","b"}, "-")
Repeat string strings.Repeat("a", 2) // aa
Lower string strings.ToLower("TEST")
Trim whitespace in two sides strings.TrimSpace("\t Hello world!\n ")
Trim trailing whitespace strings.TrimRight("\t Hello world!\n ", "\n ")
Concact string fmt.Sprintf("%s%s", str1, str2)
Reference Link: package strings

1.12 GOLANG INTEGER/FLOAT

Name Comment
Int max MaxInt32 = 1<<31 – 1 golang math
Int min MinInt32 = -1 << 31 golang math
Pass int as reference sample code

1.13 GOLANG ENV

Name Comment
GOPATH It is called as the workspace directory for Go programs
GOROOT The location of your Go installation. No need to set any more
go env Show a full list of environment variables
Reference Link: GOPATH, GOROOT, GOBIN

1.14 GOLANG PACKAGE MANAGEMENT

Name Comment
go mod Link: go modules
go get fix GO111MODULE=off go get -fix ./...
go mod replace url go mod edit -replace…

1.15 GOLANG ASCII

Name Comment
get character ascii byte('0')
ascii offset fmt.Println(string('B' + byte('a')-byte('A')))

1.16 GOLANG DICT/HASHMAP/MAP

Name Comment
Create dict map[string]int{"a": 1, "b": 2}
Create dict make(map[string]int)
Check existence _, ok := m[k]
Delete key delete(m, "k1")
Create a map of lists m := make(map[string][]string)
Get keys of a map Loop the dictionary and append the key to a list
Use (x, y) as hashmap key m[[2]int{2, 2}] = true, code/example-hashmap-arraykey.go

1.17 GOLANG NETWORKING

Name Comment
golang http code/example-http.go

1.18 GOLANG GOROUTINES

Name Comment
Basic goroutine code/example-goroutine.go

1.19 GOLANG INTEFACE

Name Comment
Hash map with both key and value dynamic map[interface{}]interface{}
Define and use interface code/example-interface.go
Convert map[interface {}]interface {} to map[string]string code/interface-conversion.go

1.20 GOLANG FILES & FOLDERS

Name Comment
Read files code/example-read-file.go
Write files code/example-write-file.go

1.21 GOLANG MATH

Name Comment
pow(2, 3) int(math.Pow(2, 3)) // Default is float64
sqrt math.Sqrt(100)
Get rand rand.Intn(100), rand.Float64()

1.22 GOLANG BIT OPERATOR & MATH

Name Comment
Shift left fmt.Print(1 << 10) // 1024
Shift right fmt.Print(1024 >> 3) // 128

1.23 GOLANG BBD TESTING

Name Summary
ginkgo BDD Testing Framework for Go http://onsi.github.io/ginkgo/
Ubuntu install ginkgo apt-get install golang-ginkgo-dev
gomega Ginkgo’s Preferred Matcher Library
Add tags to tests // +build availability, go test -v --tags=availability ./test/e2e/...

1.24 GOLANG MISC

Name Comment
Golang sleep time.Sleep(4* time.Second)
Golang logging import "log", log.Info, log.Print, log.Error(err)
Golang print function name runtime.Callers