Terraform

Ce projet a pour but de déployer une VM sur le cloud afin d'y installer l'application demofastapi:simple construite ici.

La VM sera déployée sur AWS, mais tous les plans terraform pourront être utilisés avec un plan gratuit AWS.

Avant d'executer les exemples, il est nécessaire de:

• Installer terraform et terragrunt sur votre machine
• Génerer une access key et une secret key sur votre compte AWS
• Remplir le fichier env avec ces clés.

Ces exemples ont pour but de montrer plusieurs façons d'organiser du code terraform, mais pas d'expliquer la syntaxe terraform pour autant.

1. Plan simple

La façon la plus simple d'utiliser Terraform est de créer un fichier plan.tf contenant à la fois les informations sur le provider terraform et la description de l'infrastructure.

Un tel fichier peut être trouvé dans 1_simple/plan.tf.

Terraform peut ensuite être utilisé sur ce fichier:

# Load AWS access and secret keys
source env
cd 1_simple
# Initialize project and install required providers
terraform init
# Previsualize actions required
terraform plan
# Actually execute actions (press yes when asked for confirmation)
terraform apply
# List deployed objects
terraform state list
# Print info on a specific object
terraform state show aws_instance.app_server
# Destroy infrastructure (press yes when asked for confirmation)
terraform destroy

Résultat des commandes terraform

➜ terraform init
Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 6.0"...
- Installing hashicorp/aws v6.16.0...
- Installed hashicorp/aws v6.16.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

➜ terraform plan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_instance.app_server will be created
  + resource "aws_instance" "app_server" {
      + ami                                  = "ami-04b5bfc32945e6d28"
      + arn                                  = (known after apply)
      + associate_public_ip_address          = (known after apply)
      + availability_zone                    = (known after apply)
      + disable_api_stop                     = (known after apply)
      + disable_api_termination              = (known after apply)
      + ebs_optimized                        = (known after apply)
      + enable_primary_ipv6                  = (known after apply)
      + force_destroy                        = false
      + get_password_data                    = false
      + host_id                              = (known after apply)
      + host_resource_group_arn              = (known after apply)
      + iam_instance_profile                 = (known after apply)
      + id                                   = (known after apply)
      + instance_initiated_shutdown_behavior = (known after apply)
      + instance_lifecycle                   = (known after apply)
      + instance_state                       = (known after apply)
      + instance_type                        = "t3.small"
      + ipv6_address_count                   = (known after apply)
      + ipv6_addresses                       = (known after apply)
      + key_name                             = (known after apply)
      + monitoring                           = (known after apply)
      + outpost_arn                          = (known after apply)
      + password_data                        = (known after apply)
      + placement_group                      = (known after apply)
      + placement_group_id                   = (known after apply)
      + placement_partition_number           = (known after apply)
      + primary_network_interface_id         = (known after apply)
      + private_dns                          = (known after apply)
      + private_ip                           = (known after apply)
      + public_dns                           = (known after apply)
      + public_ip                            = (known after apply)
      + region                               = "eu-west-3"
      + secondary_private_ips                = (known after apply)
      + security_groups                      = (known after apply)
      + source_dest_check                    = true
      + spot_instance_request_id             = (known after apply)
      + subnet_id                            = (known after apply)
      + tags                                 = {
          + "Name" = "demofastapi"
        }
      + tags_all                             = {
          + "Name" = "demofastapi"
        }
      + tenancy                              = (known after apply)
      + user_data_base64                     = (known after apply)
      + user_data_replace_on_change          = false
      + vpc_security_group_ids               = (known after apply)

      + capacity_reservation_specification (known after apply)

      + cpu_options (known after apply)

      + ebs_block_device (known after apply)

      + enclave_options (known after apply)

      + ephemeral_block_device (known after apply)

      + instance_market_options (known after apply)

      + maintenance_options (known after apply)

      + metadata_options (known after apply)

      + network_interface (known after apply)

      + primary_network_interface (known after apply)

      + private_dns_name_options (known after apply)

      + root_block_device (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.

➜ terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_instance.app_server will be created
  + resource "aws_instance" "app_server" {
      + ami                                  = "ami-04b5bfc32945e6d28"
      + arn                                  = (known after apply)
      + associate_public_ip_address          = (known after apply)
      + availability_zone                    = (known after apply)
      + disable_api_stop                     = (known after apply)
      + disable_api_termination              = (known after apply)
      + ebs_optimized                        = (known after apply)
      + enable_primary_ipv6                  = (known after apply)
      + force_destroy                        = false
      + get_password_data                    = false
      + host_id                              = (known after apply)
      + host_resource_group_arn              = (known after apply)
      + iam_instance_profile                 = (known after apply)
      + id                                   = (known after apply)
      + instance_initiated_shutdown_behavior = (known after apply)
      + instance_lifecycle                   = (known after apply)
      + instance_state                       = (known after apply)
      + instance_type                        = "t3.small"
      + ipv6_address_count                   = (known after apply)
      + ipv6_addresses                       = (known after apply)
      + key_name                             = (known after apply)
      + monitoring                           = (known after apply)
      + outpost_arn                          = (known after apply)
      + password_data                        = (known after apply)
      + placement_group                      = (known after apply)
      + placement_group_id                   = (known after apply)
      + placement_partition_number           = (known after apply)
      + primary_network_interface_id         = (known after apply)
      + private_dns                          = (known after apply)
      + private_ip                           = (known after apply)
      + public_dns                           = (known after apply)
      + public_ip                            = (known after apply)
      + region                               = "eu-west-3"
      + secondary_private_ips                = (known after apply)
      + security_groups                      = (known after apply)
      + source_dest_check                    = true
      + spot_instance_request_id             = (known after apply)
      + subnet_id                            = (known after apply)
      + tags                                 = {
          + "Name" = "demofastapi"
        }
      + tags_all                             = {
          + "Name" = "demofastapi"
        }
      + tenancy                              = (known after apply)
      + user_data_base64                     = (known after apply)
      + user_data_replace_on_change          = false
      + vpc_security_group_ids               = (known after apply)

      + capacity_reservation_specification (known after apply)

      + cpu_options (known after apply)

      + ebs_block_device (known after apply)

      + enclave_options (known after apply)

      + ephemeral_block_device (known after apply)

      + instance_market_options (known after apply)

      + maintenance_options (known after apply)

      + metadata_options (known after apply)

      + network_interface (known after apply)

      + primary_network_interface (known after apply)

      + private_dns_name_options (known after apply)

      + root_block_device (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_instance.app_server: Creating...
aws_instance.app_server: Still creating... [00m10s elapsed]
aws_instance.app_server: Creation complete after 13s [id=i-0934d1f4a862fde94]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

➜ terraform state list
aws_instance.app_server

➜ terraform state show aws_instance.app_server
# aws_instance.app_server:
resource "aws_instance" "app_server" {
    ami                                  = "ami-04b5bfc32945e6d28"
    arn                                  = "arn:aws:ec2:eu-west-3:683918607876:instance/i-0934d1f4a862fde94"
    associate_public_ip_address          = true
    availability_zone                    = "eu-west-3c"
    disable_api_stop                     = false
    disable_api_termination              = false
    ebs_optimized                        = false
    force_destroy                        = false
    get_password_data                    = false
    hibernation                          = false
    host_id                              = null
    iam_instance_profile                 = null
    id                                   = "i-0934d1f4a862fde94"
    instance_initiated_shutdown_behavior = "stop"
    instance_lifecycle                   = null
    instance_state                       = "running"
    instance_type                        = "t3.small"
    ipv6_address_count                   = 0
    ipv6_addresses                       = []
    key_name                             = null
    monitoring                           = false
    outpost_arn                          = null
    password_data                        = null
    placement_group                      = null
    placement_group_id                   = null
    placement_partition_number           = 0
    primary_network_interface_id         = "eni-0723dc008c408bcf1"
    private_dns                          = "ip-172-31-38-241.eu-west-3.compute.internal"
    private_ip                           = "172.31.38.241"
    public_dns                           = "ec2-51-44-17-153.eu-west-3.compute.amazonaws.com"
    public_ip                            = "51.44.17.153"
    region                               = "eu-west-3"
    secondary_private_ips                = []
    security_groups                      = [
        "default",
    ]
    source_dest_check                    = true
    spot_instance_request_id             = null
    subnet_id                            = "subnet-0924043b35bc8bf28"
    tags                                 = {
        "Name" = "demofastapi"
    }
    tags_all                             = {
        "Name" = "demofastapi"
    }
    tenancy                              = "default"
    user_data_replace_on_change          = false
    vpc_security_group_ids               = [
        "sg-075a43d9922e4cfb0",
    ]

    capacity_reservation_specification {
        capacity_reservation_preference = "open"
    }

    cpu_options {
        amd_sev_snp      = null
        core_count       = 1
        threads_per_core = 2
    }

    credit_specification {
        cpu_credits = "unlimited"
    }

    enclave_options {
        enabled = false
    }

    maintenance_options {
        auto_recovery = "default"
    }

    metadata_options {
        http_endpoint               = "enabled"
        http_protocol_ipv6          = "disabled"
        http_put_response_hop_limit = 2
        http_tokens                 = "required"
        instance_metadata_tags      = "disabled"
    }

    primary_network_interface {
        delete_on_termination = true
        network_interface_id  = "eni-0723dc008c408bcf1"
    }

    private_dns_name_options {
        enable_resource_name_dns_a_record    = false
        enable_resource_name_dns_aaaa_record = false
        hostname_type                        = "ip-name"
    }

    root_block_device {
        delete_on_termination = true
        device_name           = "/dev/sda1"
        encrypted             = false
        iops                  = 3000
        kms_key_id            = null
        tags                  = {}
        tags_all              = {}
        throughput            = 125
        volume_id             = "vol-0c7e98245354de201"
        volume_size           = 8
        volume_type           = "gp3"
    }
}

➜ terraform destroy
aws_instance.app_server: Refreshing state... [id=i-0934d1f4a862fde94]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  # aws_instance.app_server will be destroyed
  - resource "aws_instance" "app_server" {
      - ami                                  = "ami-04b5bfc32945e6d28" -> null
      - arn                                  = "arn:aws:ec2:eu-west-3:683918607876:instance/i-0934d1f4a862fde94" -> null
      - associate_public_ip_address          = true -> null
      - availability_zone                    = "eu-west-3c" -> null
      - disable_api_stop                     = false -> null
      - disable_api_termination              = false -> null
      - ebs_optimized                        = false -> null
      - force_destroy                        = false -> null
      - get_password_data                    = false -> null
      - hibernation                          = false -> null
      - id                                   = "i-0934d1f4a862fde94" -> null
      - instance_initiated_shutdown_behavior = "stop" -> null
      - instance_state                       = "running" -> null
      - instance_type                        = "t3.small" -> null
      - ipv6_address_count                   = 0 -> null
      - ipv6_addresses                       = [] -> null
      - monitoring                           = false -> null
      - placement_partition_number           = 0 -> null
      - primary_network_interface_id         = "eni-0723dc008c408bcf1" -> null
      - private_dns                          = "ip-172-31-38-241.eu-west-3.compute.internal" -> null
      - private_ip                           = "172.31.38.241" -> null
      - public_dns                           = "ec2-51-44-17-153.eu-west-3.compute.amazonaws.com" -> null
      - public_ip                            = "51.44.17.153" -> null
      - region                               = "eu-west-3" -> null
      - secondary_private_ips                = [] -> null
      - security_groups                      = [
          - "default",
        ] -> null
      - source_dest_check                    = true -> null
      - subnet_id                            = "subnet-0924043b35bc8bf28" -> null
      - tags                                 = {
          - "Name" = "demofastapi"
        } -> null
      - tags_all                             = {
          - "Name" = "demofastapi"
        } -> null
      - tenancy                              = "default" -> null
      - user_data_replace_on_change          = false -> null
      - vpc_security_group_ids               = [
          - "sg-075a43d9922e4cfb0",
        ] -> null
        # (9 unchanged attributes hidden)

      - capacity_reservation_specification {
          - capacity_reservation_preference = "open" -> null
        }

      - cpu_options {
          - core_count       = 1 -> null
          - threads_per_core = 2 -> null
            # (1 unchanged attribute hidden)
        }

      - credit_specification {
          - cpu_credits = "unlimited" -> null
        }

      - enclave_options {
          - enabled = false -> null
        }

      - maintenance_options {
          - auto_recovery = "default" -> null
        }

      - metadata_options {
          - http_endpoint               = "enabled" -> null
          - http_protocol_ipv6          = "disabled" -> null
          - http_put_response_hop_limit = 2 -> null
          - http_tokens                 = "required" -> null
          - instance_metadata_tags      = "disabled" -> null
        }

      - primary_network_interface {
          - delete_on_termination = true -> null
          - network_interface_id  = "eni-0723dc008c408bcf1" -> null
        }

      - private_dns_name_options {
          - enable_resource_name_dns_a_record    = false -> null
          - enable_resource_name_dns_aaaa_record = false -> null
          - hostname_type                        = "ip-name" -> null
        }

      - root_block_device {
          - delete_on_termination = true -> null
          - device_name           = "/dev/sda1" -> null
          - encrypted             = false -> null
          - iops                  = 3000 -> null
          - tags                  = {} -> null
          - tags_all              = {} -> null
          - throughput            = 125 -> null
          - volume_id             = "vol-0c7e98245354de201" -> null
          - volume_size           = 8 -> null
          - volume_type           = "gp3" -> null
            # (1 unchanged attribute hidden)
        }
    }

Plan: 0 to add, 0 to change, 1 to destroy.

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

aws_instance.app_server: Destroying... [id=i-0934d1f4a862fde94]
aws_instance.app_server: Still destroying... [id=i-0934d1f4a862fde94, 00m10s elapsed]
aws_instance.app_server: Still destroying... [id=i-0934d1f4a862fde94, 00m20s elapsed]
aws_instance.app_server: Destruction complete after 30s

Destroy complete! Resources: 1 destroyed.

Une fois le plan appliqué, la VM peut être visible dans l'interface web AWS :

2. Plan complet

Le plan de l'exemple précédent n'est pas suffisant pour pouvoir se connecter à la machine distante. Comme souvent sur le cloud, il va falloir créer plusieurs objets et les faire fonctionner ensemble pour obtenir une infra cohérente.

En plus de l'objet VM (aws_instance), il faut créer un objet clé SSH (aws_key_pair) et configurer le VPC par défaut (aws_default_vpc, qui correspond à un réseau virtuel) et le security group par défaut (aws_default_security_group) pour autoriser les accès SSH.

Dans l'exemple précédent, nous avons mis toutes les informations terraform dans un unique fichier, mais cela peut devenir compliqué à maintenir pour de gros plans. Ici, nous séparons les différents objets dans différents fichiers. Cela n'a pas d'incidence pour terraform qui dans tous les cas concatènera tous ces fichiers avant d'executer des commandes.

Avant d'executer ce plan, nous avons besoin de créer la clé SSH qui est utilisée dans le fichier 2_complet/compute.tf :

cd 2_complet/data
# Creation of a SSH key with **no password**, do not use in production
ssh-keygen -t rsa -N "" -C "" -f demofastapi

Le plan peut ensuite être executé comme précédement

source env
cd 2_complet
terraform init
terraform apply

Une fois la VM créée, on peut récupérer son adresse IP ainsi :

source env
cd 2_complet
terraform state show aws_instance.app_server | grep public_ip

Puis se connecter à la VM via :

cd 2_complet
ssh -i ./data/demofastapi ubuntu@<adresse_ip>

Le nom de l'utilisateur dépend de l'image (AMI) utilisée, ubuntu dans ce cas.

Pour simplifier la récupération de l'adresse IP de la machine, on peut rajouter un fichier 2_complet/output.tf avec le contenu suivant :

output "ssh_ip_address" {
  value = aws_instance.app_server.public_ip
}

Puis ré-exécuter un terraform apply, ce qui nous affichera l'adresse IP publique de notre instance à chaque execution

➜ terraform apply
aws_key_pair.ssh_access: Refreshing state... [id=demofastapi]
aws_default_vpc.default: Refreshing state... [id=vpc-04dfe48f26a3d83e8]
aws_instance.app_server: Refreshing state... [id=i-09b97a63bc3ceed35]
aws_default_security_group.default: Refreshing state... [id=sg-075a43d9922e4cfb0]

Changes to Outputs:
  + ssh_ip_address = "13.36.209.226"

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes


Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

ssh_ip_address = "13.36.209.226"

Il est possible d'afficher les outputs par la suite en executant :

source env
cd 2_complet
terraform output

Puis de supprimer cette infra via :

source env
cd 2_complet
terraform destroy

3. Module

En pratique, on voudra déployer une même infra plusieurs fois, par exemple pour avoir une instance de lab et une de production. Avec le plan précédent, cela pourrait être fait de deux manières :

• Ajouter les machines de lab dans le plan, au risque d'impacter la production en faisant des modifications sur le lab ;
• Dupliquer le plan pour créer un plan de lab, ce qui aura un impact sur la maintenabilité du projet.

Une autre solution serait de mettre la logique de déploiement dans un module, et d'avoir deux plans (lab et prod) qui feraient appels à ce module.

Dans ce but, un module correspondant à l'infra précédente a été créé ici, et le dossier 3_module contient un plan faisant appel à ce module. Il y a deux différences majeures par rapport au plan précédent :

• Le fichier main.tf a disparu, la configuration des providers étant faite dans le plan, et non dans le module ;
• Un fichier variables.tf a été créé pour décrire les variables d'entrée du module.

Comme pour un plan terraform, les différents fichiers du module pourraient être regroupés dans un seul. Terraform les concaténant dans tous les cas avant execution.

Comme précédemment, il faut d'abord créer les clés SSH :

cd 3_module/data
# Creation of a SSH key with **no password**, do not use in production
ssh-keygen -t rsa -N "" -C "" -f demofastapi

Puis le plan peut être déployé ainsi :

source env
cd 3_module
terraform init
terraform apply

La VM peut être accédée ainsi :

source env
cd 3_module
terraform output
ssh -i ./data/demofastapi ubuntu@<adresse_ip>

4. Ansible

Maintenant que nous avons une vraie machine dans le cloud, nous pouvons reprendre le dernier inventaire ansible et y ajouter notre inventaire de machines. Cet inventaire se trouve dans le dossier 4_ansible/inventory. Un premier fichier hosts.yml contient la liste de machines composant notre infrastructure triées dans des groupes (ici une machine instance dans le groupe aws). Le dossier 4_ansible/inventory/group_vars/aws contient des variables partagées par toutes les machines du groupe aws. Le dossier 4_ansible/inventory/host_vars/instance contient les variables spécifiques à la machine instance.

Pour pouvoir executer les playbooks, il faudra dorénavant spécifier où se trouve l'inventaire, mais avant, il faut inscrire l'IP de votre machine (déployée dans la section 3) dans le fichier 4_ansible/inventory/host_vars/instance/main.yml.

Puis :

cd 4_ansible
ansible-galaxy install -r requirements.yml
# Install docker on the machine
ansible-playbook -i inventory playbooks/install_docker.yml

Comme l'image demofastapi:simple n'existe que sur notre machine, il faut d'abord l'envoyer sur la machine distante. En production, il faudrait passer par un dépôt docker.

cd 3_module
# Save the docker image as an archive
docker image save demofastapi:simple -o image.tar.gz
# Send the image to the distant server
scp -i ./data/demofastapi image.tar.gz ubuntu@<ADRESSE_IP>:image.tar.gz
# Load image
ssh -i ./data/demofastapi ubuntu@<ADRESSE_IP> docker load -i image.tar.gz

Enfin, le service peut être déployé :

# Deploy the service
ansible-playbook -i inventory playbooks/deploy.yml

Et le service peut être accédé à http://<ADRESSE_IP>:8004