diff --git a/MIGRATION-2.0.md b/MIGRATION-2.0.md new file mode 100644 index 0000000..ebd6a01 --- /dev/null +++ b/MIGRATION-2.0.md @@ -0,0 +1,256 @@ +# Migrating to version 2.0 + +Version 2.0 adds EKS Auto Mode to `terraform-aws-eks` while retaining the existing managed-node-group behavior. The `auto_mode` input defaults to `null`, so upgrading an existing standard EKS deployment does not enable Auto Mode implicitly. + +The Kubernetes API endpoint remains public and private by default. The public endpoint allows `0.0.0.0/0` by default; production configurations should set `public_access_cidrs` explicitly. + +This guide covers: + +- upgrading a standard cluster from `terraform-aws-eks` v1.x; +- enabling Auto Mode on an existing standard cluster. + +## Before upgrading + +1. Upgrade the current configuration to its latest applicable v1.x release and apply it so that the starting plan is clean. +2. Check that the root configuration can use AWS provider `>= 6.59.0`. +3. Back up the Terraform state and confirm that the backup can be accessed by the team responsible for recovery. +4. Test the migration against a non-production cluster first. +5. Record the current EKS endpoint, authentication mode, `bootstrap_cluster_creator_admin_permissions` value, IP family, service CIDR, node groups, add-ons, access entries, Pod Identity associations, storage classes, and load balancers. + +For a local state file, a backup can be captured with: + +```bash +terraform state pull > terraform-state-before-eks-v2.json +``` + +Store state backups according to your security policy because state can contain sensitive values. Do not commit the backup. + +## 1. Upgrade `terraform-aws-eks` from v1.x to v2.x + +Change only the module version first. Keep the module block label unchanged: + +```hcl +module "eks" { + source = "" + version = "2.0.0" + + name = "production" + subnet_ids = var.eks_subnet_ids + + # Optional: null is the v2 default and leaves Auto Mode unmanaged. + auto_mode = null + + # Make the existing public endpoint behavior explicit during the upgrade. + endpoint_private_access = true + endpoint_public_access = true + public_access_cidrs = var.eks_public_access_cidrs +} +``` + +Then initialize and inspect the plan: + +```bash +terraform init -upgrade +terraform validate +terraform plan +``` + +No state moves are expected when the module block label is unchanged. Version 2.0 preserves the standard cluster, cluster IAM role, managed-node IAM role, managed node group, add-on, log group, capability, and Karpenter resource addresses used by v1.x. + +The following standard-cluster defaults also remain unchanged: + +- `auto_mode = null`, which does not enable or disable Auto Mode; +- `bootstrap_self_managed_addons = null`, which resolves to `true` when Auto Mode is not configured; +- a `default` EKS managed node group when `node_groups` is omitted; +- the `coredns`, `kube-proxy`, and `vpc-cni` EKS add-ons when `addons` is omitted; and +- public and private Kubernetes API access, with `public_access_cidrs = ["0.0.0.0/0"]`. + +Stop and investigate if a normal v1.x-to-v2.0 plan proposes replacing the EKS cluster or existing managed node groups. + +### Immutable network settings + +Treat `ip_family` and `service_ipv4_cidr` as creation-time settings. The AWS provider marks both attributes `ForceNew`, so any planned difference forces replacement of `aws_eks_cluster.this`. Do not use the v2.0 upgrade as an opportunity to change them unless replacing the cluster is intentional. + +Auto Mode supports IPv6, but `service_ipv4_cidr` is incompatible with `ip_family = "ipv6"`. For IPv6, omit `service_ipv4_cidr`; EKS assigns the IPv6 service range. EKS does not support changing a cluster's IP family after creation. + +### Renaming the module block + +Renaming a module block changes every Terraform address below it. Avoid the rename during the version upgrade when possible. If a rename is required, move the module state before planning. For example: + +```bash +terraform state mv 'module.eks_old' 'module.eks' +``` + +Adjust both addresses when the module uses `count`, `for_each`, or is nested under another module. Run `terraform state list` before and after the move, then confirm the plan contains no unintended destroys. + +## 2. Enable Auto Mode on an existing standard cluster + +Enable Auto Mode separately from the v2.0 version upgrade so each plan has one clear purpose. + +### Phase 1: enable access-entry authentication + +Auto Mode requires an API-capable authentication mode. Before adding `auto_mode`, inspect the existing cluster's `access_config` in Terraform state or with `aws eks describe-cluster`. Record the current `bootstrap_cluster_creator_admin_permissions` value; changing this creation-time value can force replacement of the cluster. + +If the cluster currently uses `CONFIG_MAP`, make `API_AND_CONFIG_MAP` the only functional change in a separate first apply. Explicitly preserve the cluster's current bootstrap creator permission. For example, when its current value is `true`: + +```hcl +module "eks" { + source = "" + version = "2.0.0" + + name = "production" + subnet_ids = var.eks_subnet_ids + + auto_mode = null + + access_config = { + authentication_mode = "API_AND_CONFIG_MAP" + bootstrap_cluster_creator_admin_permissions = true + } + + endpoint_private_access = true + endpoint_public_access = true + public_access_cidrs = var.eks_public_access_cidrs +} +``` + +Use `false` instead if that is the cluster's current bootstrap permission. Apply this authentication-mode transition and wait for the cluster update to finish before creating access entries or enabling Auto Mode. The transition away from `CONFIG_MAP` is one-way, so confirm equivalent administrator and automation access through `access_entries` before later removing `aws-auth` mappings. Clusters already using `API_AND_CONFIG_MAP` or `API` do not need this preliminary mode transition, but should still preserve the current bootstrap permission explicitly. + +Enabling the access-entry API on a legacy cluster automatically migrates only the original cluster creator. Before any later move to API-only authentication or removal of EKS-owned `aws-auth` mappings: + +1. List the cluster's API access entries and inspect `aws-auth` for existing managed node group and Fargate role mappings. +2. Confirm that an equivalent EKS-managed API access entry exists for every such role; otherwise keep `API_AND_CONFIG_MAP` and complete that service-managed migration first. +3. Do not add this module's managed-node role to `access_entries`. EKS owns that node identity, and the module rejects duplicate principals. + +```bash +terraform plan -out=eks-auth-mode.tfplan +terraform apply eks-auth-mode.tfplan +``` + +### Phase 2: enable Auto Mode in a hybrid configuration + +After API authentication is active, verify any installed Amazon VPC CNI, `kube-proxy`, Amazon EBS CSI driver, CSI snapshot controller, and EKS Pod Identity Agent against AWS's [current required add-on versions for enabling Auto Mode](https://docs.aws.amazon.com/eks/latest/userguide/auto-enable-existing.html). Follow the live AWS table rather than pinning the values from this guide. + +Auto Mode nodes include the Pod Identity agent, but standard managed nodes do not. If `pod_identity_associations` serve workloads that remain on managed node groups during the hybrid phase, retain or add `eks-pod-identity-agent` in `addons` until those workloads have moved off the standard nodes. + +Then enable Auto Mode and add the required human and automation principals through `access_entries`. Retain the existing node groups and add-ons during workload migration: + +```hcl +module "eks" { + source = "" + version = "2.0.0" + + name = "production" + subnet_ids = var.eks_subnet_ids + + auto_mode = {} + + access_config = { + authentication_mode = "API_AND_CONFIG_MAP" + bootstrap_cluster_creator_admin_permissions = true # Match the existing cluster. + } + + access_entries = var.eks_access_entries + + # Keep managed-node capacity and required traditional add-ons during the + # hybrid migration. Preserve every existing add-on and its configuration. + node_groups = var.node_groups + addons = merge(var.addons, { + vpc-cni = merge(var.addons["vpc-cni"], { + before_compute = true + }) + }) + + endpoint_private_access = true + endpoint_public_access = true + public_access_cidrs = var.eks_public_access_cidrs +} +``` + +For every fresh hybrid configuration where this module manages `vpc-cni`, `before_compute = true` is required so the CNI is ready before managed node groups bootstrap. Other add-ons default to the post-compute phase with `before_compute = false`. + +On an existing v1.x or v2.0 state, changing `vpc-cni` to `before_compute = true` moves it from one Terraform resource block to another. After editing the configuration but before running `terraform plan`, back up state and move the existing instance: + +```bash +terraform state mv \ + 'module.eks.aws_eks_addon.this["vpc-cni"]' \ + 'module.eks.aws_eks_addon.before_compute["vpc-cni"]' +``` + +Adjust the module path for the caller. Skip the move if the module does not manage `vpc-cni`, the state address is absent, or it is already under `aws_eks_addon.before_compute`. Confirm the subsequent plan updates the existing add-on rather than destroying and recreating it. + +An empty `auto_mode` object enables compute with the built-in `general-purpose` and `system` node pools. EKS-managed compute, load balancing, and block storage are enabled together to meet the EKS API and AWS provider invariant. + +By default, the module creates the Auto Mode node IAM role. If `auto_mode.create_node_iam_role = false`, verify the supplied external role before enabling Auto Mode. It must trust `ec2.amazonaws.com` for `sts:AssumeRole` and have `AmazonEKSWorkerNodeMinimalPolicy` plus `AmazonEC2ContainerRegistryPullOnly`, or equivalent permissions. The `auto_mode.node_iam_policy_arns` and `auto_mode.node_iam_policy_arn_map` inputs apply only to a module-created role and do not modify an external role. + +Create and review a saved plan before applying: + +```bash +terraform plan -out=eks-auto-mode.tfplan +terraform show eks-auto-mode.tfplan +``` + +The plan should update the existing cluster in place, update the cluster-role trust policy for Auto Mode, attach the required Auto Mode cluster policies, and create the dedicated Auto Mode node role and policies when the module owns that role. Stop if it proposes replacing the cluster or existing managed node groups. Once the review is complete, apply the saved plan and wait for the EKS update to finish: + +```bash +terraform apply eks-auto-mode.tfplan +``` + +Once enabled with built-in node pools, the effective Auto Mode compute `node_role_arn` is a cluster creation-time setting in the AWS provider. Changing it forces replacement of `aws_eks_cluster.this`. This includes switching between a module-created and external role, changing `auto_mode.node_iam_role_arn`, or renaming a module-created role so its ARN changes. Stop if such a plan proposes cluster replacement and migrate to a new cluster unless replacement is intentional. + +`bootstrap_self_managed_addons` is nullable in version 2.0. Its `null` default resolves to `true` when `auto_mode = null` and to `false` whenever an `auto_mode` object is configured. The setting is creation-time only, and the module ignores changes after creation. Consequently, enabling Auto Mode on an existing standard cluster does not rewrite its current value. + +With a built-in node pool, EKS creates the access entry for the compute node role as part of the service-managed configuration; the module deliberately does not create a duplicate. A custom `NodeClass` that reuses that compute role reuses the service-created access entry. When all built-in node pools are disabled with `node_pools = []`, `auto_mode.create_access_entry = null` (the default) makes the module create an `EC2` entry and `AmazonEKSAutoNodePolicy` association for a module-created or plan-known external Auto Mode role. Set this flag to `true` when an external `node_iam_role_arn` is unknown until apply so Terraform can determine the resource count during planning. Set it to `false` only when another stack owns the required entry and association. `true` requires enabled Auto Mode, no built-in pools, and a module-created or external node role; built-in pools always use the EKS-owned entry. + +A custom `NodeClass` that uses a different role must set `node_role_arn`. When that ARN is known during planning, leaving the nested `auto_mode_node_classes[*].create_access_entry = null` automatically creates the role's `EC2` access entry and associates `AmazonEKSAutoNodePolicy`. Set the nested flag to `true` explicitly when the ARN comes from a resource created in the same plan, because Terraform must know the access-entry collection during planning. Set it to `false` when reusing the global Auto Mode compute role or an access entry managed elsewhere. Access resources use the NodeClass name as their key by default; give NodeClasses that share one custom role the same stable `access_entry_key` to create only one entry and association. + +On an existing cluster, a later change between empty and non-empty `node_pools` transfers ownership of the global node role's access entry between Terraform and EKS. Isolate that change in its own plan and apply, wait for the cluster update to complete, then verify that exactly one `EC2` entry exists for the role and that `AmazonEKSAutoNodePolicy` is associated before continuing. Exercise this transition on a live non-production cluster first; mock plans cannot validate the service-side handoff. Prefer a distinct custom NodeClass role and module-managed entry when ownership ambiguity is unacceptable. + +The AWS-managed Auto Mode policies do not allow arbitrary user-defined tags on provisioned EC2, EBS, and load-balancing resources. When custom `NodeClass`, storage, or load-balancer configuration applies custom AWS tags, create a least-privilege policy for those tag keys. If the policy is created in the same plan, pass its ARN through the plan-safe map: + +```hcl +auto_mode = { + cluster_iam_policy_arn_map = { + custom_tags = aws_iam_policy.eks_auto_custom_tags.arn + } +} +``` + +The module attaches these caller-managed policies to the cluster IAM role; it does not create their policy documents. Custom worker security groups outside the EKS-created `eks-cluster-sg-*` naming pattern can similarly require additional cluster-role permissions so Auto Mode can add load-balancer ingress rules. Supply those permissions through `cluster_iam_policy_arn_map` and review AWS's [Auto Mode networking guidance](https://docs.aws.amazon.com/eks/latest/userguide/auto-networking.html). See the AWS [Auto Mode cluster IAM role guidance](https://docs.aws.amazon.com/eks/latest/userguide/auto-cluster-iam-role.html) for custom-tag permissions. + +Use `auto_mode.cluster_iam_policy_arn_map` and `auto_mode.node_iam_policy_arn_map`, or nested `pod_identity_associations[*].iam_policy_arn_map` and `capabilities[*].iam_policy_arn_map`, when values come from resources or modules created in the same plan. Their stable caller-chosen keys are known early enough for `for_each`. The corresponding legacy `_arns` sets remain supported for literal, imported, or otherwise plan-known ARNs; every set element must be known during planning because the ARN itself becomes the attachment key. Do not configure the same policy through both forms. + +Existing self-managed Karpenter and AWS Load Balancer Controller installations may overlap with Auto Mode only during a controlled transition. Explicitly migrate ownership of Karpenter `NodePool`/`EC2NodeClass` resources and load-balanced Kubernetes `Service` resources; do not leave legacy and Auto Mode controllers competing for the same resources long term. Remove each legacy controller only after its workloads and AWS resources have moved and the Auto Mode replacement is healthy. AWS's [existing-cluster migration guidance](https://docs.aws.amazon.com/eks/latest/userguide/migrate-auto.html) describes the supported ownership transitions and resources that cannot be transferred directly. + +After Auto Mode capacity is healthy: + +1. Move workloads in controlled batches and validate scheduling, disruption budgets, networking, storage, observability, and autoscaling. +2. Confirm which controller owns every load balancer and storage volume. Existing load balancers and EBS-backed workloads are not converted merely by enabling Auto Mode. +3. Drain the old managed node groups only after replacement capacity is ready. +4. Set `node_groups = {}` and `addons = {}` only when it is safe for Terraform to destroy those managed node groups and add-ons. + +A pure Auto Mode configuration is: + +```hcl +auto_mode = {} +node_groups = {} +addons = {} +``` + +The module retains its legacy managed-node IAM role resource for v1.x state compatibility, even when `node_groups = {}`. No EKS managed node group uses that role in a pure Auto Mode configuration. + +### Verify Auto Mode + +After applying, verify: + +```bash +aws eks describe-cluster --name +aws eks update-kubeconfig --name +kubectl get nodes +kubectl get nodeclasses +kubectl get nodepools +``` + +Also confirm that intended access entries and Pod Identity associations remain present, workloads are healthy, and public API access is limited to the expected CIDRs. + +Custom manifest inputs continue to produce YAML outputs; Terraform does not apply those Kubernetes manifests. Reconcile the rendered `NodeClass`, `NodePool`, `StorageClass`, and load-balancer `Service` YAML through the deployment process that owns in-cluster resources. diff --git a/README.md b/README.md index 8420efa..1f99e99 100755 --- a/README.md +++ b/README.md @@ -1,10 +1,17 @@ # Terraform AWS EKS Module +Official Terraform Registry module: [native-cube/eks/aws](https://registry.terraform.io/modules/native-cube/eks/aws/latest) + Simple Terraform module for creating an Amazon EKS cluster with: - EKS control plane IAM role - Managed node group IAM role - One or more EKS managed node groups +- Optional EKS Auto Mode compute, load balancing, and block storage +- Optional hybrid capacity with Auto Mode and managed node groups in one cluster +- Optional EKS access entries and access policy associations +- Optional EKS Pod Identity associations +- Rendered Auto Mode NodeClass, NodePool, StorageClass, and LoadBalancer Service manifests - Optional EKS Provisioned Control Plane scaling tiers - Core EKS add-ons: `coredns`, `kube-proxy`, and `vpc-cni` - Optional Amazon EKS managed capabilities: Argo CD, ACK, and KRO @@ -47,12 +54,119 @@ aws eks update-kubeconfig --name dev-eks ## Examples - `examples/minimal` - smallest practical module call using defaults for node groups, add-ons, API access, and logging. +- `examples/auto-mode` - pure EKS Auto Mode cluster with public and private API access and no managed node groups or traditional add-on resources. - `examples/basic` - simple explicit managed node group configuration. - `examples/advanced` - restricted API access, full control-plane logging, multiple node groups, expanded add-on configuration, optional IAM-backed add-ons, and optional node subnet overrides. - `examples/capabilities` - Amazon EKS managed capabilities for Argo CD, ACK, and KRO, including IAM Identity Center authentication for Argo CD and optional capability role policies. - `examples/karpenter-ready` - EKS-side Karpenter readiness with discovery tags, a Karpenter node role, and node access entry while leaving the Karpenter controller, Helm release, interruption queue, NodePool, and EC2NodeClass to a separate module. - `examples/provisioned-control-plane` - EKS Provisioned Control Plane configuration with a selectable scaling tier for predictable control-plane capacity. +## EKS Auto Mode + +Auto Mode is opt-in. The default `auto_mode = null` preserves version 1.x behavior and does not manage Auto Mode settings. An empty object enables Auto Mode compute, load balancing, and block storage together with the built-in `general-purpose` and `system` node pools: + +```hcl +module "eks" { + source = "./eks" + + name = "dev-auto" + subnet_ids = ["subnet-0123456789abcdef0", "subnet-0fedcba9876543210"] + + auto_mode = {} + + # Omit the standard module defaults for a pure Auto Mode cluster. + node_groups = {} + addons = {} +} +``` + +`bootstrap_self_managed_addons` defaults to `null`. The module resolves that to `true` for a new standard cluster and `false` whenever an Auto Mode object is configured. EKS treats the setting as creation-time only, so the module ignores later changes on an existing cluster. + +The Kubernetes API remains public and private by default. Restrict `public_access_cidrs` for production callers. + +Keep `node_groups` populated to run a hybrid cluster while workloads move to Auto Mode. Standard managed nodes still need the traditional VPC CNI, `kube-proxy`, and CoreDNS add-ons. In a fresh hybrid configuration, set the module-managed VPC CNI to install before compute so managed nodes can bootstrap: + +```hcl +auto_mode = {} + +node_groups = { + workloads = {} +} + +addons = { + vpc-cni = { + before_compute = true + } + kube-proxy = {} + coredns = {} + # Include eks-pod-identity-agent when standard nodes use Pod Identity. +} +``` + +`before_compute` defaults to `false`. Moving an existing module-managed `vpc-cni` add-on to the early phase changes its Terraform resource address; perform the state move in [MIGRATION-2.0.md](MIGRATION-2.0.md) before planning. AWS requires Auto Mode compute, load balancing, and block storage to be enabled or disabled together, so version 2.0 intentionally exposes one `auto_mode.enabled` switch rather than independent capability booleans. + +Set `auto_mode.node_pools = []` to disable the built-in pools and use custom NodePools. With `auto_mode.create_access_entry = null` (the default), the module creates the required `EC2` access entry and `AmazonEKSAutoNodePolicy` association when a module-created or plan-known external Auto Mode node role is configured. Set this flag to `true` when an external `node_iam_role_arn` is unknown until apply, or to `false` when another stack owns that entry and association. `true` is valid only while Auto Mode is enabled, built-in pools are empty, and a module-created or external node role is configured. When a built-in pool is enabled, EKS owns the compute role's entry instead. + +A custom NodeClass that names a different IAM role must also provide its ARN through `node_role_arn`. For a plan-known ARN, the nested `auto_mode_node_classes[*].create_access_entry` flag defaults to creating an entry keyed by the NodeClass name. Set that nested flag to `true` explicitly when the ARN comes from a resource created in the same plan. Give NodeClasses that share one custom role the same stable `access_entry_key` to deduplicate the entry; output keys use that value, or the NodeClass name by default. Set the nested flag to `false` when reusing the global/service-managed Auto Mode role or an externally managed entry. + +When `auto_mode.create_node_iam_role = false`, the supplied role is externally owned. Before enabling compute, ensure that it trusts `ec2.amazonaws.com` for `sts:AssumeRole` and has `AmazonEKSWorkerNodeMinimalPolicy` plus `AmazonEC2ContainerRegistryPullOnly`, or equivalent permissions. The `auto_mode.node_iam_policy_arns` and `auto_mode.node_iam_policy_arn_map` inputs attach policies only to a role created by this module; they do not modify an external role. + +Fresh configurations with either built-in pools or `node_pools = []` are supported. On an existing cluster, changing between empty and non-empty `node_pools` transfers the global node role's access-entry ownership between this module and EKS. Make that boundary change in a dedicated apply, wait for the EKS update, and verify that the role has one `EC2` entry with `AmazonEKSAutoNodePolicy` before proceeding. Test the transition against a live non-production cluster first. A distinct role for custom NodeClasses avoids transferring ownership of the built-in role's entry. + +The module can render custom Kubernetes YAML without configuring a Kubernetes provider. Apply the combined output only after the cluster is reachable: + +```bash +terraform output -raw auto_mode_kubernetes_manifest_yaml | kubectl apply -f - +``` + +To disable an enabled cluster safely, first apply `auto_mode = { enabled = false }`. After AWS finishes deleting Auto Mode compute and load balancers, set `auto_mode = null` in a second apply to remove retained Auto Mode IAM resources. Keep `access_config.authentication_mode` explicitly set to the cluster's current `API` or `API_AND_CONFIG_MAP` mode afterward; the transition away from `CONFIG_MAP` is one-way. For a cluster originally created for Auto Mode, keep `bootstrap_self_managed_addons = false` explicitly and permanently. If this module manages `vpc-cni` for managed node groups added later, set `before_compute = true`. Disabling Auto Mode is destructive; review AWS's [disable procedure](https://docs.aws.amazon.com/eks/latest/userguide/auto-disable.html) before changing an existing cluster. + +## Access Entries and Pod Identity + +Use `access_entries` to authorize IAM principals through EKS API authentication and optionally associate EKS access policies. When access entries are configured, the module defaults the authentication mode to `API`; existing `CONFIG_MAP` clusters should first move to `API_AND_CONFIG_MAP` as described in the migration guide. + +On a legacy cluster, enabling the access-entry API automatically migrates only the original cluster creator. Before later changing to API-only authentication or removing `aws-auth` mappings, verify that EKS has created equivalent API access entries for every existing managed node group and Fargate role. Keep `API_AND_CONFIG_MAP` until that check passes. Do not add this module's managed-node role to `access_entries`; EKS owns that node identity and the module rejects duplicate principals. + +Use `pod_identity_associations` to associate Kubernetes service accounts with existing IAM roles or module-created roles. Managed policy ARNs and inline JSON policies are supported for module-created roles. Auto Mode nodes include the Pod Identity agent. Standard managed nodes require the `eks-pod-identity-agent` add-on, so retain or add it while Pod Identity workloads run on those nodes in a hybrid cluster. + +## Plan-safe IAM policy attachments + +Use the keyed policy maps when a policy ARN comes from a resource or module created in the same plan. Stable caller-chosen keys let Terraform identify attachment instances before their ARN values are known: + +```hcl +auto_mode = { + cluster_iam_policy_arn_map = { + custom_tags = aws_iam_policy.eks_auto_custom_tags.arn + } + node_iam_policy_arn_map = { + node_access = aws_iam_policy.eks_auto_node_access.arn + } +} + +pod_identity_associations = { + application = { + namespace = "application" + service_account = "application" + iam_policy_arn_map = { + application_data = aws_iam_policy.application_data.arn + } + } +} + +capabilities = { + ack = { + type = "ACK" + iam_policy_arn_map = { + ack_resources = aws_iam_policy.ack_resources.arn + } + } +} +``` + +The legacy `cluster_iam_policy_arns`, `node_iam_policy_arns`, and nested `iam_policy_arns` sets remain supported for literal or otherwise plan-known ARNs. Because set elements become `for_each` keys, every ARN in a set must be known during planning. Do not put the same policy in both the set and map forms. + +Auto Mode's AWS-managed policies do not include arbitrary custom-resource tagging permissions. Custom worker security groups whose names do not match the EKS-created `eks-cluster-sg-*` pattern can also require extra cluster-role permissions so Auto Mode can add load-balancer ingress rules. Attach a least-privilege caller-managed policy through `auto_mode.cluster_iam_policy_arn_map`; see the AWS [Auto Mode networking considerations](https://docs.aws.amazon.com/eks/latest/userguide/auto-networking.html). + ## Provisioned Control Plane Set `control_plane_scaling_config` when a cluster needs predictable, pre-provisioned control-plane capacity. The component configuration arguments can customize the API server, controller manager, and scheduler: @@ -144,7 +258,9 @@ module "eks" { } ``` -When enabled, the module can create a capability IAM role trusted by `capabilities.eks.amazonaws.com` and creates all configured entries through one `aws_eks_capability.this` resource block. Attach managed policies or an inline policy to the capability role only when a capability needs access to supporting AWS services. +When enabled, the module can create a capability IAM role trusted by `capabilities.eks.amazonaws.com` and creates all configured entries through one `aws_eks_capability.this` resource block. Attach managed policies, an inline policy, or the `cloudcontrol_read_only`, `eks_read_only`, `resource_tagging`, and `secrets_read_only` presets only when a capability needs access to supporting AWS services. + +The `secrets_read_only` and `resource_tagging` presets use `Resource = "*"`, so they grant broad permissions to matching resources in the account and Region. For production, prefer a caller-supplied `inline_policy_json` or managed policy scoped to the exact resource ARNs and actions the capability needs. For `ARGOCD`, configure the nested `argocd` object with IAM Identity Center settings, optional RBAC role mappings, and optional VPC endpoint IDs. If `argocd.network_access_vpce_ids` is empty, AWS exposes the default public managed Argo CD endpoint. Supplying VPC endpoint IDs makes the Argo CD server private-only through those endpoints. @@ -194,28 +310,51 @@ This module deliberately does not install the Karpenter controller, create its c | [aws_cloudwatch_log_group.cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) | resource | | [aws_ec2_tag.karpenter_cluster_security_group](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_tag) | resource | | [aws_ec2_tag.karpenter_subnets](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_tag) | resource | +| [aws_eks_access_entry.auto_mode_node](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_access_entry) | resource | +| [aws_eks_access_entry.auto_mode_node_class](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_access_entry) | resource | | [aws_eks_access_entry.karpenter_node](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_access_entry) | resource | +| [aws_eks_access_entry.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_access_entry) | resource | +| [aws_eks_access_policy_association.auto_mode_node](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_access_policy_association) | resource | +| [aws_eks_access_policy_association.auto_mode_node_class](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_access_policy_association) | resource | +| [aws_eks_access_policy_association.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_access_policy_association) | resource | +| [aws_eks_addon.before_compute](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_addon) | resource | | [aws_eks_addon.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_addon) | resource | | [aws_eks_capability.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_capability) | resource | | [aws_eks_cluster.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_cluster) | resource | | [aws_eks_node_group.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_node_group) | resource | +| [aws_eks_pod_identity_association.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_pod_identity_association) | resource | +| [aws_iam_role.auto_mode_node](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | | [aws_iam_role.cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | | [aws_iam_role.eks_capability](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | | [aws_iam_role.karpenter_node](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | | [aws_iam_role.node](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | +| [aws_iam_role.pod_identity](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource | | [aws_iam_role_policy.eks_capability](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) | resource | +| [aws_iam_role_policy.eks_capability_preset](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) | resource | +| [aws_iam_role_policy.pod_identity](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) | resource | +| [aws_iam_role_policy_attachment.auto_mode_cluster_additional](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource | +| [aws_iam_role_policy_attachment.auto_mode_node](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource | +| [aws_iam_role_policy_attachment.auto_mode_node_additional](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource | | [aws_iam_role_policy_attachment.cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource | | [aws_iam_role_policy_attachment.eks_capability](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource | | [aws_iam_role_policy_attachment.karpenter_node](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource | | [aws_iam_role_policy_attachment.node](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource | +| [aws_iam_role_policy_attachment.pod_identity](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource | ## Inputs | Name | Description | Type | Default | Required | | ---- | ----------- | ---- | ------- | :------: | | [access\_config](#input\_access\_config) | Optional EKS access configuration for cluster authentication mode and creator admin permissions. |
object({
authentication_mode = optional(string)
bootstrap_cluster_creator_admin_permissions = optional(bool)
})
| `null` | no | -| [addons](#input\_addons) | EKS add-ons to install after the managed node groups are created. |
map(object({
configuration_values = optional(string)
pod_identity_associations = optional(list(object({
role_arn = string
service_account = string
})), [])
resolve_conflicts_on_create = optional(string, "OVERWRITE")
resolve_conflicts_on_update = optional(string, "OVERWRITE")
service_account_role_arn = optional(string)
version = optional(string)
}))
|
{
"coredns": {},
"kube-proxy": {},
"vpc-cni": {}
}
| no | -| [capabilities](#input\_capabilities) | Amazon EKS managed capabilities to create, keyed by a stable local name. Supported types are ARGOCD, ACK, and KRO. The module can create a capability IAM role per entry, or use an externally managed role ARN. |
map(object({
capability_name = optional(string)
create_iam_role = optional(bool, true)
delete_propagation_policy = optional(string, "RETAIN")
iam_policy_arns = optional(set(string), [])
iam_role_arn = optional(string)
iam_role_name = optional(string)
inline_policy_json = optional(string)
type = string
argocd = optional(object({
idc_instance_arn = optional(string)
idc_region = optional(string)
namespace = optional(string)
network_access_vpce_ids = optional(set(string), [])
rbac_role_mappings = optional(list(object({
role = string
identities = list(object({
id = string
type = string
}))
})), [])
}))
}))
| `{}` | no | +| [access\_entries](#input\_access\_entries) | Additional EKS access entries and optional access policy associations to create. |
map(object({
kubernetes_groups = optional(set(string), [])
policy_associations = optional(map(object({
access_scope = object({
namespaces = optional(set(string), [])
type = string
})
policy_arn = string
})), {})
principal_arn = string
type = optional(string, "STANDARD")
user_name = optional(string)
}))
| `{}` | no | +| [addons](#input\_addons) | EKS add-ons to install. Set before\_compute for add-ons such as vpc-cni that managed nodes need during bootstrap. |
map(object({
before_compute = optional(bool, false)
configuration_values = optional(string)
pod_identity_associations = optional(list(object({
role_arn = string
service_account = string
})), [])
resolve_conflicts_on_create = optional(string, "OVERWRITE")
resolve_conflicts_on_update = optional(string, "OVERWRITE")
service_account_role_arn = optional(string)
version = optional(string)
}))
|
{
"coredns": {},
"kube-proxy": {},
"vpc-cni": {}
}
| no | +| [auto\_mode](#input\_auto\_mode) | Optional EKS Auto Mode configuration. Null leaves Auto Mode unmanaged for backward compatibility; an empty object enables Auto Mode with its default node pools. Set enabled to false to explicitly disable previously managed Auto Mode capabilities. |
object({
cluster_iam_policy_arn_map = optional(map(string), {})
cluster_iam_policy_arns = optional(set(string), [])
create_access_entry = optional(bool)
create_node_iam_role = optional(bool, true)
enabled = optional(bool, true)
node_iam_policy_arn_map = optional(map(string), {})
node_iam_policy_arns = optional(set(string), [])
node_iam_role_arn = optional(string)
node_iam_role_name = optional(string)
node_pools = optional(set(string), ["general-purpose", "system"])
})
| `null` | no | +| [auto\_mode\_load\_balancer\_services](#input\_auto\_mode\_load\_balancer\_services) | EKS Auto Mode Network Load Balancer Service manifests to render as YAML, keyed by Kubernetes metadata.name. |
map(object({
annotations = optional(map(string), {})
external_traffic_policy = optional(string)
ip_families = optional(list(string), [])
ip_family_policy = optional(string)
labels = optional(map(string), {})
load_balancer_class = optional(string, "eks.amazonaws.com/nlb")
load_balancer_source_ranges = optional(list(string), [])
namespace = optional(string, "default")
ports = list(any)
selector = map(string)
session_affinity = optional(string)
}))
| `{}` | no | +| [auto\_mode\_node\_classes](#input\_auto\_mode\_node\_classes) | Custom EKS Auto Mode NodeClass manifests to render as YAML, keyed by Kubernetes metadata.name. The module does not apply these manifests. |
map(object({
access_entry_key = optional(string)
annotations = optional(map(string), {})
create_access_entry = optional(bool)
labels = optional(map(string), {})
node_role_arn = optional(string)
spec = any
}))
| `{}` | no | +| [auto\_mode\_node\_pools](#input\_auto\_mode\_node\_pools) | Custom EKS Auto Mode NodePool manifests to render as YAML, keyed by Kubernetes metadata.name. The module does not apply these manifests. |
map(object({
annotations = optional(map(string), {})
labels = optional(map(string), {})
spec = any
}))
| `{}` | no | +| [auto\_mode\_storage\_classes](#input\_auto\_mode\_storage\_classes) | EKS Auto Mode EBS StorageClass manifests to render as YAML, keyed by Kubernetes metadata.name. |
map(object({
allow_volume_expansion = optional(bool, true)
allowed_topologies = optional(list(any), [])
annotations = optional(map(string), {})
labels = optional(map(string), {})
mount_options = optional(list(string), [])
parameters = optional(map(string), {})
reclaim_policy = optional(string, "Delete")
volume_binding_mode = optional(string, "WaitForFirstConsumer")
}))
| `{}` | no | +| [bootstrap\_self\_managed\_addons](#input\_bootstrap\_self\_managed\_addons) | Whether EKS bootstraps self-managed networking add-ons at cluster creation. Leave null to derive true for standard clusters and false while Auto Mode is configured. This setting is create-only. | `bool` | `null` | no | +| [capabilities](#input\_capabilities) | Amazon EKS managed capabilities to create, keyed by a stable local name. Supported types are ARGOCD, ACK, and KRO. The module can create a capability IAM role per entry, or use an externally managed role ARN. |
map(object({
capability_name = optional(string)
create_iam_role = optional(bool, true)
delete_propagation_policy = optional(string, "RETAIN")
iam_policy_arn_map = optional(map(string), {})
iam_policy_arns = optional(set(string), [])
iam_policy_presets = optional(set(string), [])
iam_role_arn = optional(string)
iam_role_name = optional(string)
inline_policy_json = optional(string)
type = string
argocd = optional(object({
idc_instance_arn = optional(string)
idc_region = optional(string)
namespace = optional(string)
network_access_vpce_ids = optional(set(string), [])
rbac_role_mappings = optional(list(object({
role = string
identities = list(object({
id = string
type = string
}))
})), [])
}))
}))
| `{}` | no | | [cloudwatch\_log\_group\_kms\_key\_id](#input\_cloudwatch\_log\_group\_kms\_key\_id) | Optional KMS key ID or ARN for encrypting the EKS control plane CloudWatch log group. | `string` | `null` | no | | [cloudwatch\_log\_retention\_days](#input\_cloudwatch\_log\_retention\_days) | Retention in days for the EKS control plane CloudWatch log group. | `number` | `30` | no | | [cluster\_encryption\_config](#input\_cluster\_encryption\_config) | Optional EKS encryption configuration for Kubernetes secrets using an existing KMS key. |
object({
provider_key_arn = string
resources = optional(list(string), ["secrets"])
})
| `null` | no | @@ -226,6 +365,8 @@ This module deliberately does not install the Karpenter controller, create its c | [enabled\_cluster\_log\_types](#input\_enabled\_cluster\_log\_types) | EKS control plane log types to enable. | `list(string)` |
[
"api",
"audit",
"authenticator"
]
| no | | [endpoint\_private\_access](#input\_endpoint\_private\_access) | Whether the Kubernetes API server endpoint is reachable from within the VPC. | `bool` | `true` | no | | [endpoint\_public\_access](#input\_endpoint\_public\_access) | Whether the Kubernetes API server endpoint is reachable from the public internet. | `bool` | `true` | no | +| [force\_update\_version](#input\_force\_update\_version) | Whether to force a Kubernetes version update when EKS cannot drain pods. | `bool` | `null` | no | +| [ip\_family](#input\_ip\_family) | Optional Kubernetes service IP family. Valid values are ipv4 or ipv6. | `string` | `null` | no | | [karpenter](#input\_karpenter) | Optional EKS-side readiness settings for Karpenter. This module prepares AWS/EKS primitives only; install Karpenter, controller IAM, interruption handling, NodePools, and EC2NodeClasses separately. |
object({
create_access_entry = optional(bool, true)
create_node_iam_role = optional(bool, true)
enabled = optional(bool, false)
node_iam_role_arn = optional(string)
node_iam_role_name = optional(string)
subnet_ids = optional(list(string), [])
tag_cluster_security_group = optional(bool, true)
tag_subnets = optional(bool, true)
})
| `{}` | no | | [kube\_api\_server\_config](#input\_kube\_api\_server\_config) | Optional Kubernetes API server configuration for event retention and the NodePort service range. |
object({
event_ttl = optional(string)
service_node_port_range = optional(object({
max_port = optional(number)
min_port = optional(number)
}))
})
| `null` | no | | [kube\_controller\_manager\_config](#input\_kube\_controller\_manager\_config) | Optional Kubernetes controller manager configuration. HPA controller customization requires a Provisioned Control Plane tier. |
object({
horizontal_pod_autoscaler_controller_config = optional(object({
horizontal_pod_autoscaler_sync_period = optional(string)
}))
})
| `null` | no | @@ -233,37 +374,62 @@ This module deliberately does not install the Karpenter controller, create its c | [kubernetes\_version](#input\_kubernetes\_version) | Kubernetes version for the EKS cluster and managed node groups. Leave null to use the current AWS default. | `string` | `null` | no | | [name](#input\_name) | Name prefix for module-created resources. Used as the EKS cluster name when cluster\_name is null. | `string` | n/a | yes | | [node\_groups](#input\_node\_groups) | Managed node groups to create. |
map(object({
ami_type = optional(string)
capacity_type = optional(string, "ON_DEMAND")
desired_size = optional(number, 2)
disk_size = optional(number, 20)
instance_types = optional(list(string), ["t3.medium"])
labels = optional(map(string), {})
max_size = optional(number, 3)
min_size = optional(number, 1)
subnet_ids = optional(list(string), [])
update_max_unavailable = optional(number, 1)
node_repair_config = optional(object({
enabled = optional(bool)
max_parallel_nodes_repaired_count = optional(number)
max_parallel_nodes_repaired_percentage = optional(number)
max_unhealthy_node_threshold_count = optional(number)
max_unhealthy_node_threshold_percentage = optional(number)
overrides = optional(list(object({
min_repair_wait_time_mins = number
node_monitoring_condition = string
node_unhealthy_reason = string
repair_action = string
})), [])
}))
taints = optional(list(object({
effect = string
key = string
value = optional(string, "")
})), [])
}))
|
{
"default": {}
}
| no | +| [pod\_identity\_associations](#input\_pod\_identity\_associations) | EKS Pod Identity associations to create for Kubernetes service accounts. The module can create the IAM role per association, or use an externally managed role ARN. |
map(object({
create_iam_role = optional(bool, true)
disable_session_tags = optional(bool)
iam_policy_arn_map = optional(map(string), {})
iam_policy_arns = optional(set(string), [])
iam_role_arn = optional(string)
iam_role_name = optional(string)
inline_policy_json = optional(string)
namespace = string
service_account = string
tags = optional(map(string), {})
target_role_arn = optional(string)
}))
| `{}` | no | | [public\_access\_cidrs](#input\_public\_access\_cidrs) | CIDR blocks that can access the public Kubernetes API endpoint. | `list(string)` |
[
"0.0.0.0/0"
]
| no | | [service\_ipv4\_cidr](#input\_service\_ipv4\_cidr) | Optional Kubernetes service IPv4 CIDR. Set only when you need a non-default service CIDR. | `string` | `null` | no | | [subnet\_ids](#input\_subnet\_ids) | Subnet IDs for the EKS control plane and default node groups. Use at least two subnets in different Availability Zones. | `list(string)` | n/a | yes | | [tags](#input\_tags) | Tags to apply to created resources. | `map(string)` | `{}` | no | +| [upgrade\_policy\_support\_type](#input\_upgrade\_policy\_support\_type) | Optional Kubernetes version support policy. Valid values are STANDARD and EXTENDED. | `string` | `null` | no | ## Outputs | Name | Description | | ---- | ----------- | +| [access\_entry\_arns](#output\_access\_entry\_arns) | EKS access entry ARNs by access entry key. | | [addon\_arns](#output\_addon\_arns) | EKS add-on ARNs by add-on name. | | [argocd\_idc\_managed\_application\_arns](#output\_argocd\_idc\_managed\_application\_arns) | IAM Identity Center managed application ARNs by ARGOCD capability key. | | [argocd\_server\_urls](#output\_argocd\_server\_urls) | Managed Argo CD server URLs by ARGOCD capability key. | +| [auto\_mode\_enabled](#output\_auto\_mode\_enabled) | Whether EKS Auto Mode is enabled by this module configuration. | +| [auto\_mode\_kubernetes\_manifest\_yaml](#output\_auto\_mode\_kubernetes\_manifest\_yaml) | All rendered EKS Auto Mode Kubernetes manifests joined into a single multi-document YAML string. | +| [auto\_mode\_kubernetes\_manifests](#output\_auto\_mode\_kubernetes\_manifests) | All rendered EKS Auto Mode Kubernetes YAML manifests by kind/name. | +| [auto\_mode\_load\_balancer\_service\_manifests](#output\_auto\_mode\_load\_balancer\_service\_manifests) | Rendered EKS Auto Mode LoadBalancer Service YAML manifests by name. | +| [auto\_mode\_node\_access\_entry\_arn](#output\_auto\_mode\_node\_access\_entry\_arn) | EKS access entry ARN created for the Auto Mode node role when built-in node pools are disabled. Built-in node pools register their role automatically. | +| [auto\_mode\_node\_class\_access\_entry\_arns](#output\_auto\_mode\_node\_class\_access\_entry\_arns) | EKS access entry ARNs created for custom Auto Mode NodeClass IAM roles, keyed by access\_entry\_key (or NodeClass name by default). | +| [auto\_mode\_node\_class\_manifests](#output\_auto\_mode\_node\_class\_manifests) | Rendered EKS Auto Mode NodeClass YAML manifests by name. | +| [auto\_mode\_node\_iam\_role\_arn](#output\_auto\_mode\_node\_iam\_role\_arn) | IAM role ARN used by EKS Auto Mode managed compute when Auto Mode is configured. | +| [auto\_mode\_node\_iam\_role\_name](#output\_auto\_mode\_node\_iam\_role\_name) | IAM role name used by EKS Auto Mode managed compute when Auto Mode is configured. | +| [auto\_mode\_node\_pool\_manifests](#output\_auto\_mode\_node\_pool\_manifests) | Rendered EKS Auto Mode NodePool YAML manifests by name. | +| [auto\_mode\_storage\_class\_manifests](#output\_auto\_mode\_storage\_class\_manifests) | Rendered EKS Auto Mode StorageClass YAML manifests by name. | | [capability\_arns](#output\_capability\_arns) | Amazon EKS capability ARNs by capability key. | +| [capability\_iam\_policy\_preset\_names](#output\_capability\_iam\_policy\_preset\_names) | Capability IAM policy preset names attached by capability key. | | [capability\_iam\_role\_arns](#output\_capability\_iam\_role\_arns) | IAM role ARNs used by Amazon EKS capabilities by capability key. | | [capability\_iam\_role\_names](#output\_capability\_iam\_role\_names) | IAM role names used by Amazon EKS capabilities by capability key. | | [capability\_names](#output\_capability\_names) | Amazon EKS capability names by capability key. | | [capability\_versions](#output\_capability\_versions) | Amazon EKS capability software versions by capability key. | | [cluster\_arn](#output\_cluster\_arn) | EKS cluster ARN. | | [cluster\_certificate\_authority\_data](#output\_cluster\_certificate\_authority\_data) | Base64-encoded cluster certificate authority data. | +| [cluster\_created\_at](#output\_cluster\_created\_at) | Timestamp when the EKS cluster was created. | | [cluster\_endpoint](#output\_cluster\_endpoint) | Kubernetes API server endpoint. | | [cluster\_iam\_role\_arn](#output\_cluster\_iam\_role\_arn) | IAM role ARN used by the EKS control plane. | | [cluster\_log\_group\_name](#output\_cluster\_log\_group\_name) | CloudWatch log group for EKS control plane logs, if cluster logs are enabled. | | [cluster\_name](#output\_cluster\_name) | EKS cluster name. | | [cluster\_oidc\_issuer\_url](#output\_cluster\_oidc\_issuer\_url) | OIDC issuer URL for the EKS cluster. | +| [cluster\_platform\_version](#output\_cluster\_platform\_version) | EKS platform version. | | [cluster\_security\_group\_id](#output\_cluster\_security\_group\_id) | Security group created by EKS for the cluster. | +| [cluster\_status](#output\_cluster\_status) | EKS cluster status. | +| [cluster\_tags\_all](#output\_cluster\_tags\_all) | All tags applied to the EKS cluster, including provider default tags. | +| [cluster\_version](#output\_cluster\_version) | Kubernetes version running on the EKS cluster. | | [karpenter\_discovery\_tag\_key](#output\_karpenter\_discovery\_tag\_key) | Tag key used by Karpenter discovery selectors when Karpenter readiness is enabled. | | [karpenter\_discovery\_tag\_value](#output\_karpenter\_discovery\_tag\_value) | Tag value used by Karpenter discovery selectors when Karpenter readiness is enabled. | | [karpenter\_node\_iam\_role\_arn](#output\_karpenter\_node\_iam\_role\_arn) | IAM role ARN for Karpenter-launched worker nodes when Karpenter readiness is enabled. | | [karpenter\_node\_iam\_role\_name](#output\_karpenter\_node\_iam\_role\_name) | IAM role name for Karpenter EC2NodeClass role configuration when Karpenter readiness is enabled. | | [node\_group\_arns](#output\_node\_group\_arns) | Managed node group ARNs by node group key. | | [node\_iam\_role\_arn](#output\_node\_iam\_role\_arn) | IAM role ARN used by managed node groups. | +| [pod\_identity\_association\_arns](#output\_pod\_identity\_association\_arns) | EKS Pod Identity association ARNs by association key. | +| [pod\_identity\_association\_ids](#output\_pod\_identity\_association\_ids) | EKS Pod Identity association IDs by association key. | +| [pod\_identity\_external\_ids](#output\_pod\_identity\_external\_ids) | External IDs generated for EKS Pod Identity associations by association key. | +| [pod\_identity\_iam\_role\_arns](#output\_pod\_identity\_iam\_role\_arns) | IAM role ARNs used by EKS Pod Identity associations by association key. | +| [pod\_identity\_iam\_role\_names](#output\_pod\_identity\_iam\_role\_names) | IAM role names used by EKS Pod Identity associations by association key. | | [update\_kubeconfig\_command](#output\_update\_kubeconfig\_command) | AWS CLI command to configure kubectl for this cluster. | diff --git a/data.tf b/data.tf index bbdf45c..c3b16ac 100755 --- a/data.tf +++ b/data.tf @@ -9,7 +9,7 @@ data "aws_iam_policy_document" "cluster_assume_role" { identifiers = ["eks.amazonaws.com"] } - actions = ["sts:AssumeRole"] + actions = local.cluster_assume_role_actions } } @@ -41,3 +41,19 @@ data "aws_iam_policy_document" "capability_assume_role" { ] } } + +data "aws_iam_policy_document" "pod_identity_assume_role" { + statement { + effect = "Allow" + + principals { + type = "Service" + identifiers = ["pods.eks.amazonaws.com"] + } + + actions = [ + "sts:AssumeRole", + "sts:TagSession" + ] + } +} diff --git a/examples/auto-mode/README.md b/examples/auto-mode/README.md new file mode 100644 index 0000000..be7125c --- /dev/null +++ b/examples/auto-mode/README.md @@ -0,0 +1,48 @@ +# EKS Auto Mode Example + +This example creates an EKS cluster that uses Auto Mode for compute, load balancing, and block storage. It enables the EKS built-in `general-purpose` and `system` node pools and deliberately sets both `node_groups` and `addons` to empty maps. + +Because an Auto Mode object is configured, the nullable `bootstrap_self_managed_addons` input resolves to `false` when this new cluster is created. EKS treats that setting as creation-time only. + +The Kubernetes API endpoint is public and private. The example defaults `public_access_cidrs` to `0.0.0.0/0` for ease of evaluation; production callers should replace that value with the narrowest CIDRs that need API access. + +Provide at least two existing subnet IDs in different Availability Zones before planning: + +```bash +terraform init +terraform plan \ + -var='subnet_ids=["subnet-0123456789abcdef0","subnet-0fedcba9876543210"]' \ + -var='public_access_cidrs=["203.0.113.10/32"]' +``` + +After applying the plan, configure `kubectl` with the rendered command: + +```bash +terraform output -raw update_kubeconfig_command +``` + +The module can also render custom Auto Mode `NodeClass`, `NodePool`, `StorageClass`, and load-balancer `Service` YAML through `auto_mode_node_classes`, `auto_mode_node_pools`, `auto_mode_storage_classes`, and `auto_mode_load_balancer_services`. It returns the YAML as outputs but does not apply it, because the module does not configure a Kubernetes provider. + +Set `access_entries` to grant principals access through the EKS API authentication mode. Set `pod_identity_associations` when workloads need EKS Pod Identity; the module can create a role for an association or use an existing role. Auto Mode nodes include the Pod Identity agent, so this pure Auto Mode example does not need the `eks-pod-identity-agent` add-on. Hybrid clusters must retain or add that add-on for Pod Identity workloads running on standard managed nodes. A fresh hybrid configuration must also set `addons["vpc-cni"].before_compute = true`; moving an existing add-on to that phase requires the state move documented in [`MIGRATION-2.0.md`](../../MIGRATION-2.0.md). + +EKS creates the access entry for the node role used by the built-in node pools. The module does not create a duplicate. A custom `NodeClass` using a separate IAM role should set `node_role_arn`; for a plan-known ARN, the module automatically creates an `EC2` access entry and `AmazonEKSAutoNodePolicy` association keyed by the NodeClass name. Set `create_access_entry = true` explicitly when the ARN comes from a resource created in the same plan. Give NodeClasses sharing one role the same stable `access_entry_key` to create one shared entry; the output uses that key. Set `create_access_entry = false` when reusing the built-in compute role or an entry managed elsewhere. + +After Auto Mode is enabled with built-in node pools, any planned change to its effective compute node role ARN forces EKS cluster replacement. Keep the role ARN stable for the cluster's lifetime. + +AWS-managed Auto Mode policies do not grant permission to propagate arbitrary user-defined tags to provisioned AWS resources. Use a plan-safe map when the policy is created alongside the cluster: + +```hcl +auto_mode = { + cluster_iam_policy_arn_map = { + custom_tags = aws_iam_policy.eks_auto_custom_tags.arn + } +} +``` + +Use `cluster_iam_policy_arns` only for ARNs already known during planning. The same map/set distinction applies to `auto_mode.node_iam_policy_arn_map`/`node_iam_policy_arns` and to `iam_policy_arn_map`/`iam_policy_arns` in Pod Identity and capability entries. + +Custom worker security groups outside the EKS-created `eks-cluster-sg-*` naming pattern can require additional cluster-role permissions for Auto Mode to add load-balancer ingress rules. Supply a least-privilege policy through `cluster_iam_policy_arn_map` and review AWS's [Auto Mode networking guidance](https://docs.aws.amazon.com/eks/latest/userguide/auto-networking.html). + +Choose the network family only when creating the cluster. The provider marks `ip_family` and `service_ipv4_cidr` `ForceNew`, so any planned difference forces replacement of the EKS cluster. Auto Mode supports IPv6, but `service_ipv4_cidr` cannot be set with `ip_family = "ipv6"`; omit it and let EKS assign the IPv6 service range. + +Disabling Auto Mode requires two applies and destroys Auto Mode-managed compute and load balancers. Review AWS's [disable procedure](https://docs.aws.amazon.com/eks/latest/userguide/auto-disable.html) before changing an enabled cluster to `auto_mode = null`, and keep `bootstrap_self_managed_addons = false` permanently for this Auto-created cluster. diff --git a/examples/auto-mode/main.tf b/examples/auto-mode/main.tf new file mode 100644 index 0000000..9f483db --- /dev/null +++ b/examples/auto-mode/main.tf @@ -0,0 +1,39 @@ +terraform { + required_version = ">= 1.5.0" + + required_providers { + aws = { + source = "hashicorp/aws" + version = ">= 6.59.0" + } + } +} + +provider "aws" { + region = var.region +} + +module "eks" { + source = "../.." + + name = var.name + cluster_name = var.cluster_name + kubernetes_version = var.kubernetes_version + subnet_ids = var.subnet_ids + + # An empty object enables Auto Mode with the EKS built-in + # general-purpose and system node pools. + auto_mode = {} + + # This is a pure Auto Mode cluster: do not create managed node groups or + # install the traditional EKS add-on resources. + node_groups = {} + addons = {} + + # Version 2.0 keeps the module's public-and-private endpoint behavior. + endpoint_private_access = true + endpoint_public_access = true + public_access_cidrs = var.public_access_cidrs + + tags = var.tags +} diff --git a/examples/auto-mode/outputs.tf b/examples/auto-mode/outputs.tf new file mode 100644 index 0000000..404ea0b --- /dev/null +++ b/examples/auto-mode/outputs.tf @@ -0,0 +1,19 @@ +output "cluster_name" { + description = "EKS cluster name." + value = module.eks.cluster_name +} + +output "cluster_endpoint" { + description = "Kubernetes API server endpoint." + value = module.eks.cluster_endpoint +} + +output "auto_mode_node_iam_role_arn" { + description = "IAM role ARN used by EKS Auto Mode managed compute." + value = module.eks.auto_mode_node_iam_role_arn +} + +output "update_kubeconfig_command" { + description = "AWS CLI command to configure kubectl for this cluster." + value = module.eks.update_kubeconfig_command +} diff --git a/examples/auto-mode/variables.tf b/examples/auto-mode/variables.tf new file mode 100644 index 0000000..38f3188 --- /dev/null +++ b/examples/auto-mode/variables.tf @@ -0,0 +1,43 @@ +variable "region" { + description = "AWS region to deploy into." + type = string + default = "eu-west-2" +} + +variable "name" { + description = "Name prefix for module-created resources." + type = string + default = "example-auto-mode" +} + +variable "cluster_name" { + description = "Optional EKS cluster name. When null, name is used as the cluster name." + type = string + default = null +} + +variable "kubernetes_version" { + description = "Kubernetes version for the EKS cluster. Leave null to use the current AWS default." + type = string + default = null +} + +variable "subnet_ids" { + description = "Existing subnet IDs for the EKS control plane and Auto Mode compute. Prefer private subnets with NAT egress." + type = list(string) +} + +variable "public_access_cidrs" { + description = "CIDR blocks allowed to reach the public Kubernetes API endpoint. Restrict this in production." + type = list(string) + default = ["0.0.0.0/0"] +} + +variable "tags" { + description = "Tags applied to resources." + type = map(string) + default = { + Environment = "example" + Example = "auto-mode" + } +} diff --git a/iam.tf b/iam.tf index 2f9ea64..2e7cc37 100755 --- a/iam.tf +++ b/iam.tf @@ -11,6 +11,13 @@ resource "aws_iam_role_policy_attachment" "cluster" { policy_arn = each.value } +resource "aws_iam_role_policy_attachment" "auto_mode_cluster_additional" { + for_each = local.auto_mode_cluster_policy_arn_map + + role = aws_iam_role.cluster.name + policy_arn = each.value +} + resource "aws_iam_role" "node" { name = local.node_role_name assume_role_policy = data.aws_iam_policy_document.node_assume_role.json @@ -24,6 +31,54 @@ resource "aws_iam_role_policy_attachment" "node" { policy_arn = each.value } +resource "aws_iam_role" "auto_mode_node" { + count = local.auto_mode_create_node_iam_role ? 1 : 0 + + name = local.auto_mode_node_role_name + assume_role_policy = data.aws_iam_policy_document.node_assume_role.json + tags = local.common_tags +} + +resource "aws_iam_role_policy_attachment" "auto_mode_node" { + for_each = local.auto_mode_node_policy_arns + + role = aws_iam_role.auto_mode_node[0].name + policy_arn = each.value +} + +resource "aws_iam_role_policy_attachment" "auto_mode_node_additional" { + for_each = local.auto_mode_node_policy_arn_map + + role = aws_iam_role.auto_mode_node[0].name + policy_arn = each.value +} + +resource "aws_iam_role" "pod_identity" { + for_each = local.pod_identity_create_iam_roles + + name = each.value.iam_role_name + assume_role_policy = data.aws_iam_policy_document.pod_identity_assume_role.json + tags = merge(local.common_tags, each.value.tags) +} + +resource "aws_iam_role_policy_attachment" "pod_identity" { + for_each = local.pod_identity_policy_attachments + + role = aws_iam_role.pod_identity[each.value.association_key].name + policy_arn = each.value.policy_arn +} + +resource "aws_iam_role_policy" "pod_identity" { + for_each = { + for key, association in local.pod_identity_create_iam_roles : key => association + if association.inline_policy_json != null + } + + name = substr("${each.value.iam_role_name}-policy", 0, 128) + role = aws_iam_role.pod_identity[each.key].id + policy = each.value.inline_policy_json +} + resource "aws_iam_role" "eks_capability" { for_each = local.eks_capability_create_iam_roles @@ -50,6 +105,14 @@ resource "aws_iam_role_policy" "eks_capability" { policy = each.value.inline_policy_json } +resource "aws_iam_role_policy" "eks_capability_preset" { + for_each = local.eks_capability_preset_policies + + name = substr("${local.eks_capability_configs[each.value.capability_key].iam_role_name}-${each.value.preset_name}", 0, 128) + role = aws_iam_role.eks_capability[each.value.capability_key].id + policy = jsonencode(local.eks_capability_iam_policy_preset_documents[each.value.preset_name]) +} + resource "aws_iam_role" "karpenter_node" { count = local.karpenter_create_node_iam_role ? 1 : 0 diff --git a/locals.tf b/locals.tf index 3d0f6b7..a419164 100755 --- a/locals.tf +++ b/locals.tf @@ -12,9 +12,45 @@ locals { cluster_role_name = substr("${var.name}-cluster-role", 0, 64) node_role_name = substr("${var.name}-node-role", 0, 64) - cluster_policy_arns = toset([ - "arn:${data.aws_partition.current.partition}:iam::aws:policy/AmazonEKSClusterPolicy" - ]) + auto_mode_configured = var.auto_mode != null + auto_mode_enabled = try(var.auto_mode.enabled, false) + auto_mode_builtin_node_pools = local.auto_mode_enabled && length(try(var.auto_mode.node_pools, toset([]))) > 0 + auto_mode_create_node_iam_role = local.auto_mode_configured && try(var.auto_mode.create_node_iam_role, true) + auto_mode_global_node_role = local.auto_mode_create_node_iam_role || try(var.auto_mode.node_iam_role_arn, null) != null + auto_mode_create_access_entry = local.auto_mode_enabled && !local.auto_mode_builtin_node_pools && ( + try(var.auto_mode.create_access_entry, null) != null ? + var.auto_mode.create_access_entry : + local.auto_mode_global_node_role + ) + auto_mode_node_role_name = coalesce(try(var.auto_mode.node_iam_role_name, null), substr("${var.name}-auto-node-role", 0, 64)) + auto_mode_node_iam_role_arn = local.auto_mode_create_node_iam_role ? try(aws_iam_role.auto_mode_node[0].arn, null) : try(var.auto_mode.node_iam_role_arn, null) + auto_mode_external_node_role_name = try(var.auto_mode.node_iam_role_arn, null) == null ? try(var.auto_mode.node_iam_role_name, null) : coalesce( + try(var.auto_mode.node_iam_role_name, null), + reverse(split("/", var.auto_mode.node_iam_role_arn))[0] + ) + auto_mode_node_iam_role_name = local.auto_mode_global_node_role ? ( + local.auto_mode_create_node_iam_role ? try(aws_iam_role.auto_mode_node[0].name, local.auto_mode_node_role_name) : local.auto_mode_external_node_role_name + ) : null + cluster_bootstrap_self_managed_addons = coalesce(var.bootstrap_self_managed_addons, local.auto_mode_configured ? false : true) + + cluster_assume_role_actions = concat( + ["sts:AssumeRole"], + local.auto_mode_configured ? ["sts:TagSession"] : [] + ) + + cluster_policy_arns = setunion( + toset([ + "arn:${data.aws_partition.current.partition}:iam::aws:policy/AmazonEKSClusterPolicy" + ]), + local.auto_mode_configured ? toset([ + "arn:${data.aws_partition.current.partition}:iam::aws:policy/AmazonEKSBlockStoragePolicyV2", + "arn:${data.aws_partition.current.partition}:iam::aws:policy/AmazonEKSComputePolicy", + "arn:${data.aws_partition.current.partition}:iam::aws:policy/AmazonEKSLoadBalancingPolicy", + "arn:${data.aws_partition.current.partition}:iam::aws:policy/AmazonEKSNetworkingPolicy" + ]) : toset([]), + local.auto_mode_configured ? try(var.auto_mode.cluster_iam_policy_arns, toset([])) : toset([]) + ) + auto_mode_cluster_policy_arn_map = local.auto_mode_configured ? try(var.auto_mode.cluster_iam_policy_arn_map, {}) : {} node_policy_arns = toset([ "arn:${data.aws_partition.current.partition}:iam::aws:policy/AmazonEKSWorkerNodePolicy", @@ -22,14 +58,98 @@ locals { "arn:${data.aws_partition.current.partition}:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly" ]) + addons_before_compute = { + for name, addon in var.addons : name => addon + if addon.before_compute + } + + addons_after_compute = { + for name, addon in var.addons : name => addon + if !addon.before_compute + } + + auto_mode_node_policy_arns = local.auto_mode_create_node_iam_role ? setunion(toset([ + "arn:${data.aws_partition.current.partition}:iam::aws:policy/AmazonEC2ContainerRegistryPullOnly", + "arn:${data.aws_partition.current.partition}:iam::aws:policy/AmazonEKSWorkerNodeMinimalPolicy" + ]), try(var.auto_mode.node_iam_policy_arns, toset([]))) : toset([]) + auto_mode_node_policy_arn_map = local.auto_mode_create_node_iam_role ? try(var.auto_mode.node_iam_policy_arn_map, {}) : {} + + pod_identity_association_configs = { + for key, association in var.pod_identity_associations : key => { + create_iam_role = try(association.create_iam_role, true) + disable_session_tags = try(association.disable_session_tags, null) + iam_policy_arn_map = try(association.iam_policy_arn_map, {}) + iam_policy_arns = try(association.iam_policy_arns, toset([])) + iam_role_arn = try(association.iam_role_arn, null) + iam_role_name = coalesce(try(association.iam_role_name, null), substr("${local.cluster_name}-${key}-pod-identity", 0, 64)) + iam_role_name_override = try(association.iam_role_name, null) + inline_policy_json = try(association.inline_policy_json, null) + namespace = association.namespace + service_account = association.service_account + tags = try(association.tags, {}) + target_role_arn = try(association.target_role_arn, null) + } + } + + pod_identity_create_iam_roles = { + for key, association in local.pod_identity_association_configs : key => association + if association.create_iam_role + } + + pod_identity_policy_attachments = length(local.pod_identity_create_iam_roles) == 0 ? {} : merge([ + for association_key, association in local.pod_identity_create_iam_roles : merge( + { + for policy_arn in association.iam_policy_arns : "${association_key}:${policy_arn}" => { + association_key = association_key + policy_arn = policy_arn + } + }, + { + for policy_key, policy_arn in association.iam_policy_arn_map : "${association_key}:key:${policy_key}" => { + association_key = association_key + policy_arn = policy_arn + } + } + ) + ]...) + + pod_identity_iam_role_arns = { + for key, association in local.pod_identity_association_configs : + key => association.create_iam_role ? try(aws_iam_role.pod_identity[key].arn, null) : association.iam_role_arn + } + + pod_identity_external_role_names = { + for key, association in local.pod_identity_association_configs : + key => association.iam_role_arn == null ? association.iam_role_name : coalesce(association.iam_role_name_override, reverse(split("/", association.iam_role_arn))[0]) + } + + pod_identity_iam_role_names = { + for key, association in local.pod_identity_association_configs : + key => association.create_iam_role ? try(aws_iam_role.pod_identity[key].name, association.iam_role_name) : local.pod_identity_external_role_names[key] + } + + access_policy_associations = length(var.access_entries) == 0 ? {} : merge([ + for entry_key, entry in var.access_entries : { + for association_key, association in entry.policy_associations : "${entry_key}:${association_key}" => { + access_scope = association.access_scope + entry_key = entry_key + policy_arn = association.policy_arn + principal_arn = entry.principal_arn + } + } + ]...) + eks_capability_configs = { for key, capability in var.capabilities : key => { capability_name = coalesce(try(capability.capability_name, null), key) create_iam_role = try(capability.create_iam_role, true) delete_propagation_policy = try(capability.delete_propagation_policy, "RETAIN") + iam_policy_arn_map = try(capability.iam_policy_arn_map, {}) iam_policy_arns = try(capability.iam_policy_arns, toset([])) + iam_policy_presets = try(capability.iam_policy_presets, toset([])) iam_role_arn = try(capability.iam_role_arn, null) iam_role_name = coalesce(try(capability.iam_role_name, null), substr("${upper(capability.type)}CapabilityRole-${local.cluster_name}-${key}", 0, 64)) + iam_role_name_override = try(capability.iam_role_name, null) inline_policy_json = try(capability.inline_policy_json, null) type = upper(capability.type) argocd = try(capability.argocd, null) == null ? null : { @@ -42,16 +162,96 @@ locals { } } + eks_capability_iam_policy_preset_documents = { + cloudcontrol_read_only = { + Version = "2012-10-17" + Statement = [ + { + Effect = "Allow" + Action = [ + "cloudcontrol:GetResource", + "cloudcontrol:ListResourceRequests", + "cloudcontrol:ListResources" + ] + Resource = "*" + } + ] + } + eks_read_only = { + Version = "2012-10-17" + Statement = [ + { + Effect = "Allow" + Action = [ + "eks:DescribeAccessEntry", + "eks:DescribeCluster", + "eks:DescribePodIdentityAssociation", + "eks:ListAccessEntries", + "eks:ListClusters", + "eks:ListPodIdentityAssociations" + ] + Resource = "*" + } + ] + } + resource_tagging = { + Version = "2012-10-17" + Statement = [ + { + Effect = "Allow" + Action = [ + "tag:GetResources", + "tag:GetTagKeys", + "tag:GetTagValues", + "tag:TagResources", + "tag:UntagResources" + ] + Resource = "*" + } + ] + } + secrets_read_only = { + Version = "2012-10-17" + Statement = [ + { + Effect = "Allow" + Action = [ + "secretsmanager:DescribeSecret", + "secretsmanager:GetSecretValue", + "secretsmanager:ListSecrets" + ] + Resource = "*" + } + ] + } + } + eks_capabilities_enabled = length(local.eks_capability_configs) > 0 eks_capability_create_iam_roles = { for key, capability in local.eks_capability_configs : key => capability if capability.create_iam_role } eks_capability_policy_attachments = length(local.eks_capability_create_iam_roles) == 0 ? {} : merge([ + for capability_key, capability in local.eks_capability_create_iam_roles : merge( + { + for policy_arn in capability.iam_policy_arns : "${capability_key}:${policy_arn}" => { + capability_key = capability_key + policy_arn = policy_arn + } + }, + { + for policy_key, policy_arn in capability.iam_policy_arn_map : "${capability_key}:key:${policy_key}" => { + capability_key = capability_key + policy_arn = policy_arn + } + } + ) + ]...) + eks_capability_preset_policies = length(local.eks_capability_create_iam_roles) == 0 ? {} : merge([ for capability_key, capability in local.eks_capability_create_iam_roles : { - for policy_arn in capability.iam_policy_arns : "${capability_key}:${policy_arn}" => { + for preset_name in capability.iam_policy_presets : "${capability_key}:${preset_name}" => { capability_key = capability_key - policy_arn = policy_arn + preset_name = preset_name } } ]...) @@ -61,22 +261,184 @@ locals { } eks_capability_external_role_names = { for key, capability in local.eks_capability_configs : - key => capability.iam_role_arn == null ? capability.iam_role_name : coalesce(capability.iam_role_name, reverse(split("/", capability.iam_role_arn))[0]) + key => capability.iam_role_arn == null ? capability.iam_role_name : coalesce(capability.iam_role_name_override, reverse(split("/", capability.iam_role_arn))[0]) } eks_capability_iam_role_names = { for key, capability in local.eks_capability_configs : key => capability.create_iam_role ? try(aws_iam_role.eks_capability[key].name, capability.iam_role_name) : local.eks_capability_external_role_names[key] } + auto_mode_node_class_specs = { + for name, node_class in var.auto_mode_node_classes : + name => merge( + ( + try(node_class.spec.role, null) == null && + try(node_class.spec.instanceProfile, null) == null && + ( + try(node_class.node_role_arn, null) != null || + local.auto_mode_node_iam_role_name != null + ) + ) ? { + role = try(node_class.node_role_arn, null) == null ? local.auto_mode_node_iam_role_name : reverse(split("/", node_class.node_role_arn))[0] + } : {}, + node_class.spec + ) + } + + auto_mode_node_class_access_entry_configs = { + for name, node_class in var.auto_mode_node_classes : name => { + access_entry_key = coalesce(try(node_class.access_entry_key, null), name) + create_access_entry = ( + try(node_class.create_access_entry, null) != null ? + node_class.create_access_entry : + try(node_class.node_role_arn, null) != null + ) + principal_arn = try(node_class.node_role_arn, null) + } + } + auto_mode_node_class_access_entry_groups = { + for name, config in local.auto_mode_node_class_access_entry_configs : + config.access_entry_key => merge(config, { node_class_name = name })... + if config.create_access_entry + } + auto_mode_node_class_access_entries = { + for access_entry_key, configs in local.auto_mode_node_class_access_entry_groups : + access_entry_key => { + node_class_names = [for config in configs : config.node_class_name] + principal_arn = configs[0].principal_arn + } + } + + auto_mode_node_class_manifests = { + for name, node_class in var.auto_mode_node_classes : name => yamlencode({ + apiVersion = "eks.amazonaws.com/v1" + kind = "NodeClass" + metadata = merge( + { name = name }, + length(node_class.annotations) > 0 ? { annotations = node_class.annotations } : {}, + length(node_class.labels) > 0 ? { labels = node_class.labels } : {} + ) + spec = local.auto_mode_node_class_specs[name] + }) + } + + auto_mode_node_pool_manifests = { + for name, node_pool in var.auto_mode_node_pools : name => yamlencode({ + apiVersion = "karpenter.sh/v1" + kind = "NodePool" + metadata = merge( + { name = name }, + length(node_pool.annotations) > 0 ? { annotations = node_pool.annotations } : {}, + length(node_pool.labels) > 0 ? { labels = node_pool.labels } : {} + ) + spec = node_pool.spec + }) + } + + auto_mode_storage_class_manifests = { + for name, storage_class in var.auto_mode_storage_classes : name => yamlencode(merge( + { + apiVersion = "storage.k8s.io/v1" + kind = "StorageClass" + metadata = merge( + { name = name }, + length(storage_class.annotations) > 0 ? { annotations = storage_class.annotations } : {}, + length(storage_class.labels) > 0 ? { labels = storage_class.labels } : {} + ) + provisioner = "ebs.csi.eks.amazonaws.com" + parameters = merge( + { + encrypted = "true" + type = "gp3" + }, + storage_class.parameters + ) + reclaimPolicy = storage_class.reclaim_policy + volumeBindingMode = storage_class.volume_binding_mode + allowVolumeExpansion = storage_class.allow_volume_expansion + }, + length(storage_class.allowed_topologies) > 0 ? { allowedTopologies = storage_class.allowed_topologies } : {}, + length(storage_class.mount_options) > 0 ? { mountOptions = storage_class.mount_options } : {} + )) + } + + auto_mode_load_balancer_service_manifests = { + for name, service in var.auto_mode_load_balancer_services : name => yamlencode({ + apiVersion = "v1" + kind = "Service" + metadata = merge( + { + name = name + namespace = service.namespace + }, + length(service.annotations) > 0 ? { annotations = service.annotations } : {}, + length(service.labels) > 0 ? { labels = service.labels } : {} + ) + spec = merge( + { + type = "LoadBalancer" + loadBalancerClass = service.load_balancer_class + ports = service.ports + selector = service.selector + }, + service.external_traffic_policy == null ? {} : { externalTrafficPolicy = service.external_traffic_policy }, + length(service.ip_families) > 0 ? { ipFamilies = service.ip_families } : {}, + service.ip_family_policy == null ? {} : { ipFamilyPolicy = service.ip_family_policy }, + length(service.load_balancer_source_ranges) > 0 ? { loadBalancerSourceRanges = service.load_balancer_source_ranges } : {}, + service.session_affinity == null ? {} : { sessionAffinity = service.session_affinity } + ) + }) + } + + auto_mode_kubernetes_manifests = merge( + { + for name, manifest in local.auto_mode_node_class_manifests : + "nodeclass/${name}" => manifest + }, + { + for name, manifest in local.auto_mode_node_pool_manifests : + "nodepool/${name}" => manifest + }, + { + for name, manifest in local.auto_mode_storage_class_manifests : + "storageclass/${name}" => manifest + }, + { + for name, manifest in local.auto_mode_load_balancer_service_manifests : + "service/${name}" => manifest + } + ) + karpenter_enabled = var.karpenter.enabled karpenter_create_node_iam_role = local.karpenter_enabled && var.karpenter.create_node_iam_role karpenter_create_access_entry = local.karpenter_enabled && var.karpenter.create_access_entry - cluster_access_config = local.karpenter_create_access_entry || local.eks_capabilities_enabled ? { + api_authentication_required = ( + local.auto_mode_configured || + local.karpenter_create_access_entry || + local.eks_capabilities_enabled || + length(var.access_entries) > 0 || + length(local.auto_mode_node_class_access_entries) > 0 + ) + + cluster_access_config = local.api_authentication_required ? { authentication_mode = coalesce(try(var.access_config.authentication_mode, null), "API") bootstrap_cluster_creator_admin_permissions = try(var.access_config.bootstrap_cluster_creator_admin_permissions, null) } : var.access_config + managed_node_group_access_entry_enabled = ( + length(var.node_groups) > 0 && + contains(["API", "API_AND_CONFIG_MAP"], try(local.cluster_access_config.authentication_mode, "")) + ) + managed_access_entry_principal_arns = compact(concat( + [for _, entry in var.access_entries : entry.principal_arn], + [for _, entry in local.auto_mode_node_class_access_entries : entry.principal_arn], + distinct(values(local.eks_capability_iam_role_arns)), + local.managed_node_group_access_entry_enabled ? [aws_iam_role.node.arn] : [], + local.karpenter_create_access_entry ? [local.karpenter_node_role_arn] : [], + (local.auto_mode_builtin_node_pools || local.auto_mode_create_access_entry) ? [local.auto_mode_node_iam_role_arn] : [] + )) + karpenter_discovery_tag_key = "karpenter.sh/discovery" karpenter_discovery_tag_value = local.cluster_name karpenter_subnet_ids = length(var.karpenter.subnet_ids) > 0 ? var.karpenter.subnet_ids : var.subnet_ids diff --git a/main.tf b/main.tf index 7d93eee..5bab8a6 100755 --- a/main.tf +++ b/main.tf @@ -1,8 +1,10 @@ resource "aws_eks_cluster" "this" { - name = local.cluster_name - role_arn = aws_iam_role.cluster.arn - version = var.kubernetes_version - deletion_protection = var.deletion_protection + name = local.cluster_name + role_arn = aws_iam_role.cluster.arn + version = var.kubernetes_version + bootstrap_self_managed_addons = local.cluster_bootstrap_self_managed_addons + deletion_protection = var.deletion_protection + force_update_version = var.force_update_version enabled_cluster_log_types = var.enabled_cluster_log_types @@ -91,6 +93,16 @@ resource "aws_eks_cluster" "this" { } } + dynamic "compute_config" { + for_each = local.auto_mode_configured ? [var.auto_mode] : [] + + content { + enabled = local.auto_mode_enabled + node_pools = local.auto_mode_enabled ? compute_config.value.node_pools : null + node_role_arn = local.auto_mode_builtin_node_pools ? local.auto_mode_node_iam_role_arn : null + } + } + dynamic "encryption_config" { for_each = var.cluster_encryption_config == null ? [] : [var.cluster_encryption_config] @@ -104,16 +116,47 @@ resource "aws_eks_cluster" "this" { } dynamic "kubernetes_network_config" { - for_each = var.service_ipv4_cidr == null ? [] : [var.service_ipv4_cidr] + for_each = local.auto_mode_configured || var.ip_family != null || var.service_ipv4_cidr != null ? [1] : [] + + content { + ip_family = var.ip_family + service_ipv4_cidr = var.service_ipv4_cidr + + dynamic "elastic_load_balancing" { + for_each = local.auto_mode_configured ? [local.auto_mode_enabled] : [] + + content { + enabled = elastic_load_balancing.value + } + } + } + } + + dynamic "storage_config" { + for_each = local.auto_mode_configured ? [local.auto_mode_enabled] : [] content { - service_ipv4_cidr = kubernetes_network_config.value + block_storage { + enabled = storage_config.value + } + } + } + + dynamic "upgrade_policy" { + for_each = var.upgrade_policy_support_type == null ? [] : [var.upgrade_policy_support_type] + + content { + support_type = upgrade_policy.value } } tags = local.common_tags lifecycle { + # This setting is create-only. Ignoring later changes permits in-place Auto + # Mode enablement for clusters originally created with bootstrap add-ons. + ignore_changes = [bootstrap_self_managed_addons] + precondition { condition = ( try(var.kube_controller_manager_config.horizontal_pod_autoscaler_controller_config, null) == null || @@ -121,14 +164,183 @@ resource "aws_eks_cluster" "this" { ) error_message = "kube_controller_manager_config.horizontal_pod_autoscaler_controller_config requires control_plane_scaling_config.tier to be tier-xl, tier-2xl, tier-4xl, or tier-8xl." } + + precondition { + condition = ( + !local.api_authentication_required || + contains(["API", "API_AND_CONFIG_MAP"], try(local.cluster_access_config.authentication_mode, "")) + ) + error_message = "Auto Mode, EKS access entries, Karpenter access entries, and EKS capabilities require access_config.authentication_mode to be API or API_AND_CONFIG_MAP." + } + + precondition { + condition = ( + local.auto_mode_enabled || + ( + length(var.auto_mode_node_classes) == 0 && + length(var.auto_mode_node_pools) == 0 && + length(var.auto_mode_storage_classes) == 0 && + length(var.auto_mode_load_balancer_services) == 0 + ) + ) + error_message = "Auto Mode Kubernetes manifests require auto_mode.enabled to be true." + } + + precondition { + condition = alltrue([ + for _, node_class in var.auto_mode_node_classes : + try(node_class.spec.role, null) != null || + try(node_class.spec.instanceProfile, null) != null || + try(node_class.node_role_arn, null) != null || + local.auto_mode_node_iam_role_arn != null + ]) + error_message = "Each Auto Mode NodeClass must specify a role, instanceProfile, or node_role_arn unless auto_mode supplies a node IAM role." + } + + precondition { + condition = !var.endpoint_public_access || length(var.public_access_cidrs) > 0 + error_message = "public_access_cidrs must contain at least one CIDR when endpoint_public_access is true." + } + + precondition { + condition = var.ip_family != "ipv6" || var.service_ipv4_cidr == null + error_message = "service_ipv4_cidr cannot be set when ip_family is ipv6." + } + + precondition { + condition = ( + local.cluster_bootstrap_self_managed_addons || + length(var.node_groups) == 0 || + !contains(keys(var.addons), "vpc-cni") || + try(var.addons["vpc-cni"].before_compute, true) + ) + error_message = "Clusters without bootstrapped self-managed add-ons must set addons[\"vpc-cni\"].before_compute to true when this module manages VPC CNI for managed node groups." + } + + precondition { + condition = !local.auto_mode_enabled || !local.cluster_bootstrap_self_managed_addons + error_message = "EKS Auto Mode requires bootstrap_self_managed_addons to be false." + } + + precondition { + condition = length(distinct(local.managed_access_entry_principal_arns)) == length(local.managed_access_entry_principal_arns) + error_message = "Each module- or service-managed EKS access entry must use a unique principal ARN. Reuse service-managed entries, or give NodeClasses sharing a custom role the same access_entry_key." + } + + precondition { + condition = alltrue([ + for _, configs in local.auto_mode_node_class_access_entry_groups : + length(distinct([for config in configs : config.principal_arn])) == 1 + ]) + error_message = "NodeClasses sharing an access_entry_key must use the same node_role_arn." + } } depends_on = [ aws_cloudwatch_log_group.cluster, + aws_iam_role_policy_attachment.auto_mode_cluster_additional, + aws_iam_role_policy_attachment.auto_mode_node, + aws_iam_role_policy_attachment.auto_mode_node_additional, aws_iam_role_policy_attachment.cluster ] } +resource "aws_eks_access_entry" "this" { + for_each = var.access_entries + + cluster_name = aws_eks_cluster.this.name + kubernetes_groups = each.value.kubernetes_groups + principal_arn = each.value.principal_arn + tags = local.common_tags + type = each.value.type + user_name = each.value.user_name +} + +resource "aws_eks_access_policy_association" "this" { + for_each = local.access_policy_associations + + cluster_name = aws_eks_cluster.this.name + policy_arn = each.value.policy_arn + principal_arn = each.value.principal_arn + + access_scope { + namespaces = each.value.access_scope.namespaces + type = each.value.access_scope.type + } + + depends_on = [aws_eks_access_entry.this] +} + +resource "aws_eks_access_entry" "auto_mode_node" { + count = local.auto_mode_create_access_entry ? 1 : 0 + + cluster_name = aws_eks_cluster.this.name + principal_arn = local.auto_mode_node_iam_role_arn + tags = local.common_tags + type = "EC2" +} + +resource "aws_eks_access_policy_association" "auto_mode_node" { + count = local.auto_mode_create_access_entry ? 1 : 0 + + cluster_name = aws_eks_cluster.this.name + policy_arn = "arn:${data.aws_partition.current.partition}:eks::aws:cluster-access-policy/AmazonEKSAutoNodePolicy" + principal_arn = local.auto_mode_node_iam_role_arn + + access_scope { + type = "cluster" + } + + depends_on = [aws_eks_access_entry.auto_mode_node] +} + +resource "aws_eks_access_entry" "auto_mode_node_class" { + for_each = local.auto_mode_node_class_access_entries + + cluster_name = aws_eks_cluster.this.name + principal_arn = each.value.principal_arn + tags = local.common_tags + type = "EC2" + + lifecycle { + precondition { + condition = each.value.principal_arn != null + error_message = "Custom Auto Mode NodeClass access entries require auto_mode_node_classes[*].node_role_arn." + } + } +} + +resource "aws_eks_access_policy_association" "auto_mode_node_class" { + for_each = local.auto_mode_node_class_access_entries + + cluster_name = aws_eks_cluster.this.name + policy_arn = "arn:${data.aws_partition.current.partition}:eks::aws:cluster-access-policy/AmazonEKSAutoNodePolicy" + principal_arn = each.value.principal_arn + + access_scope { + type = "cluster" + } + + depends_on = [aws_eks_access_entry.auto_mode_node_class] +} + +resource "aws_eks_pod_identity_association" "this" { + for_each = local.pod_identity_association_configs + + cluster_name = aws_eks_cluster.this.name + disable_session_tags = each.value.disable_session_tags + namespace = each.value.namespace + role_arn = local.pod_identity_iam_role_arns[each.key] + service_account = each.value.service_account + tags = merge(local.common_tags, each.value.tags) + target_role_arn = each.value.target_role_arn + + depends_on = [ + aws_iam_role_policy.pod_identity, + aws_iam_role_policy_attachment.pod_identity + ] +} + resource "aws_ec2_tag" "karpenter_cluster_security_group" { count = local.karpenter_enabled && var.karpenter.tag_cluster_security_group ? 1 : 0 @@ -220,6 +432,7 @@ resource "aws_eks_capability" "this" { depends_on = [ aws_iam_role_policy.eks_capability, + aws_iam_role_policy.eks_capability_preset, aws_iam_role_policy_attachment.eks_capability ] } @@ -290,12 +503,35 @@ resource "aws_eks_node_group" "this" { ) depends_on = [ + aws_eks_addon.before_compute, aws_iam_role_policy_attachment.node ] } +resource "aws_eks_addon" "before_compute" { + for_each = local.addons_before_compute + + cluster_name = aws_eks_cluster.this.name + addon_name = each.key + addon_version = each.value.version + configuration_values = each.value.configuration_values + resolve_conflicts_on_create = each.value.resolve_conflicts_on_create + resolve_conflicts_on_update = each.value.resolve_conflicts_on_update + service_account_role_arn = each.value.service_account_role_arn + tags = local.common_tags + + dynamic "pod_identity_association" { + for_each = each.value.pod_identity_associations + + content { + role_arn = pod_identity_association.value.role_arn + service_account = pod_identity_association.value.service_account + } + } +} + resource "aws_eks_addon" "this" { - for_each = var.addons + for_each = local.addons_after_compute cluster_name = aws_eks_cluster.this.name addon_name = each.key diff --git a/outputs.tf b/outputs.tf index b02606a..50dc0f9 100755 --- a/outputs.tf +++ b/outputs.tf @@ -8,6 +8,11 @@ output "cluster_arn" { value = aws_eks_cluster.this.arn } +output "cluster_created_at" { + description = "Timestamp when the EKS cluster was created." + value = aws_eks_cluster.this.created_at +} + output "cluster_endpoint" { description = "Kubernetes API server endpoint." value = aws_eks_cluster.this.endpoint @@ -39,11 +44,116 @@ output "cluster_iam_role_arn" { value = aws_iam_role.cluster.arn } +output "cluster_platform_version" { + description = "EKS platform version." + value = aws_eks_cluster.this.platform_version +} + +output "cluster_status" { + description = "EKS cluster status." + value = aws_eks_cluster.this.status +} + +output "cluster_tags_all" { + description = "All tags applied to the EKS cluster, including provider default tags." + value = aws_eks_cluster.this.tags_all +} + +output "cluster_version" { + description = "Kubernetes version running on the EKS cluster." + value = aws_eks_cluster.this.version +} + output "node_iam_role_arn" { description = "IAM role ARN used by managed node groups." value = aws_iam_role.node.arn } +output "auto_mode_enabled" { + description = "Whether EKS Auto Mode is enabled by this module configuration." + value = local.auto_mode_enabled +} + +output "auto_mode_node_iam_role_arn" { + description = "IAM role ARN used by EKS Auto Mode managed compute when Auto Mode is configured." + value = local.auto_mode_configured ? local.auto_mode_node_iam_role_arn : null +} + +output "auto_mode_node_iam_role_name" { + description = "IAM role name used by EKS Auto Mode managed compute when Auto Mode is configured." + value = local.auto_mode_configured ? local.auto_mode_node_iam_role_name : null +} + +output "auto_mode_node_access_entry_arn" { + description = "EKS access entry ARN created for the Auto Mode node role when built-in node pools are disabled. Built-in node pools register their role automatically." + value = try(aws_eks_access_entry.auto_mode_node[0].access_entry_arn, null) +} + +output "access_entry_arns" { + description = "EKS access entry ARNs by access entry key." + value = { for key, entry in aws_eks_access_entry.this : key => entry.access_entry_arn } +} + +output "auto_mode_node_class_access_entry_arns" { + description = "EKS access entry ARNs created for custom Auto Mode NodeClass IAM roles, keyed by access_entry_key (or NodeClass name by default)." + value = { for key, entry in aws_eks_access_entry.auto_mode_node_class : key => entry.access_entry_arn } +} + +output "auto_mode_node_class_manifests" { + description = "Rendered EKS Auto Mode NodeClass YAML manifests by name." + value = local.auto_mode_node_class_manifests +} + +output "auto_mode_node_pool_manifests" { + description = "Rendered EKS Auto Mode NodePool YAML manifests by name." + value = local.auto_mode_node_pool_manifests +} + +output "auto_mode_storage_class_manifests" { + description = "Rendered EKS Auto Mode StorageClass YAML manifests by name." + value = local.auto_mode_storage_class_manifests +} + +output "auto_mode_load_balancer_service_manifests" { + description = "Rendered EKS Auto Mode LoadBalancer Service YAML manifests by name." + value = local.auto_mode_load_balancer_service_manifests +} + +output "auto_mode_kubernetes_manifests" { + description = "All rendered EKS Auto Mode Kubernetes YAML manifests by kind/name." + value = local.auto_mode_kubernetes_manifests +} + +output "auto_mode_kubernetes_manifest_yaml" { + description = "All rendered EKS Auto Mode Kubernetes manifests joined into a single multi-document YAML string." + value = length(local.auto_mode_kubernetes_manifests) == 0 ? "" : join("---\n", values(local.auto_mode_kubernetes_manifests)) +} + +output "pod_identity_association_arns" { + description = "EKS Pod Identity association ARNs by association key." + value = { for key, association in aws_eks_pod_identity_association.this : key => association.association_arn } +} + +output "pod_identity_association_ids" { + description = "EKS Pod Identity association IDs by association key." + value = { for key, association in aws_eks_pod_identity_association.this : key => association.association_id } +} + +output "pod_identity_external_ids" { + description = "External IDs generated for EKS Pod Identity associations by association key." + value = { for key, association in aws_eks_pod_identity_association.this : key => association.external_id } +} + +output "pod_identity_iam_role_arns" { + description = "IAM role ARNs used by EKS Pod Identity associations by association key." + value = local.pod_identity_iam_role_arns +} + +output "pod_identity_iam_role_names" { + description = "IAM role names used by EKS Pod Identity associations by association key." + value = local.pod_identity_iam_role_names +} + output "capability_arns" { description = "Amazon EKS capability ARNs by capability key." value = { for key, capability in aws_eks_capability.this : key => capability.arn } @@ -69,6 +179,11 @@ output "capability_iam_role_names" { value = local.eks_capability_iam_role_names } +output "capability_iam_policy_preset_names" { + description = "Capability IAM policy preset names attached by capability key." + value = { for key, capability in local.eks_capability_configs : key => capability.iam_policy_presets } +} + output "argocd_server_urls" { description = "Managed Argo CD server URLs by ARGOCD capability key." value = { @@ -112,7 +227,10 @@ output "node_group_arns" { output "addon_arns" { description = "EKS add-on ARNs by add-on name." - value = { for key, addon in aws_eks_addon.this : key => addon.arn } + value = merge( + { for key, addon in aws_eks_addon.before_compute : key => addon.arn }, + { for key, addon in aws_eks_addon.this : key => addon.arn } + ) } output "update_kubeconfig_command" { diff --git a/tests/auto-mode-cluster.tftest.hcl b/tests/auto-mode-cluster.tftest.hcl new file mode 100644 index 0000000..8e14d08 --- /dev/null +++ b/tests/auto-mode-cluster.tftest.hcl @@ -0,0 +1,482 @@ +mock_provider "aws" { + override_during = plan + + mock_data "aws_partition" { + defaults = { + partition = "aws" + } + } + + mock_data "aws_iam_policy_document" { + defaults = { + json = "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"eks.amazonaws.com\"},\"Action\":[\"sts:AssumeRole\",\"sts:TagSession\"]}]}" + } + } + + mock_resource "aws_iam_role" { + defaults = { + arn = "arn:aws:iam::123456789012:role/mock-eks-role" + name = "mock-eks-role" + } + } +} + +run "standard_mode_remains_the_default" { + command = plan + + variables { + name = "unit-standard-v2" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + } + + assert { + condition = length(aws_eks_cluster.this.compute_config) == 0 + error_message = "Omitting auto_mode must not add an Auto Mode compute configuration to an existing standard cluster." + } + + assert { + condition = length(aws_eks_cluster.this.storage_config) == 0 + error_message = "Omitting auto_mode must not add an Auto Mode storage configuration to an existing standard cluster." + } + + assert { + condition = aws_eks_cluster.this.bootstrap_self_managed_addons == true + error_message = "Standard clusters must continue bootstrapping self-managed add-ons by default." + } + + assert { + condition = length(aws_iam_role.auto_mode_node) == 0 + error_message = "Omitting auto_mode must not create an Auto Mode node IAM role." + } + + assert { + condition = output.auto_mode_enabled == false && output.auto_mode_node_iam_role_arn == null + error_message = "Auto Mode outputs must remain disabled and null for existing standard callers." + } + + assert { + condition = !contains(keys(aws_iam_role_policy_attachment.cluster), "arn:aws:iam::aws:policy/AmazonEKSBlockStoragePolicyV2") + error_message = "Omitting auto_mode must not add Auto Mode policies to the cluster role." + } + + assert { + condition = toset(keys(aws_eks_node_group.this)) == toset(["default"]) + error_message = "The standard module must retain its default managed node group." + } + + assert { + condition = toset(keys(aws_eks_addon.this)) == toset(["coredns", "kube-proxy", "vpc-cni"]) + error_message = "The standard module must retain its default EKS add-ons." + } + + assert { + condition = length(aws_eks_addon.before_compute) == 0 + error_message = "Standard default add-ons must retain their post-compute ordering." + } + + assert { + condition = aws_eks_cluster.this.vpc_config[0].endpoint_public_access == true + error_message = "The standard cluster API endpoint must remain public by default." + } +} + +run "auto_only_cluster_shape" { + command = plan + + variables { + name = "unit-auto-only" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = {} + node_groups = {} + addons = {} + } + + assert { + condition = aws_eks_cluster.this.bootstrap_self_managed_addons == false + error_message = "An Auto-only cluster must not bootstrap the legacy self-managed add-ons." + } + + assert { + condition = aws_eks_cluster.this.access_config[0].authentication_mode == "API" + error_message = "Auto Mode must derive API authentication when access_config is omitted." + } + + assert { + condition = aws_eks_cluster.this.compute_config[0].enabled == true + error_message = "Auto Mode compute must be enabled." + } + + assert { + condition = toset(aws_eks_cluster.this.compute_config[0].node_pools) == toset(["general-purpose", "system"]) + error_message = "Auto Mode must enable the general-purpose and system built-in node pools by default." + } + + assert { + condition = aws_eks_cluster.this.kubernetes_network_config[0].elastic_load_balancing[0].enabled == true + error_message = "Auto Mode load balancing must be enabled with Auto Mode compute." + } + + assert { + condition = aws_eks_cluster.this.storage_config[0].block_storage[0].enabled == true + error_message = "Auto Mode block storage must be enabled with Auto Mode compute." + } + + assert { + condition = length(aws_iam_role.auto_mode_node) == 1 + error_message = "Auto Mode must create a dedicated node IAM role by default." + } + + assert { + condition = aws_iam_role.auto_mode_node[0].name == "unit-auto-only-auto-node-role" + error_message = "The default Auto Mode node role name must be predictable." + } + + assert { + condition = output.auto_mode_enabled == true && output.auto_mode_node_iam_role_name == "unit-auto-only-auto-node-role" + error_message = "Auto Mode outputs must expose the enabled state and node role name." + } + + assert { + condition = contains(keys(aws_iam_role_policy_attachment.auto_mode_node), "arn:aws:iam::aws:policy/AmazonEKSWorkerNodeMinimalPolicy") + error_message = "The Auto Mode node role must include AmazonEKSWorkerNodeMinimalPolicy." + } + + assert { + condition = contains(keys(aws_iam_role_policy_attachment.auto_mode_node), "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryPullOnly") + error_message = "The Auto Mode node role must include AmazonEC2ContainerRegistryPullOnly." + } + + assert { + condition = contains(keys(aws_iam_role_policy_attachment.cluster), "arn:aws:iam::aws:policy/AmazonEKSBlockStoragePolicyV2") + error_message = "The cluster role must use AmazonEKSBlockStoragePolicyV2 for Auto Mode." + } + + assert { + condition = !contains(keys(aws_iam_role_policy_attachment.cluster), "arn:aws:iam::aws:policy/AmazonEKSBlockStoragePolicy") + error_message = "The superseded AmazonEKSBlockStoragePolicy must not be attached for Auto Mode." + } + + assert { + condition = contains(keys(aws_iam_role_policy_attachment.cluster), "arn:aws:iam::aws:policy/AmazonEKSComputePolicy") + error_message = "The cluster role must include AmazonEKSComputePolicy for Auto Mode." + } + + assert { + condition = contains(keys(aws_iam_role_policy_attachment.cluster), "arn:aws:iam::aws:policy/AmazonEKSLoadBalancingPolicy") + error_message = "The cluster role must include AmazonEKSLoadBalancingPolicy for Auto Mode." + } + + assert { + condition = contains(keys(aws_iam_role_policy_attachment.cluster), "arn:aws:iam::aws:policy/AmazonEKSNetworkingPolicy") + error_message = "The cluster role must include AmazonEKSNetworkingPolicy for Auto Mode." + } + + assert { + condition = contains(flatten([for statement in data.aws_iam_policy_document.cluster_assume_role.statement : statement.actions]), "sts:TagSession") + error_message = "The EKS cluster role trust policy must permit sts:TagSession for Auto Mode." + } + + assert { + condition = length(aws_eks_node_group.this) == 0 && length(aws_eks_addon.this) == 0 + error_message = "An Auto-only caller must be able to omit managed node groups and traditional add-ons explicitly." + } + + assert { + condition = aws_eks_cluster.this.vpc_config[0].endpoint_public_access == true + error_message = "Auto Mode support must keep the cluster API endpoint public by default." + } + + assert { + condition = contains(aws_eks_cluster.this.vpc_config[0].public_access_cidrs, "0.0.0.0/0") + error_message = "Auto Mode support must retain the module's default public API CIDR." + } +} + +run "auto_mode_without_builtin_node_pools" { + command = plan + + variables { + name = "unit-auto-custom-pools" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = { + node_pools = [] + } + node_groups = {} + addons = {} + } + + assert { + condition = aws_eks_cluster.this.compute_config[0].enabled == true + error_message = "Auto Mode must remain enabled when no built-in node pools are selected." + } + + assert { + condition = length(aws_eks_cluster.this.compute_config[0].node_pools) == 0 + error_message = "An empty node_pools set must disable both built-in Auto Mode node pools." + } + + assert { + condition = aws_eks_cluster.this.compute_config[0].node_role_arn == null + error_message = "EKS compute_config must omit node_role_arn when no built-in node pools are selected." + } + + assert { + condition = length(aws_iam_role.auto_mode_node) == 1 + error_message = "The optional Auto Mode node role must remain available for custom NodeClass manifests." + } + + assert { + condition = length(aws_eks_access_entry.auto_mode_node) == 1 && aws_eks_access_entry.auto_mode_node[0].type == "EC2" + error_message = "Without built-in node pools, the Auto Mode node role must receive an explicit EC2 access entry for custom NodeClasses." + } + + assert { + condition = aws_eks_cluster.this.kubernetes_network_config[0].elastic_load_balancing[0].enabled == true && aws_eks_cluster.this.storage_config[0].block_storage[0].enabled == true + error_message = "Disabling built-in node pools must not disable other Auto Mode capabilities." + } +} + +run "auto_mode_external_global_access_entry_can_be_suppressed" { + command = plan + + variables { + name = "unit-auto-external-entry" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = { + create_access_entry = false + create_node_iam_role = false + node_iam_role_arn = "arn:aws:iam::123456789012:role/external-auto-node" + node_pools = [] + } + node_groups = {} + addons = {} + + access_entries = { + external_auto_node = { + principal_arn = "arn:aws:iam::123456789012:role/external-auto-node" + type = "EC2" + policy_associations = { + auto_node = { + policy_arn = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSAutoNodePolicy" + access_scope = { + type = "cluster" + } + } + } + } + } + } + + assert { + condition = length(aws_eks_access_entry.auto_mode_node) == 0 && length(aws_eks_access_policy_association.auto_mode_node) == 0 + error_message = "auto_mode.create_access_entry=false must suppress the module's global Auto Mode EC2 access resources." + } + + assert { + condition = aws_eks_access_entry.this["external_auto_node"].principal_arn == "arn:aws:iam::123456789012:role/external-auto-node" && aws_eks_access_entry.this["external_auto_node"].type == "EC2" + error_message = "Suppressing the global entry must permit the same external role to be managed through access_entries." + } + + assert { + condition = aws_eks_access_policy_association.this["external_auto_node:auto_node"].policy_arn == "arn:aws:eks::aws:cluster-access-policy/AmazonEKSAutoNodePolicy" + error_message = "The caller-managed external role must retain its Auto Node policy association." + } +} + +run "auto_mode_external_global_access_entry_can_be_forced" { + command = plan + + variables { + name = "unit-auto-forced-entry" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = { + create_access_entry = true + create_node_iam_role = false + node_iam_role_arn = "arn:aws:iam::123456789012:role/external-auto-node" + node_pools = [] + } + node_groups = {} + addons = {} + } + + assert { + condition = length(aws_eks_access_entry.auto_mode_node) == 1 && aws_eks_access_entry.auto_mode_node[0].principal_arn == "arn:aws:iam::123456789012:role/external-auto-node" && aws_eks_access_entry.auto_mode_node[0].type == "EC2" + error_message = "auto_mode.create_access_entry=true must create the global EC2 access entry for an external role." + } + + assert { + condition = length(aws_eks_access_policy_association.auto_mode_node) == 1 && aws_eks_access_policy_association.auto_mode_node[0].policy_arn == "arn:aws:eks::aws:cluster-access-policy/AmazonEKSAutoNodePolicy" + error_message = "A forced global entry must receive AmazonEKSAutoNodePolicy." + } +} + +run "auto_mode_supports_hybrid_compute" { + command = plan + + variables { + name = "unit-auto-hybrid" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = { + node_pools = ["system"] + } + + node_groups = { + workloads = { + desired_size = 1 + min_size = 1 + max_size = 2 + } + } + + addons = { + coredns = {} + vpc-cni = { + before_compute = true + } + } + } + + override_resource { + target = aws_iam_role.node + override_during = plan + values = { + arn = "arn:aws:iam::123456789012:role/unit-auto-hybrid-managed-node-role" + } + } + + assert { + condition = aws_eks_cluster.this.compute_config[0].enabled == true + error_message = "Hybrid clusters must keep Auto Mode compute enabled." + } + + assert { + condition = toset(aws_eks_cluster.this.compute_config[0].node_pools) == toset(["system"]) + error_message = "Hybrid clusters must honor the selected built-in Auto Mode node pools." + } + + assert { + condition = toset(keys(aws_eks_node_group.this)) == toset(["workloads"]) + error_message = "Hybrid clusters must retain caller-configured managed node groups." + } + + assert { + condition = toset(keys(aws_eks_addon.this)) == toset(["coredns"]) + error_message = "Hybrid clusters must create post-compute add-ons in the regular add-on resource." + } + + assert { + condition = toset(keys(aws_eks_addon.before_compute)) == toset(["vpc-cni"]) + error_message = "Hybrid clusters must create the VPC CNI before managed node-group compute." + } + + assert { + condition = toset(keys(output.addon_arns)) == toset(["coredns", "vpc-cni"]) + error_message = "The add-on ARN output must merge before-compute and post-compute add-ons." + } + + assert { + condition = aws_iam_role.node.name == "unit-auto-hybrid-node-role" && length(aws_iam_role.auto_mode_node) == 1 + error_message = "Hybrid clusters must keep separate managed-node-group and Auto Mode node IAM roles." + } + + assert { + condition = aws_eks_cluster.this.vpc_config[0].endpoint_public_access == true + error_message = "Hybrid Auto Mode support must keep the cluster API endpoint public by default." + } +} + +run "auto_mode_can_be_explicitly_disabled" { + command = plan + + variables { + name = "unit-auto-disabled" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = { + enabled = false + } + addons = { + coredns = {} + kube-proxy = {} + vpc-cni = { + before_compute = true + } + } + } + + assert { + condition = aws_eks_cluster.this.compute_config[0].enabled == false + error_message = "Explicit disablement must send compute enabled=false to EKS." + } + + assert { + condition = aws_eks_cluster.this.bootstrap_self_managed_addons == false + error_message = "An explicitly configured Auto Mode lifecycle must keep bootstrap self-managed add-ons disabled." + } + + assert { + condition = aws_eks_cluster.this.access_config[0].authentication_mode == "API" + error_message = "The explicit-disable transition must continue managing the API authentication required by Auto Mode." + } + + assert { + condition = aws_eks_cluster.this.compute_config[0].node_role_arn == null && try(length(aws_eks_cluster.this.compute_config[0].node_pools), 0) == 0 + error_message = "Explicit disablement must clear Auto Mode's node role and built-in node pools." + } + + assert { + condition = aws_eks_cluster.this.kubernetes_network_config[0].elastic_load_balancing[0].enabled == false + error_message = "Explicit disablement must send load balancing enabled=false to EKS." + } + + assert { + condition = aws_eks_cluster.this.storage_config[0].block_storage[0].enabled == false + error_message = "Explicit disablement must send block storage enabled=false to EKS." + } + + assert { + condition = length(aws_iam_role.auto_mode_node) == 1 + error_message = "The explicit-disable transition must retain the Auto Mode node role until auto_mode is set back to null." + } + + assert { + condition = contains(keys(aws_iam_role_policy_attachment.cluster), "arn:aws:iam::aws:policy/AmazonEKSBlockStoragePolicyV2") + error_message = "The explicit-disable transition must retain Auto Mode cluster policies until auto_mode is set back to null." + } + + assert { + condition = toset(keys(aws_eks_node_group.this)) == toset(["default"]) + error_message = "Disabling Auto Mode must leave standard managed node groups available." + } + + assert { + condition = aws_eks_cluster.this.vpc_config[0].endpoint_public_access == true + error_message = "Disabling Auto Mode must not make the cluster endpoint private." + } +} diff --git a/tests/auto-mode-integrations.tftest.hcl b/tests/auto-mode-integrations.tftest.hcl new file mode 100644 index 0000000..44b9e05 --- /dev/null +++ b/tests/auto-mode-integrations.tftest.hcl @@ -0,0 +1,535 @@ +mock_provider "aws" { + override_during = plan + + mock_data "aws_partition" { + defaults = { + partition = "aws" + } + } + + mock_data "aws_iam_policy_document" { + defaults = { + json = "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"eks.amazonaws.com\"},\"Action\":[\"sts:AssumeRole\",\"sts:TagSession\"]}]}" + } + } + + mock_resource "aws_iam_role" { + defaults = { + arn = "arn:aws:iam::123456789012:role/mock-eks-role" + name = "mock-eks-role" + } + } +} + +run "auto_mode_access_pod_identity_and_manifests" { + command = plan + + variables { + name = "unit-auto-integrations" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = { + cluster_iam_policy_arn_map = { + custom_cluster = "arn:aws:iam::aws:policy/AmazonEKSVPCResourceController" + } + node_iam_policy_arn_map = { + ssm = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore" + } + node_iam_policy_arns = [ + "arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy" + ] + } + node_groups = {} + addons = {} + + access_entries = { + platform_admin = { + principal_arn = "arn:aws:iam::123456789012:role/platform-admin" + policy_associations = { + admin = { + policy_arn = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy" + access_scope = { + type = "cluster" + } + } + } + } + + namespace_viewer = { + principal_arn = "arn:aws:iam::123456789012:role/namespace-viewer" + policy_associations = { + view = { + policy_arn = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSViewPolicy" + access_scope = { + type = "namespace" + namespaces = ["apps"] + } + } + } + } + + custom_compute = { + principal_arn = "arn:aws:iam::123456789012:role/generic-ec2-node" + type = "EC2" + policy_associations = { + auto_node = { + policy_arn = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSAutoNodePolicy" + access_scope = { + type = "cluster" + } + } + } + } + } + + pod_identity_associations = { + web = { + namespace = "apps" + service_account = "web" + iam_policy_arn_map = { + xray = "arn:aws:iam::aws:policy/AWSXRayDaemonWriteAccess" + } + iam_policy_arns = [ + "arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy" + ] + } + + external = { + create_iam_role = false + iam_role_arn = "arn:aws:iam::123456789012:role/external-pod-role" + namespace = "apps" + service_account = "external" + } + } + + auto_mode_node_classes = { + custom = { + node_role_arn = "arn:aws:iam::123456789012:role/CustomAutoNodeRole" + spec = { + role = "CustomAutoNodeRole" + subnetSelectorTerms = [ + { + tags = { + Tier = "private" + } + } + ] + } + } + + } + + auto_mode_node_pools = { + custom = { + spec = { + template = { + spec = { + nodeClassRef = { + group = "eks.amazonaws.com" + kind = "NodeClass" + name = "custom" + } + requirements = [ + { + key = "karpenter.sh/capacity-type" + operator = "In" + values = ["on-demand"] + } + ] + } + } + } + } + } + + auto_mode_storage_classes = { + "gp3-encrypted" = { + parameters = { + iops = "5000" + type = "gp3" + } + } + } + + auto_mode_load_balancer_services = { + web = { + namespace = "apps" + selector = { + app = "web" + } + ports = [ + { + name = "http" + port = 80 + protocol = "TCP" + targetPort = 8080 + } + ] + } + } + } + + assert { + condition = aws_eks_cluster.this.access_config[0].authentication_mode == "API" + error_message = "Auto Mode integrations must use API authentication by default." + } + + assert { + condition = toset(keys(aws_eks_access_entry.this)) == toset(["custom_compute", "namespace_viewer", "platform_admin"]) + error_message = "Every configured generic access entry must be created." + } + + assert { + condition = aws_eks_access_entry.this["custom_compute"].type == "EC2" + error_message = "Generic access entries must support the EC2 type used by Auto Mode custom compute." + } + + assert { + condition = aws_eks_access_policy_association.this["custom_compute:auto_node"].policy_arn == "arn:aws:eks::aws:cluster-access-policy/AmazonEKSAutoNodePolicy" + error_message = "Generic EC2 access entries must support the cluster-scoped AmazonEKSAutoNodePolicy association." + } + + assert { + condition = aws_eks_access_policy_association.this["platform_admin:admin"].access_scope[0].type == "cluster" + error_message = "Cluster-scoped access policy associations must be preserved." + } + + assert { + condition = contains(aws_eks_access_policy_association.this["namespace_viewer:view"].access_scope[0].namespaces, "apps") + error_message = "Namespace-scoped access policy associations must preserve their namespaces." + } + + assert { + condition = contains(keys(aws_iam_role_policy_attachment.auto_mode_node), "arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy") + error_message = "Caller-supplied Auto Mode node policy sets must remain supported." + } + + assert { + condition = aws_iam_role_policy_attachment.auto_mode_cluster_additional["custom_cluster"].policy_arn == "arn:aws:iam::aws:policy/AmazonEKSVPCResourceController" + error_message = "Auto Mode cluster policy maps must use caller-supplied stable attachment keys." + } + + assert { + condition = aws_iam_role_policy_attachment.auto_mode_node_additional["ssm"].policy_arn == "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore" + error_message = "Auto Mode node policy maps must use caller-supplied stable attachment keys." + } + + assert { + condition = yamldecode(local.auto_mode_node_class_manifests["custom"]).kind == "NodeClass" + error_message = "The module must render custom Auto Mode NodeClass YAML." + } + + assert { + condition = yamldecode(local.auto_mode_node_class_manifests["custom"]).spec.role == "CustomAutoNodeRole" + error_message = "A custom NodeClass must preserve its explicitly configured node role name." + } + + assert { + condition = yamldecode(local.auto_mode_node_pool_manifests["custom"]).kind == "NodePool" + error_message = "The module must render custom Auto Mode NodePool YAML." + } + + assert { + condition = yamldecode(local.auto_mode_storage_class_manifests["gp3-encrypted"]).provisioner == "ebs.csi.eks.amazonaws.com" + error_message = "The module must render EKS Auto Mode EBS StorageClass YAML." + } + + assert { + condition = yamldecode(local.auto_mode_storage_class_manifests["gp3-encrypted"]).parameters.encrypted == "true" + error_message = "Rendered Auto Mode storage classes must default to encryption." + } + + assert { + condition = yamldecode(local.auto_mode_load_balancer_service_manifests["web"]).spec.loadBalancerClass == "eks.amazonaws.com/nlb" + error_message = "The module must render EKS Auto Mode NLB Service YAML." + } + + assert { + condition = toset(keys(local.auto_mode_kubernetes_manifests)) == toset(["nodeclass/custom", "nodepool/custom", "service/web", "storageclass/gp3-encrypted"]) + error_message = "The aggregate manifest map must contain every rendered Auto Mode object." + } + + assert { + condition = output.auto_mode_kubernetes_manifests == local.auto_mode_kubernetes_manifests && output.auto_mode_kubernetes_manifest_yaml != "" + error_message = "The module outputs must expose both map and multi-document forms of the rendered manifests." + } + + assert { + condition = aws_eks_access_entry.auto_mode_node_class["custom"].type == "EC2" && aws_eks_access_entry.auto_mode_node_class["custom"].principal_arn == "arn:aws:iam::123456789012:role/CustomAutoNodeRole" + error_message = "A NodeClass with a known role ARN must derive an EC2 access entry keyed by its NodeClass name." + } + + assert { + condition = aws_eks_access_policy_association.auto_mode_node_class["custom"].policy_arn == "arn:aws:eks::aws:cluster-access-policy/AmazonEKSAutoNodePolicy" + error_message = "The derived NodeClass access entry key must also key its AmazonEKSAutoNodePolicy association." + } + + assert { + condition = contains(keys(output.auto_mode_node_class_access_entry_arns), "custom") + error_message = "The NodeClass access-entry output must use the derived stable NodeClass key." + } + + assert { + condition = length(aws_eks_access_entry.auto_mode_node) == 0 + error_message = "The module must not duplicate the access entry that EKS manages for built-in Auto Mode node pools." + } + + assert { + condition = toset(keys(aws_eks_pod_identity_association.this)) == toset(["external", "web"]) + error_message = "Every configured Pod Identity association must be created." + } + + assert { + condition = aws_eks_pod_identity_association.this["web"].namespace == "apps" && aws_eks_pod_identity_association.this["web"].service_account == "web" + error_message = "Pod Identity must target the configured namespace and service account." + } + + assert { + condition = contains(keys(aws_iam_role_policy_attachment.pod_identity), "web:arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy") + error_message = "Pod Identity policy sets must remain supported." + } + + assert { + condition = aws_iam_role_policy_attachment.pod_identity["web:key:xray"].policy_arn == "arn:aws:iam::aws:policy/AWSXRayDaemonWriteAccess" + error_message = "Pod Identity policy maps must create attachments with stable association and policy keys." + } + + assert { + condition = length(aws_iam_role.pod_identity) == 1 + error_message = "Pod Identity must create IAM roles only for associations that request one." + } + + assert { + condition = aws_eks_pod_identity_association.this["external"].role_arn == "arn:aws:iam::123456789012:role/external-pod-role" + error_message = "Pod Identity must support externally managed IAM roles." + } + + assert { + condition = output.pod_identity_iam_role_arns["external"] == "arn:aws:iam::123456789012:role/external-pod-role" + error_message = "Pod Identity outputs must expose external IAM role ARNs." + } + + assert { + condition = output.pod_identity_iam_role_names["external"] == "external-pod-role" + error_message = "Pod Identity outputs must derive an external role name from the final ARN path segment." + } + + assert { + condition = aws_eks_cluster.this.vpc_config[0].endpoint_public_access == true + error_message = "Adding Auto Mode integrations must keep the public cluster endpoint enabled." + } +} + +run "auto_mode_node_class_defaults_to_module_role" { + command = plan + + variables { + name = "unit-auto-node-class-default" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = {} + node_groups = {} + addons = {} + + auto_mode_node_classes = { + defaulted = { + spec = { + ephemeralStorage = { + size = "60Gi" + } + } + } + } + } + + assert { + condition = yamldecode(local.auto_mode_node_class_manifests["defaulted"]).spec.role == "unit-auto-node-class-default-auto-node-role" + error_message = "A custom NodeClass must default to the module-created Auto Mode node role name." + } + + assert { + condition = length(aws_eks_access_entry.auto_mode_node_class) == 0 + error_message = "Omitting create_access_entry and node_role_arn must avoid an access entry while still rendering the NodeClass with the global role." + } +} + +run "auto_mode_custom_only_without_global_node_role" { + command = plan + + variables { + name = "unit-auto-custom-only" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = { + create_node_iam_role = false + node_pools = [] + } + node_groups = {} + addons = {} + + auto_mode_node_classes = { + custom = { + create_access_entry = true + node_role_arn = "arn:aws:iam::123456789012:role/CustomOnlyAutoNodeRole" + spec = { + role = "CustomOnlyAutoNodeRole" + } + } + } + + auto_mode_node_pools = { + custom = { + spec = { + template = { + spec = { + nodeClassRef = { + group = "eks.amazonaws.com" + kind = "NodeClass" + name = "custom" + } + } + } + } + } + } + } + + assert { + condition = aws_eks_cluster.this.compute_config[0].enabled == true && length(aws_eks_cluster.this.compute_config[0].node_pools) == 0 + error_message = "Custom-only Auto Mode must remain enabled without built-in node pools." + } + + assert { + condition = aws_eks_cluster.this.compute_config[0].node_role_arn == null + error_message = "Custom-only Auto Mode must omit the global node role from compute_config." + } + + assert { + condition = length(aws_iam_role.auto_mode_node) == 0 && length(aws_eks_access_entry.auto_mode_node) == 0 + error_message = "Custom-only Auto Mode must not create a global node role or its singleton access entry." + } + + assert { + condition = aws_eks_access_entry.auto_mode_node_class["custom"].principal_arn == "arn:aws:iam::123456789012:role/CustomOnlyAutoNodeRole" + error_message = "Custom-only Auto Mode must honor an explicitly requested NodeClass access entry." + } +} + +run "auto_mode_custom_only_derives_role_from_node_class_arn" { + command = plan + + variables { + name = "unit-auto-custom-only-derived-role" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = { + create_node_iam_role = false + node_pools = [] + } + node_groups = {} + addons = {} + + auto_mode_node_classes = { + custom = { + node_role_arn = "arn:aws:iam::123456789012:role/platform/CustomOnlyDerivedRole" + spec = { + ephemeralStorage = { + size = "80Gi" + } + } + } + } + } + + assert { + condition = yamldecode(local.auto_mode_node_class_manifests["custom"]).spec.role == "CustomOnlyDerivedRole" + error_message = "A custom-only NodeClass must derive spec.role from its node_role_arn when no global Auto Mode role exists." + } + + assert { + condition = aws_eks_access_entry.auto_mode_node_class["custom"].principal_arn == "arn:aws:iam::123456789012:role/platform/CustomOnlyDerivedRole" + error_message = "A custom-only NodeClass with a plan-known node_role_arn must create its EC2 access entry." + } +} + +run "auto_mode_external_role_and_cluster_options" { + command = plan + + variables { + name = "unit-auto-external" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = { + create_node_iam_role = false + node_iam_role_arn = "arn:aws:iam::123456789012:role/platform/EKSAutoNodeRole" + node_pools = ["system"] + } + node_groups = {} + addons = {} + + force_update_version = true + ip_family = "ipv4" + service_ipv4_cidr = "172.20.0.0/16" + upgrade_policy_support_type = "STANDARD" + } + + assert { + condition = length(aws_iam_role.auto_mode_node) == 0 + error_message = "External Auto Mode node role mode must not create an IAM role." + } + + assert { + condition = aws_eks_cluster.this.compute_config[0].node_role_arn == "arn:aws:iam::123456789012:role/platform/EKSAutoNodeRole" + error_message = "External Auto Mode node role mode must pass the supplied ARN to EKS." + } + + assert { + condition = output.auto_mode_node_iam_role_arn == "arn:aws:iam::123456789012:role/platform/EKSAutoNodeRole" && output.auto_mode_node_iam_role_name == "EKSAutoNodeRole" + error_message = "Auto Mode outputs must expose the external node role ARN and derived role name." + } + + assert { + condition = toset(aws_eks_cluster.this.compute_config[0].node_pools) == toset(["system"]) + error_message = "External role mode must retain the configured built-in node pools." + } + + assert { + condition = aws_eks_cluster.this.force_update_version == true + error_message = "The cluster force-update option must be configurable." + } + + assert { + condition = aws_eks_cluster.this.kubernetes_network_config[0].ip_family == "ipv4" + error_message = "The Kubernetes service IP family must be configurable." + } + + assert { + condition = aws_eks_cluster.this.kubernetes_network_config[0].service_ipv4_cidr == "172.20.0.0/16" + error_message = "The Kubernetes service IPv4 CIDR must be configurable." + } + + assert { + condition = aws_eks_cluster.this.upgrade_policy[0].support_type == "STANDARD" + error_message = "The Kubernetes version support policy must be configurable." + } + + assert { + condition = aws_eks_cluster.this.vpc_config[0].endpoint_public_access == true + error_message = "Custom Auto Mode settings must keep the public cluster endpoint enabled by default." + } +} diff --git a/tests/auto-mode-validation.tftest.hcl b/tests/auto-mode-validation.tftest.hcl new file mode 100644 index 0000000..4cceabf --- /dev/null +++ b/tests/auto-mode-validation.tftest.hcl @@ -0,0 +1,644 @@ +mock_provider "aws" { + override_during = plan + + mock_data "aws_partition" { + defaults = { + partition = "aws" + } + } + + mock_data "aws_iam_policy_document" { + defaults = { + json = "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"eks.amazonaws.com\"},\"Action\":[\"sts:AssumeRole\",\"sts:TagSession\"]}]}" + } + } + + mock_resource "aws_iam_role" { + defaults = { + arn = "arn:aws:iam::123456789012:role/mock-eks-role" + name = "mock-eks-role" + } + } +} + +run "rejects_auto_mode_external_role_without_arn" { + command = plan + + variables { + name = "unit-invalid-auto-role" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = { + create_node_iam_role = false + } + } + + expect_failures = [ + var.auto_mode + ] +} + +run "rejects_forced_auto_mode_access_entry_with_builtin_node_pools" { + command = plan + + variables { + name = "unit-invalid-auto-entry-builtins" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = { + create_access_entry = true + } + } + + expect_failures = [ + var.auto_mode + ] +} + +run "rejects_forced_auto_mode_access_entry_without_role" { + command = plan + + variables { + name = "unit-invalid-auto-entry-role" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = { + create_access_entry = true + create_node_iam_role = false + node_pools = [] + } + } + + expect_failures = [ + var.auto_mode + ] +} + +run "rejects_invalid_auto_mode_node_pool" { + command = plan + + variables { + name = "unit-invalid-auto-pool" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = { + node_pools = ["gpu"] + } + } + + expect_failures = [ + var.auto_mode + ] +} + +run "rejects_invalid_auto_mode_node_role_name" { + command = plan + + variables { + name = "unit-invalid-auto-role-name" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = { + node_iam_role_name = "role name with spaces" + } + } + + expect_failures = [ + var.auto_mode + ] +} + +run "rejects_auto_mode_with_config_map_authentication" { + command = plan + + variables { + name = "unit-invalid-auto-auth" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = {} + access_config = { + authentication_mode = "CONFIG_MAP" + } + } + + expect_failures = [ + aws_eks_cluster.this + ] +} + +run "rejects_auto_mode_manifests_while_disabled" { + command = plan + + variables { + name = "unit-invalid-disabled-manifests" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = { + enabled = false + } + auto_mode_node_pools = { + custom = { + spec = {} + } + } + } + + expect_failures = [ + aws_eks_cluster.this + ] +} + +run "rejects_auto_mode_with_bootstrap_self_managed_addons" { + command = plan + + variables { + name = "unit-invalid-auto-bootstrap-addons" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = {} + bootstrap_self_managed_addons = true + node_groups = {} + addons = {} + } + + expect_failures = [ + aws_eks_cluster.this + ] +} + +run "rejects_invalid_generic_access_entry_type" { + command = plan + + variables { + name = "unit-invalid-access-entry-type" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + access_entries = { + invalid = { + principal_arn = "arn:aws:iam::123456789012:role/invalid" + type = "ROOT" + } + } + } + + expect_failures = [ + var.access_entries + ] +} + +run "rejects_kubernetes_identity_fields_on_nonstandard_access_entry" { + command = plan + + variables { + name = "unit-invalid-node-access-fields" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + access_entries = { + node = { + kubernetes_groups = ["system:nodes"] + principal_arn = "arn:aws:iam::123456789012:role/worker-node" + type = "EC2_LINUX" + user_name = "system:node:custom" + } + } + } + + expect_failures = [ + var.access_entries + ] +} + +run "rejects_invalid_generic_access_scope" { + command = plan + + variables { + name = "unit-invalid-access-scope" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + access_entries = { + platform_admin = { + principal_arn = "arn:aws:iam::123456789012:role/platform-admin" + policy_associations = { + admin = { + policy_arn = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy" + access_scope = { + type = "invalid" + } + } + } + } + } + } + + expect_failures = [ + var.access_entries + ] +} + +run "rejects_access_entries_with_config_map_authentication" { + command = plan + + variables { + name = "unit-invalid-access-auth" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + access_config = { + authentication_mode = "CONFIG_MAP" + } + + access_entries = { + platform_admin = { + principal_arn = "arn:aws:iam::123456789012:role/platform-admin" + } + } + } + + expect_failures = [ + aws_eks_cluster.this + ] +} + +run "rejects_access_entry_for_managed_node_group_role" { + command = plan + + variables { + name = "unit-invalid-managed-node-role-entry" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + access_config = { + authentication_mode = "API" + } + + access_entries = { + duplicate_node = { + principal_arn = "arn:aws:iam::123456789012:role/mock-eks-role" + } + } + } + + expect_failures = [ + aws_eks_cluster.this + ] +} + +run "rejects_hybrid_vpc_cni_after_compute" { + command = plan + + variables { + name = "unit-invalid-hybrid-cni-order" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = {} + node_groups = { + workloads = {} + } + addons = { + vpc-cni = {} + } + } + + override_resource { + target = aws_iam_role.node + override_during = plan + values = { + arn = "arn:aws:iam::123456789012:role/unit-invalid-hybrid-cni-managed-node-role" + } + } + + expect_failures = [ + aws_eks_cluster.this + ] +} + +run "rejects_pod_identity_external_role_without_arn" { + command = plan + + variables { + name = "unit-invalid-pod-role" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + pod_identity_associations = { + web = { + create_iam_role = false + namespace = "apps" + service_account = "web" + } + } + } + + expect_failures = [ + var.pod_identity_associations + ] +} + +run "rejects_invalid_pod_identity_namespace" { + command = plan + + variables { + name = "unit-invalid-pod-namespace" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + pod_identity_associations = { + web = { + namespace = "Invalid_Namespace" + service_account = "web" + } + } + } + + expect_failures = [ + var.pod_identity_associations + ] +} + +run "rejects_invalid_auto_mode_node_class_name" { + command = plan + + variables { + name = "unit-invalid-node-class-name" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode_node_classes = { + default = { + spec = {} + } + } + } + + expect_failures = [ + var.auto_mode_node_classes + ] +} + +run "deduplicates_auto_mode_node_class_role_entries" { + command = plan + + variables { + name = "unit-duplicate-node-class-role" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = {} + node_groups = {} + addons = {} + + auto_mode_node_classes = { + first = { + access_entry_key = "shared-role" + node_role_arn = "arn:aws:iam::123456789012:role/SharedAutoNodeRole" + spec = { + role = "SharedAutoNodeRole" + } + } + second = { + access_entry_key = "shared-role" + node_role_arn = "arn:aws:iam::123456789012:role/SharedAutoNodeRole" + spec = { + role = "SharedAutoNodeRole" + } + } + } + } + + assert { + condition = ( + length(aws_eks_access_entry.auto_mode_node_class) == 1 && + aws_eks_access_entry.auto_mode_node_class["shared-role"].principal_arn == "arn:aws:iam::123456789012:role/SharedAutoNodeRole" + ) + error_message = "NodeClasses sharing a role and access_entry_key must receive one access entry under that stable key." + } + + assert { + condition = ( + contains(keys(aws_eks_access_policy_association.auto_mode_node_class), "shared-role") && + contains(keys(output.auto_mode_node_class_access_entry_arns), "shared-role") + ) + error_message = "The shared stable key must propagate to the policy association and access-entry output." + } +} + +run "rejects_shared_node_class_access_key_with_different_roles" { + command = plan + + variables { + name = "unit-conflicting-node-class-access-key" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = {} + node_groups = {} + addons = {} + + auto_mode_node_classes = { + first = { + access_entry_key = "shared-key" + node_role_arn = "arn:aws:iam::123456789012:role/FirstAutoNodeRole" + spec = { + role = "FirstAutoNodeRole" + } + } + second = { + access_entry_key = "shared-key" + node_role_arn = "arn:aws:iam::123456789012:role/SecondAutoNodeRole" + spec = { + role = "SecondAutoNodeRole" + } + } + } + } + + expect_failures = [ + aws_eks_cluster.this + ] +} + +run "rejects_explicit_node_class_access_entry_without_arn" { + command = plan + + variables { + name = "unit-node-class-entry-without-arn" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode = {} + node_groups = {} + addons = {} + + auto_mode_node_classes = { + custom = { + create_access_entry = true + spec = { + role = "CustomAutoNodeRole" + } + } + } + } + + expect_failures = [ + var.auto_mode_node_classes + ] +} + +run "rejects_invalid_auto_mode_node_pool_name" { + command = plan + + variables { + name = "unit-invalid-node-pool-name" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode_node_pools = { + "Invalid_Pool" = { + spec = {} + } + } + } + + expect_failures = [ + var.auto_mode_node_pools + ] +} + +run "rejects_invalid_auto_mode_storage_class_name" { + command = plan + + variables { + name = "unit-invalid-storage-class-name" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode_storage_classes = { + ".invalid" = {} + } + } + + expect_failures = [ + var.auto_mode_storage_classes + ] +} + +run "rejects_invalid_auto_mode_service_name" { + command = plan + + variables { + name = "unit-invalid-service-name" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + auto_mode_load_balancer_services = { + "Invalid_Service" = { + ports = [ + { + port = 80 + } + ] + selector = { + app = "web" + } + } + } + } + + expect_failures = [ + var.auto_mode_load_balancer_services + ] +} + +run "rejects_invalid_ip_family" { + command = plan + + variables { + name = "unit-invalid-ip-family-v2" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + ip_family = "dualstack" + } + + expect_failures = [ + var.ip_family + ] +} + +run "rejects_invalid_upgrade_policy" { + command = plan + + variables { + name = "unit-invalid-upgrade-policy" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + upgrade_policy_support_type = "PREMIUM" + } + + expect_failures = [ + var.upgrade_policy_support_type + ] +} diff --git a/tests/capabilities.tftest.hcl b/tests/capabilities.tftest.hcl index 8a412b3..b058305 100644 --- a/tests/capabilities.tftest.hcl +++ b/tests/capabilities.tftest.hcl @@ -88,6 +88,9 @@ run "capabilities_cluster_shape" { iam_policy_arns = [ "arn:aws:iam::aws:policy/SecretsManagerReadWrite" ] + iam_policy_arn_map = { + read_only = "arn:aws:iam::aws:policy/ReadOnlyAccess" + } } ack = { @@ -102,6 +105,30 @@ run "capabilities_cluster_shape" { } } + override_resource { + target = aws_iam_role.node + override_during = plan + values = { + arn = "arn:aws:iam::123456789012:role/unit-capabilities-managed-node-role" + } + } + + override_resource { + target = aws_iam_role.eks_capability["argocd"] + override_during = plan + values = { + arn = "arn:aws:iam::123456789012:role/unit-capabilities-argocd-role" + } + } + + override_resource { + target = aws_iam_role.eks_capability["ack"] + override_during = plan + values = { + arn = "arn:aws:iam::123456789012:role/unit-capabilities-ack-role" + } + } + assert { condition = aws_eks_cluster.this.access_config[0].authentication_mode == "API" error_message = "EKS capabilities should derive API authentication when access_config is omitted." @@ -169,11 +196,85 @@ run "capabilities_cluster_shape" { assert { condition = contains(keys(aws_iam_role_policy_attachment.eks_capability), "argocd:arn:aws:iam::aws:policy/SecretsManagerReadWrite") - error_message = "Configured capability managed policies should be attached to the created IAM role." + error_message = "Configured capability policy sets should remain supported." + } + + assert { + condition = aws_iam_role_policy_attachment.eks_capability["argocd:key:read_only"].policy_arn == "arn:aws:iam::aws:policy/ReadOnlyAccess" + error_message = "Capability policy maps should create attachments with stable capability and policy keys." } assert { condition = aws_eks_capability.this["kro"].role_arn == "arn:aws:iam::123456789012:role/platform/KROCapabilityRole-external" error_message = "External capability role mode should use the supplied IAM role ARN." } + + assert { + condition = output.capability_iam_role_names["kro"] == "KROCapabilityRole-external" + error_message = "Capability outputs should derive an external role name from the final ARN path segment." + } +} + +run "rejects_capability_role_reused_by_generic_access_entry" { + command = plan + + variables { + name = "unit-capability-access-collision" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + access_entries = { + platform_admin = { + principal_arn = "arn:aws:iam::123456789012:role/shared-capability-access-role" + } + } + + capabilities = { + ack = { + type = "ACK" + create_iam_role = false + iam_role_arn = "arn:aws:iam::123456789012:role/shared-capability-access-role" + } + } + } + + expect_failures = [ + aws_eks_cluster.this + ] +} + +run "allows_capabilities_to_share_external_role" { + command = plan + + variables { + name = "unit-capability-role-collision" + subnet_ids = [ + "subnet-0123456789abcdef0", + "subnet-0fedcba9876543210" + ] + + capabilities = { + ack = { + type = "ACK" + create_iam_role = false + iam_role_arn = "arn:aws:iam::123456789012:role/shared-capability-role" + } + + kro = { + type = "KRO" + create_iam_role = false + iam_role_arn = "arn:aws:iam::123456789012:role/shared-capability-role" + } + } + } + + assert { + condition = ( + aws_eks_capability.this["ack"].role_arn == "arn:aws:iam::123456789012:role/shared-capability-role" && + aws_eks_capability.this["kro"].role_arn == "arn:aws:iam::123456789012:role/shared-capability-role" + ) + error_message = "EKS capabilities must be able to share an external capability role while EKS manages the shared access entry." + } } diff --git a/tests/karpenter.tftest.hcl b/tests/karpenter.tftest.hcl index 02dde13..f8a47c1 100755 --- a/tests/karpenter.tftest.hcl +++ b/tests/karpenter.tftest.hcl @@ -68,6 +68,14 @@ run "karpenter_ready_cluster_shape" { } } + override_resource { + target = aws_iam_role.node + override_during = plan + values = { + arn = "arn:aws:iam::123456789012:role/unit-karpenter-managed-node-role" + } + } + assert { condition = aws_eks_cluster.this.access_config[0].authentication_mode == "API" error_message = "Karpenter access entries should derive API authentication when access_config is omitted." diff --git a/tests/validation.tftest.hcl b/tests/validation.tftest.hcl index ae86821..5b56965 100755 --- a/tests/validation.tftest.hcl +++ b/tests/validation.tftest.hcl @@ -307,8 +307,16 @@ run "rejects_karpenter_access_entry_with_config_map_authentication" { } } + override_resource { + target = aws_iam_role.node + override_during = plan + values = { + arn = "arn:aws:iam::123456789012:role/unit-invalid-karpenter-auth-managed-node-role" + } + } + expect_failures = [ - aws_eks_access_entry.karpenter_node + aws_eks_cluster.this ] } @@ -523,6 +531,6 @@ run "rejects_capability_with_config_map_authentication" { } expect_failures = [ - aws_eks_capability.this + aws_eks_cluster.this ] } diff --git a/variables.tf b/variables.tf index 6ea8d5a..e384d6a 100755 --- a/variables.tf +++ b/variables.tf @@ -83,6 +83,18 @@ variable "deletion_protection" { default = null } +variable "bootstrap_self_managed_addons" { + description = "Whether EKS bootstraps self-managed networking add-ons at cluster creation. Leave null to derive true for standard clusters and false while Auto Mode is configured. This setting is create-only." + type = bool + default = null +} + +variable "force_update_version" { + description = "Whether to force a Kubernetes version update when EKS cannot drain pods." + type = bool + default = null +} + variable "control_plane_scaling_config" { description = "Optional EKS Provisioned Control Plane scaling configuration. Leave null to use the standard control plane scaling tier." type = object({ @@ -259,12 +271,98 @@ variable "cloudwatch_log_group_kms_key_id" { default = null } +variable "auto_mode" { + description = "Optional EKS Auto Mode configuration. Null leaves Auto Mode unmanaged for backward compatibility; an empty object enables Auto Mode with its default node pools. Set enabled to false to explicitly disable previously managed Auto Mode capabilities." + type = object({ + cluster_iam_policy_arn_map = optional(map(string), {}) + cluster_iam_policy_arns = optional(set(string), []) + create_access_entry = optional(bool) + create_node_iam_role = optional(bool, true) + enabled = optional(bool, true) + node_iam_policy_arn_map = optional(map(string), {}) + node_iam_policy_arns = optional(set(string), []) + node_iam_role_arn = optional(string) + node_iam_role_name = optional(string) + node_pools = optional(set(string), ["general-purpose", "system"]) + }) + default = null + + validation { + condition = var.auto_mode == null ? true : ( + !var.auto_mode.enabled || + length(var.auto_mode.node_pools) == 0 || + var.auto_mode.create_node_iam_role || + var.auto_mode.node_iam_role_arn != null + ) + error_message = "When Auto Mode built-in node pools are enabled and create_node_iam_role is false, auto_mode.node_iam_role_arn must be set." + } + + validation { + condition = var.auto_mode == null ? true : ( + var.auto_mode.create_access_entry != true || ( + var.auto_mode.enabled && + length(var.auto_mode.node_pools) == 0 && + (var.auto_mode.create_node_iam_role || var.auto_mode.node_iam_role_arn != null) + ) + ) + error_message = "auto_mode.create_access_entry=true requires enabled Auto Mode without built-in node pools and a module-created or external node IAM role." + } + + validation { + condition = var.auto_mode == null ? true : ( + alltrue([ + for node_pool in var.auto_mode.node_pools : + contains(["general-purpose", "system"], node_pool) + ]) + ) + error_message = "auto_mode.node_pools supports only general-purpose and system." + } + + validation { + condition = var.auto_mode == null ? true : ( + var.auto_mode.node_iam_role_name == null || + can(regex("^[A-Za-z0-9+=,.@_-]{1,64}$", var.auto_mode.node_iam_role_name)) + ) + error_message = "auto_mode.node_iam_role_name must be 1-64 characters and contain only IAM role name characters." + } + + validation { + condition = var.auto_mode == null ? true : alltrue([ + for key in concat(keys(var.auto_mode.cluster_iam_policy_arn_map), keys(var.auto_mode.node_iam_policy_arn_map)) : + can(regex("^[A-Za-z0-9][A-Za-z0-9_.-]{0,127}$", key)) + ]) + error_message = "auto_mode IAM policy map keys must be stable identifiers containing only letters, numbers, periods, underscores, and hyphens." + } +} + +variable "ip_family" { + description = "Optional Kubernetes service IP family. Valid values are ipv4 or ipv6." + type = string + default = null + + validation { + condition = var.ip_family == null || contains(["ipv4", "ipv6"], var.ip_family) + error_message = "ip_family must be ipv4 or ipv6." + } +} + variable "service_ipv4_cidr" { description = "Optional Kubernetes service IPv4 CIDR. Set only when you need a non-default service CIDR." type = string default = null } +variable "upgrade_policy_support_type" { + description = "Optional Kubernetes version support policy. Valid values are STANDARD and EXTENDED." + type = string + default = null + + validation { + condition = var.upgrade_policy_support_type == null || contains(["STANDARD", "EXTENDED"], var.upgrade_policy_support_type) + error_message = "upgrade_policy_support_type must be STANDARD or EXTENDED." + } +} + variable "node_groups" { description = "Managed node groups to create." type = map(object({ @@ -337,8 +435,9 @@ variable "node_groups" { } variable "addons" { - description = "EKS add-ons to install after the managed node groups are created." + description = "EKS add-ons to install. Set before_compute for add-ons such as vpc-cni that managed nodes need during bootstrap." type = map(object({ + before_compute = optional(bool, false) configuration_values = optional(string) pod_identity_associations = optional(list(object({ role_arn = string @@ -356,13 +455,322 @@ variable "addons" { } } +variable "access_entries" { + description = "Additional EKS access entries and optional access policy associations to create." + type = map(object({ + kubernetes_groups = optional(set(string), []) + policy_associations = optional(map(object({ + access_scope = object({ + namespaces = optional(set(string), []) + type = string + }) + policy_arn = string + })), {}) + principal_arn = string + type = optional(string, "STANDARD") + user_name = optional(string) + })) + default = {} + + validation { + condition = alltrue([ + for _, entry in var.access_entries : + contains(["STANDARD", "EC2", "EC2_LINUX", "EC2_WINDOWS", "FARGATE_LINUX", "HYBRID_LINUX"], entry.type) + ]) + error_message = "access_entries[*].type must be STANDARD, EC2, EC2_LINUX, EC2_WINDOWS, FARGATE_LINUX, or HYBRID_LINUX." + } + + validation { + condition = alltrue(flatten([ + for _, entry in var.access_entries : [ + for _, association in entry.policy_associations : + contains(["cluster", "namespace"], association.access_scope.type) + ] + ])) + error_message = "access_entries[*].policy_associations[*].access_scope.type must be cluster or namespace." + } + + validation { + condition = alltrue(flatten([ + for _, entry in var.access_entries : [ + for _, association in entry.policy_associations : + association.access_scope.type == "namespace" || length(association.access_scope.namespaces) == 0 + ] + ])) + error_message = "access_entries[*].policy_associations[*].access_scope.namespaces must be empty when access_scope.type is cluster." + } + + validation { + condition = alltrue(flatten([ + for _, entry in var.access_entries : [ + for _, association in entry.policy_associations : + association.access_scope.type != "namespace" || length(association.access_scope.namespaces) > 0 + ] + ])) + error_message = "access_entries[*].policy_associations[*].access_scope.namespaces must contain at least one namespace when access_scope.type is namespace." + } + + validation { + condition = alltrue([ + for _, entry in var.access_entries : + entry.type == "STANDARD" || + length(entry.policy_associations) == 0 || + ( + entry.type == "EC2" && + alltrue([ + for _, association in entry.policy_associations : + can(regex(":eks::aws:cluster-access-policy/AmazonEKSAutoNodePolicy$", association.policy_arn)) && + association.access_scope.type == "cluster" + ]) + ) + ]) + error_message = "Only STANDARD access entries may define arbitrary policy associations. EC2 entries may associate only AmazonEKSAutoNodePolicy at cluster scope." + } + + validation { + condition = alltrue([ + for _, entry in var.access_entries : + entry.type == "STANDARD" || (length(entry.kubernetes_groups) == 0 && entry.user_name == null) + ]) + error_message = "Only STANDARD access entries may define kubernetes_groups or user_name." + } + + validation { + condition = length(distinct([for _, entry in var.access_entries : entry.principal_arn])) == length(var.access_entries) + error_message = "access_entries must not contain duplicate principal ARNs." + } +} + +variable "auto_mode_node_classes" { + description = "Custom EKS Auto Mode NodeClass manifests to render as YAML, keyed by Kubernetes metadata.name. The module does not apply these manifests." + type = map(object({ + access_entry_key = optional(string) + annotations = optional(map(string), {}) + create_access_entry = optional(bool) + labels = optional(map(string), {}) + node_role_arn = optional(string) + spec = any + })) + default = {} + + validation { + condition = alltrue([ + for name, _ in var.auto_mode_node_classes : + can(regex("^[a-z0-9]([-a-z0-9]*[a-z0-9])?$", name)) && length(name) <= 63 && name != "default" + ]) + error_message = "auto_mode_node_classes keys must be DNS label names up to 63 characters and must not be default." + } + + validation { + condition = alltrue([ + for _, node_class in var.auto_mode_node_classes : + !(try(node_class.spec.role, null) != null && try(node_class.spec.instanceProfile, null) != null) + ]) + error_message = "auto_mode_node_classes[*].spec must not set both role and instanceProfile." + } + + validation { + condition = alltrue([ + for _, node_class in var.auto_mode_node_classes : + node_class.create_access_entry == false || ( + try(node_class.spec.role, null) == null && + try(node_class.spec.instanceProfile, null) == null + ) || node_class.node_role_arn != null + ]) + error_message = "A NodeClass with a custom spec.role or spec.instanceProfile must set node_role_arn unless create_access_entry=false." + } + + validation { + condition = alltrue([ + for _, node_class in var.auto_mode_node_classes : + node_class.create_access_entry != true || node_class.node_role_arn != null + ]) + error_message = "auto_mode_node_classes[*].create_access_entry=true requires node_role_arn." + } + + validation { + condition = alltrue([ + for name, node_class in var.auto_mode_node_classes : + node_class.access_entry_key == null || ( + can(regex("^[A-Za-z0-9][A-Za-z0-9_.-]{0,127}$", node_class.access_entry_key)) && + node_class.node_role_arn != null && + node_class.create_access_entry != false + ) + ]) + error_message = "auto_mode_node_classes[*].access_entry_key must be a stable identifier and requires an enabled access entry with node_role_arn." + } + + validation { + condition = alltrue([ + for _, node_class in var.auto_mode_node_classes : + node_class.node_role_arn == null || + try(node_class.spec.role, null) == null || + reverse(split("/", node_class.node_role_arn))[0] == node_class.spec.role + ]) + error_message = "auto_mode_node_classes[*].spec.role must match the role name in node_role_arn when both are set." + } +} + +variable "auto_mode_node_pools" { + description = "Custom EKS Auto Mode NodePool manifests to render as YAML, keyed by Kubernetes metadata.name. The module does not apply these manifests." + type = map(object({ + annotations = optional(map(string), {}) + labels = optional(map(string), {}) + spec = any + })) + default = {} + + validation { + condition = alltrue([ + for name, _ in var.auto_mode_node_pools : + can(regex("^[a-z0-9]([-a-z0-9]*[a-z0-9])?$", name)) && length(name) <= 63 + ]) + error_message = "auto_mode_node_pools keys must be DNS label names up to 63 characters." + } +} + +variable "auto_mode_storage_classes" { + description = "EKS Auto Mode EBS StorageClass manifests to render as YAML, keyed by Kubernetes metadata.name." + type = map(object({ + allow_volume_expansion = optional(bool, true) + allowed_topologies = optional(list(any), []) + annotations = optional(map(string), {}) + labels = optional(map(string), {}) + mount_options = optional(list(string), []) + parameters = optional(map(string), {}) + reclaim_policy = optional(string, "Delete") + volume_binding_mode = optional(string, "WaitForFirstConsumer") + })) + default = {} + + validation { + condition = alltrue([ + for name, _ in var.auto_mode_storage_classes : + can(regex("^[a-z0-9]([-a-z0-9.]*[a-z0-9])?$", name)) && length(name) <= 253 + ]) + error_message = "auto_mode_storage_classes keys must be valid Kubernetes storage class names up to 253 characters." + } + + validation { + condition = alltrue([ + for _, storage_class in var.auto_mode_storage_classes : + contains(["Delete", "Retain"], storage_class.reclaim_policy) + ]) + error_message = "auto_mode_storage_classes[*].reclaim_policy must be Delete or Retain." + } + + validation { + condition = alltrue([ + for _, storage_class in var.auto_mode_storage_classes : + contains(["Immediate", "WaitForFirstConsumer"], storage_class.volume_binding_mode) + ]) + error_message = "auto_mode_storage_classes[*].volume_binding_mode must be Immediate or WaitForFirstConsumer." + } +} + +variable "auto_mode_load_balancer_services" { + description = "EKS Auto Mode Network Load Balancer Service manifests to render as YAML, keyed by Kubernetes metadata.name." + type = map(object({ + annotations = optional(map(string), {}) + external_traffic_policy = optional(string) + ip_families = optional(list(string), []) + ip_family_policy = optional(string) + labels = optional(map(string), {}) + load_balancer_class = optional(string, "eks.amazonaws.com/nlb") + load_balancer_source_ranges = optional(list(string), []) + namespace = optional(string, "default") + ports = list(any) + selector = map(string) + session_affinity = optional(string) + })) + default = {} + + validation { + condition = alltrue([ + for name, _ in var.auto_mode_load_balancer_services : + can(regex("^[a-z0-9]([-a-z0-9]*[a-z0-9])?$", name)) && length(name) <= 63 + ]) + error_message = "auto_mode_load_balancer_services keys must be DNS label names up to 63 characters." + } + + validation { + condition = alltrue([ + for _, service in var.auto_mode_load_balancer_services : + can(regex("^[a-z0-9]([-a-z0-9]*[a-z0-9])?$", service.namespace)) && length(service.namespace) <= 63 + ]) + error_message = "auto_mode_load_balancer_services[*].namespace must be a DNS label up to 63 characters." + } +} + +variable "pod_identity_associations" { + description = "EKS Pod Identity associations to create for Kubernetes service accounts. The module can create the IAM role per association, or use an externally managed role ARN." + type = map(object({ + create_iam_role = optional(bool, true) + disable_session_tags = optional(bool) + iam_policy_arn_map = optional(map(string), {}) + iam_policy_arns = optional(set(string), []) + iam_role_arn = optional(string) + iam_role_name = optional(string) + inline_policy_json = optional(string) + namespace = string + service_account = string + tags = optional(map(string), {}) + target_role_arn = optional(string) + })) + default = {} + + validation { + condition = alltrue([ + for _, association in var.pod_identity_associations : + association.create_iam_role || association.iam_role_arn != null + ]) + error_message = "When pod_identity_associations[*].create_iam_role is false, pod_identity_associations[*].iam_role_arn must be set." + } + + validation { + condition = alltrue([ + for _, association in var.pod_identity_associations : + can(regex("^[a-z0-9]([-a-z0-9]*[a-z0-9])?$", association.namespace)) && length(association.namespace) <= 63 + ]) + error_message = "pod_identity_associations[*].namespace must be a DNS label up to 63 characters." + } + + validation { + condition = alltrue([ + for _, association in var.pod_identity_associations : + can(regex("^[a-z0-9]([-a-z0-9]*[a-z0-9])?$", association.service_account)) && length(association.service_account) <= 63 + ]) + error_message = "pod_identity_associations[*].service_account must be a DNS label up to 63 characters." + } + + validation { + condition = alltrue([ + for _, association in var.pod_identity_associations : + association.iam_role_name == null || can(regex("^[A-Za-z0-9+=,.@_-]{1,64}$", association.iam_role_name)) + ]) + error_message = "pod_identity_associations[*].iam_role_name must be 1-64 characters and contain only IAM role name characters." + } + + validation { + condition = alltrue(flatten([ + for _, association in var.pod_identity_associations : [ + for key in keys(association.iam_policy_arn_map) : + can(regex("^[A-Za-z0-9][A-Za-z0-9_.-]{0,127}$", key)) + ] + ])) + error_message = "pod_identity_associations[*].iam_policy_arn_map keys must be stable identifiers containing only letters, numbers, periods, underscores, and hyphens." + } +} + variable "capabilities" { description = "Amazon EKS managed capabilities to create, keyed by a stable local name. Supported types are ARGOCD, ACK, and KRO. The module can create a capability IAM role per entry, or use an externally managed role ARN." type = map(object({ capability_name = optional(string) create_iam_role = optional(bool, true) delete_propagation_policy = optional(string, "RETAIN") + iam_policy_arn_map = optional(map(string), {}) iam_policy_arns = optional(set(string), []) + iam_policy_presets = optional(set(string), []) iam_role_arn = optional(string) iam_role_name = optional(string) inline_policy_json = optional(string) @@ -391,6 +799,21 @@ variable "capabilities" { error_message = "capabilities entries must use type ACK, ARGOCD, or KRO." } + validation { + condition = length(distinct([for _, capability in var.capabilities : upper(capability.type)])) == length(var.capabilities) + error_message = "Only one capability of each type may be configured per EKS cluster." + } + + validation { + condition = alltrue(flatten([ + for _, capability in var.capabilities : [ + for preset in capability.iam_policy_presets : + contains(["cloudcontrol_read_only", "eks_read_only", "resource_tagging", "secrets_read_only"], preset) + ] + ])) + error_message = "capabilities[*].iam_policy_presets supports cloudcontrol_read_only, eks_read_only, resource_tagging, and secrets_read_only." + } + validation { condition = alltrue([ for _, capability in var.capabilities : @@ -423,6 +846,16 @@ variable "capabilities" { error_message = "capabilities[*].iam_role_name must be 1-64 characters and contain only IAM role name characters." } + validation { + condition = alltrue(flatten([ + for _, capability in var.capabilities : [ + for key in keys(capability.iam_policy_arn_map) : + can(regex("^[A-Za-z0-9][A-Za-z0-9_.-]{0,127}$", key)) + ] + ])) + error_message = "capabilities[*].iam_policy_arn_map keys must be stable identifiers containing only letters, numbers, periods, underscores, and hyphens." + } + validation { condition = alltrue([ for _, capability in var.capabilities :