Amazon EKS: User Guide
Amazon EKS: User Guide
Amazon EKS: User Guide
User Guide
Amazon EKS User Guide
Table of Contents
What Is Amazon EKS? ......................................................................................................................... 1
How Does Amazon EKS Work? ..................................................................................................... 2
Getting Started .................................................................................................................................. 3
Amazon EKS Prerequisites ........................................................................................................... 3
Create your Amazon EKS Service Role .................................................................................. 3
Create your Amazon EKS Cluster VPC ................................................................................... 3
Install and Configure kubectl for Amazon EKS ....................................................................... 4
(Optional) Download and Install the Latest AWS CLI ............................................................... 7
Step 1: Create Your Amazon EKS Cluster ....................................................................................... 7
Step 2: Configure kubectl for Amazon EKS ................................................................................. 9
Step 3: Launch and Configure Amazon EKS Worker Nodes ............................................................. 12
Step 4: Launch a Guest Book Application .................................................................................... 14
Step 5: Cleaning Up Guest Book Objects ..................................................................................... 16
Clusters ........................................................................................................................................... 17
Creating a Cluster .................................................................................................................... 17
Deleting a Cluster .................................................................................................................... 20
Worker Nodes .................................................................................................................................. 21
Amazon EKS-Optimized AMI ...................................................................................................... 21
Amazon EKS-Optimized AMI Build Scripts ........................................................................... 22
Launching Amazon EKS Worker Nodes ........................................................................................ 22
Storage Classes ................................................................................................................................ 25
Networking ...................................................................................................................................... 27
Cluster VPC Considerations ........................................................................................................ 27
VPC Tagging Requirement ................................................................................................. 27
Subnet Tagging Requirement ............................................................................................. 28
Cluster Security Group Considerations ......................................................................................... 28
Pod Networking ....................................................................................................................... 29
Installing Calico on Amazon EKS ................................................................................................ 31
Stars Policy Demo ............................................................................................................ 31
Managing Cluster Authentication ........................................................................................................ 36
Configure kubectl for Amazon EKS ............................................................................................ 36
Create a kubeconfig for Amazon EKS ....................................................................................... 39
Managing Users or IAM Roles for your Cluster .............................................................................. 41
Service Limits .................................................................................................................................. 45
IAM Policies, Roles, and Permissions ................................................................................................... 46
Policy Structure ........................................................................................................................ 46
Policy Syntax ................................................................................................................... 46
Actions for Amazon EKS .................................................................................................... 47
Testing Permissions .......................................................................................................... 47
Creating IAM Policies ................................................................................................................ 48
Amazon EKS Service IAM Role ................................................................................................... 49
Tutorial: Deploy Kubernetes Dashboard ............................................................................................... 52
Prerequisites ............................................................................................................................ 53
Step 1: Deploy the Dashboard ................................................................................................... 54
Step 2: Create an eks-admin Service Account and Cluster Role Binding .......................................... 55
Step 3: Connect to the Dashboard .............................................................................................. 56
Step 4: Next Steps ................................................................................................................... 57
Tutorial: Creating a VPC for Amazon EKS ............................................................................................ 58
Step 1: Create an Elastic IP Address for Your NAT Gateway ............................................................ 58
Step 2: Run the VPC Wizard ...................................................................................................... 58
Step 3: Create Additional Subnets .............................................................................................. 59
Step 4: Create a Control Plane Security Group ............................................................................. 59
Next Steps ............................................................................................................................... 60
CloudTrail ........................................................................................................................................ 61
iii
Amazon EKS User Guide
iv
Amazon EKS User Guide
Amazon EKS runs Kubernetes control plane instances across multiple Availability Zones to ensure high
availability. Amazon EKS automatically detects and replaces unhealthy control plane instances, and it
provides automated version upgrades and patching for them.
Amazon EKS is also integrated with many AWS services to provide scalability and security for your
applications, including the following:
Amazon EKS runs up-to-date versions of the open-source Kubernetes software, so you can use all the
existing plugins and tooling from the Kubernetes community. Applications running on Amazon EKS are
fully compatible with applications running on any standard Kubernetes environment, whether running
in on-premises data centers or public clouds. This means that you can easily migrate any standard
Kubernetes application to Amazon EKS without any code modification required.
1
Amazon EKS User Guide
How Does Amazon EKS Work?
1. First, create an Amazon EKS cluster in the AWS Management Console or with the AWS CLI or one of
the AWS SDKs.
2. Then, launch worker nodes that register with the Amazon EKS cluster. We provide you with an AWS
CloudFormation template that automatically configures your nodes.
3. When your cluster is ready, you can configure your favorite Kubernetes tools (such as kubectl) to
communicate with your cluster.
4. Deploy and manage applications on your Amazon EKS cluster the same way that you would with any
other Kubernetes environment.
For more information about creating your required resources and your first Amazon EKS cluster, see
Getting Started with Amazon EKS (p. 3).
2
Amazon EKS User Guide
Amazon EKS Prerequisites
You must also create a VPC with and a security group for your cluster to use. Although the VPC and
security groups can be used for multiple EKS clusters, we recommend that you use a separate VPC for
each EKS cluster to provide better network isolation.
This section also helps you to install the kubectl binary and configure it to work with Amazon EKS.
https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/amazon-eks-vpc-
sample.yaml
3
Amazon EKS User Guide
Install and Configure kubectl for Amazon EKS
6. On the Specify Details page, fill out the parameters accordingly, and then choose Next.
• Stack name: Choose a stack name for your AWS CloudFormation stack. For example, you can call
it eks-vpc.
• VpcBlock: Choose a CIDR range for your VPC. You may leave the default value.
• Subnet01Block: Choose a CIDR range for subnet 1. You may leave the default value.
• Subnet02Block: Choose a CIDR range for subnet 2. You may leave the default value.
• Subnet03Block: Choose a CIDR range for subnet 3. You may leave the default value.
7. (Optional) On the Options page, tag your stack resources. Choose Next.
8. On the Review page, choose Create.
9. When your stack is created, select it in the console and choose Outputs.
10. Record the SecurityGroups value for the security group that was created. You need this when you
create your EKS cluster; this security group is applied to the cross-account elastic network interfaces
that are created in your subnets that allow the Amazon EKS control plane to communicate with your
worker nodes.
11. Record the VpcId for the subnets that were created. You need this when you launch your worker
node group template.
12. Record the SubnetIds for the subnets that were created. You need this when you create your EKS
cluster; these are the subnets that your worker nodes are launched into.
If you do not already have a local kubectl version 1.10 client on your system, you can use the steps
below to install one. You can also refer to the Kubernetes documentation to install kubectl.
1. Download and install kubectl for your operating system. Amazon EKS vends kubectl binaries that
you can use, or you can follow the instructions in the Kubernetes documentation to install.
• To install the Amazon EKS-vended version of kubectl:
• Linux: https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/linux/
amd64/kubectl
• MacOS: https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/
darwin/amd64/kubectl
• Windows: https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/
windows/amd64/kubectl.exe
Use the command below to download the binary, substituting the correct URL for your
platform. The example below is for macOS clients.
4
Amazon EKS User Guide
Install and Configure kubectl for Amazon EKS
b. (Optional) Verify the downloaded binary with the MD5 sum provided in the same bucket
prefix, substituting the correct URL for your platform. The example below is to download
the MD5 sum for macOS clients.
chmod +x ./kubectl
d. Copy the binary to a folder in your $PATH. If you have already installed a version of kubectl
(from Homebrew or Apt), then we recommend creating a $HOME/bin/kubectl and
ensuring that $HOME/bin comes first in your $PATH.
e. (Optional) Add the $HOME/bin path to your shell initialization file so that it is configured
when you open a shell.
• Or, to install kubectl using the Kubernetes documentation, see Install and Set Up kubectl in the
Kubernetes documentation.
2. After you install kubectl, you can verify its version with the following command:
Example output:
Amazon EKS vends heptio-authenticator-aws binaries that you can use, or you can use go get
to fetch the binary from the AWS IAM Authenticator for Kubernetes project on GitHub for other
operating systems.
• To download and install the Amazon EKS-vended heptio-authenticator-aws binary for
Linux:
• Linux: https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/linux/
amd64/heptio-authenticator-aws
• MacOS: https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/
darwin/amd64/heptio-authenticator-aws
• Windows: https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/
windows/amd64/heptio-authenticator-aws.exe
Use the command below to download the binary, substituting the correct URL for your
platform. The example below is for macOS clients.
b. (Optional) Verify the downloaded binary with the MD5 sum provided in the same bucket
prefix, substituting the correct URL for your platform. The example below is to download
the MD5 sum for macOS clients.
chmod +x ./heptio-authenticator-aws
heptio-authenticator-aws help
a. Install the Go programming language for your operating system if you do not already have
go installed. For more information, see Install the Go tools in the Go documentation.
b. Use go get to install the aws-iam-authenticator binary.
6
Amazon EKS User Guide
(Optional) Download and Install the Latest AWS CLI
go get -u -v github.com/kubernetes-sigs/aws-iam-authenticator/cmd/aws-iam-
authenticator
aws-iam-authenticator help
7
Amazon EKS User Guide
Step 1: Create Your Amazon EKS Cluster
Note
If your IAM user does not have administrative privileges, you must explicitly add permissions
for that user to call the Amazon EKS API operations. For more information, see Creating
Amazon EKS IAM Policies (p. 48).
3. On the Create cluster page, fill in the following fields and then choose Create:
Note
You may receive an error that one of the Availability Zones in your request does not have
sufficient capacity to create an Amazon EKS cluster. If this happens, the error output
contains the Availability Zones that can support a new cluster. Retry creating your cluster
with at least two subnets that are located in the supported Availability Zones for your
account.
4. On the Clusters page, choose the name of your newly created cluster to view the cluster
information.
5. The Status field shows CREATING until the cluster provisioning process completes. When your
cluster provisioning is complete (usually less than 10 minutes), and note the API server endpoint
and Certificate authority values. These are used in your kubectl configuration.
1. Create your cluster with the following command. Substitute your cluster name, the Amazon
Resource Name (ARN) of your Amazon EKS service role that you created in Create your Amazon EKS
Service Role (p. 3), and the subnet and security group IDs for the VPC that you created in Create
your Amazon EKS Cluster VPC (p. 3).
Important
You must use IAM user credentials for this step, not root credentials. If you create your
Amazon EKS cluster using root credentials, you cannot authenticate to the cluster. For more
information, see How Users Sign In to Your Account in the IAM User Guide.
8
Amazon EKS User Guide
Step 2: Configure kubectl for Amazon EKS
Note
If your IAM user does not have administrative privileges, you must explicitly add permissions
for that user to call the Amazon EKS API operations. For more information, see Creating
Amazon EKS IAM Policies (p. 48).
Output:
{
"cluster": {
"name": "devel",
"arn": "arn:aws:eks:us-west-2:111122223333:cluster/devel",
"createdAt": 1527785885.159,
"version": "1.10",
"roleArn": "arn:aws:iam::111122223333:role/eks-service-role-
AWSServiceRoleForAmazonEKS-AFNL4H8HB71F",
"resourcesVpcConfig": {
"subnetIds": [
"subnet-a9189fe2",
"subnet-50432629"
],
"securityGroupIds": [
"sg-f5c54184"
],
"vpcId": "vpc-a54041dc"
},
"status": "CREATING",
"certificateAuthority": {}
}
}
2. Cluster provisioning usually takes less than 10 minutes. You can query the status of your cluster with
the following command. When your cluster status is ACTIVE, you can proceed.
9
Amazon EKS User Guide
Step 2: Configure kubectl for Amazon EKS
Amazon EKS uses the Heptio Authenticator with kubectl for cluster authentication, which uses the same
default AWS credential provider chain as the AWS CLI and AWS SDKs. If you have installed the AWS CLI
on your system, then by default the Heptio authenticator will use the same credentials that are returned
with the following command:
For more information, see Configuring the AWS CLI in the AWS Command Line Interface User Guide.
To instead have the Heptio authenticator assume a role to perform cluster operations, uncomment the -
r and <role-arn> lines and substitute an IAM role ARN to use with your user.
If you manage multiple AWS credential profiles, you can either set the AWS_PROFILE variable in your
shell or specify the profile name in an environment variable value for the authenticator to use in your
kubeconfig as shown in the procedure below.
If you do not have an existing configuration, or to add the Amazon EKS cluster without modifying your
existing configuration files, you can use the following procedure to add the Amazon EKS cluster to your
configuration.
When your cluster provisioning is complete, retrieve the endpoint and certificateAuthority.data
values with the following commands. These must be added to your kubectl configuration so that you
can communicate with your cluster.
1. Retrieve the endpoint for your cluster. Use this for the <endpoint-url> in your kubeconfig file.
2. Retrieve the certificateAuthority.data for your cluster. Use this for the <base64-encoded-
ca-cert> in your kubeconfig file.
mkdir -p ~/.kube
2. Open your favorite text editor and copy the kubeconfig code block below into it.
apiVersion: v1
clusters:
- cluster:
server: <endpoint-url>
certificate-authority-data: <base64-encoded-ca-cert>
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: aws
name: aws
current-context: aws
kind: Config
preferences: {}
10
Amazon EKS User Guide
Step 2: Configure kubectl for Amazon EKS
users:
- name: aws
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
command: heptio-authenticator-aws
args:
- "token"
- "-i"
- "<cluster-name>"
# - "-r"
# - "<role-arn>"
# env:
# - name: AWS_PROFILE
# value: "<aws-profile>"
3. Replace the <endpoint-url> with the endpoint URL that was created for your cluster.
4. Replace the <base64-encoded-ca-cert> with the certificateAuthority.data that was
created for your cluster.
5. Replace the <cluster-name> with your cluster name.
6. (Optional) To have the Heptio authenticator assume a role to perform cluster operations (instead
of the default AWS credential provider chain), uncomment the -r and <role-arn> lines and
substitute an IAM role ARN to use with your user.
7. (Optional) To have the Heptio authenticator always use a specific named AWS credential profile
(instead of the default AWS credential provider chain), uncomment the env lines and substitute
<aws-profile> with the profile name to use.
8. Save the file to the default kubectl folder, with your cluster name in the file name. For example, if
your cluster name is devel, save the file to ~/.kube/config-devel.
9. Add that file path to your KUBECONFIG environment variable so that kubectl knows where to look
for your cluster configuration.
export KUBECONFIG=$KUBECONFIG:~/.kube/config-devel
10. (Optional) Add the configuration to your shell initialization file so that it is configured when you
open a shell.
Note
If you receive the error "heptio-authenticator-aws": executable file not
found in $PATH, then your kubectl is not configured for Amazon EKS. For more
information, see Configure kubectl for Amazon EKS (p. 36).
Output:
11
Amazon EKS User Guide
Step 3: Launch and Configure Amazon EKS Worker Nodes
https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/amazon-eks-
nodegroup.yaml
6. On the Specify Details page, fill out the following parameters accordingly, and choose Next.
• Stack name: Choose a stack name for your AWS CloudFormation stack. For example, you can call
it <cluster-name>-worker-nodes.
• ClusterName: Enter the name that you used when you created your Amazon EKS cluster.
Important
This name must exactly match the name you used in Step 1: Create Your Amazon EKS
Cluster (p. 7); otherwise, your worker nodes cannot join the cluster.
• ClusterControlPlaneSecurityGroup: Choose the SecurityGroups value from the AWS
CloudFormation output that you generated with Create your Amazon EKS Cluster VPC (p. 3).
• NodeGroupName: Enter a name for your node group that is included in your Auto Scaling node
group name.
• NodeAutoScalingGroupMinSize: Enter the minimum number of nodes that your worker node
Auto Scaling group can scale in to.
• NodeAutoScalingGroupMaxSize: Enter the maximum number of nodes that your worker node
Auto Scaling group can scale out to.
• NodeInstanceType: Choose an instance type for your worker nodes.
• NodeImageId: Enter the current Amazon EKS worker node AMI ID for your Region.
12
Amazon EKS User Guide
Step 3: Launch and Configure Amazon EKS Worker Nodes
Note
The Amazon EKS worker node AMI is based on Amazon Linux 2. You can track security
or privacy events for Amazon Linux 2 at the Amazon Linux Security Center or subscribe
to the associated RSS feed. Security and privacy events include an overview of the issue,
what packages are affected, and how to update your instances to correct the issue.
• KeyName: Enter the name of an Amazon EC2 SSH key pair that you can use to connect using SSH
into your worker nodes with after they launch.
• VpcId: Enter the ID for the VPC that you created in Create your Amazon EKS Cluster
VPC (p. 3).
• Subnets: Choose the subnets that you created in Create your Amazon EKS Cluster VPC (p. 3).
7. On the Options page, you can choose to tag your stack resources. Choose Next.
8. On the Review page, review your information, acknowledge that the stack might create IAM
resources, and then choose Create.
9. When your stack has finished creating, select it in the console and choose the Outputs tab.
10. Record the NodeInstanceRole for the node group that was created. You need this when you
configure your Amazon EKS worker nodes.
curl -O https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/aws-auth-
cm.yaml
b. Open the file with your favorite text editor. Replace the <ARN of instance role (not
instance profile)> snippet with the NodeInstanceRole value that you recorded in the
previous procedure, and save the file.
Important
Do not modify any other lines in this file.
apiVersion: v1
kind: ConfigMap
metadata:
name: aws-auth
namespace: kube-system
data:
mapRoles: |
- rolearn: <ARN of instance role (not instance profile)>
username: system:node:{{EC2PrivateDNSName}}
groups:
- system:bootstrappers
- system:nodes
c. Apply the configuration. This command may take a few minutes to finish.
Note
If you receive the error "heptio-authenticator-aws": executable file not
found in $PATH, then your kubectl is not configured for Amazon EKS. For more
information, see Configure kubectl for Amazon EKS (p. 36).
13
Amazon EKS User Guide
Step 4: Launch a Guest Book Application
2. Watch the status of your nodes and wait for them to reach the Ready status.
Note
If you receive the error "heptio-authenticator-aws": executable file not
found in $PATH, then your kubectl is not configured for Amazon EKS. For more
information, see Configure kubectl for Amazon EKS (p. 36).
Output:
Output:
Output:
Output:
14
Amazon EKS User Guide
Step 4: Launch a Guest Book Application
Output:
Output:
7. Query the services in your cluster and wait until the External IP column for the guestbook service
is populated.
Note
It may take several minutes before the IP address is available.
15
Amazon EKS User Guide
Step 5: Cleaning Up Guest Book Objects
Important
If you are unable to connect to the external IP address with your browser, be sure that your
corporate firewall is not blocking non-standards ports, like 3000. You can try switching to a
guest network to verify.
Note
If you receive the error "heptio-authenticator-aws": executable file not found
in $PATH, then your kubectl is not configured for Amazon EKS. For more information, see
Configure kubectl for Amazon EKS (p. 36).
16
Amazon EKS User Guide
Creating a Cluster
The Amazon EKS control plane consists of control plane nodes that run the Kubernetes software, like
etcd and the Kubernetes API server. The control plane runs in an account managed by AWS, and the
Kubernetes API is exposed via the Amazon EKS endpoint associated with your cluster.
Amazon EKS worker nodes run in your AWS account and connect to your cluster's control plane via the
API server endpoint and a certificate file that is created for your cluster.
The cluster control plane is provisioned across multiple Availability Zones and fronted by an Elastic Load
Balancing Network Load Balancer. Amazon EKS also provisions elastic network interfaces in your VPC
subnets to provide connectivity from the control plane instances to the worker nodes (for example, to
support kubectl exec, logs, and proxy data flows).
Topics
• Creating an Amazon EKS Cluster (p. 17)
• Deleting a Cluster (p. 20)
If this is your first time creating an Amazon EKS cluster, we recommend that you follow our Getting
Started with Amazon EKS (p. 3) guide instead, which provides a complete end-to-end walkthrough from
creating an Amazon EKS cluster to deploying a sample Kubernetes application.
• You have created a VPC and a dedicated security group that meets the requirements for an Amazon
EKS cluster. For more information, see Cluster VPC Considerations (p. 27) and Cluster Security
Group Considerations (p. 28). The Getting Started with Amazon EKS (p. 3) guide creates a VPC
that meets the requirements, or you can also follow Tutorial: Creating a VPC with Public and Private
Subnets for Your Amazon EKS Cluster (p. 58) to create one manually.
• You have created an Amazon EKS service role to apply to your cluster. The Getting Started with
Amazon EKS (p. 3) guide creates a service role for you, or you can also follow Amazon EKS Service IAM
Role (p. 49) to create one manually.
Important
When an Amazon EKS cluster is created, the IAM entity (user or role) that creates the cluster is
added to the Kubernetes RBAC authorization table as the administrator. Initially, only that IAM
user can make calls to the Kubernetes API server using kubectl. Also, the Heptio Authenticator
uses the AWS SDK for Go to authenticate against your Amazon EKS cluster. If you use the
console to create the cluster, you must ensure that the same IAM user credentials are in the AWS
SDK credential chain when you are running kubectl commands on your cluster.
If you install and configure the AWS CLI, you can configure the IAM credentials for your user.
These also work for the Heptio Authenticator. If the AWS CLI is configured properly for your
user, then the Heptio Authenticator can find those credentials as well. For more information, see
Configuring the AWS CLI in the AWS Command Line Interface User Guide.
17
Amazon EKS User Guide
Creating a Cluster
Note
You may receive an error that one of the Availability Zones in your request does not have
sufficient capacity to create an Amazon EKS cluster. If this happens, the error output
contains the Availability Zones that can support a new cluster. Retry creating your cluster
with at least two subnets that are located in the supported Availability Zones for your
account.
4. On the Clusters page, choose the name of your newly created cluster to view the cluster
information.
5. The Status field shows CREATING until the cluster provisioning process completes. When your
cluster provisioning is complete (usually less than 10 minutes), and note the API server endpoint
and Certificate authority values. These are used in your kubectl configuration.
6. Now that you have created your cluster, follow the procedures in Configure kubectl for Amazon
EKS (p. 36) and Create a kubeconfig for Amazon EKS (p. 39) to enable communication with
your new cluster.
1. Create your cluster with the following command. Substitute your cluster name, the Amazon
Resource Name (ARN) of your Amazon EKS service role that you created in Create your Amazon EKS
18
Amazon EKS User Guide
Creating a Cluster
Service Role (p. 3), and the subnet and security group IDs for the VPC you created in Create your
Amazon EKS Cluster VPC (p. 3).
Important
You must use IAM user credentials for this step, not root credentials. If you create your
Amazon EKS cluster using root credentials, you cannot authenticate to the cluster. For more
information, see How Users Sign In to Your Account in the IAM User Guide.
Note
If your IAM user does not have administrative privileges, you must explicitly add permissions
for that user to call the Amazon EKS API operations. For more information, see Creating
Amazon EKS IAM Policies (p. 48).
Output:
{
"cluster": {
"name": "devel",
"arn": "arn:aws:eks:us-west-2:111122223333:cluster/devel",
"createdAt": 1527785885.159,
"version": "1.10",
"roleArn": "arn:aws:iam::111122223333:role/eks-service-role-
AWSServiceRoleForAmazonEKS-AFNL4H8HB71F",
"resourcesVpcConfig": {
"subnetIds": [
"subnet-a9189fe2",
"subnet-50432629"
],
"securityGroupIds": [
"sg-f5c54184"
],
"vpcId": "vpc-a54041dc"
},
"status": "CREATING",
"certificateAuthority": {}
}
}
Note
You may receive an error that one of the Availability Zones in your request does not have
sufficient capacity to create an Amazon EKS cluster. If this happens, the error output
contains the Availability Zones that can support a new cluster. Retry creating your cluster
with at least two subnets that are located in the supported Availability Zones for your
account.
2. Cluster provisioning usually takes less than 10 minutes. You can query the status of your cluster with
the following command. When your cluster status is ACTIVE, you can proceed.
19
Amazon EKS User Guide
Deleting a Cluster
4. Now that you have created your cluster, follow the procedures in Configure kubectl for Amazon
EKS (p. 36) and Create a kubeconfig for Amazon EKS (p. 39) to enable communication with
your new cluster.
Deleting a Cluster
When you are done using an Amazon EKS cluster, you should delete the resources associated with it so
that you do not incur any unnecessary costs.
Important
If you have active services in your cluster that are associated with a load balancer, you must
delete those services before deleting the cluster so that the load balancers are deleted properly.
Otherwise, you can have orphaned resources in your VPC that prevent you from being able to
delete the VPC.
2. Delete any services that have an associated EXTERNAL-IP value. These services are fronted by an
Elastic Load Balancing load balancer, and you must delete them in Kubernetes to allow the load
balancer and associated resources to be properly released.
a. Select the VPC stack to delete and choose Actions, Delete Stack.
b. On the Delete Stack confirmation screen, choose Yes, Delete.
20
Amazon EKS User Guide
Amazon EKS-Optimized AMI
Worker Nodes
Worker machines in Kubernetes are called nodes. Amazon EKS worker nodes run in your AWS account
and connect to your cluster's control plane via the cluster API server endpoint.
Amazon EKS worker nodes are standard Amazon EC2 instances, and you are billed for them based on
normal EC2 On-Demand prices. For more information, see Amazon EC2 Pricing.
By default, Amazon EKS provides AWS CloudFormation templates to spin up worker nodes in your
Amazon EKS cluster. This AMI is built on top of Amazon Linux 2, and is configured to serve as the
base image for Amazon EKS worker nodes. The AMI is configured to work with Amazon EKS out of
the box, and it includes Docker, kubelet, and the AWS IAM Authenticator. The AMI also launches with
specialized Amazon EC2 user data that allows it to discover and connect to your cluster's control plane
automatically.
Note
You can track security or privacy events for Amazon Linux 2 at the Amazon Linux Security Center
or subscribe to the associated RSS feed. Security and privacy events include an overview of the
issue, what packages are affected, and how to update your instances to correct the issue.
The AWS CloudFormation worker node template launches your worker nodes with specialized Amazon
EC2 user data that allows them to discover and connect to your cluster's control plane automatically. For
more information, see Launching Amazon EKS Worker Nodes (p. 22).
For more information about worker nodes from a general Kubernetes perspective, see Nodes in the
Kubernetes documentation.
Topics
• Amazon EKS-Optimized AMI (p. 21)
• Launching Amazon EKS Worker Nodes (p. 22)
The AMI IDs for the latest Amazon EKS-optimized AMI are shown in the following table.
The AWS CloudFormation worker node template launches your worker nodes with specialized Amazon
EC2 user data that allows them to discover and connect to your cluster's control plane automatically. For
more information, see Launching Amazon EKS Worker Nodes (p. 22).
21
Amazon EKS User Guide
Amazon EKS-Optimized AMI Build Scripts
The Amazon EKS-optimized AMI is built on top of Amazon Linux 2, specifically for use as a worker node
in Amazon EKS clusters. You can use this repository to view the specifics of how the Amazon EKS team
configures kubelet, Docker, the AWS IAM Authenticator for Kubernetes, and more.
The build scripts repository includes a HashiCorp Packer template and build scripts to generate an AMI.
These scripts are the source of truth for Amazon EKS-optimized AMI builds, so you can follow the GitHub
repository to monitor changes to our AMIs. For example, perhaps you want your own AMI to use the
same version of Docker that the EKS team uses for the official AMI.
Additionally, the GitHub repository contains our Amazon EKS worker node AWS CloudFormation
templates. These templates make it easier to spin up an instance running the Amazon EKS-optimized
AMI and register it with a cluster. The templates also contain Amazon EC2 user data that runs at boot
time to configure your instance's certificate data, control plane endpoint, cluster name, and more.
If this is your first time launching Amazon EKS worker nodes, we recommend that you follow our Getting
Started with Amazon EKS (p. 3) guide instead. The guide provides a complete end-to-end walkthrough
from creating an Amazon EKS cluster to deploying a sample Kubernetes application.
Important
Amazon EKS worker nodes are standard Amazon EC2 instances, and you are billed for them
based on normal Amazon EC2 On-Demand Instance prices. For more information, see Amazon
EC2 Pricing.
• You have created a VPC and security group that meets the requirements for an Amazon EKS
cluster. For more information, see Cluster VPC Considerations (p. 27) and Cluster Security Group
Considerations (p. 28). The Getting Started with Amazon EKS (p. 3) guide creates a VPC that meets
the requirements, or you can also follow Tutorial: Creating a VPC with Public and Private Subnets for
Your Amazon EKS Cluster (p. 58) to create one manually.
• You have created an Amazon EKS cluster and specified that it use the above VPC and security group.
For more information, see Creating an Amazon EKS Cluster (p. 17).
22
Amazon EKS User Guide
Launching Amazon EKS Worker Nodes
https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/amazon-eks-
nodegroup.yaml
6. On the Specify Details page, fill out the following parameters accordingly, and choose Next:
• Stack name: Choose a stack name for your AWS CloudFormation stack. For example, you can call
it <cluster-name>-worker-nodes.
• ClusterName: Enter the name that you used when you created your Amazon EKS cluster.
Important
This name must exactly match your Amazon EKS cluster name. Otherwise, your worker
nodes will be unable to join it.
• ClusterControlPlaneSecurityGroup: Enter the security group or groups that you used when you
created your Amazon EKS cluster. This AWS CloudFormation template creates a worker node
security group that allows traffic to and from the cluster control plane security group specified.
• NodeGroupName: Enter a name for your node group that is included in your Auto Scaling node
group name.
• NodeAutoScalingGroupMinSize: Enter the minimum number of nodes to which your worker node
Auto Scaling group can scale in.
• NodeAutoScalingGroupMaxSize: Enter the maximum number of nodes to which your worker
node Auto Scaling group can scale out.
• NodeInstanceType: Choose an instance type for your worker nodes.
• NodeImageId: Enter the current Amazon EKS worker node AMI ID for your Region.
Note
The Amazon EKS worker node AMI is based on Amazon Linux 2. You can track security
or privacy events for Amazon Linux 2 at the Amazon Linux Security Center or subscribe
to the associated RSS feed. Security and privacy events include an overview of the issue,
what packages are affected, and how to update your instances to correct the issue.
• KeyName: Enter the name of an Amazon EC2 SSH key pair that you can use to connect using SSH
into your worker nodes with after they launch.
• VpcId: Enter the ID for the VPC that your worker nodes should launch into.
• Subnets: Choose the subnets within the above VPC that your worker nodes should launch into.
7. On the Options page, you can choose to tag your stack resources. Choose Next.
8. On the Review page, review your information, acknowledge that the stack might create IAM
resources, and then choose Create.
9. When your stack has finished creating, select it in the console and choose Outputs.
10. Record the NodeInstanceRole for the node group that was created. You need this when you
configure your Amazon EKS worker nodes.
1. Download, edit, and apply the AWS IAM Authenticator configuration map.
23
Amazon EKS User Guide
Launching Amazon EKS Worker Nodes
curl -O https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/aws-auth-
cm.yaml
b. Open the file with your favorite text editor. Replace the <ARN of instance role (not
instance profile)> snippet with the NodeInstanceRole value that you recorded in the
previous procedure, and save the file.
Important
Do not modify any other lines in this file.
apiVersion: v1
kind: ConfigMap
metadata:
name: aws-auth
namespace: kube-system
data:
mapRoles: |
- rolearn: <ARN of instance role (not instance profile)>
username: system:node:{{EC2PrivateDNSName}}
groups:
- system:bootstrappers
- system:nodes
c. Apply the configuration. This command may take a few minutes to finish.
Note
If you receive the error "heptio-authenticator-aws": executable file not
found in $PATH, then your kubectl is not configured for Amazon EKS. For more
information, see Configure kubectl for Amazon EKS (p. 36).
2. Watch the status of your nodes and wait for them to reach the Ready status.
24
Amazon EKS User Guide
Storage Classes
Amazon EKS clusters are not created with any storage classes. You must define storage classes for your
cluster to use and you should define a default storage class for your persistent volume claims. For more
information, see Storage Classes in the Kubernetes documentation.
1. Create an AWS storage class manifest file for your storage class. The below example defines a
storage class called gp2 that uses the Amazon EBS gp2 volume type. For more information about
the options available for AWS storage classes, see AWS in the Kubernetes documentation. For this
example, the file is called gp2-storage-class.yaml.
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: gp2
provisioner: kubernetes.io/aws-ebs
parameters:
type: gp2
reclaimPolicy: Retain
mountOptions:
- debug
2. Use kubectl to create the storage class from the manifest file.
Output:
1. List the existing storage classes for your cluster. A storage class must be defined before you can set it
as a default.
Output:
2. Choose a storage class and set it as your default by setting the storageclass.kubernetes.io/
is-default-class=true annotation.
Output:
25
Amazon EKS User Guide
Output:
26
Amazon EKS User Guide
Cluster VPC Considerations
Topics
• Cluster VPC Considerations (p. 27)
• Cluster Security Group Considerations (p. 28)
• Pod Networking (p. 29)
• Installing Calico on Amazon EKS (p. 31)
The subnets that you pass when you create the cluster influence where Amazon EKS places elastic
network interfaces that are used for the control plane to worker node communication.
It is possible to specify only public or private subnets when you create your cluster, but there are some
limitations associated with these configurations:
• Private-only: Everything runs in a private subnet and Kubernetes cannot create internet-facing load
balancers for your pods.
• Public-only: Everything runs in a public subnet, including your worker nodes.
Amazon EKS creates an elastic network interface in your private subnets to facilitate communication
to your worker nodes. This communication channel supports Kubernetes functionality such as kubectl
exec and kubectl logs. The security group that you specify when you create your cluster is applied to the
elastic network interfaces that are created for your cluster control plane.
Key Value
kubernetes.io/cluster/<cluster-name> shared
• Key: The <cluster-name> value matches your Amazon EKS cluster's name.
• Value: The shared value allows more than one cluster to use this VPC.
27
Amazon EKS User Guide
Subnet Tagging Requirement
Key Value
kubernetes.io/cluster/<cluster-name> shared
The security group for the worker nodes and the security group for the control plane communication to
the worker nodes have been set up to prevent communication to privileged ports in the worker nodes.
If your applications require added inbound or outbound access from the control plane or worker nodes,
you must add these rules to the security groups associated with your cluster. For more information, see
Security Groups for Your VPC in the Amazon VPC User Guide.
Note
To allow proxy functionality on privileged ports or to run the CNCF conformance tests yourself,
you must edit the security groups for your control plane and the worker nodes. The security
group on the worker nodes side need to allow inbound access for ports 0-66535 from the
control plane, and the control plane side needs to allow outbound access to the worker nodes
on ports 0-65535.
The following tables show the minimum required and recommended security group settings for the
control plane and worker node security groups for your cluster:
28
Amazon EKS User Guide
Pod Networking
Pod Networking
Amazon EKS supports native VPC networking via the Amazon VPC CNI plugin for Kubernetes. Using this
CNI plugin allows Kubernetes pods to have the same IP address inside the pod as they do on the VPC
network. This CNI plugin is an open-source project that is maintained on GitHub.
29
Amazon EKS User Guide
Pod Networking
The CNI plugin is responsible for allocating VPC IP addresses to Kubernetes nodes and configuring the
necessary networking for pods on each node. The plugin consists of two primary components:
30
Amazon EKS User Guide
Installing Calico on Amazon EKS
• The L-IPAM daemon is responsible for attaching elastic network interfaces to instances, assigning
secondary IP addresses to elastic network interfaces, and maintaining a "warm pool" of IP addresses on
each node for assignment to Kubernetes pods when they are scheduled.
• The CNI plugin itself is responsible for wiring the host network (for example, configuring the interfaces
and virtual Ethernet pairs) and adding the correct interface to the pod namespace.
For more information about the design and networking configuration, see Proposal: CNI plugin for
Kubernetes networking over AWS VPC.
Elastic network interface and secondary IP address limitations by Amazon EC2 instance types are
applicable. In general, larger instances can support more IP addresses. For more information, see IP
Addresses Per Network Interface Per Instance Type in the Amazon EC2 User Guide for Linux Instances.
1. Apply the Calico manifest from the aws/amazon-vpc-cni-k8s GitHub project. This manifest
creates daemon sets in the kube-system namespace.
2. Watch the kube-system daemon sets and wait for the calico-node daemon set to have the
DESIRED number of pods in the READY state. When this happens, Calico is working.
Output:
Before you create any network policies, all services can communicate bidirectionally. After you apply the
network policies, you can see that the client can only communicate with the front-end service, and the
backend can only communicate with the frontend.
31
Amazon EKS User Guide
Stars Policy Demo
3. To connect to the management UI, forward your local port 9001 to the management-ui service
running on your cluster:
4. Open a browser on your local system and point it to http://localhost:9001/. You should see the
management UI. The C node is the client service, the F node is the front-end service, and the B node
is the back-end service. Each node has full communication access to all other nodes (as indicated by
the bold, colored lines).
32
Amazon EKS User Guide
Stars Policy Demo
5. Apply the following network policies to isolate the services from each other:
6. Refresh your browser, and you can see that the management UI can no longer reach any of the
nodes, so they don't show up in the UI.
7. Apply the following network policies to allow the management UI to access the services:
33
Amazon EKS User Guide
Stars Policy Demo
8. Refresh your browser, and you can see that the management UI can reach the nodes again, but the
nodes cannot communicate with each other.
9. Apply the following network policy to allow traffic from the front-end service to the back-end
service:
10. Apply the following network policy to allow traffic from the client namespace to the front-end
service:
34
Amazon EKS User Guide
Stars Policy Demo
11. (Optional) When you are done with the demo, you can delete its resources with the following
command:
35
Amazon EKS User Guide
Configure kubectl for Amazon EKS
Topics
• Configure kubectl for Amazon EKS (p. 36)
• Create a kubeconfig for Amazon EKS (p. 39)
• Managing Users or IAM Roles for your Cluster (p. 41)
1. Download and install kubectl for your operating system. Amazon EKS vends kubectl binaries that
you can use, or you can follow the instructions in the Kubernetes documentation to install.
• To install the Amazon EKS-vended version of kubectl:
36
Amazon EKS User Guide
Configure kubectl for Amazon EKS
• Linux: https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/linux/
amd64/kubectl
• MacOS: https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/
darwin/amd64/kubectl
• Windows: https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/
windows/amd64/kubectl.exe
Use the command below to download the binary, substituting the correct URL for your
platform. The example below is for macOS clients.
b. (Optional) Verify the downloaded binary with the MD5 sum provided in the same bucket
prefix, substituting the correct URL for your platform. The example below is to download
the MD5 sum for macOS clients.
chmod +x ./kubectl
d. Copy the binary to a folder in your $PATH. If you have already installed a version of kubectl
(from Homebrew or Apt), then we recommend creating a $HOME/bin/kubectl and
ensuring that $HOME/bin comes first in your $PATH.
e. (Optional) Add the $HOME/bin path to your shell initialization file so that it is configured
when you open a shell.
• Or, to install kubectl using the Kubernetes documentation, see Install and Set Up kubectl in the
Kubernetes documentation.
2. After you install kubectl, you can verify its version with the following command:
Example output:
37
Amazon EKS User Guide
Configure kubectl for Amazon EKS
Amazon EKS vends heptio-authenticator-aws binaries that you can use, or you can use go get
to fetch the binary from the AWS IAM Authenticator for Kubernetes project on GitHub for other
operating systems.
• To download and install the Amazon EKS-vended heptio-authenticator-aws binary for
Linux:
• Linux: https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/linux/
amd64/heptio-authenticator-aws
• MacOS: https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/
darwin/amd64/heptio-authenticator-aws
• Windows: https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/bin/
windows/amd64/heptio-authenticator-aws.exe
Use the command below to download the binary, substituting the correct URL for your
platform. The example below is for macOS clients.
b. (Optional) Verify the downloaded binary with the MD5 sum provided in the same bucket
prefix, substituting the correct URL for your platform. The example below is to download
the MD5 sum for macOS clients.
chmod +x ./heptio-authenticator-aws
38
Amazon EKS User Guide
Create a kubeconfig for Amazon EKS
heptio-authenticator-aws help
a. Install the Go programming language for your operating system if you do not already have
go installed. For more information, see Install the Go tools in the Go documentation.
b. Use go get to install the aws-iam-authenticator binary.
go get -u -v github.com/kubernetes-sigs/aws-iam-authenticator/cmd/aws-iam-
authenticator
aws-iam-authenticator help
Amazon EKS uses the Heptio Authenticator with kubectl for cluster authentication, which uses the same
default AWS credential provider chain as the AWS CLI and AWS SDKs. If you have installed the AWS CLI
on your system, then by default the Heptio authenticator will use the same credentials that are returned
with the following command:
For more information, see Configuring the AWS CLI in the AWS Command Line Interface User Guide.
To instead have the Heptio authenticator assume a role to perform cluster operations, uncomment the -
r and <role-arn> lines and substitute an IAM role ARN to use with your user.
39
Amazon EKS User Guide
Create a kubeconfig for Amazon EKS
If you manage multiple AWS credential profiles, you can either set the AWS_PROFILE variable in your
shell or specify the profile name in an environment variable value for the authenticator to use in your
kubeconfig as shown in the procedure below.
If you do not have an existing configuration, or to add the Amazon EKS cluster without modifying your
existing configuration files, you can use the following procedure to add the Amazon EKS cluster to your
configuration.
When your cluster provisioning is complete, retrieve the endpoint and certificateAuthority.data
values with the following commands. These must be added to your kubectl configuration so that you
can communicate with your cluster.
1. Retrieve the endpoint for your cluster. Use this for the <endpoint-url> in your kubeconfig file.
2. Retrieve the certificateAuthority.data for your cluster. Use this for the <base64-encoded-
ca-cert> in your kubeconfig file.
mkdir -p ~/.kube
2. Open your favorite text editor and copy the kubeconfig code block below into it.
apiVersion: v1
clusters:
- cluster:
server: <endpoint-url>
certificate-authority-data: <base64-encoded-ca-cert>
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: aws
name: aws
current-context: aws
kind: Config
preferences: {}
users:
- name: aws
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
command: heptio-authenticator-aws
args:
- "token"
- "-i"
- "<cluster-name>"
# - "-r"
# - "<role-arn>"
# env:
# - name: AWS_PROFILE
40
Amazon EKS User Guide
Managing Users or IAM Roles for your Cluster
# value: "<aws-profile>"
3. Replace the <endpoint-url> with the endpoint URL that was created for your cluster.
4. Replace the <base64-encoded-ca-cert> with the certificateAuthority.data that was
created for your cluster.
5. Replace the <cluster-name> with your cluster name.
6. (Optional) To have the Heptio authenticator assume a role to perform cluster operations (instead
of the default AWS credential provider chain), uncomment the -r and <role-arn> lines and
substitute an IAM role ARN to use with your user.
7. (Optional) To have the Heptio authenticator always use a specific named AWS credential profile
(instead of the default AWS credential provider chain), uncomment the env lines and substitute
<aws-profile> with the profile name to use.
8. Save the file to the default kubectl folder, with your cluster name in the file name. For example, if
your cluster name is devel, save the file to ~/.kube/config-devel.
9. Add that file path to your KUBECONFIG environment variable so that kubectl knows where to look
for your cluster configuration.
export KUBECONFIG=$KUBECONFIG:~/.kube/config-devel
10. (Optional) Add the configuration to your shell initialization file so that it is configured when you
open a shell.
Note
If you receive the error "heptio-authenticator-aws": executable file not
found in $PATH, then your kubectl is not configured for Amazon EKS. For more
information, see Configure kubectl for Amazon EKS (p. 36).
Output:
The aws-auth ConfigMap is applied as part of the Getting Started with Amazon EKS (p. 3) guide which
provides a complete end-to-end walkthrough from creating an Amazon EKS cluster to deploying a
41
Amazon EKS User Guide
Managing Users or IAM Roles for your Cluster
sample Kubernetes application. It is initially created to allow your worker nodes to join your cluster, but
you also use this ConfigMap to add RBAC access to IAM users and roles. If you have not launched worker
nodes and applied the aws-auth ConfigMap, you can do so with the following procedure.
If you receive an error stating "Error from server (NotFound): configmaps "aws-auth"
not found", then proceed with the following steps to apply the stock ConfigMap.
2. Download, edit, and apply the AWS authenticator configuration map.
curl -O https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/aws-auth-
cm.yaml
b. Open the file with your favorite text editor. Replace the <ARN of instance role (not
instance profile)> snippet with the NodeInstanceRole value that you recorded in the
previous procedure, and save the file.
Important
Do not modify any other lines in this file.
apiVersion: v1
kind: ConfigMap
metadata:
name: aws-auth
namespace: kube-system
data:
mapRoles: |
- rolearn: <ARN of instance role (not instance profile)>
username: system:node:{{EC2PrivateDNSName}}
groups:
- system:bootstrappers
- system:nodes
c. Apply the configuration. This command may take a few minutes to finish.
Note
If you receive the error "heptio-authenticator-aws": executable file not
found in $PATH, then your kubectl is not configured for Amazon EKS. For more
information, see Configure kubectl for Amazon EKS (p. 36).
3. Watch the status of your nodes and wait for them to reach the Ready status.
1. Ensure that the AWS credentials that kubectl is using are already authorized for your cluster. The
IAM user that created the cluster has these permissions by default.
2. Open the aws-auth ConfigMap.
42
Amazon EKS User Guide
Managing Users or IAM Roles for your Cluster
Note
If you receive an error stating "Error from server (NotFound): configmaps "aws-
auth" not found", then use the previous procedure to apply the stock ConfigMap.
Example ConfigMap:
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will
be
# reopened with the relevant failures.
#
apiVersion: v1
data:
mapRoles: |
- rolearn: arn:aws:iam::111122223333:role/doc-test-worker-nodes-NodeInstanceRole-
WDO5P42N3ETB
username: system:node:{{EC2PrivateDNSName}}
groups:
- system:bootstrappers
- system:nodes
kind: ConfigMap
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","data":{"mapRoles":"- rolearn: arn:aws:iam::111122223333:role/
doc-test-worker-nodes-NodeInstanceRole-WDO5P42N3ETB\n username: system:node:
{{EC2PrivateDNSName}}\n groups:\n - system:bootstrappers\n -
system:nodes\n"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"aws-
auth","namespace":"kube-system"}}
creationTimestamp: 2018-04-04T18:49:10Z
name: aws-auth
namespace: kube-system
resourceVersion: "780"
selfLink: /api/v1/namespaces/kube-system/configmaps/aws-auth
uid: dcc31de5-3838-11e8-af26-02e00430057c
• To add an IAM user: add the user details to the mapUsers section of the ConfigMap, under
data. Add this section if it does not already exist in the file. Each entry supports the following
parameters:
• userarn: The ARN of the IAM user to add.
• username: The user name within Kubernetes to map to the IAM user. By default, the user name
is the ARN of the IAM user.
• groups: A list of groups within Kubernetes to which the user is mapped to.
• To add an IAM role: add the role details to the mapRoles section of the ConfigMap, under
data. Add this section if it does not already exist in the file. Each entry supports the following
parameters:
• rolearn: The ARN of the IAM role to add.
• username: The user name within Kubernetes to map to the IAM role. By default, the user name
is the ARN of the IAM role.
• groups: A list of groups within Kubernetes to which the role is mapped.
• To add an AWS account to be auto-mapped: add the AWS account ID (enclosed in quotation
marks) to the mapAccounts section of the ConfigMap, under data. Add this section if it does not
already exist in the file. Every AWS user and AWS role in that account is automatically mapped
43
Amazon EKS User Guide
Managing Users or IAM Roles for your Cluster
to a user in the Kubernetes cluster with the Amazon Resource Name (ARN) of that user or role as
the username. However, no permissions are provided in RBAC by this action alone; you must still
create role bindings in your cluster to provide these entities permissions.
• A mapRoles section that adds the worker node instance role so that worker nodes can register
themselves with the cluster.
• A mapUsers section with the AWS users admin from the default AWS account, and ops-user
from another AWS account. Both users are added to the system:masters group.
• A mapAccounts section with the AWS account, 111122223333.
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will
be
# reopened with the relevant failures.
#
apiVersion: v1
data:
mapRoles: |
- rolearn: arn:aws:iam::555555555555:role/devel-worker-nodes-
NodeInstanceRole-74RF4UBDUKL6
username: system:node:{{EC2PrivateDNSName}}
groups:
- system:bootstrappers
- system:nodes
mapUsers: |
- userarn: arn:aws:iam::555555555555:user/admin
username: admin
groups:
- system:masters
- userarn: arn:aws:iam::111122223333:user/ops-user
username: ops-user
groups:
- system:masters
mapAccounts: |
- "111122223333"
44
Amazon EKS User Guide
The following table provides limitations for Amazon EKS that cannot be changed.
45
Amazon EKS User Guide
Policy Structure
When you attach a policy to a user or group of users, it allows or denies the users permission to perform
the specified tasks on the specified resources. For more information, see Permissions and Policies in the
IAM User Guide. For more information about managing and creating custom IAM policies, see Managing
IAM Policies.
Likewise, Amazon EKS makes calls to other AWS services on your behalf, so the service must authenticate
with your credentials. This authentication is accomplished by creating an IAM role and policy that can
provide these permissions and then associating that role with your compute environments when you
create them. For more information, see Amazon EKS Service IAM Role (p. 49) and also IAM Roles in the
IAM User Guide.
Getting Started
An IAM policy must grant or deny permissions to use one or more Amazon EKS actions.
Topics
• Policy Structure (p. 46)
• Creating Amazon EKS IAM Policies (p. 48)
• Amazon EKS Service IAM Role (p. 49)
Policy Structure
The following topics explain the structure of an IAM policy.
Topics
• Policy Syntax (p. 46)
• Actions for Amazon EKS (p. 47)
• Checking That Users Have the Required Permissions (p. 47)
Policy Syntax
An IAM policy is a JSON document that consists of one or more statements. Each statement is structured
as follows:
{
"Statement":[{
"Effect":"effect",
46
Amazon EKS User Guide
Actions for Amazon EKS
"Action":"action",
"Resource":"arn",
"Condition":{
"condition":{
"key":"value"
}
}
}
]
}
• Effect: The effect can be Allow or Deny. By default, IAM users don't have permission to use resources
and API actions, so all requests are denied. An explicit allow overrides the default. An explicit deny
overrides any allows.
• Action: The action is the specific API action for which you are granting or denying permission.
• Resource: The resource that's affected by the action. Amazon EKS API operations currently do not
support resource level permissions, so you must use the * wildcard to specify that all resources can be
affected by the action.
• Condition: Conditions are optional. They can be used to control when your policy is in effect.
For more information about example IAM policy statements for Amazon EKS, see Creating Amazon EKS
IAM Policies (p. 48).
To specify multiple actions in a single statement, separate them with commas as follows:
You can also specify multiple actions using wildcards. For example, you can specify all actions whose
name begins with the word "Describe" as follows:
"Action": "eks:Describe*"
To specify all Amazon EKS API actions, use the * wildcard as follows:
"Action": "eks:*"
First, create an IAM user for testing purposes, and then attach the IAM policy that you created to the test
user. Then, make a request as the test user. You can make test requests in the console or with the AWS
CLI.
47
Amazon EKS User Guide
Creating IAM Policies
Note
You can also test your policies with the IAM Policy Simulator. For more information on the policy
simulator, see Working with the IAM Policy Simulator in the IAM User Guide.
If the policy doesn't grant the user the permissions that you expected, or is overly permissive, you can
adjust the policy as needed and retest until you get the desired results.
Important
It can take several minutes for policy changes to propagate before they take effect. Therefore,
we recommend that you allow five minutes to pass before you test your policy updates.
If an authorization check fails, the request returns an encoded message with diagnostic information. You
can decode the message using the DecodeAuthorizationMessage action. For more information, see
DecodeAuthorizationMessage in the AWS Security Token Service API Reference, and decode-authorization-
message in the AWS CLI Command Reference.
When you attach a policy to a user or group of users, it allows or denies the users permission to perform
the specified tasks on the specified resources. For more information, see Permissions and Policies in the
IAM User Guide. For more information about managing and creating custom IAM policies, see Managing
IAM Policies.
If your IAM user does not have administrative privileges, you must explicitly add permissions for that user
to call the Amazon EKS API operations.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"eks:*"
],
"Resource": "*"
}
]
}
48
Amazon EKS User Guide
Amazon EKS Service IAM Role
• AmazonEKSServicePolicy
• AmazonEKSClusterPolicy
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:CreateNetworkInterface",
"ec2:CreateNetworkInterfacePermission",
"ec2:DeleteNetworkInterface",
"ec2:DescribeInstances",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeVpcs",
"ec2:ModifyNetworkInterfaceAttribute",
"iam:ListAttachedRolePolicies"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"ec2:CreateTags",
"ec2:DeleteTags"
],
"Resource": [
"arn:aws:ec2:*:*:vpc/*",
"arn:aws:ec2:*:*:subnet/*"
]
}
]
}
{
"Version": "2012-10-17",
"Statement": [
49
Amazon EKS User Guide
Amazon EKS Service IAM Role
{
"Effect": "Allow",
"Action": [
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:UpdateAutoScalingGroup",
"ec2:AttachVolume",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:CreateRoute",
"ec2:CreateSecurityGroup",
"ec2:CreateTags",
"ec2:CreateVolume",
"ec2:DeleteRoute",
"ec2:DeleteSecurityGroup",
"ec2:DeleteVolume",
"ec2:DescribeInstances",
"ec2:DescribeRouteTables",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeVolumes",
"ec2:DescribeVolumesModifications",
"ec2:DescribeVpcs",
"ec2:DetachVolume",
"ec2:ModifyInstanceAttribute",
"ec2:ModifyVolume",
"ec2:RevokeSecurityGroupIngress",
"elasticloadbalancing:AddTags",
"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer",
"elasticloadbalancing:AttachLoadBalancerToSubnets",
"elasticloadbalancing:ConfigureHealthCheck",
"elasticloadbalancing:CreateListener",
"elasticloadbalancing:CreateLoadBalancer",
"elasticloadbalancing:CreateLoadBalancerListeners",
"elasticloadbalancing:CreateLoadBalancerPolicy",
"elasticloadbalancing:CreateTargetGroup",
"elasticloadbalancing:DeleteListener",
"elasticloadbalancing:DeleteLoadBalancer",
"elasticloadbalancing:DeleteLoadBalancerListeners",
"elasticloadbalancing:DeleteTargetGroup",
"elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
"elasticloadbalancing:DeregisterTargets",
"elasticloadbalancing:DescribeListeners",
"elasticloadbalancing:DescribeLoadBalancerAttributes",
"elasticloadbalancing:DescribeLoadBalancerPolicies",
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:DescribeTargetGroupAttributes",
"elasticloadbalancing:DescribeTargetGroups",
"elasticloadbalancing:DescribeTargetHealth",
"elasticloadbalancing:DetachLoadBalancerFromSubnets",
"elasticloadbalancing:ModifyListener",
"elasticloadbalancing:ModifyLoadBalancerAttributes",
"elasticloadbalancing:ModifyTargetGroup",
"elasticloadbalancing:ModifyTargetGroupAttributes",
"elasticloadbalancing:RegisterInstancesWithLoadBalancer",
"elasticloadbalancing:RegisterTargets",
"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer",
"elasticloadbalancing:SetLoadBalancerPoliciesOfListener",
"kms:DescribeKey"
],
"Resource": "*"
}
]
}
You can use the following procedure to check and see if your account already has the Amazon EKS
service role and to attach the managed IAM policy if needed.
50
Amazon EKS User Guide
Amazon EKS Service IAM Role
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "eks.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
51
Amazon EKS User Guide
52
Amazon EKS User Guide
Prerequisites
Prerequisites
This tutorial assumes the following:
53
Amazon EKS User Guide
Step 1: Deploy the Dashboard
• You have created an Amazon EKS cluster by following the steps in Getting Started with Amazon
EKS (p. 3).
• The security groups for your control plane elastic network interfaces and worker nodes follow the
recommended settings in Cluster Security Group Considerations (p. 28).
• You are using a kubectl client that is configured to communicate with your Amazon EKS cluster (p. 9).
Output:
2. Deploy heapster to enable container cluster monitoring and performance analysis on your cluster:
Output:
Output:
Output:
54
Amazon EKS User Guide
Step 2: Create an eks-admin Service
Account and Cluster Role Binding
apiVersion: v1
kind: ServiceAccount
metadata:
name: eks-admin
namespace: kube-system
Output:
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: eks-admin
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: eks-admin
namespace: kube-system
Output:
55
Amazon EKS User Guide
Step 3: Connect to the Dashboard
1. Retrieve an authentication token for the eks-admin service account. Copy the
<authentication_token> value from the output. You use this token to connect to the dashboard.
kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep eks-
admin | awk '{print $1}')
Output:
Name: eks-admin-token-b5zv4
Namespace: kube-system
Labels: <none>
Annotations: kubernetes.io/service-account.name=eks-admin
kubernetes.io/service-account.uid=bcfe66ac-39be-11e8-97e8-026dce96b6e8
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1025 bytes
namespace: 11 bytes
token: <authentication_token>
kubectl proxy
3. Open the following link with a web browser to access the dashboard endpoint: http://
localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/
4. Choose Token, paste the <authentication_token> output from the previous command into the
Token field, and choose SIGN IN.
56
Amazon EKS User Guide
Step 4: Next Steps
Note
It may take a few minutes before CPU and memory metrics appear in the dashboard.
57
Amazon EKS User Guide
Step 1: Create an Elastic IP Address for Your NAT Gateway
Topics
• Step 1: Create an Elastic IP Address for Your NAT Gateway (p. 58)
• Step 2: Run the VPC Wizard (p. 58)
• Step 3: Create Additional Subnets (p. 59)
• Step 4: Create a Control Plane Security Group (p. 59)
• Next Steps (p. 60)
58
Amazon EKS User Guide
Step 3: Create Additional Subnets
The worker node AWS CloudFormation template used in Step 3: Launch and Configure Amazon EKS
Worker Nodes (p. 12) creates a worker node security group, and it applies the necessary rules to allow
communication with the control plane automatically, but you must specify the control plane security
group when you create a stack from that template.
59
Amazon EKS User Guide
Next Steps
1. In the left navigation pane, for Filter by VPC, select your VPC and choose Security Groups, Create
Security Group.
Note
If you don't see your new VPC here, refresh the page to pick it up.
2. Fill in the following fields and choose Yes, Create:
• For Name tag, provide a name for your security group. For example, <cluster-name>-
control-plane.
• For Description, provide a description of your security group to help you identify it later.
• For VPC, choose the VPC that you are using for your Amazon EKS cluster.
Next Steps
After you have created your VPC, you can try the Getting Started with Amazon EKS (p. 3) walkthrough,
but you can skip the Create your Amazon EKS Cluster VPC (p. 3) section and use these subnets and
security groups for your cluster.
60
Amazon EKS User Guide
Amazon EKS Information in CloudTrail
To learn more about CloudTrail, including how to configure and enable it, see the AWS CloudTrail User
Guide.
Every event or log entry contains information about who generated the request. The identity
information helps you determine the following:
• Whether the request was made with root or IAM user credentials.
• Whether the request was made with temporary security credentials for a role or federated user.
• Whether the request was made by another AWS service.
You can also create a trail and store your log files in your Amazon S3 bucket for as long as you want, and
define Amazon S3 lifecycle rules to archive or delete log files automatically. By default, your log files are
encrypted with Amazon S3 server-side encryption (SSE).
To be notified of log file delivery, configure CloudTrail to publish Amazon SNS notifications when new
log files are delivered. For more information, see Configuring Amazon SNS Notifications for CloudTrail.
You can also aggregate Amazon EKS log files from multiple AWS Regions and multiple AWS accounts
into a single Amazon S3 bucket.
For more information, see Receiving CloudTrail Log Files from Multiple Regions and Receiving CloudTrail
Log Files from Multiple Accounts.
61
Amazon EKS User Guide
Understanding Amazon EKS Log File Entries
The following example shows a CloudTrail log entry that demonstrates the CreateCluster action.
{
"eventVersion": "1.05",
"userIdentity": {
"type": "IAMUser",
"principalId": "AKIAIOSFODNN7EXAMPLE",
"arn": "arn:aws:iam::111122223333:user/ericn",
"accountId": "111122223333",
"accessKeyId": "AKIAIOSFODNN7EXAMPLE",
"userName": "ericn"
},
"eventTime": "2018-05-28T19:16:43Z",
"eventSource": "eks.amazonaws.com",
"eventName": "CreateCluster",
"awsRegion": "us-west-2",
"sourceIPAddress": "205.251.233.178",
"userAgent": "PostmanRuntime/6.4.0",
"requestParameters": {
"resourcesVpcConfig": {
"subnetIds": [
"subnet-a670c2df",
"subnet-4f8c5004"
]
},
"roleArn": "arn:aws:iam::111122223333:role/AWSServiceRoleForAmazonEKS-CAC1G1VH3ZKZ",
"clusterName": "test"
},
"responseElements": {
"cluster": {
"clusterName": "test",
"status": "CREATING",
"createdAt": 1527535003.208,
"certificateAuthority": {},
"arn": "arn:aws:eks:us-west-2:111122223333:cluster/test",
"roleArn": "arn:aws:iam::111122223333:role/AWSServiceRoleForAmazonEKS-CAC1G1VH3ZKZ",
"version": "1.10",
"resourcesVpcConfig": {
"securityGroupIds": [],
"vpcId": "vpc-21277358",
"subnetIds": [
"subnet-a670c2df",
"subnet-4f8c5004"
]
}
}
},
"requestID": "a7a0735d-62ab-11e8-9f79-81ce5b2b7d37",
"eventID": "eab22523-174a-499c-9dd6-91e7be3ff8e3",
"readOnly": false,
"eventType": "AwsApiCall",
"recipientAccountId": "111122223333"
}
62
Amazon EKS User Guide
Insufficient Capacity
Insufficient Capacity
If you receive the following error while attempting to create an Amazon EKS cluster, then one of the
Availability Zones you specified does not have sufficient capacity to support a cluster.
Retry creating your cluster with subnets in your cluster VPC that are hosted in the Availability Zones
returned by this error message.
• The aws-auth-cm.yaml file does not have the correct IAM role ARN for your worker nodes. Ensure
that the worker node IAM role ARN (not the instance profile ARN) is specified in your aws-auth-
cm.yaml file. For more information, see Launching Amazon EKS Worker Nodes (p. 22).
• The ClusterName in your worker node AWS CloudFormation template does not exactly match the
name of the cluster you want your worker nodes to join. Passing an incorrect value to this field results
in an incorrect configuration of the worker node's /var/lib/kubelet/kubeconfig file, and the
nodes will not join the cluster.
63
Amazon EKS User Guide
Amazon EKS initial release Initial documentation for service June 5, 2018
launch
64
Amazon EKS User Guide
AWS Glossary
For the latest AWS terminology, see the AWS Glossary in the AWS General Reference.
65