Manage AWS Elastic IPs (EIPs) and Elastic Network Interfaces (ENIs) as Custom Resources in your Kubernetes cluster and assign them your pods.
- Your pod IPs must be allocated from your VPC subnets. This is the default setup on AWS EKS by using the AWS VPC CNI plugin.
- If you wish egress traffic to be sourced from assigned EIPs: In AWS VPC CNI plugin,
AWS_VPC_K8S_CNI_EXTERNALSNATmust be set totrueorAWS_VPC_K8S_CNI_EXCLUDE_SNAT_CIDRSmust include the destination CIDR's. - Your worker nodes must reside in a public subnet with an internet gateway attached.
Create an IAM role with the policy here.
Run:
$ helm install --namespace kube-system --set aws.region=us-east-1 oci://ghcr.io/goto-opensource/k8s-aws-operator --version v1.0.0 # adjust versionIf you want to use IAM roles for service accounts, add the required trust relationship with your cluster to the IAM role and add the corresponding annotation on the service account (e.g. by setting the Helm value serviceAccount.annotations."eks.amazonaws.com/role-arn" accordingly).
Create a new file example.yaml:
apiVersion: aws.k8s.logmein.com/v1alpha1
kind: EIP
metadata:
name: my-eip
spec:
tags:
owner: My teamApply it:
$ kubectl apply -f example.yaml
eip.aws.k8s.logmein.com/my-eip createdDescribe it:
$ kubectl get eip my-eip
NAME STATE PUBLIC IP POD
my-eip allocated 34.228.250.93Request a random address from a BYOIP address pool:
apiVersion: aws.k8s.logmein.com/v1alpha1
kind: EIP
# ...
spec:
publicIPv4Pool: <your pool ID here>
# ...Request a specific address from a BYOIP address pool:
apiVersion: aws.k8s.logmein.com/v1alpha1
kind: EIP
# ...
spec:
publicIPv4Address: 12.34.56.78
# ...Adjust example.yaml to include an assignment section:
apiVersion: aws.k8s.logmein.com/v1alpha1
kind: EIP
metadata:
name: my-eip
spec:
tags:
owner: My team
assignment:
podName: some-podApply it:
$ kubectl apply -f example.yaml
eip.aws.k8s.logmein.com/my-eip configuredDescribe it:
$ kubectl get eip my-eip
NAME STATE PUBLIC IP POD
my-eip assigned 34.228.250.93 my-podAllocating and assigning can also be done in one step.
Remove the assignment section again and reapply the manifest.
$ kubectl delete eip my-eip
eip.aws.k8s.logmein.com/my-eip deletedUnassigning and releasing can also be done in one step.
You can use an initContainer as part of your pod definition to create the EIP custom resource. This requires that your pod has RBAC permissions to create EIP resources.
apiVersion: v1
kind: ServiceAccount
metadata:
name: eip-user
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: eip-user-role
rules:
- apiGroups:
- aws.k8s.logmein.com
resources:
- eips
verbs:
- '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: eip-user-rolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: eip-user-role
subjects:
- kind: ServiceAccount
name: eip-user
---
apiVersion: apps/v1
kind: Deployment
# ...
spec:
# ...
template:
spec:
# ...
serviceAccountName: eip-user
initContainers:
- name: init-eip
image: <some image that has kubectl>
env:
- name: MY_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: MY_POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
command:
- /bin/sh
- -c
- |
# allocate and assign EIP
cat <<EOS | kubectl apply -f-
apiVersion: aws.k8s.logmein.com/v1alpha1
kind: EIP
metadata:
name: $(MY_POD_NAME)
namespace: $(MY_POD_NAMESPACE)
spec:
tags:
owner: My team
pod: $(MY_POD_NAME)
namespace: $(MY_POD_NAMESPACE)
assignment:
podName: $(MY_POD_NAME)
EOS
# wait for EIP to be assigned
while [ "$(kubectl get eip $(MY_POD_NAME) -o jsonpath='{.status.state}')" != "assigned" ]
do
sleep 1
doneYou can ensure that an EIP is released when your pod is terminated by including ownerReferences in your EIP resource. Setting blockOwnerDeletion: true prevents the pod from vanishing until the EIP is unassigned and released.
apiVersion: aws.k8s.logmein.com/v1alpha1
kind: EIP
metadata:
name: my-eip
ownerReferences:
- apiVersion: v1
kind: Pod
name: some-pod
uid: ... # put the UID of the pod here
blockOwnerDeletion: true
spec:
tags:
owner: My team
assignment:
podName: some-podTo be documented