Categories
AWS Compute

Lambda endpoint configuration

AWS Lambda is serverless managed service. Lambda code will run without managing any server and provisioning any hardware. This post is not for Lambda feature but to Lambda Endpoint configuration. I am intend to keep that way.

Endpoints are used to connect AWS service programmatically. This connection uses “AWS privatelink“. In case organization does not want to expose VPC to public internet to communicate with AWS Lambda, endpoint comes to rescue. Endpoint is regional service.

Endpoint are two types –

  • Interface
  • Gateway

I am using Endpoint Interface in my design. I have created two lambda function for testing. When endpoint is created and configured, we need to associate subnet with endpoint. Endpoint creates interface in environment and add allocate ip in all subnet where Endpoint is associated with. This interface\ip is been used for lambda invoking. Both lambda function will use same interface ip for communication.

Architecture Diagram image-1

Note : This is one of the usecase of endpoint. Main, usecase for endpoint is sharing lambda function or any other service as Software as service. Clients can share this service across AWS accounts or even across organization.

Above design, we can still implemented by defining VPC interface while creating Lamda function. Drawback here, if you have 10 lambda function to share in the VPC, we need 10 ip from subnet which is overkill. With endpoints you need just 1 ip to invoke all lambda function for given AWS account.

Source Code

Download “lambda-endpoint-connection.yaml” file from below link –

https://github.com/yogeshagrawal11/cloud/tree/master/aws/lambda/endpoint

Prerequisite

Create user with access to run formation template.

Install AWS SDK and configure AWS environment. I am using “us-west-2” region. In case if you want use different region please use appropriate AMI from parameter. (My apologies, I have added all regions ami in parameter)

Open file “lambda-endpoint-connection.yaml” in any text editor and change following ur systemip. default value is ssh is allowed from all instances

Change Default value to your ip/32 format – Image 2

Create and Download Ec2 instance keypair and update keypair name in this field. Download keypair key file same location as cloud formation template.

update correct EC2 keypair name – Image 3

Add correct VPC cidr information. If not, default will be used.

Implementation

Cloud formation template will create following resources(not all resource mentioned in list) –

  • VpcName : VPC for this test
  • Subnet1 : Subnet totally private except to ssh
  • AppInternetGateway : Internet gateway used just to connect my system with EC2 instance.
  • AppSecurityGroup : Allows port no 22 from my system to EC2 and allows all communication within VPC
  • EC2AccessLambdaRole : This role allows EC2 instance to invoke lambda function.
  • LambdaRole : This role allows Lambda function to create log groups in cloud watch to check print output in cloud watch
  • RootInstanceProfile : Instance profile for instance. Uses EC2AccessLambdaRole for assuming permission
  • EC2Instance : Instance to invoke lambda function
  • LambdaFunction : First lambda function
  • SecoundLambdaFunction : Second lambda function
  • LambdaVPCEndpoint : Lambda vpc endpoint

Run following command to validate template is working fine

aws cloudformation validate-template –template-body file://lambda-endpoint-connection.yaml

Create Stack by executing following command –

aws cloudformation create-stack –stack-name lambdaEndpoint –template-body –capabilities CAPABILITY_IAM file://lambda-endpoint-connection.yaml

Stack creation via CLI – Image 4

This will create stack in background it will take couple of minutes. Check your stack is created successful in Events section of cloud formation.

Stack creation events – Image 5

Ensure Stack is created successfully.

Stack creation completed – Image 6

Stack outputs are saved in key value pair. Take a note of Instance publicIP. We need this output for ssh into EC2 instance to check lambda access. Take note of FirstLambdaFunction and SecondLambdaFunction values we need these value to invoke Lambda function.

Stack output – Image 7

Ensure two lambda functions created successfully. Keep a note of both function name. We need function name for invoking from our EC2 instance.

Lambda function – Image 8

VPC – Endpoint configuration is created. EC2 instance internally created via private DNS name. That name is derived as <servicename>.<region_name>.amazonaws.com.

In out case servicename is “lambda”.

Endpoint configuration – Image 9

Endpoint assigns ip address in all allocated subnets. In our case we have assigned VPC to just one subnet so that it assigns single ip address. IP 10.1.1.78 is part of subnet where endpoint is associated with.

Endpoint Subnet association – Image 10

Assign security group to endpoint. In case need to stop access for any EC2 instances to access lambda function. This security group can be used for security reason. We can use Iam policy as well to restrict access from invoking Lambda function.

Endpoint security group configuration – Image 11

Policy definition. Full access allows any user or service to access lambda function. I highly recommend to restrict access from any services or EC2 instance via Endpoint policy and security group.

Endpoint policy – Image 12

Endpoint creates network interface in VPC environment. IP is assigned to this network interface.

Network interface – Image 13

Subnet ip count also shows ip counts is reduced for /24 masked subnet.

VPC subnet configuration – Image 14

Routed table has route with internet gateway to connect my system via ssh.

Subnet route table configuration – Image 15

Security group only allows access to port 22 from entire world and all ports are open within VPC communication for inbound and outbound traffic.

Security group inbound rule – Image 16
Security group outbound rule – Image 17

Login to newly created instance use same keypair that created during pre-req phase –

SSH to system – image 18

Configure AWS with region “us-west-2” or select any region you may like to select.

Configure AWS using us-west-2 – Image 19

To check list of functions using “aws lambda list-functions

Lambda function – Image 20

To invoke function use following command. We dont have access to any external https connection but we are still able to access lambda function.

  • aws lambda invoke –function-name <first_function_name> first_response.json
  • aws lambda invoke –function-name <second_function_name> second_response.json

Output from lambda function is saved into json format.

Invoke lambda function – Image 21

Reading output files. “body” key matches output from lambda function.

Output from lambda function – Image 22

Cloud watch events. If payload is defined while invoking function, this will be visible in cloud watch event as well.

Output from Cloud watch events – Image 23

Clean-up

Delete stack to cleanup. Enter following command –

  • aws cloudformation delete-stack –stack-name lambdaEndpoint

Conclusion

Lambda Endpoint is new feature and connects lambda via AWS privatelinks via AWS internal network. Again, security is elevated as no need to open your VPC to external traffic for lambda execution. Great way to use Lambda for function as service or using Lambda across multiple AWS accounts across organization.

Enjoy !!! Keep Building !!!!

Categories
AWS Networking

AWS Transit Gateway

Networking is a big challenge with growing demands on diversified environments and creating datacenter across the world. Limit is just imagination. Enterprises works around different sites, different geography but common vein that join those environments are Network. With growing demands, it’s getting complicated to manage routes between sites. AWS Transit Gateway(TGW) is born to make Network Engineers life easy. TGW helps with following features –

  • Connect multiple VPC network environment together for given account
  • Connect multiple VPC network across multiple AWS account
  • Inter region connectivity across multiple VPC
  • Connect on-premise datacenter with VPC network via VPN or
  • Connect multiple cloud environment via VPN using BGP.

Benefits of Transit Gateway

Easy Connectivity : AWS Transit Gateway is cloud router and help easy deployment of network. Routes can\will be easily propagated into environment after adding new network to TGW.

Better visibility and control : AWS Transit Gateway Network Manager used to monitor Amazon VPC’s and edge locations from central location. This will helps to identify and react on network issue quickly.

Flexible multicast : TGW supports multicast. Multicast helps sending same content to multiple destinations.

Better Security: Amazon VPC and TGW traffic always remain on Amazon private environment. Data is encrypted. Data is also protected for common network exploits.

Inter-region peering : AWS Transit Gateway inter-region peering allows customers to route traffic across AWS Regions using the AWS global network. Inter-region peering provides a simple and cost-effective way to share resources between AWS Regions or replicate data for geographic redundancy.

Transit Gateway Components

There are 4 major components in transit gateway –

Attachments : Attach network component to gateway. Attachments will be added to single route table. Following are network devices can be connected to TGW –

  • Amazon VPC
  • An AWS Direct Connect gateway
  • Peering connection with another transit gateway
  • VPN connection to on-prem or multi-cloud network

Transit gateway route table : Default route table will be created. TGW can have multiple route table. Route table defines boundary for connection. Attachments will be added to route table. Given route table can have multiple attachment where as attachment can only be added to single route table.

Route table includes dynamic and static routes. It determine next stop by given destination ip.

Association : To attach your attachment to route table we use association. Each attachment is associated with single route table but route table can have multiple attachments.

Route Propagation : All VPC and VPN associated to route table can dynamically propagate routes to route table. If VPN configured with BGP protocol then routes from VPN network can automatically propagated to transit gateway. For VPC one must create static routes to send traffic to transit gateway. Peering attachments does not dynamically added routes to route table so we need to add static routes.

Architecture Design

We are going to test following TGW scenarios. In this architecture design I am creating “management VPC” that will be shared for entire organization. This VPC can be used for Active Directory, DNS, DHCP or NTP like common services for organization.

Project_VPC1 and Project_VPC2 will be able communicate with each other and managent_vpc. Private_VPC is isolated network(private project) and will not be able to communicate with project VPC’s, but should be able to communicate with management_vpc.

Following is architecture for this design –

Design document – Image 1

Pre-requisites

  • Region name
  • Ami ID – “ami id” depend upon region
  • Instance Role. We dont need this one explicitly, as we are not accessing any services or environment from instances.
  • Instance key pair : Create instance key pair. Add key pair name in parameterstore. Parameter name should be – “ec2-keypair”. Value should be name of your keypair name.

Source code

https://github.com/yogeshagrawal11/cloud/tree/master/aws/Network/Transit%20Gateway

Cost

This implementation has cost associated with it. With attached configuration testing is done within 1 hr then it will not take more than 50 cents.

For latest charges, look for AWS price calculator

Implementation

First time to Terraform check this blog to get started –

https://cloudtechsavvy.com/2020/09/20/terraform-initial-setup/

Run following command to start terraform –

  • ./terraform init
  • ./terraform plan
  • ./terraform apply –auto-approve

I am using terraform for implementation. Following is output for terraform –

Total 47(not 45) devices are configured.

4 VPC created

4 subnet created if you observe available ips are 1 less because one of the ip in a subnet will be used by transit gateway for data transfer and routing.

4 Route table created. Each route table will use transit gateway as target for other VPC network.

Security group – These are most important configuration configuration in real world. For DNS you will allow port 53 or AD server open appropriate ports. In my case, I am using ping for checking communication.

Private VPC will only able to communicate with management network.

Project VPC will able to communicate with other VPC where as not able to communicate with Private VPC.

Note : We don’t have to explicitly, block network in project VPC this should be blocked by transit gateway as we are not going to add propagation.

Transit Gateway created. Remember if 64512 ASN is used by existing VPN then this can be added as parameter to change it.

DNS support enables help to reach out cloud with dns names rather than ip address ,certainly a useful feature.

Transit gateway can be shared with other transit gateway for inter regional data transfer for VPC’s over Amazon private network. Its advisable to “disable” auto accept shared for security reason.

Default route table is created and all VPC not explicitly attached will be attached to default route table.

Each VPN needed to add to transit gateway as attachment.

Each route table is created. Route table can be created as per segregation one needed into environment. In my case I am creating 3 route table for 4 VPC’s. Generally in Enterprise environment, we do create 5 route table. Separate route table to backup and security environment.

Since project VPC1 and project VPC2 should have same network requirement so I added them to same route table.

Management Route table

Management route table has management VPC attachment. Propagation added from all network which needed to communicate with management VPC. In this case, management VPC should be able to communicate with all other network so added propagation from all networks. This will add all routes propagated automatically.

Private Network Route table

Private VPC attached to private network route table. Private network should be able to communicated with management VPC so added propagation for management VPC. Also route for management VPC is added automatically after propagation.

Project Route table

Project route table do have attachment from both project VPC. Propagation added for other project VPC network and management network. Respective routes are added.

Testing Environment

Management server is able to ping both private and project environment instance.

Project VPC can talk to management VPC and other project VPC’s but not with private VPC

Private VPC able to talk to management vpc but not able to communicated with any project VPC’s. That makes private VPC private within organization.

Delete terraform configuration

To delete terrform configuration. Ensure all resources are destroyed

./terraform destroy –auto-approve

Conclusion

Transit gateway is tool to connect multiple VPC, VPN and direct connect network to make communication over private network. Transit gateway can be used to isolate network traffic. This makes routing comparatively easy.

SD-WAN partner solution can be used to automate adding new remote site into AWS network.

References

https://docs.aws.amazon.com/index.html