CSV, JSON conversion into Parquet
In this world of data explosion, every company working on consolidate data into common data format. Apache Parquet is a columnar storage format available to any project in the Hadoop ecosystem, regardless of the choice of data processing framework, data model or programming language. Parquet is built from the ground up with complex nested data structures in […]
Randomizer template for Cloud formation
An old colleague of mine reached out to me for creating a random string within cloud formation. If one has not used it in past it can get tricky. I wish amazon created a function for the same but then how would I have showcased my love to SERVERLESS with this blog. I will be […]
Lambda endpoint configuration
AWS Lambda is serverless managed service. Lambda code will run without managing any server and provisioning any hardware. This post is not for Lambda feature but to Lambda Endpoint configuration. I am intend to keep that way. Endpoints are used to connect AWS service programmatically. This connection uses “AWS privatelink“. In case organization does not […]
AWS Transit Gateway
Networking is a big challenge with growing demands on diversified environments and creating datacenter across the world. Limit is just imagination. Enterprises works around different sites, different geography but common vein that join those environments are Network. With growing demands, it’s getting complicated to manage routes between sites. AWS Transit Gateway(TGW) is born to make […]
Image Processing with Lambda/AWS API Gateway
We are clicking pics every day and the Image datastore industry is spreading its way to our lifestyle. Massive amounts of images are kept on adding every day. In this story, I like to present a tool to search for images of a given object or celebrity like Google images. Don’t get me wrong, this […]
Three Tier Architecture with AWS
Three tier application configuration using AWS resources. Application load balancer, Web server(application tier) and Database tier using Dynamodb
EFS and EC2 instance creation using Terraform templating
Automating implementation and reducing time to deploy complex environments is key. In this story, I am planning to get one of the environments that fairly used in the industry to map NFS FS over multiple subnets. This is a very basic configuration but complexity starts when you wanted to use the same template for deploying […]
Convert object-oriented data to Nosql Dynamodb — 101
The IoT Ecosystem is buzz words and needed lots of data management. We receive data but how to make use of data is the most important. This design is a very small portion of a bigger portfolio. Much more application can be integrated into this design. There are many ways to perform this transformation. Athena […]
AWS EC2 instance management — Starting and Storage instances using tagging
Perform shutting and starting of non-production instances during non-business hours. That will help with cost-saving.
