Problem We have multiple Spark jobs running during ETL phase of our application. Some of them are really large and complicated jobs which takes hours to complete. One of such job, which uses GraphX & Pregal libraries, started failing after running for sometime. We use Spark History server to debug any jobs after they are… Read More Missing Spark History logs
Learn how to create a MySQL instance in AWS RDS. RDS stands for Relational Database Service which is the DBaaS solution from AWS. It enables you to create database instances in the cloud with just few clicks. Such instances can be made publicly accessible or private so that it can be accessed only by other applications… Read More Creating AWS RDS MySQL instance in 5min
Back in 2008, at the LinuxWorld and Next Generation Data Center Expo, Stateless Computing was suggested as the future in Cloud. A stateless application does not rely on the session state of any kind to process a client request. For such an application you should be able to say that, for any request “this request… Read More What is better than State-less?
Quick introduction to AWS Elastic Load Balancer (ELB) and explaining how to set-up an ELB with two web server EC2 instances. The video also shows ELB in action with one EC2 instance failure. Please watch and subscribe to my YouTube channel.
Recently an emerging SMB from cookware manufacturing sector approached me to consult about some of the IT challenges they were facing. They have a simple website designed in WordPress and hosted in BigRock & a new website is under development. A limited number of corporate email accounts are also created in the same domain. They also have an ERP… Read More Setting up web-server, mail server & DB in AWS
Long back Google has reportedthat a 500ms (half a second) delay caused 20% drop in their traffic and Amazon founda 100ms of extra latency dropped their sales by 1%. So in any software application which uses persistent data, introducing a cache is a very common technique to improve the performance by reducing the latency. But… Read More Application Performance Improvement with Cache
VPC – Virtual Private Cloud – is the virtual network in AWS where you launch your EC2 instances. VPC is a powerful yet easy to use concept. This post gives you a heads up with basic VPC concepts. For more information refer the VPC user guide. You can create a VPC in simple clicks and… Read More AWS Networking (VPC) simplified
SSH using secure key is actually very simple but if you look around web the explanations are sometimes confusing. The concept is simple, similar to our door keys – to get access you should have the key. Similarly SSH Keys are used to connect from one machine to another machine without providing a password. So… Read More SSH Keys
Here I am discussing about how a client-server model can work in a NAT enviornment. Network Address Translation (NAT) It works as your office phone extentions, to the outside world the call is coming from a single number but inside there are many phone lines. NAT works completely on the network level, meaning that… Read More Connectivy in NAT evniorment