Problem We have multiple Spark jobs running during ETL phase of our application. Some of them are really large and complicated jobs which takes hours to complete. One of such job, which uses GraphX & Pregal libraries, started failing after running for sometime. We use Spark History server to debug any jobs after they are… Read More Missing Spark History logs
Back in 2008, at the LinuxWorld and Next Generation Data Center Expo, Stateless Computing was suggested as the future in Cloud. A stateless application does not rely on the session state of any kind to process a client request. For such an application you should be able to say that, for any request “this request… Read More What is better than State-less?
In this video I am explaining how you can launch an EC2 instance in AWS. EC2 stands for Elastic Compute cloud and its the term used by Amazon Web Service (AWS) for their virtual machines. Please watch and subscribe to my YouTube channel.
Here is a video I have prepared which gives you a quick introduction to Virtualization in 5 mins. Please watch and subscribe to my YouTube channel.
Recently an emerging SMB from cookware manufacturing sector approached me to consult about some of the IT challenges they were facing. They have a simple website designed in WordPress and hosted in BigRock & a new website is under development. A limited number of corporate email accounts are also created in the same domain. They also have an ERP… Read More Setting up web-server, mail server & DB in AWS
Long back Google has reportedthat a 500ms (half a second) delay caused 20% drop in their traffic and Amazon founda 100ms of extra latency dropped their sales by 1%. So in any software application which uses persistent data, introducing a cache is a very common technique to improve the performance by reducing the latency. But… Read More Application Performance Improvement with Cache
VPC – Virtual Private Cloud – is the virtual network in AWS where you launch your EC2 instances. VPC is a powerful yet easy to use concept. This post gives you a heads up with basic VPC concepts. For more information refer the VPC user guide. You can create a VPC in simple clicks and… Read More AWS Networking (VPC) simplified
Storage is an interesting domain to learn and work with. Here is my PPT on some very basics of Storage from a DR perspective. The three basic types of Storage – DAS, NAS & SAN – are discussed at a high-level first followed by SAN in more detail.
Most of the people always consider the Java heap size as the RAM usage a Java application, but this is wrong. Actually your java application is not the process which is running when you start it, but its an instance of JVM which starts. Your application code is supplied as an argument to the JVM… Read More Heap Memory vs RAM usage
I have started reading “Java Concurrency In Practice” which is the fourth one I am reading from this list of “Top 5 Java Books”. Really enjoyed the first three – Head First Java, Head First Design Patterns and Effective Java. Putting down the summery of the first chapter from the book. Pros: Makes use… Read More Pros and Cons of Multi-threading