Software, College Life and Clarity for successful careers
HDFS Basic Commands -t = sort by time -r = list in reverse -S = sort by size -R = recursively show all files under that particular directory(in this case, /user/hdpuser) -p = create if not exists hadoop fs -rmdir /user/hdpuser/dir1 (remove empty directory) hadoop fs -rm -R /user/hdpuser/dir2 (non empty directory) hadoop fs -mv…
We will be discussing interaction with HDFS using the command-line interface. For this, we can use two command prefaces: ➡ hadoop fs➡ hdfs dfs There are some basic differences between hadoop fs and hdfs dfs, but for all practical purposes of this post, both are the same. List files/directories ➡ hadoop fs -ls <dirname> (Directory…
In the previous post, we discussed HDFS and how Hadoop handles Datanode failure. In this post, we will discuss how Hadoop handles Namenode failure. Hadoop 1.0 -> Namenode is a single point of failure. Failure handling is highly costly and time consuming due to recovery of edit logs. Hadoop 2.0 -> Introduction of Secondary Namenode…
HDFS stands for Hadoop Distributed File System. It is a distributed file system that is designed to store and manage large datasets across multiple computers in a cluster. It is highly fault-tolerant and can handle large amounts of data with high throughput, making it ideal for big data processing. HDFS architecture NN(NameNode) – Name Node stores the…