Top 10 Hadoop Shell Commands

Top 10 Hadoop Shell Commands

• 1.
  Subject : DMBI(2170715) Active Learning Assignment Topic Name: HADOOP SHELL COMMANDS Branch : Computer DIV. : C - 1 Enrollment No. : 160120107145 Prepared by : - Simoni Shah Guided By: Prof.Margil Shah Gandhinagar Institute of Technology(012)
• 2.
  The Apache Hadoopsoftware library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. The Hadoop Distributed File System (HDFS) is the primary data storage system used by Hadoop applications. The File System (FS) shell includes various shell-like commands that directly interact with the Hadoop Distributed File System (HDFS) as well as other file systems that Hadoop supports Hadoop
• 3.
  Hadoop Shell Commands 1.Hadoop Version Syntax : hadoop fs version This Hadoop command prints the Hadoop version 2. Create Directory Syntax : hadoop fs –mkdir<paths> Example : hadoop fs –mkdir /user/dir1 This command simply creates a directory at the given path.
• 4.
  Conti… 3. Displays alist Syntax : hadoop fs -ls <path> Example : hadoop fs -ls /user/dir1 This Hadoop HDFS ls command displays a list of the contents of a directory specified by path provided by the user. 4. Display Content Syntax : hadoop -cat <file-name> Example : hadoop fs-cat /user/dataflair/dir1/sample This Hadoop fs shell command displays the contents of the filename on consol.
• 5.
  5. Move thefile or directory Syntax : hadoop fs -mv <src> <dest> Example : hadoop fs -mv /user/dataflair/dir1/purchases.txt /user/dataflair/dir2 This basic HDFS command moves the file or directory indicated by the source to destination. 6. Copy the file or directory Syntax : hadoop fs -cp <src> <dest> Example : hadoop fs -cp /user/dataflair/dir2/purchases.txt /user/dataflair/dir1 This Hadoop File system shell command copies the file or directory identified by the source to destination. Conti…
• 6.
  Conti… 7. Remove afile or directory Syntax : hadoop fs –rm <arg> Example : hadoop fs –rm /user/dir1/abc.txt This removes the file or directory mentioned in the command. 8. Upload a file Syntax : hadoop fs –put <localsrc> … <dest_path> Example : hadoop fs –put /home/dir1/abc.txt /user/dir1 This command copies a single src file or multiple src files from local file system to Hadoop file system.
• 7.
  Conti… 9. Download afile Syntax : hadoop fs –get <dest_path> … <localsrc> Example : hadoop fs –get /user/dir1/abc.txt /home/dir1 This command copies a single src file or multiple src files from Hadoop file system to local file system . 10. Display last few line Syntax : hadoop fs –tail <path> Example : hadoop fs –tail /user/dir1/abc.txt This displays the last few lines present in the given file.