Register Now

Login

Lost Password

Lost your password? Please enter your email address. You will receive a link and will create a new password via email.

Login

Register Now

It will take less than 1 minute to register for lifetime. Bonus Tip - We don't send OTP to your email id Make Sure to use your own email id for free books and giveaways

HDFS and YARN command cheat sheet

Topic – HDFS and YARN command cheat sheet
Let’s look into some of the top HDFS commands that you might require while working in a Big Data infrastructure.
HDFS and YARN command cheat sheetat sheet

HDFS and YARN command cheat sheet
HDFS and YARN command cheat sheet

HDFS and YARN command cheat sheet

  1. hadoop fs -ls
    It lists the files in hadoop home directory
  2. hadoop fs -ls -R/
    It lists the files in a recurrsive order
  3. hadoop fs -ls/
    It lists the files in hadoop root directory
  4. hadoop fs -ls -t -r
    It lists the files in reverse order, sorted by time
  5. hadoop fs -ls -s
    It lists the files in the descending order of size
  6. hadoop fs -ls /user |grep booking
    It searches the files with the name booking
  7. hadoop fs -tail /user/thedatamonk/restaurant.txt
    It will display the last 10 rows of the file restaurant.txt
  8. hadoop fs -rmdir /user/thedatamonk/direct
    It will remove directories
  9. hadoop fs -rm -R /user/thedatamonk/direct
    It will remove empty and non-empty directories as well
  10. hadoop fs -cp <complete file location 1> <complete directory location>
    It will move a file from one location to another directory.
    It copies a file from one location to another
  11. hadoop fs -mv <complete file location 1> <complete folder location>
    It cut and paste file from one location to another directory
  12. hadoop fs -copyFromLocal <Local folder location> <HDFS folder location>
    It copies file from local to HDFS location. Remember, the keyword copyFromLocal is case sensitive.
  13. hadoop fs -put <Local folder location> <HDFS folder location>
    Another command to copy files from local to HDFS location. All the words are keywords.
  14. hadoop fs -copyToLocal <HDFS data location> <Local folder location>
    It copies from HDFS to Local or desktop. Remember, the key word copyToLocal is case sensitive.
  15. hadoop fs -get <HDFS data location> <Local folder location>
    Another command to copy from HDFS to Local or desktop.
  16. hadoop fs -df -h <location>
    The above command gives the free space in the disk.
    -h converts the space into bytes
  17. hadoop fs -du -h <location>
    The above command gets the used space in the disk
  18. hadoop fs -touchz
    To create an empty file on the file system
  19. hadoop fs -cat
    It copies files to stdout

    YARN Commands
  20. yarn node -list
    list nodes in the yarn cluster
  21. yarn node -status <node id>
    It gives status of a node (memory used, free, number of containers, etc) for (first column from command above)
  22. yarn application -list
    It gives a list of Yarn applications and their state
  23. yarn logs -applicationId <app id>
    It dumps the logs for a particular application
  24. hdfs dfsadmin -safemode get
    It finds out if you’re in safe mode
  25. hdfs dfsadmin -report
    It finds out how much disk space is
    used, free, under-replicated, etc.

The Data Monk Interview Books – Don’t Miss

Now we are also available on our website where you can directly download the PDF of the topic you are interested in. On Amazon, each book costs ~299, on our website we have put it at a 60-80% discount. There are ~4000 solved interview questions prepared for you.

10 e-book bundle with 1400 interview questions spread across SQL, Python, Statistics, Case Studies, and Machine Learning Algorithms – Ideal for 0-3 years experienced candidates

23 E-book with ~2000 interview questions spread across AWS, SQL, Python, 10+ ML algorithms, MS Excel, and Case Studies – Complete Package for someone between 0 to 8 years of experience (The above 10 e-book bundle has a completely different set of e-books)

12 E-books for 12 Machine Learning algorithms with 1000+ interview questions – For those candidates who want to include any Machine Learning Algorithm in their resume and to learn/revise the important concepts. These 12 e-books are a part of the 23 e-book package

Individual 50+ e-books on separate topics

Important Resources to crack interviews (Mostly Free)

There are a few things that might be very useful for your preparation

The Data Monk Youtube channel – Here you will get only those videos that are asked in interviews with Data Analysts, Data Scientists, Machine Learning Engineers, Business Intelligence Engineers, Analytics managers, etc.
Go through the watchlist which makes you uncomfortable:-

All the list of 200 videos
Complete Python Playlist for Data Science
Company-wise Data Science Interview Questions – Must Watch
All important Machine Learning Algorithm with code in Python
Complete Python Numpy Playlist
Complete Python Pandas Playlist
SQL Complete Playlist
Case Study and Guesstimates Complete Playlist
Complete Playlist of Statistics

Keep Learning !!

Thanks,
Nitin Kamal

About TheDataMonkGrand Master

I am the Co-Founder of The Data Monk. I have a total of 6+ years of analytics experience 3+ years at Mu Sigma 2 years at OYO 1 year and counting at The Data Monk I am an active trader and a logically sarcastic idiot :)

Follow Me