HDFS and YARN command cheat sheet

Topic – HDFS and YARN command cheat sheet
Let’s look into some of the top HDFS commands that you might require while working in a Big Data infrastructure.
HDFS and YARN command cheat sheetat sheet

HDFS and YARN command cheat sheet
HDFS and YARN command cheat sheet

HDFS and YARN command cheat sheet

  1. hadoop fs -ls
    It lists the files in hadoop home directory
  2. hadoop fs -ls -R/
    It lists the files in a recurrsive order
  3. hadoop fs -ls/
    It lists the files in hadoop root directory
  4. hadoop fs -ls -t -r
    It lists the files in reverse order, sorted by time
  5. hadoop fs -ls -s
    It lists the files in the descending order of size
  6. hadoop fs -ls /user |grep booking
    It searches the files with the name booking
  7. hadoop fs -tail /user/thedatamonk/restaurant.txt
    It will display the last 10 rows of the file restaurant.txt
  8. hadoop fs -rmdir /user/thedatamonk/direct
    It will remove directories
  9. hadoop fs -rm -R /user/thedatamonk/direct
    It will remove empty and non-empty directories as well
  10. hadoop fs -cp <complete file location 1> <complete directory location>
    It will move a file from one location to another directory.
    It copies a file from one location to another
  11. hadoop fs -mv <complete file location 1> <complete folder location>
    It cut and paste file from one location to another directory
  12. hadoop fs -copyFromLocal <Local folder location> <HDFS folder location>
    It copies file from local to HDFS location. Remember, the keyword copyFromLocal is case sensitive.
  13. hadoop fs -put <Local folder location> <HDFS folder location>
    Another command to copy files from local to HDFS location. All the words are keywords.
  14. hadoop fs -copyToLocal <HDFS data location> <Local folder location>
    It copies from HDFS to Local or desktop. Remember, the key word copyToLocal is case sensitive.
  15. hadoop fs -get <HDFS data location> <Local folder location>
    Another command to copy from HDFS to Local or desktop.
  16. hadoop fs -df -h <location>
    The above command gives the free space in the disk.
    -h converts the space into bytes
  17. hadoop fs -du -h <location>
    The above command gets the used space in the disk
  18. hadoop fs -touchz
    To create an empty file on the file system
  19. hadoop fs -cat
    It copies files to stdout

    YARN Commands
  20. yarn node -list
    list nodes in the yarn cluster
  21. yarn node -status <node id>
    It gives status of a node (memory used, free, number of containers, etc) for (first column from command above)
  22. yarn application -list
    It gives a list of Yarn applications and their state
  23. yarn logs -applicationId <app id>
    It dumps the logs for a particular application
  24. hdfs dfsadmin -safemode get
    It finds out if you’re in safe mode
  25. hdfs dfsadmin -report
    It finds out how much disk space is
    used, free, under-replicated, etc.

The Data Monk services

We are well known for our interview books and have 70+ e-book across Amazon and The Data Monk e-shop page . Following are best-seller combo packs and services that we are providing as of now

  1. YouTube channel covering all the interview-related important topics in SQL, Python, MS Excel, Machine Learning Algorithm, Statistics, and Direct Interview Questions
    Link – The Data Monk Youtube Channel
  2. Website – ~2000 completed solved Interview questions in SQL, Python, ML, and Case Study
    Link – The Data Monk website
  3. E-book shop – We have 70+ e-books available on our website and 3 bundles covering 2000+ solved interview questions. Do check it out
    Link – The Data E-shop Page
  4. Instagram Page – It covers only Most asked Questions and concepts (100+ posts). We have 100+ most asked interview topics explained in simple terms
    Link – The Data Monk Instagram page
  5. Mock Interviews/Career Guidance/Mentorship/Resume Making
    Book a slot on Top Mate

The Data Monk e-books

We know that each domain requires a different type of preparation, so we have divided our books in the same way:

1. 2200 Interview Questions to become Full Stack Analytics Professional – 2200 Most Asked Interview Questions
2.Data Scientist and Machine Learning Engineer -> 23 e-books covering all the ML Algorithms Interview Questions
3. 30 Days Analytics Course – Most Asked Interview Questions from 30 crucial topics

You can check out all the other e-books on our e-shop page – Do not miss it


For any information related to courses or e-books, please send an email to nitinkamal132@gmail.com

Author: TheDataMonk

I am the Co-Founder of The Data Monk. I have a total of 6+ years of analytics experience 3+ years at Mu Sigma 2 years at OYO 1 year and counting at The Data Monk I am an active trader and a logically sarcastic idiot :)