HDFS and YARN command cheat sheet
Topic – HDFS and YARN command cheat sheet
Let’s look into some of the top HDFS commands that you might require while working in a Big Data infrastructure.
HDFS and YARN command cheat sheetat sheet
HDFS and YARN command cheat sheet
- hadoop fs -ls
It lists the files in hadoop home directory - hadoop fs -ls -R/
It lists the files in a recurrsive order - hadoop fs -ls/
It lists the files in hadoop root directory - hadoop fs -ls -t -r
It lists the files in reverse order, sorted by time - hadoop fs -ls -s
It lists the files in the descending order of size - hadoop fs -ls /user |grep booking
It searches the files with the name booking - hadoop fs -tail /user/thedatamonk/restaurant.txt
It will display the last 10 rows of the file restaurant.txt - hadoop fs -rmdir /user/thedatamonk/direct
It will remove directories - hadoop fs -rm -R /user/thedatamonk/direct
It will remove empty and non-empty directories as well - hadoop fs -cp <complete file location 1> <complete directory location>
It will move a file from one location to another directory.
It copies a file from one location to another - hadoop fs -mv <complete file location 1> <complete folder location>
It cut and paste file from one location to another directory - hadoop fs -copyFromLocal <Local folder location> <HDFS folder location>
It copies file from local to HDFS location. Remember, the keyword copyFromLocal is case sensitive. - hadoop fs -put <Local folder location> <HDFS folder location>
Another command to copy files from local to HDFS location. All the words are keywords. - hadoop fs -copyToLocal <HDFS data location> <Local folder location>
It copies from HDFS to Local or desktop. Remember, the key word copyToLocal is case sensitive. - hadoop fs -get <HDFS data location> <Local folder location>
Another command to copy from HDFS to Local or desktop. - hadoop fs -df -h <location>
The above command gives the free space in the disk.
-h converts the space into bytes - hadoop fs -du -h <location>
The above command gets the used space in the disk - hadoop fs -touchz
To create an empty file on the file system - hadoop fs -cat
It copies files to stdout
YARN Commands - yarn node -list
list nodes in the yarn cluster - yarn node -status <node id>
It gives status of a node (memory used, free, number of containers, etc) for (first column from command above) - yarn application -list
It gives a list of Yarn applications and their state - yarn logs -applicationId <app id>
It dumps the logs for a particular application - hdfs dfsadmin -safemode get
It finds out if you’re in safe mode - hdfs dfsadmin -report
It finds out how much disk space is used, free, under-replicated, etc.
The Data Monk services
We are well known for our interview books and have 70+ e-book across Amazon and The Data Monk e-shop page . Following are best-seller combo packs and services that we are providing as of now
- YouTube channel covering all the interview-related important topics in SQL, Python, MS Excel, Machine Learning Algorithm, Statistics, and Direct Interview Questions
Link – The Data Monk Youtube Channel - Website – ~2000 completed solved Interview questions in SQL, Python, ML, and Case Study
Link – The Data Monk website - E-book shop – We have 70+ e-books available on our website and 3 bundles covering 2000+ solved interview questions. Do check it out
Link – The Data E-shop Page - Instagram Page – It covers only Most asked Questions and concepts (100+ posts). We have 100+ most asked interview topics explained in simple terms
Link – The Data Monk Instagram page - Mock Interviews/Career Guidance/Mentorship/Resume Making
Book a slot on Top Mate
The Data Monk e-books
We know that each domain requires a different type of preparation, so we have divided our books in the same way:
1. 2200 Interview Questions to become Full Stack Analytics Professional – 2200 Most Asked Interview Questions
2.Data Scientist and Machine Learning Engineer -> 23 e-books covering all the ML Algorithms Interview Questions
3. 30 Days Analytics Course – Most Asked Interview Questions from 30 crucial topics
You can check out all the other e-books on our e-shop page – Do not miss it
For any information related to courses or e-books, please send an email to [email protected]
