This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
cluster:121 [2013/09/16 13:41] hmeij [Hadoop Summary] |
cluster:121 [2013/09/16 14:37] hmeij |
||
---|---|---|---|
Line 4: | Line 4: | ||
==== Hadoop Summary ==== | ==== Hadoop Summary ==== | ||
- | Our production Hadoop Cluster is based on [[http:// | + | Our production Hadoop Cluster is based on [[http:// |
* namenode (that is login node): whitetail.wesleyan.edu | * namenode (that is login node): whitetail.wesleyan.edu | ||
* whitetail also runs the Hadoop Scheduler and Health Monitor | * whitetail also runs the Hadoop Scheduler and Health Monitor | ||
- | * Health Status | + | * [[http:// |
- | * Job Tracker | + | * [[http:// |
* ssh to it directly or from any of our other tails | * ssh to it directly or from any of our other tails | ||
* resources: access to 600 GB of memory and 1.75 TB of Hadoop' | * resources: access to 600 GB of memory and 1.75 TB of Hadoop' | ||
Line 15: | Line 15: | ||
* HDFS is not backed up! | * HDFS is not backed up! | ||
- | | + | |
- | * Be sure to down load your results to / | + | * Be sure to down load your results to / |
* Data to be shared (dictionaries, | * Data to be shared (dictionaries, | ||
Line 25: | Line 25: | ||
* python | * python | ||
* perl (Hadoop:: | * perl (Hadoop:: | ||
- | * R+RHadoop (rmr2, rhbase) | + | * R+RHadoop (rmr2, rhdfs, rhbase) |
* Hbase (noSQL database) | * Hbase (noSQL database) | ||
+ | * [[http:// | ||
+ | * [[http:// | ||
* MySQL | * MySQL | ||
+ | * request a database to be set up | ||
- | + | Note: the permissions are bit weird in HDFS but I think it is sorted out. | |
+ | * If this turns into a problem we'll let everybody run as user hdfs ... | ||
Other useful pages | Other useful pages |