hadoop randomwriter not reading configuration file
So with randomwriter sample, you want to write data less than 10 GB. You have searched all around the net but have not found the solution and almost gave up? :p
Ok, so I am sharing what worked for me.
My environment
- Windows 8
- HDInsight Developer preview released on March 25, 2013 (https://www.robin.eu.org/bigdata/installing-microsoft-hdinsight-developer-preview/)
Configuration test file (RandomWriter-conf.xml)
<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>test.randomwriter.maps_per_host</name> <value>2</value> </property> <property> <name>test.randomwrite.bytes_per_map</name> <value>1048576</value> </property> <property> <name>test.randomwrite.min_key</name> <value>10</value> </property> <property> <name>test.randomwrite.max_key</name> <value>1000</value> </property> <property> <name>test.randomwrite.min_value</name> <value>0</value> </property> <property> <name>test.randomwrite.max_value</name> <value>20000</value> </property> </configuration>
Command line
$ hadoop jar hadoop-examples-1.1.0-SNAPSHOT.jar randomwriter -conf RandomWriter-conf.xml /random-output
Other thoughts
- You might have missed reading the help (like me, yeah I know :p ), check with following command
$ hadoop jar hadoop-examples-1.1.0-SNAPSHOT.jar randomwriter