Knowledge is Power
Stuck with creating Hive tables on compressed files? Well the documentation on apache.org suggests that Hive natively supports compressed file – https://cwiki.apache.org/confluence/display/Hive/CompressedStorage Lets try that out. Store a snappy compressed file on HDFS. … thinking, I do not have such file… Wait!...
There are times when we want to parse a string that is actually a JSON. Usually that could be done with built in functions of Hive such as get_json_object(). Though get_json_object cannot parse JSON Array from my experience. These array...
At times we want access to tmux and we are just stuck due to admin restrictions or root access. There are three important parts to setting up tmux, It is dependent on libevent Compiling and installing on non-system folders require customized...
Using JsonSerDe in Hive Download JSON Serde – https://github.com/rcongiu/Hive-JSON-Serde Compile command for hive 1.2.1 – “mvn -Pcdh5 -Dcdh5.hive.version=1.2.1 clean package” . change hive version per the environment Copy json-serde/target/json-serde-1.3.8-SNAPSHOT-jar-with-dependencies.jar (or similar) to hive/lib Restart hive Sample JSON with test HiveQLs...
Add below lines to file ~/.ssh/config Host * ServerAliveCountMax 3 ServerAliveInterval 10 TCPKeepAlive yes The first line specifies that this applies to all hosts The second and third specify that SSH2 keepalive messages should be sent every 10 seconds during...
Fetch DNS servers list networksetup -getdnsservers Wi-Fi Set DNS servers list Format: networksetup -setdnsservers (Network Service) (DNS IP list) networksetup -setdnsservers Wi-Fi 208.67.222.222 208.67.220.220 8.8.8.8 8.8.4.4 References: http://osxdaily.com/2015/06/02/change-dns-command-line-mac-os-x/ Related posts: MAC terminal shortcut keys How-to: Install Hue on a Mac umask in...
Something that is very much desired while working on shell. I love to have this feature of $ssh ser[TAB] Using brew is my preferred way, just type in – $brew install bash-completion Do not forget to add the text in...
Source: https://community.hortonworks.com/articles/16846/how-to-identify-what-is-consuming-space-in-hdfs.html Script #!/usr/bin/env bash max_depth=5 largest_root_dirs=$(hdfs dfs -du -s ‘/*’ | sort -nr | perl -ane ‘print “$F[1] “‘) printf “%15s %s\n” “bytes” “directory” for ld in $largest_root_dirs; do printf “%15.0f %s\n” $(hdfs dfs -du -s $ld| cut -d’ ‘...
More
