Better WSL terminal
In your search of a Linux terminal on Windows you might have stumbled across projects such as cmder and WSL. There are many more projects like those. Personally for me this was the only configuration (in this post) that ended...
adhocshare Blog
Hadoop like a PRO
Lookup yarn queue of a user from bash This bash function will lookup capacity scheduler XML and return queues for the user getYarnQueue() { grep $1 -B 1 /etc/hadoop/conf/capacity-scheduler.xml | awk -F’.’ ‘/name/{print $(NF-1)}’ } Works on Hortonworks HDP. Usage:...
Java unsupported major minor version 52.0
If you are reading this then perhaps you are having java version mismatch. Several apps based on implementations may present you with errors such as class not found error (yes! That’s what R did to me) or class version error –...
Merge multiple consecutive lines
At times there is a need to merge multiple consecutive lines to one. paste command makes it very easy to merge lines. Syntax: paste -d’,’ – – < input_file See the example below, robin@Home-XPS:~$ echo -e ‘1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12’ 1 2 3...
Chocolatey package manager for Windows
You will find a few similarities with Homebrew (https://brew.sh/) and linuxbrew (http://linuxbrew.sh/). Chocolatey (https://chocolatey.org/) is a similar package manager for windows. Installation Installation command is a little weird than other OS. Inside an administrative command prompt, type below >”%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe” -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command “iex ((New-Object System.Net.WebClient).DownloadString(‘https://chocolatey.org/install.ps1’))” && SET “PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin”...
Etrade Pro Strategy Scanner
WN=Under+$5,+High+Vol+High+Volatility&Sh_CMU=on&Sh_BBU=on&Sh_SV=on&QSV=10&Sh_NR7_1=on&MaxPrice=5&MinVol3M=500000&MinRV=1.5&MinVWVP=0.3&MinRange120P=2&MinRange5DP=3&MinFCD=0.01&MinBB=1.5&X_NYSE=on&X_ARCA=on&X_AMEX=on&XN=on The objective Use the above etrade scanner for finding, Stocks NOT on the OTCBB or Pinks, US big board stocks only, SP between $.01 and $5.00. Bullish stocks having high volume and volatility This is good for taking...
Hive msck repair not working
Okay, so msck repair is not working and you saw something as below, 0: jdbc:hive2://hive_server:10000> msck repair table mytable; Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask (state=08S01,code=1) It seems to appear because of higher...
DistCp Between HA Clusters
Source: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_administration/content/distcp_between_ha_clusters.html DistCp Between HA Clusters To copy data between HA clusters, use the dfs.internal.nameservices property in the hdfs-site.xml file to explicitly specify the name services belonging to the local cluster, while continuing to use the dfs.nameservices property to specify all of the name services in...
Hive export to CSV
Bash function to export Hive table data to local CSV file Usage: hive_export_csv <db.table> <output.csv> [queue] Recommendation: Add to .bash_profile hive_export_csv () { if [ -z “$2” ]; then echo “Bad arguments. Usage: ${FUNCNAME[0]} <db.table> <output.csv> [queue]” else uuid=$(uuidgen)...
HiveAccessControlException Permission denied user [user] does not have [WRITE] privilege on …
Source: https://community.hortonworks.com/questions/112754/insert-overwrite-directory-beeline.html Error Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [user] does not have [WRITE] privilege on [/tmp/*] (state=42000,code=40000) The above error appears, even though you’ve setup ranger policies, hdfs policies are set up. You’ve checked everything and...
