Knowledge is Power
This is a quick and raw method to setup alerts when the filesystem fill above threshold. Pre-requisites Monitored filesystems should be consistent, meaning available across all nodes passwordless ssh should be setup between the nodes. Node where the alert script...
In your search of a Linux terminal on Windows you might have stumbled across projects such as cmder and WSL. There are many more projects like those. Personally for me this was the only configuration (in this post) that ended...
Lookup yarn queue of a user from bash This bash function will lookup capacity scheduler XML and return queues for the user getYarnQueue() { grep $1 -B 1 /etc/hadoop/conf/capacity-scheduler.xml | awk -F’.’ ‘/name/{print $(NF-1)}’ } Works on Hortonworks HDP. Usage:...
At times there is a need to merge multiple consecutive lines to one. paste command makes it very easy to merge lines. Syntax: paste -d’,’ – – < input_file See the example below, robin@Home-XPS:~$ echo -e ‘1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12’ 1 2 3...
Top 20 files by size Concatenate multiple files skipping n rows Compress last X days data in one archive Compress files older than X days individually Merging multiple lines Find latest files recursively Get rows starting and ending between a...
Bash tricks Check bash version in script if ((BASH_VERSINFO[0] < 4)) then echo “Sorry, you need at least bash-4.0 to run this script.” exit 1 fi Reading columns from a file in bash array table_collection=( $(cut -d $’\t’ -f1...
What? An older bash version? Are you seeing and still not believing on something like below, MacBook-Pro:~$ bash –version GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin16) Copyright (C) 2007 Free Software Foundation, Inc. Well that is my MBP as of Mar 29,...
Split program output to console and to a file Basic usage $ command | tee out_file.txt Appending to the output file using -a $ command | tee -a out_file.txt Redirecting stderr and stdout $ command |& tee -a out_file.txt Example...
AWK syntax: awk [-Fs] “program” [file1 file2…] # commands come from DOS cmdline awk ‘program{print “foo”}’ file1 # single quotes around double quotes # NB: Don’t use single quotes alone if the embedded info will contain the # vertical bar...
Chart of similar operations with sed and awk ——————————————– string ====== sed “s/from/to/” awk ‘{sub(“from”,”to”); print}’ sed “s/from/to/g” awk ‘{gsub(“from”,”to”); print}’ sed “s/from/to/3” awk ‘{$0=gensub(“from”,”to”,3); print}’ regex ===== sed “s/reg.*$/_&_/” awk ‘{sub(/reg.*$/, “_&_”); print}’ sed “s/reg[ex]/YY/g” awk ‘{gsub(/reg[ex]/, “YY”); print}’...
More
