Knowledge is Power
Lookup yarn queue of a user from bash This bash function will lookup capacity scheduler XML and return queues for the user getYarnQueue() { grep $1 -B 1 /etc/hadoop/conf/capacity-scheduler.xml | awk -F’.’ ‘/name/{print $(NF-1)}’ } Works on Hortonworks HDP. Usage:...
Source: https://community.hortonworks.com/questions/112754/insert-overwrite-directory-beeline.html Error Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [user] does not have [WRITE] privilege on [/tmp/*] (state=42000,code=40000) The above error appears, even though you’ve setup ranger policies, hdfs policies are set up. You’ve checked everything and...
Source: https://docs.hortonworks.com/HDPDocuments/SS1/SmartSense-1.4.0/bk_installation/content/SSL_troubleshooting.html SmartSense SSL Troubleshooting SmartSense components use SSL for protecting communications between the HST server and agents, and between the HST server and SmartSense Gateway. If installation issues arise, you can reset these SSL certificates. HST Server To reset the...
Ambari configuration over REST Ambari configuration over REST API Need to login to ambari Access below URL, http://ambari-host:8080/api/v1/services/AMBARI/components/AMBARI_SERVER Related posts: Adding compression codec to Hortonworks data platform Permanently add jars to hadoop HDFS disk consumption – Find what...
Syntax used for connecting to Hbase using sqlline through phoenix is as listed below. ./sqlline.py 10.10.20.60:2181:/hbase-unsecure 10.10.20.60 – My Zookeeper 2181 – Zookeeper client port hbase-unsecure – value configured as zookeeper.znode.parent in hbase-site.xml Sample query /usr/hdp/current/phoenix-client/bin/psql.py zookeeperserver:2181:/hbase-unsecure /usr/hdp/current/phoenix-client/doc/examples/WEB_STAT.sql...
Table Partitioning in MySQL: (Version 5.1.6 or above) Note: Before starting backup/restore please stop all running application which usage XA_ACCESS_AUDIT table. this will be help for keeping snapshot of XA_ACCESS_AUDIT for particular timestamp. Table Partitioning in MySQL:- Partitioned tables created...
Looking to add custom SerDe and custom or third party codecs to Hortonworks HDP? Only auxlib folder trick worked for me after having tried lot of alternatives. The places where we need to add that auxlib folder containing JARs is,...
Problems Following are some problems we might come across while working in a large setup of hadoop clusters, Namenode restarts taking long time (http://nn-host:50070/dfshealth.html#tab-startup-progress) Namenode startup goes to safemode for a long time after restart Best practices for Namenode &...
Lately I tried installing xz/lzma codec on my local VM setup. The compression ratios are pretty awesome. Won’t do a benchmark here, try it out yourself 😉 Steps Download codec JAR – https://github.com/yongtang/hadoop-xz or https://mvnrepository.com/artifact/io.sensesecure/hadoop-xz Copy downloaded JAR to HDPs’ libs...
Once you sync users in Apache Ranger they will stay in the database even if we sync ranger users from a different source. All those users will clutter up the Ranger user interface. Following two scripts will help in deleting...
More
