Hello World with Apache Sentry

Hello World with Apache Sentry

Apache Sentry

Sentry is Apache project for role based authorization in hadoop. Sentry works pretty well with Apache Hive.In this blog we will talk about creating a policy in Sentry using Beeline(HiveServer2) shell.



Pre-requisite : I am having Cloudera VM with Sentry installed on it.

Hive authorization is done by creating policies in sentry.
Sentry policy can be created by Sentry Admins. We need to create sentry admin group and add that group into Sentry Admin list using cloudera manager(in sentry-site.xml). Lets create user sentryAdmin with group as sentryAdmin. Fire below command on linux.

useradd sentryAdmin

Now lets Add this group to sentry admin list.

Go to Cloudera Manager - Sentry - Configuration .
Select Sentry(Service-wide) from Scope and Main from cataegory.
Add sentryAdmin in Admins Groups(sentry.service.admin.group)
Restart Sentry service.



Its time to create a policy for user. Now lets say that I have a database in Hive and I want to give read permission to group employee1
.
Now I will create a policy for employee1 using sentryAdmin.

Sentry policy creation is a three step process.
  • Role Creation
  • Assign Role to Group
  • Assign Permission to Role
Lets follow below steps :

Create user employee1 and by default in linux it will be assigned to employee1 group.

useradd employee1

Login to Beeline using employee1

beeline
!connect jdbc:hive2://localhost:10000
username : employee1
password : <Keep it Blank>

Type show databases to make employee1 does not have permission to read anything at this moment.


Login to Beeline using sentryAdmin

beeline
!connect jdbc:hive2://localhost:10000
username : sentryAdmin
password : <Keep it Blank>

Create Role in sentry.

CREATE ROLE Role1;


Assign this role to groups for granting required privileges.

GRANT ROLE Role1 TO GROUP employee1;

Grant/REVOKE privileges to created Role on required objects (Database, Table, column etc.)


GRANT select ON database department TO ROLE Role1;

Now Re -login with employee1 and try show databases command in beeline. You should be able to see department database in output.


Happy Coding....!!!!!

Commentaires

Enregistrer un commentaire

Posts les plus consultés de ce blog

Controlling Parallelism in Spark by controlling the input partitions by controlling the input partitions

Image
It is useful to be able to control the degree of parallelism when using Spark. Spark provides a very convenient method to increase the degree of parallelism which should be adequate in practice. This blog entry of the "Inside Spark" series describes the knobs Spark uses to control the degree of parallelism. Controlling the number of partitions in the " local " mode Spark is designed ground up to enable unit testing. To control the degree of parallelism in the local mode simply utilize the function JavaRDD<String> org.apache.spark.api.java.JavaSparkContext.parallelize(List<String> list, int numSlices) The following snippet of code demonstrates this usage- public class SparkControlPartitionSizeLocally { public static void main ( String [] args ) { JavaSparkContext sc = new JavaSparkContext ( "local" , "localpartitionsizecontrol" ); String input = "four score and sev
Lire la suite

Spark performance optimization: shuffle tuning

Image
Article directory 1 shuffle tuning 1.1 Summary of tuning 1.2 Overview of ShuffleManager Development 1.3 HashShuffleManager operating principle 1.3.1 Unoptured HashShuffleManager 1.3.2 Optimized HashShuffleManager 1.4 SortShuffleManager operating principle 1.4.1 General operating mechanism 1.4.2 bypass running mechanism 1.5 shuffle related parameters tuning 1.5.1 spark.shuffle.file.buffer 1.5.2 spark.reducer.maxSizeInFlight 1.5.3 spark.shuffle.io.maxRetries 1.5.4 spark.shuffle.io.retryWait 1.5.5 spark.shuffle.memoryFraction 1.5.6 spark.shuffle.manager 1.5.7 spark.shuffle.sort.bypassMergeThreshold 1.5.8 spark.shuffle.consolidateFiles 2 write in the last words Shuffle Summary of tuning Most of the performance of Spark operations is mainly consumed in the shuffle link, because the link contains a large number of disk IO, serialization, network data transmission and othe
Lire la suite

Spark optimization

Image
One operation and maintenance 1. Master hang up, standby restart is also invalid Master defaults to 512M of memory, when the task in the cluster is particularly high, it will hang, because the master will read each task event log log to generate spark ui, the memory will naturally OOM, you can run the log See that the master of the start through the HA will naturally fail for this reason. solve Increase the Master's memory spark-env.sh , set in the master node spark-env.sh : export SPARK_DAEMON_MEMORY 10g # 根据你的实际情况 Reduce the job information stored in the Master memory spark.ui.retainedJobs 500 # 默认都是1000 spark.ui.retainedStages 500 Hang up or suspend Sometimes we will see the web node in the web ui disappear or in the dead state, the task of running the node will report a variety of lost worker errors, causing the same reasons and the above, worker memory to save a lot of ui The information leads to gc when the heartbeat is lost
Lire la suite