Hortonworks Data Platform (HDP) provides centralized enterprise services for comprehensive security to enable end-to-end protection, access, compliance and auditing of data in motion and at rest. HDP's centralized architecture-with Apache Hadoop YARN at its core-also enables consistent operations to enable provisioning, management, monitoring and deployment of Hadoop clusters for a reliable enterprise-ready data lake.
But comprehensive security and consistent operations go together, and neither is possible in isolation.
We published two blogs recently announcing Ambari 2.0 and its new ability to manage rolling upgrades. This post will look at those innovations through the security lens, because security, like operations, is a core requirement for enterprise-ready Hadoop.
Security in Hadoop Today
HDP offers comprehensive security, across all batch, interactive, or real-time workloads and access patterns. Hortonworks is focused on delivering comprehensive security across 5 pillars, namely centralized administration, authentication, authorization, audit, and data protection.
HDP provides comprehensive security by way of three key services:
-
Kerberos is an MIT standard adopted by the open source community to authenticate users attempting to access Hadoop.
-
Apache Ranger provides centralized security administration for HDFS, Hive, HBase, Storm and Knox as well as fine-grain access control.
-
Apache Knox provides perimeter security for API access and REST services.
Security Setup with Ambari 2.0
Ambari 2.0 represents a significant milestone in the community's ongoing work to make Hadoop enterprise-ready with easy security setup and administration. Now Ambari 2.0 can help administrators automate Kerberos setup for a cluster, install KDC and create service principles. Administrators can also use Ambari to install Ranger admin and enable the Ranger plugin with a few clicks.
Automated Kerberos integration
Before Ambari 2.0, the Kerberos integration in Hadoop required a combination of manual steps to install and manage these important components:
-
KDC (key distribution center),
-
User and service principles (identities) and
-
Respective keytabs (tokens).
With Ambari 2.0, the entire Kerberos setup process is automated, now with the following:
-
A step-by-step wizard to setup the Kerberos infrastructure
-
Integration with existing MIT KDC or Active Directory infrastructure
-
Deployment, configuration and management of Kerberos Clients
-
First time setup as well as ongoing management for adding new services or nodes
-
Automated creation of principals
-
Automated generation and distribution of keytabs
-
Support for regeneration of keytabs
Ambari 2.0 can automate Kerberos deployment and management for existing clusters already using Kerberos, as well as for users looking to install a new cluster.
This Kerberos Overview documentation for Ambari 2.0 contains an overview and step-by-step details on Kerberos setup.
Automated Ranger deployment
Hortonworks introduced Apache Ranger to deliver the vision of coordinated security across Hadoop with centralized administration, fine-grain access control and audit. Apache Ranger's first release included enhancements to existing capabilities in the original code base developed at XA Secure and added support for audit storage in HDFS, support for Apache Storm and Knox authorization and auditing, and also REST APIs for managing policies.
With Ambari 2.0, administrators can now easily add comprehensive security through Ranger to either an existing or new cluster. Ambari 2.0 adds in the following benefits to Ranger:
-
Automated install of Ranger policy administrator and user sync. The policy database (mySQL or Oracle) can be configured and user sync can be integrated with LDAP/AD or Unix.
-
Easy one-click setup of the Ranger plugin for HDFS, Hive , HBase, Storm and Knox
-
Ability to start/stop services through the Ambari UI
-
Ability to disable plugins through the Ambari UI
The following screen shots show a user adding Ranger service via Ambari.
Hortonworks continues to lead open-source innovation to enable comprehensive data security for Hadoop-making it easier for security administrators to protect their clusters. With Ambari 2.0, we added the automated install and administration of the HDP cluster's security infrastructure, with support for installing Kerberos, Apache Knox and Apache Ranger.
This innovation highlights what Hortonworks customers appreciate about our 100% open-source Apache Hadoop platform. HDP provides centralized enterprise services for comprehensive security and consistent operations to enable provisioning, management, monitoring and deployment of secure Hadoop clusters.
Hadoop is ready for the enterprise-providing any data, for any application, anywhere.
More About Comprehensive Security and Consistent Operations in HDP
Read recent Ambari posts