a. Implementing, managing and administering the overall Hadoop infrastructure.
b. Must be proficient in installation and configuration of Hadoop clusters based on Hortonworks and Cloudera.
c. Work closely with the data science team, support team, BI team and application teams to make sure that all the big data applications are highly available and performing as expected.
d. Responsible for capacity planning and estimating the requirements for lowering or increasing the capacity of the Hadoop cluster.
e. Strong security administration skills including Kerberos, LDAP integration, Sentry and Ranger.
f. Performance Tuning of the clusters.
g. Monitoring the cluster connectivity and performance.
h. Backup and recovery tasks
i. Strong Linux administration skills.
j. Experience integrating with external tools like Elastic Search, Neo4j, etc.
Good to have
a. Knowledge on AWS, Azure and Google cloud platforms
b. Python or Scala
c. Puppet or Chef