Role: Big Data/ Hadoop Architect
BU: Insights and Data
Hands on big data architecture end to end project execution
• 14+ years of software development experience building large scale distributed data processing systems/application or large scale internet systems
• Experience of at least 3 years in architecting Big Data solution at enterprise scale with at least one end to end implementation
• Strong understanding & experience of Hadoop eco system such as HDFS, MapReduce, Yarn, Spark, Scala, Hive, HBase, Phoenix, Zookeeper, Pig, Hadoop streaming, Sqoop
• Knowledge of Hadoop Security, Data Management and Governance
• Ability to articulate pros & cons of “TO-BE” design/architecture decisions across a wide spectrum of factors
• Work closely with Operations team to size, scale and tune existing and new architecture.
• Experience working in core development Big Data projects, Should be able to perform hands-on development activities particularly in Spark, HBase / Cassandra, Hive, Shell Scripts.
• Responsible for designing, developing, testing, tuning and building a large-scale data processing system
• Troubleshoot and develop on Hadoop technologies including HDFS, Hive, HBase, Phoenix, Spark, Scala Map Reduce and Hadoop ETL development via tools such as Talend
• Must have strong knowledge on Hadoop security components like Kerberos, SSL, and Encryption using TDE etc.
• Ability to engage in senior-level technology discussions.
• The ideal candidate is pro-active, shows an ability to see the big picture and can prioritize the right work items in order to optimize the overall team output.
• Should have worked in agile environments and good to have exposure to Devops
• Excellent oral and written communications skills
• Hadoop Certified Developer/Architect will have added advantage
- Should be able to benchmark systems, analyze system bottlenecks and propose solutions to eliminate them
- Should be able to clearly articulate pros and cons of various technologies and platforms
- Should be able to document use cases, solutions and recommendations
- Must have good knowledge on third party tools supporting big data
- Must possess good knowledge on NoSQL database systems