• Developing distributed computing Big Data applications using Spark, Elastic Search on MapR Hadoop
• Help drive cross team design / development via technical leadership / mentoring
• The Senior/Lead Hadoop Developer is responsible for designing, developing, testing, tuning and building a large-scale data processing system, for Data Ingestion and Data products that allow the Client to improve quality, velocity and monetization of our data assets for both Operational Applications and Analytical needs.
• This position supports this goal with strong experience in software engineering and development of solutions within the Hadoop Ecosystem.
• Troubleshoot and develop on Hadoop technologies including HDFS, Hive, Pig, Flume, HBase, Spark, Impala and Hadoop ETL development via tools such as Talend
• Experience in analyzing text, streams with emerging Hadoop-based Big Data, NoSQL.
• Analyzing data with Hive, Pig and HBase.
• 6+ years experience in UNIX and Shell Scripting.
• 3+ years experience in job scheduling tools like AutoSys.
• 3+ years experience in Pig and Hive Queries.
• 3+ years experience Hand on experience with Oozie.
• 3+ years experience in Importing and exporting the data using Sqoop from HDFS to Relational Database systems/mainframe and vice-versa.
• 3+ years experience in MapReduce.
• Experience working with Spark for data manipulation, preparation, cleansing.
• Java Experience is a strong plus