Job Role: Big Data Engineer
- Relevant experience specifically working with Cloudera stack is a plus
- Overall IT Experience – 4 to 6 years
- Relevant Experience – around 2 years
- Designation – Associate Consultant, Software Engineer
- Good understanding of Object oriented design, Design Patterns
- Has done development or debugging on Linux/ Unix platforms
- Proven understanding with Hadoop, HBase, Hive, Pig, and HBase
- Hadoop development engineer with around 2 years of strong hands on hadoop experience with good communication skills
- Good to have experience with Storm, Kafka, Spark Streaming, Spark MLlib, Spark and Map Reduce
- Knowledge of workflow/schedulers like Oozie
- Good understanding in working with any of the NO SQL Databases such as HBase.
- Proficient with SVN, build tools like maven, continuous integration tools like Jenkins
- Expertise in Hadoop echo systems HDFS (Cloudera preferred)
- Working knowledge of data loading tools like Flume, Sqoop.
- Well versed with SDLC methodologies.
- Motivation to learn innovative trade of programming, debugging and deploying
- Self-starter, with excellent self-study skills and growth aspirations
- Develop as per the architecture and design documents
- Tuning and code fix where required.
- Monitoring performance
- Adhere to tasks and timelines defined.
- Report issues/challenges in timely manner
- Work on ingestion and acquisition methodologies defined in architecture and provide inputs to improve the same.
- Follow industry best practices and own tasks end to end.
- Develop Low level design documents
- Ensure client review comments are addressed in time
- Loading from disparate data sets.
- Pre-processing using Hive and Pig.
- Maintain security and data privacy.
- Propose best practices/standards.
Experience: 2 to 4 years