• 5 + Years of Experience
• Solid core Java (1.6 +) experience.
• Hands on experience in Spark, Spark-SQL, Dataset and Dataframe.
• Knowledge on various tuning and optimization techniques in Spark.
• Hands on experience in Hadoop ecosystems like Hive / Apace Drill , handling different file formats like parquet , avro, ORC etc.
• IDE: Eclipse.
• Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming.
• Nice to have: worked in project following Agile practices
• Development and maintenance of data analytics solutions using the CIB Data Hub platform
• Integrating any Hadoop / Big Data tools and frameworks required to provide requested capabilities and data pipeline
• Build frameworks, assets and reusable components that could be used across projects
• Implementing data transformation process
• Implement IT Data Governance (Update Data Dictionary and Meta-model, enrich data lineage, develop data quality rules provided by Data Management officers)
Other skill Required
• Nice to have knowledge in Banking domain.
• Spark streaming and Kafka.
• Structured, organized and a good communicator
• Willing to share knowledge and skills with other developers within the team
• Delivery-focused with a good eye for detail
• Exhibits positive interpersonal and team skills
• Must be able to work closely with business analysts
• Whilst able to work independently, should be a true team player.