Big Data | 4 to 6 years | Bengaluru

Job Description

  • Proven commercial experience as a Data Engineer / Big Data Developer for 3+ years preferably in building data lake solution by ingesting and processing data from various source systems
  • Demonstrated software engineering / Big Data development background using Python (preferred), Java, Scala
  • Experience with multiple Big data technologies and concepts such as HDFS, NiFi, Kafka, Hive, Spark, Spark streaming, HBase, EMR and GCP
  • Background within Hadoop eco system (Cloudera, Spark, Scala and PySpark)
  • Have worked on Spark performance tuning, troubleshooting Spark Job
  • Strong in concepts of object-oriented programming, memory management e.g. garbage collection
  • Worked on building ETL pipeline

Primary Skills

  • In-depth understanding of Data Management practices and Database technologies
  • Ability to work in team in diverse, fast-paced Agile environment
  • Apply DevOps, Continuous Integration and Continuous Delivery principles to build automated pipelines for deployment and production assurance on the data platform
  • Knowledge of building self-contained applications using Docker and OpenShift

Secondary Skills

  • Share knowledge with immediate peers and build communities and connections that promote better technical practices across the organisation
  • Implement test cases and test automation
  • Experience in building various frameworks for enterprise data lake is highly desirable
  • Good data analytics skills by using SQL to troubleshoot data issues

Ref:

632203

Posted on:

April 20, 2021

Experience level:

Experienced

Contract type:

Permanent

Location:

Bangalore

Department:

Financial Services