Big Data Engineer

Job Title: Big Data Engineer

Location:
Minneapolis, MN

Position
Type:  FTE/Permanent

 

Roles
& Responsibilities:

Looking for
a Big Data Engineer who will work on collecting, storing, processing, and
analyzing of huge sets of data. The primary focus will be on choosing optimal
solutions to use for these purposes, then maintaining, implementing, and
monitoring them.

Will also be
responsible for integrating them with the architecture used across the company.
Responsibilities

Selecting
and integrating any Big Data tools and frameworks required to provide requested
capabilities

Implementing
ETL process

Monitoring
performance and advising any necessary infrastructure changes

Defining
data retention policies Skills and Qualifications

Proficient
understanding of distributed computing principles

Management
of Hadoop cluster, with all included services

Ability to
solve any ongoing issues with operating the cluster

Experience
with building stream-processing systems, using solutions such as Storm or
Spark-Streaming

Good
knowledge of Big Data querying tools, such as Pig, Hive, and Impala

 

Role
Specific Skills:

Proficiency with Hadoop v2, MapReduce, HDFS

Experience with NoSQL databases, such as
HBase, Cassandra, MongoDB

 

Experience
& Exposure Required:

Experience
with Spark

Experience
with integration of data from multiple data sources

Experience
with NoSQL databases, such as HBase, Cassandra, MongoDB

Knowledge of
various ETL techniques and frameworks, such as Flume

Experience
with various messaging systems, such as Kafka or RabbitMQ

Experience
with Big Data ML toolkits, such as Mahout, SparkML, or H2O

Experience
with Cloudera/MapR/Hortonworks

 

Desirable
Skills:

Good
understanding of Lambda Architecture, along with its advantages and drawbacks

Bachelor’s
degree.

 

Capgemini America, Inc and its U.S. affiliates are EEO/AA employers. Capgemini conducts all
employment-related activities without regard to race, religion, color, national
origin, age, sex, marital status, sexual orientation, gender
identity/expression, disability, citizenship status, genetics, or status as a
Vietnam-era, special disabled and other covered veteran status.

Click the following
link for more information on your rights as an Applicant:
http://www.capgemini.com/resources/equal-employment-opportunity-is-the-law