Software Engineer Lead

Software Engineer Lead – Hadoop Platform Lead

Job Title: Hadoop Platform Support Lead:
Location: Remote/ Toronto Canada 

Job Description:
We are looking for a Hadoop Platform Support Lead to lead the implementation and ongoing administration of Hadoop infrastructure.
Aligning with the engineering team to propose and deploy new hardware and software environments required for Hadoop and to expand existing environments.
Working with AD teams to setup and monitor Hadoop users.
This job includes setting adding approved Active Directory Groups and testing HDFS Hive Pig and MapReduce access.
Cluster maintenance as well as creation and removal of nodes using tools using Ambari.
Performance tuning of Hadoop clusters and Hadoop MapReduce routines.
Screen Hadoop cluster job performances and capacity planning.
Monitor Hadoop cluster connectivity and security 
Manage and review Hadoop log files File system management and monitoring HDFS support and maintenance.
Collaborating with application teams to install operating system and Hadoop updates patches version upgrades when required.
Performance monitoring and tuning
Key Responsibilities 
Provide production support for ETL batch jobs running traditional data warehousing and Hadoop Data Lake environments 
Hadoop platform monitoring Operations and Monitoring System Administration Security Administration 
Recoverability Performance Management HDFS Quota Management and YARN Resource Management capacity planning and forecast platform Advisory 
Troubleshoot and analyze production job failures related to data network file delivery and server and application issues independently and provide solutions to recovery Participate in root cause analysis and preventative actions to avoid recurring incidents Participate in the build out of automation to prevent problem recurrence with the goal of automating response to all non exceptional service conditions Detailed knowledge in Cloudera CDH and CDP Cluster maintenance as well as creation and removal of nodes using tools like Cloudera Manager as well as performance tuning of Hadoop clusters and Hadoop MapReduce Impala Design and implement column schemas of HBase apply different HDFS formats and structure like Parquet Avro etc to speed up analytics Fine tune Hadoop applications for high performance and efficiency Solve and debug any Hadoop ecosystem run time issues Equivalent experience in Service Management i e Release management Incident management as well as experience with deployment toolsets like Odyssey Nolio Git Data visualization Tableau Good problem solving skills with production support experience and an ability to participate within a multidisciplinary distributed team Collaborate with ETL development and support team members onsite and offshore to resolve complex data design issues provide optimal solutions that meet business requirements and benefits system performance Work on P1 issues during General Shift Hadoop platform monitoring Operations and Monitoring System Administration Security Administration Recoverability Performance Management HDFS Quota Management and YARN Resource Management capacity planning and forecast platform Advisory Prepare and update Runbook Jira Confluence ServiceNow ticket Liaising with Application and Infrastructure support teams throughout the incident lifecycle Working collaboratively with the head of platform domains.

About Capgemini

About Capgemini:

Capgemini is a global leader in consulting, digital transformation, technology and engineering services.The Group is at the forefront of innovation to address the entire breadth of clients’ opportunities in the evolving world of cloud, digital and platforms. 
Building on its strong 50-year+ heritage and deep industry-specific expertise, Capgemini enables organizations to realize their business ambitions through an array of services from strategy to operations. Capgemini is driven by the conviction that the business value of technology comes from and through people. Today, it is a multicultural company of 270,000 team members in almost 50 countries. With Altran, the Group reported 2019 combined revenues of €17billion.

What We Offer:

What we Offer:
Your career matters to you and is important to us too. Because your goals and needs are constantly evolving, we offer visibility, leeway and support to help you grow and progress in your career. This approach builds notably on our comprehensive competency framework, our personal development, training and career management programs, and our University innovative and business-focused learning curriculums. 
We promote a culture of diversity. We believe working with talented individuals from different backgrounds and points of view is a strategic advantage and an ongoing opportunity. Diversity enriches our creative solutions and adds value for our clients. 
With the digital tech sector growing at a rapid pace and women significantly underrepresented in the industry, we are determined to inspire and recruit more women into technology and build diverse teams that reflect the clients we serve.
Our Shared values have been at the heart of the group since our formation. They are honesty, boldness, trust, freedom, team spirit, modesty and fun. These values influence the way we meet client needs while respecting the regulatory requirements of each country in which we operate, and the way we promote ethically sound practices within Capgemini and in our partnerships. 
Capgemini is committed to building a workforce of employees with diverse backgrounds and work experiences. We strongly encourage women, veterans and active military service personnel to apply. 
Capgemini is an Equal Opportunity Employer encouraging diversity in the workplace. All qualified applicants will receive consideration for employment without regard to race, national origin, gender identity/expression, age, religion, disability, sexual orientation, genetics, veteran status, marital status or any other characteristic protected by law. 
This is a general description of the Duties, Responsibilities and Qualifications required for this position. Physical, mental, sensory or environmental demands may be referenced in an attempt to communicate the manner in which this position traditionally is performed. Whenever necessary to provide individuals with disabilities an equal employment opportunity, Capgemini will consider reasonable accommodations that might involve varying job requirements and/or changing the way this job is performed, provided that such accommodations do not pose an undue hardship. 
Click the following link for more information on your rights as an Applicant – 
Applicants for employment in the US must have valid work authorization that does not now and/or will not in the future require sponsorship of a visa for employment authorization in the US by Capgemini. 




Posted on:

June 18, 2021

Experience level:

Experienced (non-manager)

Contract type:




Business units:



Financial Services


By continuing to navigate on this website, you accept the use of cookies.

For more information and to change the setting of cookies on your computer, please read our Privacy Policy.


Close cookie information