Job Responsibilities

Role: ETL Python Developer

JD:

Develop data processing / ETL pipelines using Python ETL frameworks and technologies on Amazon AWS. 
• Understand source data formats, validation and transformation rules, output file formats
• Map source system data to data warehouse data model
• Develop and test ETL processes using Python and Spark to migrate mainframe files to AWS Redshift
• Define and capture metadata and rules associated with ETL processes
• Adapt ETL processes to accommodate changes in source systems and new business user requirements
• Provide support and maintenance on ETL processes and documentation
• 4+ years’ experience coding in Python, 2+ years building ETL processes and pipelines in Python and Spark
• Experience building Python REST APIs
• Hadoop Big Data knowledge – Hive metastore; storage partitioning schemes on S3 and HDFS
• Experience with Amazon AWS, EMR, Redshift 
• Demonstrated knowledge of data models and relational database designs
• Demonstrated knowledge of source and target data structures, ETL processes and products
• Demonstrated knowledge of SQL.
• Experience working as part of a globally dispersed team (on-shore/offshore)
• Experience with Agile development
• Experience with data migration

Apply now