- Working experience with Spark Hive Message Queue or Pub Sub Streaming technologies
- Experience developing data pipelines using mix of languages Python SQL etc and open source frameworks to implement data ingest processing and analytics technologies
- Deep experience in developing data processing data manipulation tasks using PySpark such as reading data from external sources merge data perform data enrichment and load in to target data destinations
- Experience leveraging open source big data processing frameworks such as Apache Spark Hadoop and streaming technologies.
- PySpark, Python
- AWS & Azure
- Excellent verbal and written communication and interpersonal skills
- Ability to work independently and within a team environment