Data Engineer (IT / Coding / Automation) Cape Town, South Africa

Reference Number: CDTLKDE

 

Job Description:

Our client is currently looking for a Data Engineer to join the team to help build a world class data platform which enables the team to access to the data they need to guide their decisions and actions every day. The successful candidate will be helping solve data challenges across a range of systems, from real-time event streams powering their CRM and other data integrations to traditional batch workloads underpinning our warehouse and machine learning models.

 

Responsibilities:

Over the last couple of years, they managed to build a complex, end to end data platform that powers many use cases across the company. As the company continues to grow, they’re looking to make it more robust, scalable and efficient, to make sure it scales with the business. To help make this happen, you will be responsible for:

• Automating the ingestion of data from internal and external sources into our data warehouse (Amazon Redshift)
• Data cleansing and preparation using Airflow and Spark (either in Scala or Python)
• Building and maintaining daily ETL jobs
• Creating structured tables in the data warehouse to make the data clear, understandable and accessible to analysts and team members
• Writing scripts to automate the ingestion of daily reports into the data warehouse

 

Requirements:

• Proficiency with Python, Java or Scala
• Experience of data orchestration using tools such as Airflow or similar
• ETL experience using Spark, with Scala or Python, and experience of EMR
• Knowledge of the end to end data pipeline, with a focus on data cleaning, structuring and transformation
• Understanding of AWS data stack preferred (S3, MSK, EMR, Glue, Redshift, Lambda, Kinesis)
• A certification or bachelors degree in computer science, data analytics, big data or similar
• 3+ years experience as a data engineer or similar
• An analytical mind with an eye for detail and demonstrated history of solving complex data challenges

 

If you wish to apply for the position, please send your CV to Chalden Du Toit at cdutoit@caglobalint.com

 

Please visit www.caglobalint.com for more exciting opportunities.

 

Chalden Du Toit

Recruitment Consultant

CA Finance

 

CA Global will respond to short-listed candidates only. If you have not had any response in two weeks, please consider your application unsuccessful however your CV will be kept on our database for any other suitable positions.