JOB Description
- You are strong in coding
- You have a strong expertise in Data Engineering
- You have a deep understanding of data modelling and experience with data engineering tools and platforms such as Kafka, Spark, and Hadoop
- You have built large-scale data pipelines and data-centric applications using Big Data tooling like Hadoop, Spark, Hive, Oozie, and Airflow in a production setting.
- You’ve tackled challenges of persisting, working with, and exposing metadata from data engineering processes using tools such as Apache Atlas, Cloudera Navigator, etc.
- Hands-on experience building Data Engineering tooling with the Microsoft Azure Data Engineering and Analytics stacks including ADLS, Azure Synapse Analytics, Polybase, ADF, Azure Event Hub, Azure Databricks, Active Directory, and PowerBI.
- Hands-on experience with event streaming with modern event streaming tooling like Pulsar, Kakfa, Kinesis. Understanding of when streaming vs. batch processing is appropriate, and tradeoffs in a given context
- Hands-on experience with MPP query engines like Presto, Dremio, and Spark SQL.
- You are comfortable applying data security strategy to solve business problems