Data Platform Engineer

Katowice, śląskie
Stała
Pełny etat

17 dni temu

ING Hubs Poland is hiring!We are looking for you, if you:

have 5+ years of experience in a Platform Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field,
have experience with advanced Azure Dev/Ops pipelines,
have experience using one or more of the following software components:

- The Cloudera Eco System (HDFS, HIVE, NIFI)
- Management of the bigdata clusters using CM, Ranger, KNOX and HUE
- Produce and Consume streaming data (KAFKA)
- Experience with other storage layers (e.g. S3, CEPH) * know development languages such as Python, Ansible and Bash,

know how to build Cloud agnostic data capabilities,
have knowledge about public Cloud (GCP, Azure),
know monitoring tools like Grafana, Kibana, Prometheus and Elasticsearch.
have experience building and optimizing ‘big data’ data pipelines, architectures and data sets. In both batch and real-time data integration,
have knowledge of message queuing and stream processing,
have worked in a DevOps environment: CI/CD with Azure DevOps, Test automation, Docker

- Agile Scrum way of working.You'll get extra points for:

excellent communication skills, both verbally and in writing, to be able to align with various stakeholders and cross border squads,
results oriented, determined and having a self-driven attitude.

Your responsibilities:

define and implement the infrastructure required for optimal loading of data from a wide variety of data sources using big data technologies,
operate and maintain a critical environment,
setup and tune the monitoring of the infrastructure including data pipelines,
identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc,
work with stakeholders including the Data and Design teams to assist with data-related technical issues and support their data infrastructure needs,
keep our data separated and secure across international boundaries through multiple data centers.

About the team:Have you ever build the best Data Platform ever?If your answer is “yes” then you are the Data Platform engineer we are looking for.You can help us to build, expand and maintain this together with a team of experts. You know about batch and streaming ingestion and that security and data management in a data environment are must haves. You will work in a team where you learn from others and share your knowledge, have fun and make ING data driven.Our ideal candidate is an experienced engineer managing large data environments based on the Cloudera Data Platform (Hadoop), Cloudera Flow Manager (CFM) and other storage layer technologies.The Platform Engineer will actively support our software developers on data initiatives and will ensure optimal data delivery.Experience in the Agile methodologies would be a perfect fit, able to slice and dice work/ideas in clear stories and tasks.The role naming convention in the global ING job architecture will be “Engineer II”.

ING

Aplikuj