Descripción del puesto
We are looking for a savvy Senior Data Engineer to join our Data Science & Management
team. The hire will be responsible for expanding and optimizing our data and data
pipeline architecture, as well as optimizing data flow and collection for cross functional
teams.
The ideal candidate is an experienced data pipeline builder and data modeler who enjoys
optimizing data systems and building them from the ground up. The Senior Data
Engineer will support our product owners, product analysts, data analysts and data
scientists on data initiatives and will ensure optimal data delivery architecture is
consistent throughout ongoing work. They must be comfortable supporting the data
needs of multiple teams, systems and products and work in an agile minded
team.
The right candidate will be excited by the prospect of designing, developing, and
maintaining our company’s data architecture to support our next generation of products
and data initiatives.
The project consists in the migration of approximately 1,000 pipelines to Snowflake. The
ideal candidate should have experience with Active Batch, Talend, Snowflake, Docker,
and GitHub.
Requisitos
- Being a Latam resident.
- Bachelor’s degree required, Master’s an asset, in Software Engineering,
Computer Science, or equivalent work experience in a Technology or business
environment
● Minimum of 7 years of experience working in developing and following structured
work processes in data engineering.
● Minimum of 7 years of experience in integration solutions development with data
integration tools (e.g., Talend, Azure Data Factory).
● Minimum of 3 years of experience with real-time data streaming tools like Kafka,
or any similar tools.
● Experience with cloud platforms including Azure, AWS, or GCP.
● Experience with data warehousing platforms such as Snowflake or Databricks.
● High proficiency in SQL, Python, and other programming languages.
● Highly proficient in data management, governance, data design and database
architecture. Proven track record of manipulating, processing, and extracting
value from large, disconnected datasets.
● Proficient in data modelling and data architecture with experience using
WhereScape RED/3D an asset.
● Experience with big data tools: Hadoop, Spark, Kafka, etc.
● Experience with relational SQL and NoSQL databases, including Postgres and
Cassandra.
● Experience with data pipeline and workflow management tools: Azkaban, Luigi,
Airflow, etc.
● Experience with AWS cloud services: EC2, EMR, RDS, Redshift
● Experience with stream-processing systems: Storm, Spark-Streaming, etc.
● Experience with object-oriented/object function scripting languages: Python,
Java, C++, Scala, etc.
● Highly proficient in data modelling, data integrations, data orchestration, and
supporting methodologies.
● Highly proficient in leading large-scale projects or significant project steps and
communicating progress/approach with technical/non-technical peers/clients and
leaders.
● Highly proficient in multiple programming languages and coding. Excellent ability
to design and engineer moderately complex enterprise solutions.
● Working knowledge of message queuing, stream processing, and highly scalable
‘big data’ data stores.
● Strong project management and organizational skills.