Senior Data Platform Engineer
2024-09-23
Brazil
Sigma Software
Company Description
We're seeking a Senior Data Engineer to drive innovation by enhancing Python services, optimizing system architecture, and managing cutting-edge data pipelines with AWS and Snowflake.
As a Data Engineer, you will be working on our core systems, engines, and product services using the most sophisticated cutting-edge technologies to solve challenging problems that are unique to the DEI world. We are working with a highly dynamic AdTech ecosystem to protect the world from malvertising. The solution is to be a part of a portfolio that includes various software products. Being our team member means being open-minded and friendly, supportive, ready to accept the challenge, take responsibility, and never stop learning. We need to be one step ahead of attackers, which means being smarter, quicker, and looking beyond the surface. We are passionate about the people we hire to protect the internet. As the central unit driving product strategy and execution, we work closely with business, engineering, and research to coordinate all product efforts.
Job Description
Enhance and Maintain Python Services:
Update services to increase frequency and integrate paid endpoints
Optimize System Architecture:
Replace Kafka and Snowflake components to reduce costs
Manage file transfers between local storage, AWS S3, and Snowflake
Code Review and Optimization:
Review and optimize code in Python and Java
Collaborate Effectively:
Work with the new manager, Scrum Master, and team members to ensure project success
Qualifications
7+ years of hands-on experience in the Software Development field and/or Big Data
Solid experience with Python and Java
Experience working with SQL databases
Experience with AWS and S3 clouds
Strong experience with building data pipelines
Strong communication skills and at least a C1 level of English
Proficient in Airflow
WOULD BE A PLUS
Knowledge of Snowflake
Additional Information
PERSONAL PROFILE
Ability to work independently and proactively, including the capability to ping others for necessary information or if information is not provided on time
Responsibility for taking ownership of tasks and ensuring they are completed successfully
We're seeking a Senior Data Engineer to drive innovation by enhancing Python services, optimizing system architecture, and managing cutting-edge data pipelines with AWS and Snowflake.
As a Data Engineer, you will be working on our core systems, engines, and product services using the most sophisticated cutting-edge technologies to solve challenging problems that are unique to the DEI world. We are working with a highly dynamic AdTech ecosystem to protect the world from malvertising. The solution is to be a part of a portfolio that includes various software products. Being our team member means being open-minded and friendly, supportive, ready to accept the challenge, take responsibility, and never stop learning. We need to be one step ahead of attackers, which means being smarter, quicker, and looking beyond the surface. We are passionate about the people we hire to protect the internet. As the central unit driving product strategy and execution, we work closely with business, engineering, and research to coordinate all product efforts.
Job Description
Enhance and Maintain Python Services:
Update services to increase frequency and integrate paid endpoints
Optimize System Architecture:
Replace Kafka and Snowflake components to reduce costs
Manage file transfers between local storage, AWS S3, and Snowflake
Code Review and Optimization:
Review and optimize code in Python and Java
Collaborate Effectively:
Work with the new manager, Scrum Master, and team members to ensure project success
Qualifications
7+ years of hands-on experience in the Software Development field and/or Big Data
Solid experience with Python and Java
Experience working with SQL databases
Experience with AWS and S3 clouds
Strong experience with building data pipelines
Strong communication skills and at least a C1 level of English
Proficient in Airflow
WOULD BE A PLUS
Knowledge of Snowflake
Additional Information
PERSONAL PROFILE
Ability to work independently and proactively, including the capability to ping others for necessary information or if information is not provided on time
Responsibility for taking ownership of tasks and ensuring they are completed successfully