Senior Data Engineer (AWS+Python)

May 3

Apply Now

Get a free AI resume review

3Pillar Global

WebsiteLinkedInAll Job Openings

Building digital businesses, together.

Product Strategy • Product Development • Product Architecture • Mobile Apps • Big Data

1001 - 5000

💰 Private Equity Round on 2021-10

Description

• Design, develop, and maintain new data pipelines using AWS services and other relevant technologies to ensure data availability and quality. • Build and maintain RESTful APIs to facilitate data access and integration with other systems. • Develop and optimize SQL queries to extract, transform, and load data efficiently. • Manage multiple projects concurrently, prioritize tasks effectively, and meet project deadlines. • Work with large datasets, ensuring data quality, accuracy, and performance • Collaborate with technical teams and business stakeholders to understand data requirements and translate them into technical solutions. • Implement data transformation, integration, and validation processes to support analytics and reporting needs • Optimize and fine-tune data pipelines for improved speed, reliability, and efficiency • Implement best practices for data storage, retrieval, and archival to ensure data accessibility and security • Troubleshoot and resolve data-related issues, collaborating with the team to identify root causes • Document data processes, data lineage, and technical specifications for future reference • Participate in code reviews, ensuring adherence to coding standards and best practices • Collaborate with DevOps teams to automate deployment and monitoring of data pipelines • Leverage cloud services, with a preference for AWS, to design and implement scalable and cost-effective solutions. • Adapt quickly to new technologies, tools, and methodologies to stay current in the ever-evolving data engineering landscape.

Requirements

• Proven experience in data engineering with a strong track record of designing, developing, and maintaining data pipelines using AWS services and other relevant technologies to ensure data availability and quality. • Proven experience with Python (data processing and analysis) and with AWS services (for data storage, computation, and analytics). • Strong understanding of cloud concepts and services (AWS preferred) to design and implement scalable and cost-effective solutions. • Strong skills in SQL query language (to develop and optimize SQL queries to extract, transform, and load data efficiently). • Strong skills in REST APIs to facilitate data access and integration with other systems. • Good knowledge of data architecture, data modeling, and data warehousing concepts. • Experience in DevOps practices, showcasing hands-on experience in CI/CD automation (Terraform , Jenkins etc) • Must be a fast learner in order to adapt quickly to new technologies, tools, and methodologies and to stay current in the ever-evolving data engineering landscape. • Fluent in English to facilitate communication with a diverse and global team.

Benefits

• flexible work environment – whether it's the office, your home, or a blend of both. From interviews to onboarding, we embody a remote-first approach. • part of a global team, learning from top talent around the world and across cultures, speaking English everyday. Our global workforce enables our team to leverage global resources to accomplish our work in efficient and effective teams. • big on your well-being – as a company, we spend a whole trimester in our annual cycle focused on wellbeing. Whether it is taking advantage of fitness offerings, mental health plans (country-dependent), or simply leveraging generous time off, we want all of our team members operating at their best. • professional services model enables accelerated career growth and development opportunities - across projects, offerings, and industries. • equal opportunity employer. It goes without saying that we live by values like Intrinsic Dignity and Open Collaboration to create cutting-edge technology AND reinforce our commitment to diversity - globally and locally. • be a part of a global tech community! Check out our Linkedin site and Careers page to learn more about what it’s like to be part of our #oneteam!