Senior Data Engineer

Job not on LinkedIn

🔥 0 minutes ago

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

Aspire, Jordan

WebsiteLinkedInAll Job Openings

201 - 500 employees

Founded 2002

💼 Consulting

🏥 Healthcare

📣 Marketing

Consulting • Healthcare • Marketing

Aspire is reimagining and redefining what being a technology partner in the region means. With a commitment to exceeding expectations and providing exceptional quality work, Aspire consists of a group of technology experts with diverse backgrounds and expertise. The company offers a wide range of digital consulting and technology services, including digital strategy, automation, and cybersecurity solutions, while also focusing on social responsibility initiatives. Aspire serves various sectors such as financial services, healthcare, media, e-commerce, and retail, positioning itself as a trusted digital partner across the MENA region and the US.

📋 Description

• Design, implement, and manage complex DAGs and SubDAGs in Apache Airflow • Use Operators, macros, connections, variables, and XCom for efficient task execution • Conduct unit testing for Airflow workflows • Perform file operations and data movement between HDFS and S3 • Manage Hive external tables, update table DDL, and handle partition operations • Execute Hadoop commands and navigate HDFS • Use Hive on Tez for optimized query execution • Use AWS command line tools for S3 file operations • Implement and manage secrets with Ansible Vault and AWS Secrets Manager • Develop and optimize data processing tasks using PySpark • Use Livy for Spark job execution • Import and export data between Microsoft SQL Server and HDFS using Sqoop • Apply dimensional modeling principles, define facts and dimensions, and manage surrogate keys • Implement Git branching strategies • Raise and review Merge Requests in GitLab • Use SSIS for migrating data workflows • Model data in Snowflake using external and internal tables • Work with views and materialized views in Snowflake • Collaborate with cross-functional teams to deliver high-quality data solutions • Focus on data quality, performance optimization, security, and compliance

🎯 Requirements

• Bachelor's degree in Computer Science, Information Technology, or a related field • Proficiency in Python 2.7 and Python 3.6+ • Experience in unit testing and test-driven development • Familiarity with secrets management tools and techniques • Knowledge of data modeling principles and practices • Strong understanding of Git version control and GitLab workflows • Awareness or knowledge of IT security best practices under ISO, SOC, or similar standards • Experience with SSIS for data migration workflows • Experience with Snowflake, including external/internal tables, views, and materialized views • Familiarity with Hive on Tez • Experience with AWS command line tools and S3 operations • Knowledge of Livy for Spark job execution • Knowledge of Apache Airflow, PySpark, Hive, AWS, Snowflake, HDFS, Sqoop, Microsoft SQL Server, and SSIS • Excellent problem-solving and communication skills

🏖️ Benefits

• Competitive long-term total compensation package with salary and performance-based bonus • Remote-first organization with flexibility • Opportunities to learn and grow through technical and non-technical training programs • Global exposure through products with international teams and clients • Attendance at virtual and in-person international technology conferences • Nursery reimbursement benefit • Aspire Wellness Program • Exposure to an IT environment adhering to rigorous ISO / SOC security and compliance standards