Data Engineer

11 hours ago

🤠 Texas – Remote

Although this job is listed in Texas, this company may be open to hiring remote in other states.

💵 $96.3k - $125k / year

⏰ Full Time

🟠 Senior

🔴 Lead

🚰 Data Engineer

🦅 H1B Visa Sponsor

Apply Now

Receive Emails with Similar Jobs

Conduent

WebsiteLinkedInAll Job Openings

B2B • eCommerce • Government

Conduent is a leading provider of technology-led solutions aimed at enhancing customer experiences and improving operational efficiency for businesses and government agencies. The company offers a wide range of services, including customer experience management, finance and accounting solutions, human capital management, integrated digital solutions, and specialized services for healthcare and public sector clients. By leveraging automation and analytics, Conduent helps organizations streamline processes and drive business success.

10,000+ employees

Founded 2017

🤝 B2B

🛍️ eCommerce

🏛️ Government

💰 Venture Round on 2009-01

📋 Description

• Engineer robust AZURE data factory pipelines using Linked Services/Datasets to ingest high volume, diverse data from disparate source systems into Azure BLOB storage • Design and implementation of scalable data pipelines using Azure Databricks and Azure Data factory, adhering to Medallion architecture principles to ensure data integrity, consistency, and performance • Develop complex & advanced transformation logic using PySpark (Python), user-defined functions (UDFs) within Azure Databricks to design and implement data processing and ETL workflow, enabling efficient transformation of heterogeneous source data into an integrated advanced Benefit and Eligibility system • Creating modular and reusable notebooks in Azure Databricks utilizing functions, parameters, widgets for dynamic data processing and various python libraries such Panda, numpy, Pyspark SQL, SQL alchemy etc. • Developing highly optimized Databrick notebooks using PySpark (Python & SQL) for advanced data joining, data filtering, pre-aggregation, and processing of large data sets. • Working with SQL Server and Oracle, writing optimized queries with CTEs, ANSI SQL, window functions, Views, materialized Views, stored procedures and Oracle-specific syntax to perform data profiling, validation, balancing, reconciliation to ensure data accuracy (data quality) and data completeness. • Solving complex data engineering challenges including Delta merge for hundreds of millions of records, deduplication, performance tuning, and improving ETL run time efficiency using partitioning strategies, Z-Ordering etc. • Closely working with Client and Business analysts to define and assess complex data conversion and data integration requirements, translating business needs into technical solutions.

🎯 Requirements

• Bachelors Degree in Computer Science or related field • Total experience 10+ as data analyst/engineer • Federal government or State level experience • 2 years experience in Health and Huma Services programs (SNAP and TANF)

🏖️ Benefits

• Health insurance coverage • Voluntary dental and vision programs • Life and disability insurance • Retirement savings plan • Paid holidays • Paid time off (PTO) or vacation or sick time • Flexible working conditions