Senior Data Engineer

February 2

Apply Now

Get a free AI resume review

The CCS Companies

WebsiteLinkedInAll Job Openings

Providing business process outsourcing solutions to America's leading brands for over 50 years and counting!

First & Third Party Debt Collection • Subrogation Outsourcing • Insurance Processing Expertise • Automated Voice Messaging • Call Center Outsourcing

501 - 1000

Description

• Work with the agile team to participate in agile ceremonies like grooming, planning, standup, retrospective, demos • Actively contribute to grooming, standup, create & update tasks estimate and status • Design and develop modern data ingestion pipeline utilizing tools like ADF (Azure Data Factory), Databricks, MS-SQL • Extensively work on Azure Cloud Platform • Write complex queries, stored procedures, functions, for various job execution, manipulating data • Create highly configurable design to apply various business rules, filters, mapping based on various clients • Work with data architects and business analysts to create a logical data model and create DDL scripts for physical model creation • Read data files (flat, excel, csv, json etc) from sftp servers, validate the data utilizing various business rules and configurations • Design and code various data architecture component like data validation, cleansing, de-duping, Symantec layer • Work on large data to ensure configurable ingestion of data, dynamic rule & validation of data, cleansing, transforming and loading into the data warehouse • Design and implement data validation and quality checks to ensure the accuracy and completeness of the data in the data warehouse • Perform performance of queries and data processing, identify and resolve any issues looking at query plans, create appropriate indexes, resolve dead locks and create table hints • Understand data partition, performance, multi-threading/processing of high volume data • Participate in design discussions, data analysis, data model creation etc • Product quality design diagrams (using MS-Visio, Draw.io etc) and documentations (MS-Work, Excel etc) • Maintain compliance with Inovalon’s policies, procedures and mission statement

Requirements

• Minimum of 10 years industry experience working in data & reporting area, knowledge of healthcare data will be a plus • 10+ years working on data process/pipeline, ETL, Data Workflow, Query Plans & Optimization • 10+ experience in MS SQL, T-SQL, ETL Jobs • 5+ years working on databricks, PySpark/SparkSQL, data frames, unity catalog • 5+ years’ experience in writing query plans, indexes etc • 5+ years’ experience working on Azure Cloud is preferred utilizing ADF (Azure Data Factory), Azure Delta Lake, Azure SQL Server, Data Sync, Log Insights and Analytics • Experience working with Role based security at database, ETL jobs, data exports level • Experience with HIPPA and PHI will be a plus • Ability to effectively communicate with internal and external customers • Excellent verbal and written communication skills • Excellent computer proficiency (MS Office – Word, Excel and Outlook) • Must be able to work under pressure and meet deadlines; and • Ability to work independently and to carry out tasks to completion following standard accepted practices • BS degree in Computer Science or Computer Engineering, Business, or equivalent experience

Benefits

• Sedentary work (i.e., sitting for long periods of time); Exerting up to 10 pounds of force occasionally and/or negligible amount of force; Frequently or constantly to lift, carry push, pull or otherwise move objects and repetitive motions; Subject to inside environmental conditions; and Some travel (less than 10%) may be required for this position, primarily for training and collaboration purposes.