Senior Data Engineer – Data Quality, Observability

🕒 vor 15 Tagen

Jetzt Bewerben

Ähnliche Remote-Jobs finden

E-Mail-Benachrichtigung für neue Jobs einrichten

📊 Überprüfen Sie Ihre Lebenslauf-Bewertung für diese Stelle

Verbessern Sie Ihre Chancen auf ein Vorstellungsgespräch, indem Sie Ihre Lebenslauf-Bewertung vor der Bewerbung überprüfen.

Lebenslauf hochladen

Softgic

WebsiteLinkedInAlle Stellen

51 - 200 Mitarbeiter

Gegründet 2011

💼 Beratung

🤝 B2B

🤖 Künstliche Intelligenz

Consulting • B2B • Artificial Intelligence

Softgic is a digital and cognitive transformation services company that provides custom software development, automation, cloud and cybersecurity, and emerging-technology solutions to business clients. With more than a decade of experience, Softgic offers specialized “labs” for Solutions Design (UX, Agile, process design), Build (coding, low-code, mobile, DevOps), Metaverse (VR/AR, blockchain, NFTs, virtual economy), Data (Big Data, BI, AI/ML), Automation (RPA, BPM) and Go Live (cloud, IT operations, cybersecurity). The firm emphasizes tailored, enterprise-focused services, dedicated teams, and project-based engagements to help clients modernize infrastructure, streamline processes, and adopt advanced technologies.

Beschreibung

• Design and implement a scalable data quality framework across the platform. • Lead the implementation and operationalization of GX Core (Great Expectations) as the primary data validation framework. • Develop and maintain reusable data quality rules using a Rule-as-Code approach. • Create automated validation checks for business-critical datasets and workflows. • Implement data observability, monitoring, alerting, and reporting solutions. • Define and maintain data lineage across key business domains. • Design validation processes for data completeness, accuracy, integrity, consistency, reconciliation, freshness, and anomaly detection. • Integrate data quality validations into CI/CD pipelines and release processes. • Develop dashboards and reports to monitor data quality trends and operational health. • Investigate root causes of recurring data issues and implement preventive solutions. • Collaborate with Data Engineering, Application Engineering, QA, Product, and Support teams to establish ownership and governance for data quality. • Define standards for governance, validation frequency, remediation workflows, and quality metrics. • Continuously improve data quality processes and establish long-term observability best practices.

🎯 Anforderungen

• 5+ years of experience as a Data Engineer or in similar data engineering roles. • Strong experience designing and implementing enterprise Data Quality frameworks. • Hands-on experience with GX Core (Great Expectations) or similar tools such as Soda. • Strong SQL skills and experience working with Aurora PostgreSQL and Amazon Redshift. • Experience designing data validation rules, reconciliation processes, and observability solutions. • Experience building and maintaining ETL pipelines and large-scale data workflows. • Strong understanding of data modeling, referential integrity, synchronization, and batch processing. • Experience integrating data validation into CI/CD pipelines. • Experience with Git and engineering best practices such as Rule-as-Code. • Experience building dashboards, alerts, and reporting for operational monitoring. • Strong analytical and problem-solving skills with experience performing root cause analysis. • Experience collaborating with cross-functional engineering teams. • Excellent communication and documentation skills.

🏖️ Vorteile

• Fully remote position. • Opportunity to build enterprise-scale data quality and observability solutions. • High-impact role with ownership over data quality strategy and engineering best practices. • Collaborative environment working alongside Data Engineering, QA, Product, and Application Engineering teams. • Opportunity to work with modern data validation, observability, and cloud data technologies while driving continuous improvement across the platform.