Senior Data Engineer

Job not on LinkedIn

3 days ago

Apply Now

Receive Emails with Similar Jobs

GIGABIT Technologies Private Limited

WebsiteLinkedInAll Job Openings

Cybersecurity • IT Security • SaaS

GIGABIT Technologies Private Limited is a leading provider of comprehensive IT solutions, specializing in cybersecurity, infrastructure design, and managed IT services. With a focus on proactive cybersecurity measures, the company offers services including real-time threat detection, data security, and incident response to ensure businesses remain secure and resilient. GIGABIT Technologies also designs tailored storage and network solutions, helping organizations enhance their IT infrastructure while maximizing operational efficiency and compliance.

51 - 200 employees

Founded 2010

🔒 Cybersecurity

☁️ SaaS

📋 Description

• Software development • Utilize Autoloader and Kafka for data ingestion • Work with ADLS for data storage • Perform data transformation using PySpark and Structured Streaming • Manage workflow orchestration with Databricks Workflows • Handle data import/export using Kafka in Azure EventHub • Implement data governance using Unity Catalog with RBAC and data quality tools like Great Expectations

🎯 Requirements

• SQL Server Knowledge • Python Knowledge • Github Knowledge • Solid understanding of distributed data processing with Spark • Experience with Structured Streaming, Autoloader, UDFs, pandas • Experience with Databricks Live Tables • Experience with Databricks Asset Bundles • Experience with Kafka / Azure EventHub • Proficiency in Python • Strong knowledge of SQL • English

🏖️ Benefits

• An exciting growth journey and a never-ending ambition to develop as a company • A friendly, supportive and humble working culture where the team always comes first • Opportunities for professional development and personal growth • Passionate, high-performing and genuine colleagues • We care about the important things: results and well-being • Referral Programm- Refer a colleague program: Enjoy an amazing reward if you recommend a friend or colleague to GIGA IT and they get hired • 15 business days PTO and flexibility programm • Local Holidays • Remote work - You can work from anywhere! • Notebook • Work hours: Monday to Friday, from 8 a.m. to 5 p.m.