Data Engineer – AI

October 21

🏄 California – Remote

Although this job is listed in California, this company may be open to hiring remote in other states.

⏰ Full Time

🟠 Senior

🔴 Lead

🚰 Data Engineer

Apply Now

Receive Emails with Similar Jobs

Upbound

WebsiteLinkedInAll Job Openings

Cloud Computing • SaaS • Enterprise

Upbound is a company that specializes in cloud native infrastructure management aimed at developers and platform teams. By leveraging Crossplane, a cloud native control plane framework, Upbound provides tools and services that simplify the management of cloud infrastructure resources through APIs. These solutions empower organizations to adopt self-service deployment strategies, enhance developer productivity, and ensure secure and compliant cloud operations. Upbound manages Crossplane operations, allowing platform teams to focus on building cloud native platforms rather than managing infrastructure life cycles.

11 - 50 employees

Founded 2017

☁️ SaaS

🏢 Enterprise

📋 Description

• Define and drive the technical vision for data platforms that support AI-powered features in Crossplane and Upbound Spaces • Lead the design of data pipelines that transform infrastructure and data into training datasets for ML models • Architect vector search and RAG systems that leverage Crossplane Control Planes & Upbound Marketplace as a knowledge store • Build data infrastructure that processes resources, extensions, and compositions for semantic search • Establish frameworks for collecting, processing, and analyzing infrastructure configuration data • Design data pipelines that handle Crossplane-specific data • Create infrastructure for indexing and searching Upbound Marketplace content, documentation, and community patterns • Develop metrics and monitoring for AI features integrated with Upbound's control plane architecture • Design data systems that power AI agents for infrastructure provisioning & operations, helping users generate and optimize Crossplane compositions • Create feature engineering platforms that extract signals from control plane operations, resource status, and reconciliation patterns • Implement data infrastructure for training models that predict infrastructure failures, optimize resource allocation, and suggest configuration improvements • Drive the development of knowledge graph representations of infrastructure dependencies and relationships

🎯 Requirements

• 10+ years of software/data engineering experience with at least 4 years in technical leadership roles • Proven track record building data platforms that support production systems at scale • Deep expertise in both traditional data engineering (Spark, Airflow, data lakes) and ML-specific infrastructure (feature stores, model serving) • Experience with vector databases (Pinecone, Weaviate, Qdrant, Milvus, pgvector, Opensearch, ElasticSearch) • Demonstrated experience with LLM applications, including RAG architectures and semantic search implementations • Understanding of Kubernetes, cloud-native architectures, and infrastructure-as-code principles • Strong understanding of data requirements for AI/ML systems: training pipelines, feature stores, and inference infrastructure • Hands-on experience building knowledge bases and semantic search systems for technical documentation and code • Experience with embedding models for code and technical documentation • Knowledge of time-series data processing for infrastructure metrics and events • Understanding of graph databases and their application to infrastructure dependency modeling

🏖️ Benefits

• Health insurance • 401(k) matching • Flexible work hours • Paid time off • Remote work options