Distinguished Engineer

Unstructured is a company defining the standard for enterprise data transformation in the age of LLMs and generative AI. They have raised over $65M, have a popular open-source toolkit used by thousands of companies, and power AI workflows across sectors. They are building the backbone of generative AI and infrastructure for industry-wide intelligence.

Location: San Francisco, Bay Area, San Jose, Seattle, Sacramento

Responsibilities

• Define and evolve the end-to-end architecture for Unstructured’s data transformation and retrieval platform.
• Build and scale distributed systems that process massive volumes of unstructured data across diverse formats and sources.
• Serve as the company-wide authority on Kubernetes orchestration, cluster design, performance tuning, and reliability.
• Lead Python architecture and best practices—ensuring performance, modularity, and maintainability across services.
• Design and optimize Postgres schemas, queries, and indexing strategies to support large-scale metadata and retrieval pipelines.
• Mentor senior engineers through design reviews and code guidance, raising the bar for technical excellence across the org.
• Partner with the infrastructure and product teams to translate research prototypes into production-grade systems.
• Evaluate emerging technologies and open-source tools in LLM infrastructure, retrieval, and orchestration—deciding where and how to integrate them.

Requirements

• 10+ years of software engineering experience with a focus on distributed systems, infrastructure, or data architecture.
• Expertise in Python—capable of building frameworks and performance-critical services from scratch.
• Deep Kubernetes expertise; ability to design, deploy, and debug at scale.
• Fluency in Postgres—understanding query planning, partitioning, and tuning for high-throughput environments.
• Obsessed with clean, scalable architecture and leading design reviews.
• Experience in high-performance data or AI/ML systems—especially retrieval pipelines, embeddings, or hybrid workloads.
• Thrives in fast-moving, ambiguous environments where technical depth and judgment matter more than process.

Benefits

• Surrounded by smart, kind, low-ego colleagues.
• Company offsites, tech swag, and tools for remote work.
• Meaningful work in AI, tackling technical challenges powering production systems used by thousands, including nearly half of the Fortune 500.
• Holistic support including medical, dental, vision, life, disability insurance, unlimited PTO, parental leave, 401(k), and referral incentives.
• Opportunity to build foundational technology and influence company operations.