Parspec automates quoting, product selection, and submittal prep for construction distributors and agents using multimodal NLP and computer-vision to extract requirements and surface compliant products from a manufacturer-sourced catalog.
ABOUT PARSPEC Parspec is building the AI and digital infrastructure for the construction materials supply chain. Construction is a $15 trillion industry, yet the systems that underpin the buying and selling of materials remain fragmented, manual, and disconnected. Distributors and rep agencies rely on spreadsheets, PDFs, phone calls, and siloed tools to find new products and quote and manage projects; creating delays, errors, and margin erosion across the supply chain. Parspec is an AI-native platform that powers how construction products are discovered, bought, and sold. Trusted by more than 300 MEP distributors and rep agencies, Parspec helps project-driven businesses bid faster, win more work, and operate more profitably. By combining product intelligence, AI-powered workflows, and a connected ecosystem, Parspec is laying the foundation for a more intelligent, efficient construction supply chain. Founded in 2021 and headquartered in San Mateo, California, Parspec has raised $31 million from leading deep-tech and construction-technology investors. THE OPPORTUNITY We are looking for an experienced AIOps / LLMOps Engineer to help design, deploy, and manage the AI infrastructure that powers Parspec’s next-generation AI systems. This role will focus on designing, deploying and operating LLMOps/AIOps platforms for high-volume generative AI workloads, with deep emphasis on self-hosted model serving performance alongside observability, security and cost. Ensures scalable, secure AI infrastructure using async patterns on AWS. We run a high-throughput document AI pipeline: vision-language models reading construction documents, plus a fleet of smaller models (rerankers, embedders, layout detectors) serving search and extraction. Our throughput ceiling and our infrastructure bill are both set primarily by how efficiently we serve models. The core of this role is moving that ceiling. A note on fit: this is an inference-performance and AI-platform role, not an ITOps/AIOps-for-infrastructure role, not an observability-tooling role, and not a general SRE or DevOps role. Kubernetes, Terraform and CI/CD are tools you'll use daily, but they are not what we're hiring for. If your strongest work is platform engineering rather than model serving performance, this is likely the wrong fit. Preferred location: Bengaluru, with regular in-office collaboration. WHAT YOU WILL ACHIEVE AND KEY RESPONSIBILITIES MODEL SERVING PERFORMANCE AND EFFICIENCY - Own throughput and cost per unit of work for self-hosted models — tune serving stacks (vLLM, SGLang, TensorRT-LLM, TGI) at the level of scheduler configuration, batching policy and memory layout: continuous/in-flight batching, chunked prefill, max_num_seqs / max_num_batched_tokens, admission control, preemption. - Manage KV cache economics: PagedAttention block sizing, memory-utilization headroom, KV quantization, cache offload, and prefix/radix caching. Our extraction prompts carry long fixed schemas across every page, so cache hit rate is a first-class operating metric here. - Quantize models to FP8 / INT8 / INT4 (AWQ, GPTQ, W8A8) and prove quality held using an accuracy harness on real production documents. Building and maintaining that harness is part of the role. - Serve vision-language models at scale: image token budgets, tiling and dynamic resolution, DPI/resize tradeoffs, the vision encoder as an independent bottleneck, encoder output caching, page-level batching. Our workload is batch document throughput, not interactive chat. - Make and defend parallelism and topology decisions — tensor vs pipeline parallelism vs replica scale-out, including when tensor parallelism hurts (e.g. PCIe-only nodes without NVLink, where N replicas at TP=1 often beat one TP=N). - Improve accelerator packing and multi-tenancy: MIG, MPS, time-slicing, multi-LoRA serving, model load and cold-start times, weight streaming — so our small-model zoo runs co-resident efficiently rather than one model per node. - Serve non-LLM models (layout detection, embeddings, rerankers) on a real inference server such as Triton, with dynamic batching and ensembles, rather than a Python web process. - Own capacity and autoscaling policy: scale on queue depth, KV-cache utilization or TTFT rather than accelerator utilization, and be able to explain why. Spot and capacity blocks, Karpenter node pools, warm pools, bin-packing across L4 / A10G / L40S / A100 / H100, and selecting the cheapest instance that fits after quantization. - Establish benchmarking rigour: load generation using real input/output length distributions from production traffic, saturation curves and latency–throughput Pareto plots reported per-accelerator-normalized, plus performance and quality regression gates in CI. PLATFORM, RELIABILITY AND GOVERNANCE - Build document AI platforms using generative AI with asynchronous architectures — queues and event-driven workers for elastic scaling and non-blocking inference. - Architect self-hosting infrastructure for LLMs on Kubernetes/EC2 with accelerator orchestration and scheduling. - Oversee correct LLM usage, including gateways (LiteLLM, Portkey), prompt engineering support and guardrails to prevent misuse or hallucinations. - Implement observability for AI/ML pipelines: distributed tracing with OpenTelemetry and X-Ray, drift and hallucination monitoring via Prometheus/Grafana, LLM-specific metrics via Langfuse. - Manage AI/ML workflows — MLflow (including serverless SageMaker MLflow setup) and Kubeflow for experiment tracking, versioning and deployment. - Add security layers: Bedrock Guardrails (PII/toxicity), KMS encryption, IAM least-privilege, VPC endpoints, CloudTrail auditing. - Optimize AWS resources (Bedrock, SageMaker, EKS) for cost, security and performance in production AI workloads. REQUIRED SKILLS Inference performance - Production experience optimizing self-hosted model serving: continuous batching, KV cache tuning