Agent Engineer

Skills this role screens for

About the role

Lexsi is hiring for the Role of Agent Engineer!

The work

Responsibilities of the Candidate

• The core agent loop and its execution model, covering orchestration, concurrency, sub-agent coordination, cancellation, timeouts, checkpointing and resume.
• Context and state management for long-running tasks, including retention, compaction, and how an agent reasons over what it has already done.
• The shared tool protocol, its schemas, versioning, and the internal libraries every agent team builds against.
• The execution substrate. Sandboxed environments that are reproducible, snapshot-able and resource-bounded, and that behave identically for a repository task, a training run and a data analysis.
• Failure semantics. Retry policy, partial failure, idempotency, and the distinction between a recoverable error and a task that should stop and hand back.
• The trace schema every agent emits, which serves at once as our debugging surface, our training signal, and the audit record our customers keep.

Also

Evals

• Task suites for each agent type, built from real work rather than synthetic benchmarks, and the infrastructure to run them at scale in parallel.
• Verifiers and scoring. Deterministic checks where the task allows it, model-graded rubrics where it does not, and calibration of the graders themselves.
• Regression gates that run on every harness change, with cost and latency accounted alongside quality.
• The statistics to say a result is real. Seeds, variance, pass@k, confidence, and knowing how many runs are needed before anyone claims an improvement.
• Turning observed production failures into permanent test cases.
• Tooling for how we work. Trace inspection, replay, and diffing one run against another.

Who they want

Requirements

• Strong software engineering fundamentals and advanced Python. You have shipped and operated production services, not only written them.
• Concurrency and distributed systems. Async execution, worker pools and queues, retries and idempotency, backpressure, cancellation, and reasoning clearly about what happens when a step fails halfway through.
• Containers and sandboxing. Docker and OCI internals, resource isolation, reproducible environments, and an understanding of why a job that passes locally fails in a sandbox.
• Test and CI infrastructure. You have built test harnesses, run large suites in parallel, and dealt with flakiness as an engineering problem rather than an annoyance.
• Measurement literacy. Comfort with variance, sampling and significance, and healthy skepticism toward benchmark results including your own.
• Observability instincts. Distributed tracing, structured logging, and building the inspection tooling that makes non-deterministic systems debuggable.
• You debug systematically. You find out what actually happened rather than adjusting things until the symptom disappears.
• You are comfortable when problems are loosely specified and ownership is assumed rather than assigned.
• Experience with evaluation or benchmarking work for AI systems is a strong plus.
• Experience with agentic systems, LLM inference and serving, or developer tooling is a plus.
• Open source contributions we can read are a plus.