← Back

Company OS

·Bryan Lai
TechnicalEngineeringAISystemsStartups

Company OS

Deploying AI models inside an organization requires structured operational context. Generative models cannot compensate for missing documentation, fragmented data schemas, or uncoordinated feedback loops.

An effective organizational operating system makes work legible, data schemas authoritative, and customer feedback actionable.

System Legibility & Structured Audit Trails

Autonomous agents require structured operational context. Unrecorded Slack discussions, informal customer feedback, and unindexed support tickets leak context and cause repeated engineering mistakes.

Every operational workflow should maintain an explicit audit trail:

  • Support tickets with categorized telemetry and customer reproduction steps.
  • Sales notes with structured objection logging.
  • Engineering specifications with metric benchmarks and acceptance criteria.
  • Incident post-mortems with permanent regression test coverage.

Recording context in queryable, structured storage enables downstream agents to leverage historical organizational memory.

Data Analysis Is Harder Than Codegen

Code is not the only domain teams want to automate. Analytics breaks the dream faster.

People get seduced by demos: a model writes SQL, the chart appears, and the room claps. Then someone asks what the denominator means.

Software can often be bounded by unit tests and state transitions. Analytics is slipperier:

  • One latency metric lives in API logs; another lives in a dashboard tile.
  • A caveat lives in a code comment.
  • A denominator lives in an engineer's memory because timed-out requests were excluded from the pipeline last quarter.
  • A join silently changes the population.
  • A segment definition drifted two quarters ago without updating downstream slides.

The query compiles. The chart looks clean. The answer is still wrong.

It is harder to prove an analytical answer than to prove a function returned the expected value. Even if an AI generates valid SQL, you still have to defend it:

Why this join? Why this filter? Why this CTE? Why this denominator? Why this exclusion? Why this table and not the other one?

Two tables can both seem valid and appear to answer the same question while containing divergent data:

  • Table A was backfilled after an incident while Table B was not.
  • A pipeline bug corrupted one source but not the other.
  • A dbt model drifted between environments.
  • One table records event-time while another records load-time.
  • One view silently deduplicates records while another preserves raw logs.
  • A harmless-looking filter erased every timed-out search because timed-out requests never emitted search_results_rendered.

To automate analytics reliably, you need explicit semantic definitions:

  1. Metric definitions.
  2. Grain definitions.
  3. Source-of-truth tables.
  4. Reusable query shapes.
  5. A clear owner for each metric.

Tools like PyPika help construct SQL, but SQL syntax is not the hard part. The hard part is deciding what the business means—such as why search_latency_p95_v2 excludes cache hits.

In analytics, "the query ran" is not the bar. Even "the SQL is elegant" is not the bar. The bar is whether the result is explainable, defensible, and repeatable. Code can differ; meaning cannot.

First define the meaning. Then automate the work.

Closed-Loop Operational Validation

Organizational automation requires pairing internal delivery speed with external market validation:

  • Internal Loop (Execution): Scoped task definitions, automated CI/CD checks, and static verification.
  • External Loop (Validation): Production telemetry, customer retention metrics, and support ticket trends.

Targeted automation should start within a single bounded domain (such as support ticket triage or search telemetry analysis) before scaling across organizational workflows.

Edited: Refactored for conciseness and technical clarity.