Netflix’s oci-agent is a dual-agent actor-critic workflow designed to automate advanced causal information evaluation whereas sustaining human-in-the-loop oversight.
Netflix formally open-sourced oci-agent (v0.1.0 preliminary public launch) on GitHub underneath the Netflix-Skunkworks repository for Observational Causal Inference (OCI). The instrument is designed to automate repetitive or error-prone duties in causal evaluation, corresponding to sensitivity evaluation and monitoring a number of iterations, leaving higher-level duties like query framing and outcome analysis to human analysts.
The oci-agent supply code is publicly accessible on GitHub. The workflow frames OCI evaluation as goal trial emulation, treating causal estimation as discovering the optimum A/B take a look at for a given query. The system utilises a dual-agent actor-critic structure, the place an actor agent executes the evaluation plan and a critic agent critiques the outcomes and surfaces potential gaps or flaws.
Human analysts initialise the method by offering an evaluation plan and a templated Jupyter pocket book. The actor agent makes use of the plan to supply a specification, populate pocket book parameters, and execute the evaluation code.
To handle the analysis problem in causal inference (the place floor fact is absent), the workflow combines clear course of audits with human oversight. The brokers generate inspectable and re-executable artifacts, together with plans, specs, plots, and up to date Jupyter notebooks, slightly than merely offering remaining output values.
In benchmark testing on the Atlantic Causal Inference Convention (ACIC) dataset, Netflix discovered the workflow achieved aggressive efficiency in comparison with present methods.
In a Netflix case research estimating the retention impression of latest leisure varieties (e.g. video games), a baseline Claude mannequin returned an overestimate by way of easy linear regression, whereas the oci-agent workflow produced an estimate that was simply 25% of the baseline after the critic agent flagged early-adopter bias and a failed placebo take a look at.
