01
Problem
Execution algorithms decide how a large parent order gets sliced and routed across venues. Bad execution silently costs basis points that add up to real money. Most public writing on the topic either stops at closed-form models (Almgren-Chriss and friends) that were state of the art twenty years ago, or jumps straight to opaque RL benchmarks. There is little between the two that lets you sit down and see, under one framework, how a classical policy, a bandit, and an RL agent actually compare when the venues have honest latency and fees.
Execution Copilot is that middle layer. A simulator you can reason about, five policy families, and a metrics engine that reports implementation shortfall net of maker/taker fees.
02
System design
Python for orchestration and policies, C++ for an optional matching engine on the hot path. The venue simulator maintains persistent FIFO limit-order queues at each price level. Snapshots come from either a synthetic geometric-Brownian-motion generator or a JSONL replay of historical L2 captured from real exchanges.
- Backtester. Iterates over snapshots, calls the policy at each step, and hands its child orders to the venue simulator. Handles child-order lifecycle, cancels, and fills.
- Policies. TWAP, POV, Almgren-Chriss, LinUCB, Thompson Sampling, PPO, Actor-Critic, DQN. Classical policies have closed-form schedules; bandit and RL policies choose venues and slice sizes per step.
- Metrics engine. Implementation shortfall (arrival cost, timing cost, opportunity cost), markouts at 1s / 5s / 30s / 5m, fill rate, slippage, realised volatility along the trajectory, inventory risk per step.
- Optional C++ matching engine. Price-time priority over a lock-free single-producer single-consumer ring buffer, with a pybind11 binding so hot paths don't cross a process boundary.
03
Key decisions and tradeoffs
Simulated environment over historical fills. Real fills for a single account don't reflect what would have happened had the policy sliced differently. Simulation lets you A/B policies against the same market state. Cost: the model's assumptions (queue position, latency, fee schedule) are honest but they are still models.
Persistent FIFO queues over aggregated depth. Preserves queue position, which matters when a policy repeatedly cancels and reposts. Aggregated depth is faster but drops the information the bandit and RL policies need to learn from.
IS net of fees, not gross. Gross IS makes marketable policies look cheap and passive policies look expensive because it ignores maker rebates and taker fees. Reporting net changes the picture in ways worth arguing about in results.
C++ engine optional. Pure Python simulation runs the full benchmark in reasonable time. C++ is there when a policy stresses the hot path (RL training loops sample thousands of episodes; every millisecond compounds).
04
Product decisions
Backtester as the primary surface, not an API. A framework people would clone and modify beats a service they can only call. Every policy is a small class with a select method; adding a new one is a single file.
FastAPI dashboard is inspection, not the product. Live view of a running simulation for debugging. Not intended as a paper-trading dashboard, because that would imply guarantees the simulator does not make.
No fake real-time. Every metric reported has an experimental protocol behind it (30 to 50 Monte Carlo runs per condition, seed-controlled). Nothing streams as if fresh from an exchange.
05
Results
Thirty to fifty Monte Carlo runs per condition on AAPL synthetic mid-price over a one-hour horizon. IS is net of venue maker and taker fees:
| Policy | IS (bps) | Std |
|---|---|---|
| PPO | 5.68 | 3.94 |
| POV | 5.71 | 5.95 |
| Almgren-Chriss | 6.90 | 30.26 |
| LinUCB | 8.34 | 3.71 |
| Thompson Sampling | 8.35 | 3.55 |
| TWAP | 14.67 | 29.23 |
PPO and POV lead and are statistically tied on the mean. LinUCB and Thompson give up around 3 bps to the leaders but with much tighter variance, which matters if you care about worst-case fills on a single parent order. Almgren-Chriss has good mean cost but a heavy tail. TWAP is the baseline by design.
These figures are simulated. Full tables in docs/EXPERIMENTAL_RESULTS.md.
06
What I would improve next
- A queue-position estimator that learns from real fills. The simulator assumes strict FIFO by insertion time. Real queues have opaque priority (some hidden orders, some pro rata by venue). Calibrating queue-position priors from a small dataset of tagged fills would tighten the fee model.
- Adversarial venue latency. Currently every venue has a fixed latency. In practice, latency depends on venue load. Modelling a load-dependent latency term (or replaying observed per-venue latency traces) would test whether bandit policies overfit to the fixed-latency baseline.
- Regime detection layer above policy selection. Some policies dominate in specific regimes (POV in high-volume, Almgren-Chriss in trending). A supervisor that picks the policy given a regime classifier would probably beat every leaf policy.
- Deeper RL training budgets. PPO is trained enough to be competitive but not converged. Longer training and richer state (order book imbalance, microstructure features) is the obvious next lever.
