Skip to content

The worker workflow

A worker goes from an idea to a deployment in six steps. Each step has its own tool, and each one catches a different class of mistake before the next, more expensive step.

flowchart TD
    DEV["1 Develop<br/>IDecide, IProcess, ...<br/>bound by BehaviorWorkerBase"]
    UNIT["2 Unit test<br/>step the behavior on plain values;<br/>BehaviorTestHost for the CloudEvent edge"]
    LOCAL["3 Run locally<br/>LocalWorkManager: the workers wired by topics,<br/>feedback loops included"]
    DEPLOY["4 Deploy<br/>C# text: platform.DeploySourceAsync (text/x-csharp)<br/>or a nupkg (VirtufinPackageWorker)"]
    REPLAY["5 Replay and compare<br/>SubscribeAsync, then publish the same inputs;<br/>compare with the local run by content"]
    CLEAN["6 Clean up<br/>platform.CleanupAsync; orphans by tag"]
    DEV --> UNIT --> LOCAL --> DEPLOY --> REPLAY --> CLEAN
    UNIT -. "a failing case" .-> DEV
    LOCAL -. "a wrong cascade or a dead letter" .-> DEV
    REPLAY -. "a difference" .-> LOCAL
  1. Develop. Write the decision as a Virtufin.Core behavior (IDecide, IProcess, IExecutor, ...) and bind it to CloudEvents with BehaviorWorkerBase or a domain base such as StrategyWorkerBase (see behavior workers). The behavior is a plain function of state and input, so it needs no WorkManager to run.
  2. Unit test. Step the behavior on synthetic inputs and assert on its outputs. BehaviorTestHost then sends real CloudEvents through the worker's IWorker entry points. It checks the decoding, the required config, and the output topics and payloads.
  3. Run locally. Deploy the worker and its counterparts, such as an executor and a positions worker, to a LocalWorkManager. Then publish the inputs as an IObservable. The local WorkManager applies the Runtime's per-event rules and drains each input's whole cascade before the next, so a feedback loop behaves as it will on the cluster. Observe the order, fill and position topics, and the dead letters, which should be empty.
  4. Deploy. Send the worker to the cluster through the API gateway, with the same config as the local run. From a notebook it goes as C# text (text/x-csharp, compiled by the Runtime's C# source engine) with platform.DeploySourceAsync. For a published worker, use a nupkg (see packaging). Deploy consumers before producers, because a worker subscribes from the tail.
  5. Replay and compare. Subscribe to the output topics with SubscribeAsync, then publish the same inputs. Pace the replay: after each input on which the local run produced an order, wait for its position before the next input, because the cluster does not drain cascades for you. Compare the two runs by content (event time, side, quantity, limit, fills, positions), not by ids or wall-clock times.
  6. Clean up. Remove the workers and connections the run created (platform.CleanupAsync). Workers are tagged, so anything left over can be found by its tag.

The examples repository walks through all six steps for a breakout strategy, in the notebook dotnet/breakout-worker-workflow.verso.