Async jobs & mediated web fetch new¶

The runner carries a worker-thread job pool (size from lattice_runner_config.worker_threads). Jobs run off the fixed-tick main thread; their results are delivered back on the main thread inside lattice_runner_tick(), so slow I/O never stalls the simulation. The web fetch is layered on top of the pool and is mediated: every request is checked against an egress allow-list + a rate limit before any socket opens — so the server cannot be turned into an arbitrary outbound proxy (SSRF / exfiltration defence).

Threading contract

lattice_http_request, lattice_http_poll, and lattice_http_configure are main-thread calls — call them from the same thread that calls lattice_runner_tick. The on_http_result callback and lattice_http_poll only ever surface results on the tick thread.

flowchart LR
  M["main thread<br/>lattice_runner_tick"] -->|"lattice_http_request"| Q[egress gate:<br/>allow-list + rate limit]
  Q -->|rejected| R1[result ok=0<br/>status &lt; 0]
  Q -->|admitted| W[worker thread<br/>does the I/O]
  W --> D[drained back<br/>onto tick thread]
  D --> R2[on_http_result<br/>or lattice_http_poll]
  R1 --> R2

Egress policy — `lattice_http_configure`¶

LATTICE_API lattice_result lattice_http_configure(lattice_runner* r,
                                                  const lattice_http_egress_config* cfg);

Install/replace the egress policy. Until configured, the allow-list is EMPTY (every request is rejected) — egress is off by default. Role: authority (the server controls egress; an untrusted module can never widen its own). Returns: LATTICE_OK or an error.

typedef struct {
    const char* const* allow;       /* array of "host:port" strings */
    uint32_t           allow_count;
    uint32_t           rate_burst;  /* 0 => a small default */
    double             rate_per_sec;/* <= 0 => no rate limit */
} lattice_http_egress_config;

allow is an array of allow_count "host:port" C strings (e.g. "api.example.com:80"); a request whose host:port is not present is rejected immediately. An empty allow-list ⇒ deny all. rate_burst is the token-bucket capacity; rate_per_sec <= 0 disables the rate limit. The core copies the strings.

const char* allow[] = { "api.example.com:80", "127.0.0.1:8080" };
lattice_http_egress_config eg = {0};
eg.allow = allow; eg.allow_count = 2;
eg.rate_burst = 8; eg.rate_per_sec = 4.0;
lattice_http_configure(r, &eg);

`lattice_http_request`¶

LATTICE_API uint64_t lattice_http_request(lattice_runner* r, lattice_http_method method,
                                          const char* url, const uint8_t* body, uint32_t body_len);

Issue an async HTTP request. Role: authority. url is an absolute URL (http://host[:port]/path; the reference supports plaintext http only — https needs a TLS transport). body/body_len is the request body (for POST; NULL/0 for GET).

Returns: a non-zero handle on acceptance, or 0 if the request was rejected immediately (bad URL / not allow-listed / rate-limited). In both cases exactly one result is delivered later (via on_http_result or lattice_http_poll) carrying the handle; for an immediate reject the result's ok == 0 and status is the negative reason. Never blocks the tick.

typedef enum { LATTICE_HTTP_GET = 0, LATTICE_HTTP_POST = 1 } lattice_http_method;

Synthetic failure status codes¶

Returned (as a negative status) when an exchange fails before/instead of an HTTP status — distinct from real HTTP codes (always ≥ 100), so a caller can branch on status < 0.

typedef enum {
    LATTICE_HTTP_NOT_ALLOWED  = -1, /* host:port not on the egress allow-list   */
    LATTICE_HTTP_RATE_LIMITED = -2, /* rate limit exceeded                      */
    LATTICE_HTTP_BAD_URL      = -3, /* unparseable / unsupported-scheme URL     */
    LATTICE_HTTP_CONNECT_FAIL = -4, /* TCP connect failed                       */
    LATTICE_HTTP_IO_FAIL      = -5, /* send/recv failed mid-exchange            */
    LATTICE_HTTP_NO_TRANSPORT = -6  /* scheme needs a transport not configured  */
} lattice_http_status;

Reacting — `on_http_result`¶

void (*on_http_result)(void* user_data, uint64_t handle, int ok, int status,
                       const uint8_t* body, uint32_t body_len);

Fires synchronously inside lattice_runner_tick() on the main thread. handle is the value lattice_http_request returned; ok is 1 for a completed HTTP exchange or 0 for a transport/policy failure; status is the HTTP status (e.g. 200) when ok == 1, or a negative LATTICE_HTTP_* code when ok == 0; body/body_len is the response body (owned by the core, valid only for the call). Only ever fires for requests the module itself issued. A NULL pointer (the default) means "poll for results via lattice_http_poll instead".

`lattice_http_poll`¶

LATTICE_API int lattice_http_poll(lattice_runner* r,
                                  uint64_t* out_handle, int* out_ok, int* out_status,
                                  uint8_t* out_body, uint32_t out_body_cap, uint32_t* out_body_len);

Main thread: poll for the next completed request without a callback. On return-1 it fills the out params (any may be NULL): *out_handle, *out_ok (1/0), *out_status (HTTP code or negative reason). The body is copied into out_body up to out_body_cap and *out_body_len is set to the full body length (which may exceed the capacity). Returns: 1 if a result was dequeued, 0 if none is ready. Results are FIFO. Use this or on_http_result, not both.

Job-pool introspection¶

LATTICE_API uint32_t lattice_jobs_worker_count(lattice_runner* r);
LATTICE_API uint64_t lattice_jobs_submitted(lattice_runner* r);
LATTICE_API uint64_t lattice_jobs_completed(lattice_runner* r);
LATTICE_API uint64_t lattice_jobs_outstanding(lattice_runner* r);

Metrics for the job pool. Role: both. worker_count is the live worker thread count; submitted / completed count work bodies; outstanding is submitted minus delivered-to-main.

Async jobs & mediated web fetch new¶

Egress policy — lattice_http_configure¶

lattice_http_request¶