Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
d14fefa
v0.8.0
kevinsbarnard Apr 20, 2026
c61e213
build: require Python 3.11+, track uv.lock, add orjson/uvloop extras
kevinsbarnard Nov 19, 2026
3ef31d5
fix: stop work on closed connections, clean daemon shutdown, unify lo…
kevinsbarnard Nov 19, 2026
22269da
perf: event-driven fan-out, per-channel conflation, cached routing
kevinsbarnard Nov 19, 2026
ad98b51
perf: shared spy accumulator with per-connection views
kevinsbarnard Nov 19, 2026
313df9f
perf: encode each message once across clients, split worker pools
kevinsbarnard Nov 19, 2026
2b39a57
perf: faster LCM-to-JSON encoding, send text payloads pre-encoded
kevinsbarnard Nov 19, 2026
5b40b2a
feat: shared CLI with --compression, --uvloop and --processes
kevinsbarnard Nov 19, 2026
ef31678
perf: skip no-op image resize, add per-channel --max-fps for Dial
kevinsbarnard Nov 19, 2026
8f3c120
test: add load and encoding benchmarks with before/after results
kevinsbarnard Nov 19, 2026
06ffd0f
docs: document conflation, new flags, extras and spy semantics
kevinsbarnard Nov 19, 2026
ae19299
test: lint fixes for new tests
kevinsbarnard Nov 19, 2026
7fe11cb
test: vendor the LCM types used by tests so the suite is self-contained
kevinsbarnard Nov 19, 2026
8094e21
style: add project ruff config and clear existing lint findings
kevinsbarnard Nov 19, 2026
db26af7
ci: add GitHub Actions lint/test workflow and pre-commit hooks
kevinsbarnard Nov 19, 2026
1d73c4c
feat: /healthz and /metrics endpoints, configurable keepalive
kevinsbarnard Nov 19, 2026
b2355f7
build(docker): install with uv, add extras, ARGS and HEALTHCHECK
kevinsbarnard Nov 19, 2026
fd2d96c
docs: deployment, monitoring and development guidance
kevinsbarnard Nov 19, 2026
b01e5e1
ci: automate versioning and changelog with python-semantic-release
kevinsbarnard Nov 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,10 @@ uv.lock
dist/
scripts/

*__pycache__/
*__pycache__/
.venv/
.pytest_cache/
.ruff_cache/
.github/
benchmarks/
tests/
72 changes: 72 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
name: CI

on:
push:
branches: [main, develop]
pull_request:
workflow_dispatch:

permissions:
contents: read

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: astral-sh/setup-uv@v8.3.0
with:
enable-cache: true
- run: uv sync --locked --all-extras
- run: uv run ruff check .

test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.11", "3.12", "3.13", "3.14"]
steps:
- uses: actions/checkout@v7
- uses: astral-sh/setup-uv@v8.3.0
with:
enable-cache: true
python-version: ${{ matrix.python-version }}
- run: uv sync --locked --all-extras
- run: uv run pytest -q

build:
needs: [lint, test]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: astral-sh/setup-uv@v8.3.0
with:
enable-cache: true
- run: uv build

# Bumps the version (via python-semantic-release, reading conventional
# commits since the last tag), pushes the version/changelog commit and
# tag, and creates a GitHub release with the built wheel and sdist.
# main produces releases; develop produces -rc.N prereleases. Runs after
# lint+test+build so nothing broken gets tagged. Not published to PyPI.
release:
needs: [lint, test, build]
if: github.event_name == 'push' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
concurrency: release
permissions:
contents: write
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
ref: ${{ github.ref_name }}
- uses: astral-sh/setup-uv@v8.3.0
with:
enable-cache: true

- name: Release
run: uv run --group release semantic-release version
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
3 changes: 1 addition & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
poetry.lock
uv.lock
dist/
*__pycache__/
*__pycache__/
14 changes: 14 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
exclude: ^(tests/lcmtypes/|lcm_websocket_server/lib/lcm_utils/channel_stats(_list)?\.py$)
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.17.6
hooks:
- id: ruff-check
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: check-added-large-files
- id: check-toml
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
109 changes: 98 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,15 @@ pip install lcm-websocket-server
pip install .
```

Or with the `image` extension:
Or with optional extras:

```bash
pip install ".[image]"
pip install ".[image]" # Dial proxy (OpenCV)
pip install ".[orjson,uvloop]" # faster JSON encoding and event loop
```

See [Development](#wrench-development) for working on the project itself.

## :rocket: Usage

> [!TIP]
Expand All @@ -33,7 +36,7 @@ pip install ".[image]"

#### JSON Proxy

The `lcm-websocket-json-proxy` command can be used to run a server that republishes LCM messages as JSON over a WebSocket connection.
The `lcm-websocket-json-proxy` command can be used to run a server that republishes LCM messages as JSON over a WebSocket connection.

> [!NOTE]
> The `lcm-websocket-server` command has been renamed to `lcm-websocket-json-proxy`. The old name is still available for backwards compatibility, but it is recommended to use the new name.
Expand All @@ -47,15 +50,32 @@ The `lcm_packages` argument is the name of the package (or comma-separated list

#### Tuning Options

Both proxy commands accept optional tuning flags:
Both proxy commands accept these options:

| Flag | Default | Description |
|------|---------|-------------|
| `--queue-size N` | 64 | Max messages buffered per connection before the oldest are dropped |
| `--poll-interval S` | 0.1 | Sleep duration (seconds) when the message queue is empty |
| `--spy-interval S` | 1.0 | LCM spy stats emission interval (seconds) |
| `--compression {deflate,none}` | `deflate` (JSON), `none` (Dial) | WebSocket permessage-deflate. Saves bandwidth for JSON on slow links at a CPU cost per client; useless for JPEG |
| `--uvloop` | off | Use the uvloop event loop (requires the `uvloop` extra) |
| `--processes N` | 1 | Run N server processes on the same port (`SO_REUSEPORT`) to use more than one core |
| `--ping-interval S` | 20 | Seconds between WebSocket keepalive pings (0 disables) |
| `--ping-timeout S` | 20 | Seconds to wait for a pong before closing (0 disables) |

The Dial proxy also accepts:

| Flag | Default | Description |
|------|---------|-------------|
| `--max-fps F` | no cap | Max transcoded frames per second per image channel |
| `--image-workers N` | min(8, cores) | Threads for image transcoding |

Delivery is event-driven; `--poll-interval` is accepted for compatibility but ignored.

Increase `--queue-size` if you see drop warnings in the logs and your consumer can tolerate a larger backlog. Decrease `--poll-interval` for lower latency at the cost of slightly higher CPU use.
If drop warnings appear in the logs, the client is not keeping up. Options: connect with `?conflate=1` (see below), increase `--queue-size`, or subscribe to fewer channels.

Each message is encoded once and shared by all clients. Encoding runs in worker threads: JSON on a small pool and images on a separate pool, so image bursts do not delay JSON. One process is bound to roughly one core for JSON work; with `--processes N` each process has its own LCM subscription, encode cache and spy statistics.

For faster JSON encoding, install the `orjson` extra (about 2x faster than the standard library path).

**Example: `compas_lcmtypes`**

Expand Down Expand Up @@ -91,7 +111,7 @@ To run the server locally on port 8765 and republish messages on all channels (J
lcm-websocket-dial-proxy --host localhost --port 8765 --channel '.*' --quality 75 --scale 1.0
```

The Dial proxy depends on the `molars-lcmtypes` Python package to be installed. This package is not available on PyPI, so it must be built and installed manually; see the [MolaRS repository](https://github.com/CoMPASLab/molars/) for more info.
The Dial proxy depends on the `molars-lcmtypes` Python package to be installed. This package is not available on PyPI, so it must be built and installed manually; see the [MolaRS repository](https://github.com/CoMPASLab/molars/) for more info.

The `Dockerfile.dial` file can be used to build the image with the `molars-lcmtypes` package installed. For this, the Python wheel `molars_lcmtypes-0.0.0-py3-none-any.whl` must be placed at the repository root before building the image.

Expand All @@ -114,18 +134,31 @@ Clients can specify an optional `update_interval_ms` query parameter in the WebS
const ws = new WebSocket('ws://localhost:8765/.*?update_interval_ms=500');
```

This will cause the latest message for each subscribed channel to be sent to the client every 500 milliseconds.
This will cause the latest message for each subscribed channel to be sent to the client every 500 milliseconds. Between ticks only the latest message per channel is kept, so low-rate channels are never crowded out by high-rate ones.

#### Conflation

By default each connection buffers up to `--queue-size` messages and drops the oldest when full, regardless of channel. A client that cannot keep up with every message can instead request only the latest message per channel:
```javascript
const ws = new WebSocket('ws://localhost:8765/.*?conflate=1');
```

Messages are still delivered as soon as possible, but a busy channel can no longer evict a quiet one.

#### Invalid Requests

A connection with an invalid channel regex (or one longer than 1024 characters) or an invalid `update_interval_ms` is closed with code 1008 (policy violation) and a reason describing the problem.

## :bar_chart: LCM Spy Virtual Channel

The server provides a **virtual channel** called `LWS_LCM_SPY` that publishes per-channel statistics at 1 Hz on a per-connection basis. Each WebSocket client gets its own stats stream, computed from live traffic observed during that connection.
The server provides a **virtual channel** called `LWS_LCM_SPY` that publishes per-channel statistics at 1 Hz (`--spy-interval`) to every client whose channel regex matches it. Statistics are counted once per message for all traffic the server receives, independent of any client's queue. Each client's view starts when it connects: message counts are since connect, and channels appear once they have received a message since then.

### Channel Statistics

The virtual channel publishes a `channel_stats_list` message containing statistics for each active channel:
- **Hz**: Message frequency
- **Bandwidth**: Bytes per second
- **Jitter**: Variation in message intervals
- **Jitter**: Spread (max - min) of inter-arrival intervals over the last spy interval
- **Message count**: Total messages received
- **Undecodable count**: Messages that couldn't be decoded

Expand All @@ -138,7 +171,7 @@ Connect to the spy channel using the channel name in the WebSocket path:
const ws = new WebSocket('ws://localhost:8765/LWS_LCM_SPY');

ws.onmessage = (ev) => {
const stats = JSON.parse(ev.event);
const stats = JSON.parse(ev.data).event;
console.log('Channel stats:', stats.channels);
};
```
Expand All @@ -158,6 +191,41 @@ A Python example client is provided in `examples/spy_client.py`:
python examples/spy_client.py --host localhost --port 8765
```

## :satellite: Deployment

### Monitoring

Plain HTTP requests to the server port answer two endpoints (WebSocket connections on any path are unaffected):

- `GET /healthz` returns `ok`.
- `GET /metrics` returns Prometheus text: connected clients, LCM messages received, frames and bytes sent, messages dropped from full client queues, encode counts, errors and time, and worst event-loop lag since the last scrape.

A rising `lws_dropped_total` means clients are not keeping up. A sustained `lws_event_loop_lag_max_seconds` above roughly 50 ms means the process is saturated; use `--processes` or reduce load.

### LCM receive buffers

LCM messages dropped by the kernel never reach the server, so they appear in neither the logs, `/metrics`, nor the spy channel. Large or bursty traffic (especially images) needs larger UDP receive buffers than Linux defaults:

```bash
sudo sysctl -w net.core.rmem_max=16777216 net.core.rmem_default=16777216
export LCM_DEFAULT_URL="udpm://239.255.76.67:7667?ttl=0&recv_buf_size=16777216"
```

Persist the `sysctl` settings in `/etc/sysctl.d/`. Check for drops with `netstat -su` (`RcvbufErrors`).

### Matching the link

- **Local network**: compression off for best CPU efficiency.
- **Constrained links (e.g. ship to shore)**: keep `--compression deflate` for JSON; for Dial use `--scale`, `--quality` and `--max-fps` to cut image bandwidth, and raise `--ping-timeout` if large frames can take longer than 20 s to send. Clients that do not need every message should connect with `?conflate=1` or `?update_interval_ms=`.

### Scaling

One process handles about 100 JSON clients at 1.2 k msg/s (see [benchmarks](benchmarks/README.md)). Beyond that, `--processes N` adds cores. Each process has its own LCM subscription and encode cache, so image transcoding is repeated per process: keep the Dial proxy at one process and run a separate JSON proxy for large numbers of JSON clients.

### Security

The server has no authentication or encryption. When it is reachable beyond a trusted vehicle network, put it behind a reverse proxy that terminates TLS (`wss://`) and enforces access control.

## :whale: Docker

### Build
Expand Down Expand Up @@ -190,6 +258,25 @@ docker run \

Note that the `HOST`, `PORT`, and `CHANNEL` environment variables specified above are the defaults for the `mbari/lcm-websocket-server` image. These can be omitted if the defaults are acceptable.

Extra command-line flags can be passed with the `ARGS` environment variable, e.g. `-e ARGS="--uvloop --processes 4 -vv"`. The image includes the `orjson` and `uvloop` extras and a `HEALTHCHECK` that polls `/healthz`; if `HOST` is set to an address other than `0.0.0.0` or `127.0.0.1`, the health check cannot reach it.

The `LCM_PACKAGES` environment variable should be set to the name of the package (or comma-separated list of packages) that contains the LCM Python message definitions. The `/app` directory is included in the `PYTHONPATH` so that any packages mounted there (as shown with `-v` above) can be imported.

It's recommended to run with `--network=host` to avoid issues with LCM over UDP. This will allow the container to use the host's network stack.

## :wrench: Development

```bash
uv sync --all-extras
uv run pre-commit install # ruff and file hygiene hooks on commit
uv run pytest
uv run ruff check .
```

The tests are self-contained: they use LCM types vendored in `tests/lcmtypes` and the in-process `memq://` LCM provider, so no LCM network or MolaRS install is needed. CI runs lint and tests on Python 3.11 to 3.14.

Before a release, run the load benchmarks and compare against [benchmarks/README.md](benchmarks/README.md) to catch performance regressions. They need the full CoMPAS LCM types on `PYTHONPATH`:

```bash
uv run python benchmarks/load.py all
```
47 changes: 47 additions & 0 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Benchmarks

Load and microbenchmarks used to characterize the server under load.

Both scripts need the CoMPAS LCM Python types (`stdlcm`, `senlcm`, `navlcm`, `geolcm`) on `PYTHONPATH`.

```bash
export PYTHONPATH=/path/to/lcmtypes/python
uv run python benchmarks/load.py all # or: latency, json-scale, json-rate, dial, slow, periodic
uv run python benchmarks/encode.py
```

## `load.py`

Runs the real `LCMWebSocketServer` in one process. Threads stand in for the LCM thread by calling `LCMRepublisher.inject()` at fixed rates, so UDP and the LCM C library are not exercised. Clients run in 8 worker processes over loopback.

Default workload: 10 `odometry_t` channels at 100 Hz plus a 512-beam `multibeam_euclidean_t` at 20 Hz (1.22 k msg/s). Dial scenarios add a 2048x1536 Bayer `image_t` at 10 Hz.

Reported per scenario:

| Field | Meaning |
|---|---|
| `srvCPU` | Server process CPU over the window (100% = one core) |
| `deliv` | Non-image messages received per client as % of published |
| `lat` | Publish-to-receive latency from the `header_t` timestamp |
| `loopLag` | Worst event-loop stall seen by a 10 ms probe |
| `injLate` | Times the simulated LCM thread fell behind schedule |
| `dropWarn` | Queue-overflow warnings logged |
| `encodes/frame` | JPEG encodes per published image (1.00 = fully shared) |
| `spy-reported Hz` | Rates from `LWS_LCM_SPY`, to check against the source |

## Results

Deucalion (24 cores), Python 3.13, websockets 17.4, orjson installed. Baseline is v0.8.0 with default flags.

| Scenario | v0.8.0 | Now |
|---|---|---|
| JSON, 1 client | 66% delivered, p50 32 ms | 100%, p50 0.5 ms |
| JSON, 50 clients | 36%, CPU 112% | 99.9%, CPU 76% |
| JSON, 100 clients, compression on | 9% | 92% |
| JSON, 100 clients, compression off | 12% | 100%, p50 4.4 ms |
| JSON, 10 clients at 10 k msg/s | 2.8% | 38% (one core saturated) |
| Dial, 10 clients, image encodes per frame | 9.3 | 1.00 |
| Dial, 10 clients, CPU (compression off) | 162% | 42% |
| Spy Hz for a 100 Hz channel (streaming / periodic) | 64 / 12 | 100 / 100 |

Beyond about 100 JSON clients or 10 k msg/s, one process is GIL-bound; use `--processes N`.
38 changes: 38 additions & 0 deletions benchmarks/encode.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
"""
Microbenchmark of JSONHandler.handle (decode + JSON encode) on the stdlib
and orjson paths.

Usage (requires stdlcm/senlcm/navlcm/geolcm on PYTHONPATH):
uv run python benchmarks/encode.py
"""
import time

from geolcm import point_t
from lcmutils import LCMTypeRegistry
from navlcm import odometry_t
from senlcm import multibeam_euclidean_t
from stdlcm import header_t

from lcm_websocket_server.apps.json_proxy import JSONHandler
from lcm_websocket_server.lib.lcm_utils import types as T

reg = LCMTypeRegistry()
for p in ("stdlcm", "senlcm", "navlcm", "geolcm"): reg.discover(p)
hdr = header_t(); hdr.frame_id = "base"
odo = odometry_t(); odo.header = hdr
mb = multibeam_euclidean_t(); mb.header = hdr; mb.n_beams = 512
mb.flags = [0]*512; mb.range = [point_t() for _ in range(512)]; mb.intensity = [float(i) for i in range(512)]
h = JSONHandler(reg)
def bench(fn, n):
fn(); t = time.perf_counter()
for _ in range(n): fn()
return (time.perf_counter() - t) / n * 1e6
orj = T.orjson
if orj is None:
print("orjson not installed; orjson column repeats stdlib")
for name, data, n in (("odometry_t", odo.encode(), 20000), ("multibeam n=512", mb.encode(), 1000)):
T.orjson = None
a = bench(lambda data=data: h.handle("C", data), n)
T.orjson = orj
b = bench(lambda data=data: h.handle("C", data), n)
print(f"{name:<18} stdlib {a:8.1f} us orjson {b:8.1f} us")
Loading