Configuration reference
Settings for each independent remoting path.
PyTorch device
| Setting | Read by | Default / behavior |
|---|---|---|
RGPU_OPSERVER | Client | 127.0.0.1:9720 |
RGPU_ALLOW_VERSION_MISMATCH | Client | Set 1 to bypass version checking; matching major.minor is supported |
RGPU_ADVISE_AFTER | Client | 50000 messages before suggesting graph shipping |
RGPU_FLUSH_OPS | Client | 64 queued operations before sending a batch |
RGPU_FLUSH_BYTES | Client | 262144 queued encoded bytes before sending a batch |
RGPU_CONNECT_TIMEOUT | Client | 10 seconds for each connection attempt |
RGPU_RECV_TIMEOUT | Client | 300 seconds for a server response before treating the connection as lost |
RGPU_RECONNECT_SECONDS | Client | 60 seconds total for reconnect attempts |
RGPU_MAX_FRAME | Client and server | 1073741824 bytes; largest accepted encoded frame |
RGPU_TF32 | Server | Disabled; set 1 to allow TF32 matmuls and convolutions |
RGPU_SESSION_GRACE | Server | 120 seconds to retain a dropped session |
Set server variables in the environment that starts rgpu-opserver; setting RGPU_TF32 only on the laptop does not configure the remote server.
RGPU_TF32=1 rgpu-opserverTF32 trades numerical behavior for performance. Compare results at your workload's tolerance before enabling it.
rgpu-opserver --memory-fraction F caps the server at fraction F of the
GPU's memory. It needs a CUDA device. See Share one GPU.
RGPU_ALLOW_VERSION_MISMATCH=1 is a diagnostic escape hatch, not a compatibility guarantee. Do not use a mismatched benchmark run as the supported installation recipe.
The flush thresholds control when queued work is written to the socket; reaching one does not wait for the GPU or add a request-response round trip. Lower values can reduce the time before work reaches the server while increasing writes.
The client retains up to 64 MiB of sent messages so it can replay them after a connection drop. Near that limit it requests a lightweight acknowledgement of all preceding messages. This adds a round trip without synchronizing queued GPU work. A single encoded message larger than 64 MiB cannot be retained for replay.
RGPU_CONNECT_TIMEOUT applies to each connection attempt.
RGPU_RECONNECT_SECONDS is the overall recovery budget, with retry backoff.
RGPU_RECV_TIMEOUT is a backstop for a connection that stays open without
replying; a single legitimate operation may run for up to that long.
Set RGPU_MAX_FRAME in both the client and server environments when changing
it. Raising it permits larger allocations while decoding untrusted input, so
keep the default unless a valid encoded operation exceeds it.
CUDA shim
| Setting | Read by | Default / behavior |
|---|---|---|
RGPU_SERVER | Client | 127.0.0.1:9713 |
RGPU_PORT | Server | 9713 |
RGPU_VERBOSE | Shim | Log forwarded calls |
RGPU_STATS | Client | When set, print round trips, one-way calls, transferred bytes and the busiest replying APIs at process exit |
RGPU_BATCH | Client | Batching enabled; set 0 for synchronous debugging |
RGPU_FLUSH_BYTES | Client | Queued calls are sent once they reach this many bytes; default 1024. Larger sends fewer, bigger writes but starts the GPU later |
RGPU_LAZY_SYNC | Client | Unset: a stream synchronize waits unless only uploads preceded it. 0: always wait. 1: never wait |
RGPU_CUBLAS, RGPU_CUBLASLT, RGPU_CUDNN | Server | Override paths to real math libraries |
RGPU_SESSION_GRACE | Server | 120 seconds |
RGPU_MAX_SESSIONS | Server | 64, including sessions in their grace period |
RGPU_HANDSHAKE_TIMEOUT_SECONDS | Server | 10 seconds for a new connection to complete its handshake |
RGPU_MAX_PENDING_HANDSHAKES | Server | 256 simultaneous connections still completing their handshake |
RGPU_RECONNECT_SECONDS | Client | 60 seconds |
RGPU_MAX_CLIENT_THREADS | Server | 4096 live client threads per session |
RGPU_MAX_CONTEXT_STACK | Server | 64 entries per client thread |
RGPU_BATCH belongs to the CUDA path. Do not assume it controls Python opserver batching.
RGPU_LAZY_SYNC controls when cuStreamSynchronize and
cudaStreamSynchronize wait for the server. Each wait is a round trip, and a
program that synchronizes after every small upload pays the link's latency
each time. llama.cpp does so about twenty times per generated token.
- Unset (the default): a synchronize is queued, and returns at once, when only uploads were issued to its stream since that stream last waited. An upload's bytes are taken when it is issued, so there is nothing left for the caller to wait for. A synchronize after any other GPU work on the stream, such as a kernel, a memset or a cuBLAS or cuDNN call, waits as before. The one thing reported late is an upload's own failure, such as a bad destination: the next call that waits reports it.
0: every synchronize waits. Use this if a program must see an upload's failure at the synchronize.1: every synchronize is queued, whatever it follows. A failure of any queued work is then reported by the next call that waits, the work may not have finished when the synchronize returns, and code that times a kernel by the synchronize after it measures nothing.
Results do not change in any mode: the server runs every call in the order it
was issued, and reading device memory waits for everything before it.
cuCtxSynchronize and cudaDeviceSynchronize always wait.
The handshake timeout and pending-handshake limit bound connections before they become sessions. They protect the unauthenticated listener from peers that connect and then send nothing or send the handshake too slowly. They do not replace the firewall and network isolation described in deployment.
For command options, use rgpu-run --help and rgpu-opserver --help. For network restrictions, read deployment.