Telemetry storage
Telemetry storage keeps the data sets you choose in your own bucket, as Parquet files, encrypted by default, for as long as you choose. The server's dashboard reads them back, so Live activity and its sessions show records older than the server itself keeps.
Telemetry storage is on the Premium and Enterprise plans, and in the 30-day trial. The plan is needed only to set it up: turning it off never needs one.
Set it up
- Connect a bucket under Settings → Server → Scaling → Object storage: AWS S3, Cloudflare R2, Google Cloud Storage, Azure Blob Storage, or other S3-compatible storage. Backups describes the steps.
- Open Settings → Data → Telemetry storage on the hub, pick the bucket and choose the data sets.
- Files arrive under
telemetry/v1/in the bucket within a minute.
Data sets
| Data set | Default |
|---|---|
| Telemetry records | On |
| The audit log and the change log | On |
| Work items, process inventory, repository security, delivery reporting and project memory | Off |
| Captured prompts and router training examples | Off |
Captured prompts and router training examples need their own acknowledgement and encryption, and come only from computers whose privacy notice says so. The Privacy Center lists what is kept in the bucket and for how long.
Satellites' records reach the bucket through the hub, so the bucket holds everyone's.
How it's kept
- Uploaded every minute, as Parquet files, partitioned by day.
- Encrypted by default with AES-256-GCM, with a key kept in the server's credential store and included in backups. You can turn encryption off to read the files with your own analytics tools. Captured text always stays encrypted.
- Deletions follow. Records deleted on SpanDock (deleting records, a person's records, a data reset) are listed in the bucket, so readers leave them out.
- Retention per data set. You choose how long each data set stays in the bucket, or keep it with no limit.
Reading it back
With telemetry storage set up, Live activity and its sessions complete the server's own records with the bucket's. Older records are marked From storage.
- In Live activity, choose More → Show records before… and pick a time older than the server keeps. Show older goes further back.
- Open one for its full record, or an older session for its calls.
- One request reads up to 92 days back; pick an earlier time for more.
- Files read from the bucket are kept on the server in a cache of up to 2 GB. Settings → Data → Telemetry storage shows from when the server keeps records in full, and how big the cache is.
When the bucket is unreachable
Records waiting for the bucket outlive the server's retention, until the local database is over twice its size limit. Then the oldest records not uploaded yet are removed, with a red warning under the bell, until it is back under the limit. Telemetry never stops.
A failing upload, a key that's missing, or another server taking over the bucket shows a warning under the bell, and can send a notification.
Limits
- Satellites don't upload themselves; their records go through the hub.
- Only Live activity and its sessions read from the bucket so far.