pvview

pvview is a monitoring tool for solar, home-energy, temperature sensors and other arbitrary sensors published via MQTT and HTTP. It covers the whole stack from collecting and storing data (tsdb_collector.py) to providing a modern grid based dashboard UI (tsdb_server.py).

It is built around a compact day based .tsdb file format instead of a continuously running database service. The collector writes one raw file per UTC day, the server exposes those files through a REST API and browser dashboard.

There is also an InfluxDB extraction tool to one-time migrate historical InfluxDB data into the *.tsdb format (tsdb_extract_from_influxdb.py).

Typical data sources are MQTT topics from OpenDTU or sensor gateways and HTTP JSON endpoints such as Fronius solar_api responses.

Features

• Collect numeric and string values from MQTT topic filters.
• Poll JSON HTTP endpoints and map nested fields into time-series names.
• Store raw daily TSDB files such as data_2026-03-19.tsdb.
• Generate downsampled daily files from 1s through 1h for fast charts.
• Serve a browser dashboard and REST API from plain TSDB files.
• Define virtual series derived from existing series in the dashboard server.
• Migrate selected measurements and fields from InfluxDB 1.x query APIs.

Main executables

tsdb_collector.py               MQTT/HTTP collector and TSDB maintenance tool
tsdb_server.py                  REST API and dashboard server

Other tools and files

tsdb.py                         Core TSDB reader/writer implementation
tsdb_extract_from_influxdb.py   InfluxDB-to-TSDB migration tool
dashboard_ui/                   Browser dashboard served by tsdb_server.py
tsdb_collector_ui/              Browser config UI served by tsdb_collector.py
example_configs/                Example collector and InfluxDB configs
Test/                           Unit tests

Requirements

• Python >= 3.14
• uv for the examples below
• paho-mqtt when collecting from MQTT

Install dependencies:

uv sync

Run tests:

make test

Quick Start

Generate demo data and start the dashboard:

uv run python tsdb_collector.py --generate-demo-db 7
mkdir -p data
for f in demo_*.tsdb; do mv "$f" "data/${f/demo_/data_}"; done
uv run python tsdb_collector.py --downsample data/data_*.tsdb
uv run python tsdb_server.py --data-dir data

Open the dashboard at:

http://127.0.0.1:8080/

For real data, copy one of the example configs and adapt it:

cp example_configs/tsdb_collector.toml ~/.tsdb_collector.toml
uv run python tsdb_collector.py --ui

Then open:

http://127.0.0.1:8081/

Data Files

pvview uses UTC days for file names.

• Raw collector files: data_YYYY-MM-DD.tsdb
• Raw exploratory MQTT log files: mqttlog_YYYY-MM-DD.tsdb
• Downsampled numeric files: dsda_YYYY-MM-DD.<level>.tsdb
• Downsampled MQTT log files: dsmq_YYYY-MM-DD.<level>.tsdb

Downsample levels include:

1s, 5s, 15s, 1m, 5m, 15m, 1h

The TSDB file format is documented in TimeSeriesDbFileFormat.md.

tsdb_collector.py

tsdb_collector.py is the main data ingestion and maintenance tool. It can collect MQTT topics, poll configured HTTP JSON endpoints, create downsampled files, inspect TSDB files, and serve a small web UI for editing collector configuration.

By default it reads ~/.tsdb_collector.toml. The important config keys are:

mqtt_server = "192.168.1.14:1883"
data_dir = "data"
quantize_timestamps = 100

topics = [
  "solar/ac/power",
  "solar/+/+/power",
  "solar/+/+/voltage",
]

mqttlog_topics = [
  "openmqttgw/+/RTL_433toMQTT/#",
]

HTTP polling is configured under [http]. See example_configs/tsdb_collector.toml for a larger MQTT plus Fronius HTTP setup.

Collector Setup UI

Start the config UI on the default address:

uv run python tsdb_collector.py --ui

Use a custom config file and port:

uv run python tsdb_collector.py --config ./collector.toml --ui --ui-port 8091

Discover MQTT Topics

List topics seen during a 60 second window:

uv run python tsdb_collector.py --mqtt-server 192.168.1.14:1883 --list-topics --timeout 60

Filter topic names:

uv run python tsdb_collector.py --mqtt-server 192.168.1.14:1883 --list-topics --filter 'solar/*'

Monitor matching messages continuously:

uv run python tsdb_collector.py --mqtt-server 192.168.1.14:1883 --monitor --filter 'solar/*'

Print an OpenDTU summary for topics below the solar root:

uv run python tsdb_collector.py --mqtt-server 192.168.1.14:1883 --open-dtu-summary --timeout 5

Collect Data

Collect with the default config:

uv run python tsdb_collector.py

Collect into a specific data directory:

uv run python tsdb_collector.py --config ./collector.toml --collect --data-dir ./data

Collect selected MQTT topics without editing the config:

uv run python tsdb_collector.py \
  --mqtt-server 192.168.1.14:1883 \
  --data-dir ./data \
  --collect \
  --topics 'solar/ac/power' \
  --topics 'solar/+/+/yieldtotal'

Collect exploratory MQTT logs into mqttlog_*.tsdb:

uv run python tsdb_collector.py \
  --mqtt-server 192.168.1.14:1883 \
  --data-dir ./data \
  --collect \
  --mqttlog-topics 'openmqttgw/+/RTL_433toMQTT/#'

Increase logging:

uv run python tsdb_collector.py --collect -vv

Maintain and Inspect TSDB Files

Generate demo data:

uv run python tsdb_collector.py --generate-demo-db 14

Create or refresh downsampled files for one raw day:

uv run python tsdb_collector.py --downsample data/data_2026-03-19.tsdb

Downsample several files and overwrite existing outputs:

uv run python tsdb_collector.py --downsample data/data_2026-03-19.tsdb data/data_2026-03-20.tsdb --force

tsdb_server.py

tsdb_server.py serves the dashboard UI and a JSON REST API over a directory of TSDB files. It does not require a database process; it reads the .tsdb files directly from --data-dir.

Start the dashboard on the default address:

uv run python tsdb_server.py --data-dir data

Bind to all interfaces to allow connection from other machines:

uv run python tsdb_server.py --host 0.0.0.0 --port 8080 --data-dir data

Use a fallback data directory for older or archived files (e.g. from InfluxDB extracted data).

uv run python tsdb_server.py --data-dir data --data-dir2 influx_data

tsdb_extract_from_influxdb.py

tsdb_extract_from_influxdb.py is a read-only migration tool for moving selected historical measurements from InfluxDB into pvview downsampled TSDB files.

It reads a TOML config with an [influxdb] section and a [mapping] section. Mapping keys are InfluxDB measurement/field pairs. Mapping values are target pvview series names. Empty values discard a source field. Optional target scaling can be appended with ,*factor or ,/factor.

Example:

[influxdb]
url = "http://192.168.1.14:8086"
database = "housedata"
username = ""
password = ""

[mapping]
"pvdata/inv_p" = "main/inv/power"
"pvdatacum/energy_ac_produced" = "main/inv/yieldtotal,*1000"
"pvdatacum/energy_pv1" = "main/dc1/yieldtotal,*3600000"
"pvdata/unused_field" = ""

See example_configs/tsdb_extract_from_influxdb.toml for a larger migration config.

Migration Commands

Preview the effective mapping and output files without writing:

uv run python tsdb_extract_from_influxdb.py \
  --config example_configs/tsdb_extract_from_influxdb.toml \
  --data-dir data \
  --day 2026-03-19 \
  --dry-run

Extract one UTC day:

uv run python tsdb_extract_from_influxdb.py \
  --config example_configs/tsdb_extract_from_influxdb.toml \
  --data-dir data \
  --day 2026-03-19

Extract a date range by start day and number of days:

uv run python tsdb_extract_from_influxdb.py \
  --config example_configs/tsdb_extract_from_influxdb.toml \
  --data-dir data \
  --day 2026-03-01 \
  --num-days 31

Overwrite existing output files:

uv run python tsdb_extract_from_influxdb.py \
  --config example_configs/tsdb_extract_from_influxdb.toml \
  --data-dir data \
  --day 2026-03-19 \
  --force

When --day is omitted, the extractor determines candidate days from the mapped InfluxDB sources. For controlled migrations, prefer passing --day and --num-days explicitly.

The extractor writes downsampled files directly:

dsda_YYYY-MM-DD.1s.tsdb
dsda_YYYY-MM-DD.5s.tsdb
dsda_YYYY-MM-DD.15s.tsdb
dsda_YYYY-MM-DD.1m.tsdb
dsda_YYYY-MM-DD.5m.tsdb
dsda_YYYY-MM-DD.15m.tsdb
dsda_YYYY-MM-DD.1h.tsdb

Development

Run the test suite:

make test

License

See LICENSE.