Skip to content

feat: expose tedge configs to external clients - #4241

Merged
albinsuresh merged 17 commits into
thin-edge:mainfrom
albinsuresh:spec/expose-tedge-configs
Aug 14, 2026
Merged

albinsuresh merged 17 commits into
thin-edge:mainfrom
albinsuresh:spec/expose-tedge-configs

Conversation

@albinsuresh

@albinsuresh albinsuresh commented Jul 6, 2026 •

Copy link
Copy Markdown
Contributor

Proposed changes

Proposal to exposed selected tedge/mapper configs to external clients that don't have access to the config files via the same file system.

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
  • Documentation Update (if none of the other choices apply)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

#4235

Checklist

  • I have read the CONTRIBUTING doc
  • I have signed the CLA (in all commits with git commit -s. You can activate automatic signing by running just prepare-dev once)
  • I ran just format as mentioned in CODING_GUIDELINES
  • I used just check as mentioned in CODING_GUIDELINES
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Further comments

Comment on lines +43 to +49
We choose **opt-in**, for *safe by default*.
With a denylist, a newly added setting is exposed the moment it is introduced,
and a new secret leaks unless someone remembers to deny it — a silent failure.
With an allowlist, a new setting is invisible until a maintainer marks it exposable,
so the worst case is a missing value (loud, easy to fix), not a leaked secret.
Marking exposability where each setting is defined also keeps the decision
reviewable alongside the value.

@reubenmiller reubenmiller Jul 6, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure we use a special type for secrets, e.g. "Redacted" (or something like that) which could be used to disable automatic exposure via this interface. That would make a more convenient/automatic mapping of new values rather than having to explicitly opting in to each new config key

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The answer is also dependent on how many config keys we are exposing; whether we go for a "expose everything but secrets" or not.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking that we'll only expose a very limited number of configs, that we feel would be relevant to the user. For example, I don't see any reason to expose configs like device.key_path, device.cert_path, or even entire config trees like agent.*, apt.* etc that an external client would have no business with.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure we use a special type for secrets, e.g. "Redacted" (or something like that) which could be used to disable automatic exposure via this interface.

I'm fairly sure we don't have this at the moment as we store a reference to a secret (e.g. key path or password file) rather than the secret itself, so there isn't any sensitive information to redact. Though since those are just file paths, they aren't relevant for a public API as they're meaningless once they're not on the main device.

I guess with the hsm configurations we do have secrets that we store (key_pin, for example), but we don't actually have any mechanism in place to redact this info as far as I'm aware.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess with the hsm configurations we do have secrets that we store (key_pin, for example), but we don't actually have any mechanism in place to redact this info as far as I'm aware.

AFAIK secrets like device.key_pin are indeed loaded into config as regular string types, but are converted into a SecretString wrapper on the boundary with the HSM module, so they're not accidentally logger after that point.

The optimal solution would be to not have any secrets in the config (indeed putting HSM PIN there was a mistake I regret making), but since the milk is already spilt, and we can potentially have secret data in the config, it seems to me the most complete way to proceed would be to:

  1. add a mechanism to load the PIN from an external source (e.g. file so it doesn't appear in config directly)
  2. encourage users to use this new method by logging a warning when usage of the less secure secret-embedded-in-config setting is detected
  3. provide a way to disable the warning for users who don't want to change anything (probably by adding another setting)
  4. somehow cope with some users using the older setting which embeds secret in the config - either by introducing the notion of a secret field to tedge-config and having to handle it differently, or by not doing that and assuming that if the user silenced the warning, they must know what they're doing or don't care (fwiw I don't think adding this secret wrapper to the config would be that much work so we should still try it and only abandon the idea if we encounter unknown problems).

But indeed the proposal only covers step 4 and we don't have to do 1-3 for the proposed feature to work, but I just wanted to mention for completeness. I remember us talking about the secret management more generally, since there were also some other secret values that use another mechanism entirely, but if I remember correctly we haven't agreed on any general solution to that yet?

Comment thread design/decisions/0010-config-exposure-to-external-processes.md Outdated
Comment thread design/decisions/0010-config-exposure-to-external-processes.md Outdated
@reubenmiller reubenmiller added the skip-release-notes Don't include the ticket in the auto generated release notes label Jul 8, 2026

@Bravo555 Bravo555 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially I've been bummed out that because of secrets like key_pin we can't just dump the entire config as a payload to a single topic and not have to think about it again, but I see that out of 156 config options we only expose 43 of them, so I'm glad that it doesn't seem to be inhibiting the feature much. Indeed not exposing options which are considered to be, if not outright secrets, then at least considered somewhat private, is maybe better than exposing everything.

Comment thread design/decisions/0010-config-exposure-to-external-processes.md Outdated
rina23q
rina23q previously approved these changes Jul 13, 2026

@rina23q rina23q left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can move to implementation, so I would approve.

Comment thread design/decisions/0010-config-exposure-to-external-processes.md Outdated
Comment thread openspec/changes/expose-tedge-configs/design.md Outdated
Comment thread openspec/changes/expose-tedge-configs/design.md Outdated
Comment thread openspec/changes/expose-tedge-configs/design.md Outdated
Comment thread openspec/changes/expose-tedge-configs/design.md Outdated
absent
- **THEN** it SHALL take no action, so that a clear cannot echo into another clear

### Requirement: The agent serves exposed configuration over HTTP

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this specifically retained messages only (i.e. only at startup), or does it include updates when services publish those (e.g. if the mapper starts up shortly after the agent)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The config values by other services can be published at any time, as they can be started any time. The agent will then start building that service's config map in its own entity store. So, once all the config/+ retained messages for that external service is delivered to the agent, then it its config map is also complete and can be queried over HTTP. Unfortunately, there is no "readiness" cue here in terms of when all the configs of a given service is available, as that is completely dependent on the time taken for all those retained messages to be delivered and processed by the agent. So, the config endpoint becomes "available" from the moment the first config/+ message is received by the agent.

@albinsuresh
albinsuresh force-pushed the spec/expose-tedge-configs branch from c25e17a to 0a804be Compare July 21, 2026 07:09
@albinsuresh
albinsuresh requested a review from a team as a code owner July 21, 2026 07:09
@albinsuresh
albinsuresh temporarily deployed to Test Pull Request July 21, 2026 07:09 — with GitHub Actions Inactive
@github-actions

github-actions Bot commented Jul 21, 2026 •

Copy link
Copy Markdown
Contributor

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass % ⏱️ Duration
987 0 8 987 100 3h14m56.186864999s

@rina23q
rina23q dismissed their stale review July 23, 2026 09:52

I approved when only the spec document was written. So my approval is no longer appropriate.

@jarhodes314 jarhodes314 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the proposed structuring for the proposal is nice and helps frame the problem correctly. I've suggested some improvements to the config in #4729 (as a PR seemed to be the easiest way to share it along with context by way of what the resulting artifacts look like after following those config suggestions). The broad scope of those changes is to focus more on the motivation for why we want our specs to be a certain way and less on just telling the agent to write fewer words.

Comment thread openspec/changes/expose-tedge-configs/proposal.md Outdated
Comment on lines +3 to +6
The problem and the user-facing surface are in the [proposal](./proposal.md):
an external process can't read even a non-secret value like `device.id` without file access.

This document covers how that surface is built.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this usage of the term surface unusual and weird.

So even a non-secret, widely-needed value like `device.id` is out of reach for anything outside the
component that owns the config file.
The problem and the user-facing surface are in the [proposal](./proposal.md):
an external process can't read even a non-secret value like `device.id` without file access.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wording used in proposal.md is clearer:

Suggested change
an external process can't read even a non-secret value like `device.id` without file access.
A plugin, or a %%te%% component running in its own container, or a child device,
cannot read any configuration setting from the `tedge.toml` configuration file stored on the main device disk.

Comment thread openspec/config.yaml Outdated
Comment on lines +32 to +37
proposal:
- After "Why", add a "What you can do" section that introduces the feature from the
user's point of view with short usage examples — before any internal mechanism.
- Keep "What Changes" for the mechanism and code impact. Never lead the solution with an
implementation detail (a macro flag, a struct, a topic-construction rule); those belong
under "What Changes", after the reader has seen how the feature is used.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm astonished that you delegate even that to an agenr :-(

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The proposal.md generated by Openspec in this case was heavily solution oriented. So, I was just trying to tune it to our liking so that we don't have to rely on yet another document to understand the "user impact" of the proposed solution before getting into the details of the solution.

Each entity that owns `tedge_config` settings (e.g. `tedge-agent`, or a cloud mapper such as
`tedge-mapper-c8y`) publishes any setting explicitly marked exposable as a retained `config` MQTT
message under its own service topic (see the [MQTT API reference](../../references/mqtt-api.md#configuration)).
The agent collects these as an ordinary MQTT subscriber and serves them as a read-only view over HTTP.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tone of this text is not appropriate for a user manual. This is more akin to a reference or a design doc.

Suggested change
The agent collects these as an ordinary MQTT subscriber and serves them as a read-only view over HTTP.
The agent and the mappers expose part of their configuration over MQTT as a retained `config` MQTT
messages.
The agent also gather all these configuration settings and serves them as a read-only view over HTTP.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rewritten in 911c34d

Comment on lines +501 to +503
A configuration key that is not marked exposable and a key that does not exist at all are
indistinguishable here: both return `404 Not Found`, so the response reveals nothing about which
non-exposed settings exist.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Speaking about keys that are not marked exposable might confuse the users asking themselves how to expose a key.

Suggested change
A configuration key that is not marked exposable and a key that does not exist at all are
indistinguishable here: both return `404 Not Found`, so the response reveals nothing about which
non-exposed settings exist.
The agent and the mappers don't expose all their configuration settings (some are secret values, others are meaningless when used from another box).
A configuration key that is not exposed or does not exist at all are
indistinguishable here: both return `404 Not Found`, so the response reveals nothing about which
non-exposed settings exist.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-written in 911c34d

Comment thread docs/src/references/mqtt-api.md Outdated
## Configuration

A service that owns a `tedge_config` setting (`tedge-agent` for core/device settings, each cloud
mapper for its own cloud's settings) publishes any setting explicitly marked exposable as a retained

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explicitly marked exposable is confusing for the users as they have no mean to mark a setting as exposable. This is a internal concept.

Suggested change
mapper for its own cloud's settings) publishes any setting explicitly marked exposable as a retained
mapper for its own cloud's settings) publishes a non-secret subset of its setting as a retained

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-written in 911c34d

Comment thread docs/src/references/mqtt-api.md Outdated
Comment on lines +588 to +589
An empty payload means the value is unset, or was removed from configuration since the last time the
owning service started. A setting is only published this way if it was explicitly marked exposable

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are setting updates dynamically published? When the file is updated? When tedge config set is used?

What about default values - not explicitly set in tedge.toml? What is the effect of tedge config unset?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no dynamic update. The published config represents the config the service was started with. Any subsequent changes take effect (and published) only on the next restart. Default values are published and unset values are omitted. Since we switched to publishing a single config JSON document message instead of a message-per-key, that published document only contains the set values. I have updated the user facing docs to explicitly state these as well in 911c34d.

Comment thread crates/common/tedge_config/src/tedge_toml/tedge_config/config_exposure.rs Outdated
Comment thread crates/common/tedge_config/src/tedge_toml/tedge_config/config_exposure.rs Outdated

@jarhodes314 jarhodes314 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't finished reviewing, but have read through about half the files so far. Will resume tomorrow, but here are my thoughts for now.

Comment thread crates/common/tedge_config/src/tedge_toml/tedge_config/config_exposure.rs Outdated
Comment thread crates/common/tedge_config/src/tedge_toml/tedge_config/config_exposure.rs Outdated
Comment on lines +831 to +842
let exposable_paths = paths
.iter()
.filter(|path| {
path.back()
.and_then(|fog| fog.field())
.is_some_and(|field| field.exposable())
})
.collect::<Vec<_>>();
let enum_variants = exposable_paths.iter().map(|path| enum_variant(path));
let arms =
exposable_paths
.iter()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't care too much about fixing this as these macro changes will need to be reworked as part of the facet work regardless, but it feels like a smell the fact we keep iterating over the same things.

Comment thread crates/core/tedge_agent/src/entity_manager/tests.rs Outdated
Comment thread crates/core/tedge_agent/src/entity_manager/tests.rs Outdated
Comment thread crates/core/tedge_agent/src/entity_manager/tests.rs Outdated
Comment thread crates/core/tedge_agent/src/entity_manager/tests.rs Outdated
mqtt_output
.assert_received([
MqttMessage::from(("te/device/child0///config", "")).with_retain(),
MqttMessage::from(("te/device/child0//", "")).with_retain(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test feels quite noisy. A lot of what's going on here: skipping mqtt messages and asserting the deregistration message, are not actually related to what the test is trying to demonstrate. Is there a different assertion mechanism we can use (e.g. assert_received_unordered) that will allow us to trim down the test so it focuses more specifically on what we're trying to test here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved by a5fd790

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think asserting the second message is still unnecessary

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved and squashed into 4e53564

Comment thread crates/core/tedge_agent/src/http_server/entity_store.rs Outdated

@jarhodes314 jarhodes314 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more thoughts from the rest of the review.

Overall, the feature is in really good shape, just a few relatively minor things about the details of the code.

Comment thread crates/core/tedge_api/src/entity_store.rs
Comment thread crates/core/tedge_api/src/entity_store.rs Outdated
Comment thread crates/core/tedge_api/src/entity_store.rs Outdated
Comment thread crates/core/tedge_api/src/entity_store.rs Outdated
Comment thread crates/extensions/tedge_config_ext/src/actor.rs Outdated
Comment thread crates/extensions/tedge_config_ext/src/actor.rs Outdated
Comment thread crates/extensions/tedge_config_ext/src/actor.rs Outdated
Comment thread crates/extensions/tedge_config_ext/src/tests.rs Outdated
Comment thread crates/extensions/tedge_config_ext/src/tests.rs Outdated
Comment thread docs/src/references/mqtt-api.md Outdated
Add #[tedge_config(exposable)] and a generated ReadableKey::is_exposable()
accessor, following the existing help()/doc-comment pattern. This is the
opt-in marker used by the config-exposure feature to curate which settings
may be published to external clients.

Signed-off-by: Albin Suresh <albin.suresh@cumulocity.com>
Mark the initial exposable settings from design.md's allowlist table with
as pure helpers over TEdgeConfigReader, built on the existing
readable_keys()/read_string() machinery. Includes a forward-guarding test
that fails CI if a known-secret key is ever marked exposable.

Signed-off-by: Albin Suresh <albin.suresh@cumulocity.com>
Add Channel::Config{key}/ChannelFilter::Config, mirroring EntityTwinData,
and a config: BTreeMap<String, String> map on EntityMetadata, parallel to
(not merged with) twin_data. Add entity-store ingestion primitives
(ingest_config_value/clear_config_value/get_config/get_config_value) with
the same key validation as twin fragments, so config values arriving over
MQTT can be recorded without opening an external write API.

Signed-off-by: Albin Suresh <albin.suresh@cumulocity.com>
New extensions crate publishing a component's own exposable configuration
as retained per-key MQTT messages under its own service topic, one message
per value, with an empty payload for unset keys. The actor subscribes to
its own config/+ wildcard and reconciles every message it sees against its
expected state, correcting external tampering (including an empty payload
that would wipe an owned value) and clearing stale keys left behind by a
renamed, removed, or demoted setting.

Shared between tedge-agent (core config) and the cloud mappers (their own
cloud's config) instead of duplicating topic construction and reconciliation
three times over.

Signed-off-by: Albin Suresh <albin.suresh@cumulocity.com>
Compute exposed_core_config() at startup, patching mqtt.topic_root and
mqtt.device_topic_id with their CLI-effective values since the pure config
helper only sees the file value. Spawn the shared ConfigPublisherBuilder
for the agent's own service topic alongside the existing twin-manager actor.

Signed-off-by: Albin Suresh <albin.suresh@cumulocity.com>
Extend start_basic_actors, the shared helper that already spawns the health
monitor for every mapper, to also spawn the shared ConfigPublisherBuilder.
The c8y, az, and aws mappers pass their own profile-aware
exposed_cloud_config(); collectd and custom mappers, which own no cloud
config in tedge_config, pass an empty set.

Signed-off-by: Albin Suresh <albin.suresh@cumulocity.com>
Subscribe the entity store to the config channel filter alongside twin
data, ingest config/<key> messages (empty payload = removal), and clear a
deregistered entity's retained config topics the same way its twin data is
already cleared. This lets the agent collect every service's exposed
configuration, including its own, purely by subscribing.

Signed-off-by: Albin Suresh <albin.suresh@cumulocity.com>
Add GET /te/v1/entities/<service>/config and .../config/<key>, modeled on
the existing twin-data routes: a JSON object for the whole-service view, a
plain-text body for a single value, and writes (PUT/PATCH/DELETE) rejected
with 405 via the existing unmatched-channel fallback. A non-exposed key and
a non-existent key return the same 404, since the agent has no source of
truth for a key beyond what has arrived over MQTT.

Signed-off-by: Albin Suresh <albin.suresh@cumulocity.com>
Add a Robot Framework suite verifying the agent's own retained config
topics and HTTP routes, including that a secret setting (device.key_pin)
never appears on either surface, and a second suite covering a bootstrapped
c8y mapper's own retained config topics and HTTP routes, verifying the
cloud/profile qualifier is stripped and other clouds' settings are absent.

Signed-off-by: Albin Suresh <albin.suresh@cumulocity.com>
Add the config channel to the MQTT topic reference (category table,
telemetry data table, and a new Configuration section with a worked
example), and the two GET /te/v1/entities/<service>/config[/<key>] routes
to the entity HTTP API reference, cross-linked with each other.

Signed-off-by: Albin Suresh <albin.suresh@cumulocity.com>
Replace per-key retained MQTT topics with a single retained JSON
document per service on `.../config`. This simplifies the
self-correcting publisher: a stale key just drops out of the
republished document instead of needing an explicit per-key clear.
The HTTP API is unaffected — both the whole-service and single-key
routes still work, the latter now looked up in the parsed document.

tasks.md is unchecked for the sections whose implementation no longer
matches this design (channel/entity-store/publisher, MQTT ingestion,
e2e tests, MQTT docs).

Signed-off-by: Albin Suresh <albin.suresh@cumulocity.com>
Carries the revised design into code: `Channel::Config` drops its per-key
segment, the entity store gains a whole-document `set_config` replacing the
old per-key ingest/clear, and the shared publisher in tedge_config_ext
publishes and reconciles one retained JSON object per service instead of
one message per key. Stale-key clearing is no longer a separate code path,
since republishing the whole document already drops keys that fell out of
the exposed set.

The agent/mapper spawn call sites and the HTTP query side needed no changes,
since neither depended on the per-key MQTT shape.

Signed-off-by: Albin Suresh <albin.suresh@cumulocity.com>
All 33 tasks and artifacts are complete. Syncs the config-exposure
delta spec into openspec/specs and moves the change under
openspec/changes/archive/2026-08-14-expose-tedge-configs/.

Signed-off-by: Albin Suresh <albin.suresh@cumulocity.com>
@albinsuresh
albinsuresh force-pushed the spec/expose-tedge-configs branch from aae9032 to 355f5c5 Compare August 14, 2026 14:18
@albinsuresh
albinsuresh temporarily deployed to Test Pull Request August 14, 2026 14:18 — with GitHub Actions Inactive
@albinsuresh
albinsuresh added this pull request to the merge queue Aug 14, 2026
Merged via the queue into thin-edge:main with commit dc40b71 Aug 14, 2026
35 checks passed
@albinsuresh
albinsuresh deleted the spec/expose-tedge-configs branch September 27, 2026 21:06

This branch was previously deployed

2 inactive deployments
Test Auto — 355f5c55 Deployed Aug 14, 2026 by albinsuresh via Test x86_64 #8443
Test Pull Request — 355f5c55 Deployed Aug 14, 2026 by albinsuresh via Approve #8443
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

theme:configuration Theme: Configuration management

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants