Skip to content

Improvements to the 64 (62) client/PD/channel limit #526

Open

Description

@midnightveil

At the moment, the Microkit has two limitations which we will run into at some point:

  • There is a maximum number of PDs

    // Note that these values are used in the monitor so should also be changed there
    // if any of these were to change.
    pub const MAX_PDS: usize = 63;
    pub const MAX_VMS: usize = 63;

    This is limited by the fact that the monitor is a default fault endpoint for all PDs.
    It is also limited by the size of the PD names array in the monitor (this can be easily solved).

  • There is a maximum number of channels to each PD
    This is "limited" by the number of bits in an seL4_Word: for asynchronous notifications, reporting the status of 64 channels requires more 64 bits of information: in fact we limit the number of channels to 62, using the badged mechanism to also report "fault" and "PPC" identifiers.

However, whilst this is all true, these limits are not absolute:

  1. A server can easily support more than 64 different PPCs/Faults (endpoints): in seL4, endpoints form a queue.
    Whilst the top 2 bits must be reserved to distinguish fault/PPC/signal, both faults and endpoints then have the remaining 62 bits left: which gives us 2^62 different unique client identifiers, which should be more than enough.
    Part of this would also be reworking so that the "channel" is not a unique namespace between notifications/PPCs.

  2. Currently, client:server notifications have 1:1 channel IDs. That is, one client notification will set one bit in the badge of the server's notification receive, and vice versa.
    This is however, unnecessary: like in real hardware IRQs, seL4 allows multiple notifications to have the same badge, i.e. have the same ID to the server, and be distinguished out of band.
    In the simplest case, this can be done using shared memory by the user, or as is the case for many of our virtualisers in the sDDF, we don't actually need to distinguish between different clients. Alternatively, one can partition clients into "priority" classes or other groupings.

    The server to notify a client will signal a capability: we can easily have more than 64 of one particular capability, we just need to layout the CSpace such that there is one cap.

    For example: consider our network virtualiser. On a notification from a client, we process all client queues. We don't care which client did so: as this gives us the most benefit from batching of packets. To respond to a particular client, we perform a singular notification on the appropriate notification capability, which has no limit.

These two changes would allow for systems with larger number of PDs to be created.

Activity

  1. Indanz commented on Jun 15, 2026

    @Indanz

    (I wouldn't reserve one bit for faults from the global badge, I would reserve it as a sub-bit for PPC badges instead. Only notifications are limited because their values are ORed together, reserving one bit to distinguish between notifications and something else is enough.)

  2. midnightveil commented on Jul 6, 2026

    @midnightveil
    CollaboratorAuthor

    FWIW, summarising from a discussion with Zoltan: he had another place where allowing more than 1:1 channel IDs would be useful (doing partial emulation of a device IRQ in a certain situation where the device was buggy). Whether that was actually a good idea I'm slightly uncertain about, but I also see no reason to prevent.

    However, he also commented that making notified() instead pass just the badge containing the notification bits, and removing the loop over each channel in the handler loop would simplify the verification; similarly, having "one thing to do" for each return from the ReplyRecv means we could remove the deferred API in favour of returning from functions, this would also simplify verification in that sense.

    We could also add this without breaking backwards compat with a variety of wonderful C hacks (e.g. #define notified notified_one or such to not break users.

  3. added
    needs-discussionThis work needs group consensus and discussion before merging, or possibly an RFC.
    on Jul 23, 2026
  4. midnightveil commented on Aug 6, 2026

    @midnightveil
    CollaboratorAuthor

    For those who might have subscribed to this issue: please see RFC-26.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-discussionThis work needs group consensus and discussion before merging, or possibly an RFC.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions