Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

specification: Clarify the IOMMU requirements #103

Merged

Conversation

sameo
Copy link
Collaborator

@sameo sameo commented Feb 20, 2024

Fixes #69

@sameo sameo requested a review from jyao1 February 20, 2024 07:47
@sameo sameo force-pushed the topic/specification-iommu-requirements branch 2 times, most recently from c283019 to 300dd0b Compare February 20, 2024 07:49
as long as this domain has RDSM-enforced exclusive access to an IOMMU instance
RPI.

Non CoVE-IO-compliant devices might be assigned to TSM controlled supervisor
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is definition of "Non CoVE-IO-compliant"? No IDE? No TDISP?
Why we need to mention that?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is definition of "Non CoVE-IO-compliant"? No IDE? No TDISP?

Sorry, that does not make a lot of sense. I meant TEE-IO compliant. I fixed the wording now.

Why we need to mention that?

It's part of Qualcomm's comments. They're asking if assigning non TEE-IO devices to a TVM is possible/supported.

Copy link
Collaborator

@jyao1 jyao1 Feb 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is absolutely possible.
In CoVE spec (not CoVE-IO), any non-TEE-IO device can be assigned to TVM, as long as TVM understands that the communication is not secure, and uses bounce buffer for the communication.

TSM does not need to be involved here at all.
That is why I am confused on why we need to mention in this spec, because I assume this spec is to define architecture for TEE-IO, but not non-TEE-IO.

We can easily say: non TEE-IO compliant device is out of scope of this spec.

RPI.

Non CoVE-IO-compliant devices might be assigned to TSM controlled supervisor
domains, and TSMs may be requested to bind such devices to TVMs under their
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we need TSM to handle non VE-IO-compliant device?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it appears on the TSM control supervisor domain, then TSM will own and manage both the MMIO and the DMA mappings for these devices. So VMM needs to go through the TSM if it wants to assign them to a TVM.

Copy link
Collaborator

@jyao1 jyao1 Feb 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there are two different type of TSM.

  1. CoVE-IO unaware TSM - the TSM only follows CoVE spec. It does not know CoVE-IO at all. VMM can assign non CoVE-IO-compliant device to TVM directly, without TSM help. TVM can use it with old bounce buffer mechanism.
  2. CoVE-IO aware TSM - the TSM follows CoVE-IO spec. I think TSM should reject non CoVE-IO-compliant device, if the TSM knows there is something not following the spec.

I assume TSM in this spec means CoVE-IO aware TSM. As such, once TSM knows the device does not follow TEE-IO, then it should be rejected. For example, if the device has no SPDM support at all.
But VMM can assign non-TEE-IO device to TVM directly, and let TVM use insecure bounce buffer.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I think the conclusion here is that a CoVE-IO compliant TSM implementation should reject any non-TEE-IO compliant device assignment. As you said, the VMM can bypass the CoVE-IO mechanism completely through memory bounce buffering for example, and this is out of this specification scope.

For now, I simply removed the non CoVE-IO compliant device part of this PR.

@sameo sameo force-pushed the topic/specification-iommu-requirements branch from 300dd0b to 183cedf Compare February 22, 2024 06:24
@jyao1
Copy link
Collaborator

jyao1 commented Feb 22, 2024

All in all, I recommend this PR only focus on IOMMU, which is irrelevant to non-TEE-IO device.

We can have a separate PR for discuss how to support non-TEE-IO device.

Non TEE-IO devices might be assigned to TSM controlled supervisor domains, and
TSMs may be requested to bind such devices to TVMs under their control. TVMs are
eventually responsible for accepting or rejecting the binding request, depending
on their own security policies and requirements. This process is described in
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont think we need define the "the process" for non-TEE-IO device.
A non-TEE-IO device may have no TDI at all. How it can follow the Interface binding?

@sameo sameo force-pushed the topic/specification-iommu-requirements branch from 183cedf to 0ae9aea Compare March 12, 2024 06:17
And extend the glossary with IOMMU and Smmtt related terms.

Fixes riscv-non-isa#67
Fixes riscv-non-isa#69
Fixes riscv-non-isa#70
Fixes riscv-non-isa#71

Signed-off-by: Samuel Ortiz <sameo@rivosinc.com>
@sameo sameo force-pushed the topic/specification-iommu-requirements branch from 0ae9aea to 9258d06 Compare March 12, 2024 06:29
Copy link

@ozkoyuncu ozkoyuncu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this new wording and explanations clarify majority of the relevant comments. One additional suggestion would be to indicate that multiple SDSMs can have control of distinct RPIs in a given SMMU that for example aggregates multiple external device interfaces ie different sets of devices being owned/controlled by different SDSMs. I believe the archtiecture does not preclude that however not sure whether the IOMMU programming interfaces are amenable to such ownership model

@sameo
Copy link
Collaborator Author

sameo commented Mar 12, 2024

I think this new wording and explanations clarify majority of the relevant comments. One additional suggestion would be to indicate that multiple SDSMs can have control of distinct RPIs in a given SMMU that for example aggregates multiple external device interfaces

By SMMU you mean IOMMU, right? And are you thinking about the case of SDSMs controlling multiple RPIs from the same IOMMU? For example SDSM A controls RPI 0 and 1 from IOMMU N and SDSM controls RPI 2 and 3 from IOMMU N as well?

I think the IO-MTT does not make any distinctions between RPIs and physical IOMMUs, and only considers that an RPI is the programming interface for a platform IOMMU instance. Wether different IOMMU instances are part of a the same physical IOMMU or not does not really matter as long as the RDSM enforces that each RPI is accessed by only one SDSM, through MTT.

@ozkoyuncu
Copy link

I think this new wording and explanations clarify majority of the relevant comments. One additional suggestion would be to indicate that multiple SDSMs can have control of distinct RPIs in a given SMMU that for example aggregates multiple external device interfaces

By SMMU you mean IOMMU, right? And are you thinking about the case of SDSMs controlling multiple RPIs from the same IOMMU? For example SDSM A controls RPI 0 and 1 from IOMMU N and SDSM controls RPI 2 and 3 from IOMMU N as well?

I think the IO-MTT does not make any distinctions between RPIs and physical IOMMUs, and only considers that an RPI is the programming interface for a platform IOMMU instance. Wether different IOMMU instances are part of a the same physical IOMMU or not does not really matter as long as the RDSM enforces that each RPI is accessed by only one SDSM, through MTT.

Yes, I meant to say IOMMU :) and yes SDSM n controlling RPI n etc. in other words multiple SDSMs controlling their respective RPIs by virtue of RDSM assigning them in the respective MTT. Doesn't IO-MTT control the ownership of granules to the IO streams that get translated by the IOMMU ie it is ensuring that the granules are in the aperture of the respective IO device streams for that SDSM, so the first part solves the control/ownership of the translation and the second part (io-mtt) solves the assignment of granules to the respective sdsm. wanted to make sure arch does not preclude any such configuration thats all. will leave it upto you if you want to further clarify...

@sameo
Copy link
Collaborator Author

sameo commented Mar 12, 2024

Yes, I meant to say IOMMU :)

I managed the ARM to RISC-V translation ;-)

and yes SDSM n controlling RPI n etc. in other words multiple SDSMs controlling their respective RPIs by virtue of RDSM assigning them in the respective MTT. Doesn't IO-MTT control the ownership of granules to the IO streams that get translated by the IOMMU ie it is ensuring that the granules are in the aperture of the respective IO device streams for that SDSM, so the first part solves the control/ownership of the translation and the second part (io-mtt) solves the assignment of granules to the respective sdsm. wanted to make sure arch does not preclude any such configuration thats all. will leave it upto you if you want to further clarify...

I see, thanks. I'll leave it like that for now, as I feel I would duplicate even further what's already in the smmtt and iommu specs.

@sameo sameo merged commit 5838cc3 into riscv-non-isa:main Mar 12, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Qualcomm feedback] 3.2.2. Platform Hardware Components - IO Translation Agent
3 participants