[cxx-interop] Support importing value types exported from Swift

[Xazax-hun]

Whenever a Swift type is exported to C++, we create a corresponding type on the
C++ side. However, we did not support importing value types back into Swift.
This PR addresses this problem.

Unfortunately, we cannot just set the C++ type as the ClangNode of
Swift's type. Types with ClangNodes are using the foreign
representation, but in this case we want to use the native type whenever
possible (e.g., for codegen). To work this around, this PR updates
ClangTypeConverter to make sure we have the correct Clang types when we
emit function calls to C++ APIs. It also updates the importer to not
attempt to import types that are annotated with ExternalSourceSymbolAttr
attributes as Swift types.

rdar://136964764

[Xazax-hun]

This should the first address half of #76024 . I plan to address the rest in a follow-up PR as doing the same for Optional seems to pose some unique challenges.

[Xazax-hun]

@swift-ci please smoke test

[DougGregor]

Is this state change on import going to be susceptible to ordering issues? For example, could we end up querying the ClangTypeConverter for Swift's String without having seen the C++ swift::String? It might be more robust if ClangTypeConverter::convert were able to call into the Clang importer to ask "Do you have a mapping for this type?".

[Xazax-hun]

I think this should not happen now. We populate this map when we import the declarations and query it during lowering. Currently, ClangTypeConverter::convert should handle missing mappings gracefully. But I can move the code around if you think it would be more straightforward that way.

[rjmccall]

This seems like something we want to generalize, so we should set up that generalization now instead of hard-coding namespaces and identifiers.

IIRC, there are two cases in Swift-to-C++ export: some types can be directly exported, and some types need a wrapper that indirects the storage. We should design this so that the importer imports both into Swift as their native Swift type. Directly-exported types can stay as their native type in SIL. Wrapper types should be different types in SIL, and we should just implicitly bridge back and forth with the native type, like we do with Objective-C. We can start with just the directly-exported types, of course.

I think we should use attributes for this. It should be straightforward to put an attribute on the exported type that declares that it's a direct export of a Swift type. The wrapper-export thing would be nice to generalize.

[Xazax-hun]

I think we should use attributes for this.

It turns out we already annotate exported types with the ExternalSourceSymbolAttr attribute. I changed this patch to rely on this attribute instead of hardcoding the types. Let me know what you think.

[Xazax-hun]

@swift-ci please smoke test

[Xazax-hun]

@swift-ci please smoke test

[rjmccall]

I think we should use attributes for this.

It turns out we already annotate exported types with the ExternalSourceSymbolAttr attribute. I changed this patch to rely on this attribute instead of hardcoding the types. Let me know what you think.

If there's an existing attribute that exactly matches what we want, great, but we should understand what this attribute is meant for before layering assumptions on top of it. In particular, which type gets annotated with this attribute for wrapped types?

[Xazax-hun]

In particular, which type gets annotated with this attribute for wrapped types?

We add this annotation to all of the types in the interop header when exporting Swift types to C++. For example:

class SWIFT_SYMBOL("s:SS") String final {
 ...
 private:
   swift::_impl::OpaqueStorage _storage;
}

I suspect that this is what you meant by a wrapper that indirects the storage.

Wrapper types should be different types in SIL

Could you elaborate on this? What type would you expect to see in the SIL in this case and what lowered type would you expect? With this patch, we would use the native Swift type in the SIL, and would lower it to the C++ wrapper type when we call a C++ foreign function. My understanding was that this should be OK since they are layout compatible, but let me know if this is wrong.

Directly-exported types can stay as their native type in SIL

By directly exported types, do you mean foreign types that have a clang node? Or @objc types? Or is there something else?

[rjmccall]

If we're going to import something from C++ and then use it in SIL interchangeably with that Swift type, it's important that it actually have the exact same ABI in C++ and Swift. So if we generate a type in C++ that indirects the storage — and yeah, what you're describing above is exactly what I'm thinking about, although I'm a little surprised we do it for String — then okay, we can still round-trip it into Swift, but we need to treat it as a different type at the SIL layer and then bridge between those types in SILGen.

If we generate a type in C++ that exactly matches the Swift type in its ABI (which is what I mean by "directly-exported"), we can round-trip it back into both Swift and SIL in the way you're suggesting. If we generate a type in C++ that just has a pointer or an opaque value buffer in it or something, we can round-trip it into Swift in the importer, but SIL needs to model it as a different type. So the immediate questions are (1) are there actually any types that get directly exported in this way? and (2) do we already have the information that we would need in order to distinguish these two cases?

[Xazax-hun]

Thanks for the explanation!

Indeed, I dug a bit deeper, and there are two kinds of exported classes. One with an opaque pointer, like Optional:

template<class T_0_0>
class SWIFT_SYMBOL("s:Sq") Optional final {
 ...
   swift::_impl::OpaqueStorage _storage;
};

And one with an inline buffer like String (I got lost in the generated header, so I was actually wrong about the representation for String, sorry for the confusion):

class SWIFT_SYMBOL("s:SS") String final {
...
  alignas(8) char _storage[16];
};

When we print these classes, we check whether the printed Swift type has a statically known layout by checking if it has FixedTypeInfo as its type information. If it does, we use the inline buffer, otherwise we use the OpaqueStorage.

Based on my initial testing, round-tripping String worked just fine, but round-tripping Optional did not. I updated this patch to only allow round-tripping value types when their representations are using the inline buffer that should be compatible with the layout of the Swift type.

I plan to resolve the OpaqueStorage case in a follow-up PR.

[Xazax-hun]

@swift-ci please smoke test

[rjmccall]

Do you need the template specialization check here? Optional gets ruled out by your storage check below, right?

[Xazax-hun]

True, the storage check rules out swift::Optional as of today. But not necessarily in the future, if/when swift::Optional<TrivialType> has a different representation.

[rjmccall]

I don't love this attempt to pattern-match exactly what we produce in export, especially because I know that export needs to be producing different patterns than this in some cases, like when a struct contains floats. But if you're going to do it, you should really do it properly: check that there's exactly one field, check that the field has the name you expect, and check that its type is exactly a constant-size array of char.

[Xazax-hun]

Indeed, I am also not the fan of pattern matching on the contents of the record. I looked into an alternative approach, but it required significantly more plumbing (need to heap allocate an IRABIDetailsProvider somewhere and plumb through a pointer a couple of calls). I figured since we plan to support round-tripping for all types, we might prefer the simpler check for a temporary solution. (Also, not matching on some type here is relatively benign, we end up not importing it.) Let me know if you prefer the other approach even for a temporary workaround. In the meantime I made the pattern matching stricter as you suggested.

[Xazax-hun]

@swift-ci please smoke test

[Xazax-hun]

After some discussions about this feature we decided to go with another approach. I will close this PR and open a new one hopefully soon.