-
Notifications
You must be signed in to change notification settings - Fork 311
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable for Rust - Seq.map and Seq.map2 #2646
Conversation
@alexswan10k Thanks, I may have to absorb this into another PR for consistency, cause I opted for re-implementing the |
No worries @ncave we can scrap it if you have a better implementation or the likes. Getting iterators etc right is a core feature anyway so it is worth taking the time to iron it out. Let me know if there is something else on the periphery I can chip away at! |
@alexswan10k Core concepts like equality and comparison (and when to apply them to objects) still need some deep thinking, but they do need to be dealt with in order to enable all I guess we can start small and gradually work up the corner cases, we already have some checks in place.
Perhaps we can add a restrict that it also has to be CompilerGenerated? |
@alexswan10k @ncave What's Fable/src/Fable.Transforms/FableTransforms.fs Lines 88 to 103 in c223454
|
For Rust closures, we need to collect all captured (non-local to the closure) names so they can be cloned before the closure, and we need to exclude the closure arguments from that list, that's what Since we don't have a general way of getting all captured names in I guess compiler-generated |
Thanks @ncave! Interesting, ideally we should store all variable declarations in the context so when transforming a lambda we can check if an ident reference is a free var or not, although your current method should also work. How is it with references to "root" scope, do they need to be cloned as well? let foo = 5
let myMethod() =
let myInnerLambda x =
x + foo
myInnerLambda 10 Maybe here you need to add also let myMethod() =
let myInnerLambda x =
let myInnerLambda2 y =
x + y
myInnerLambda 3
myInnerLambda 10 Fable/src/Fable.Transforms/Rust/Fable2Rust.fs Lines 3286 to 3304 in c223454
|
Are you suggesting storing all arguments and local declarations in context and checking each of them if captured in the closure, using |
@alexswan10k |
Thanks @ncave. Without getting too into the weeds, the reason we need to clone everything over a boundary like this is to beat the borrow checker. All reference types are wrapped in a RefCounter https://doc.rust-lang.org/book/ch15-04-rc.html so a clone effectively gives us a new smart pointer to the same object on the heap. "Disposing" of one of these reduces the shared counter, unless it is the last one.. then it clears the memory. Poor mans GC :) Because closures cannot reason if the captured thing will live longer than the thing outside, we clone to break the dependency (captured thing is otherwise borrowed). We only need to clone Rc wrapped references in practice, as rust mandates that there can only be 1 owner who is responsible for cleaning up the memory (although there may be some nuances I have missed). |
Hi @ncave It seems that the low hanging fruits are mostly gone, I cannot seem to move anything forward any more :( Had a look at string.chunkBySize the other week and got hopelessly stuck.. then I found #1296. I guess this is blocked? Anyway, I was experimenting to see if I could innocuously turn on the following seq test, which lead me down a rabbit hole. I was wondering if you are aware of this and perhaps have any ideas..
Turns out there is a problem. The interface for this of course expects an Add trait. I figured I could get the transpiler to just output a definition for each record. Maybe something like this
Nope, turns out it is expecting a Oh no! So I did some digging, which led me to this. Turns out you cannot implement a trait on a thing if you are not in that crate because the compiler cannot guarantee it won't conflict with another definition elsewhere. So next I tried this
Problem here is this is not picked up either, because a Nice work though otherwise, its really shaping up. Excited to try a real project out when vnext is out. |
Hi Alex, I wouldn't worry about it, there are plenty of low hanging fruits left, from implementing any .NET BCL you want, to making the commented F# collection methods work, to adding support for .NET types like But that's all bike shedding at this point, until the two big elephants in the room are addressed, namely having proper imports without namespace clashing, and proper native Rust bindings (slightly lower priority only because simple bindings can already be done with some Emit and inline functions, but still badly needed at least as a concept). As far as string handling goes, I made a few deliberate choices for performance and interoperability with native Rust. Since Rust strings are For now I opted for sticking with native For sequences, yes, there are still some missing default conversions (from string to seq of char, from seq of list/array to seq of seq, etc.). For now all those can be worked around with explicit conversions (e.g. from string to array of char to seq of char), so those are on the back-burner too. For generic type constraints, the deliberate choice was to stick with F# type constraints instead of attributes, and convert them to traits, ideally ones that can be used with Thank you for your participation @alexswan10k, it's nice to have somebody to bounce ideas off. |
I have been thinking about the binding problem on and off over the weeks and have a few ideas. I am just aware this is quite a big (potentially disruptive) undertaking, but happy to maybe have a go here and perhaps lay some foundations. I guess this starts with a new Emit like attribute and probably a Fable AST representation of it.. one that has a lot more control of inbound and outbound stuff with conversions. Have you decided on what path to go yet? Either
I think UTF-8 is entirely sensible given the constraints.
Makes a lot of sense. I guess the only bit I am unsure of how to proceed with is my example above. It seems that there will be cases where you are comparing
Appreciated! Same to you, super excited with how this is progressing. Just sorry I haven't been much use recently. Will keep an eye out anyway hopefully there are some things here and there I can help chip away at. Rome wasn't built in a day. |
That's probably taking it a bit too far, unless you mean it in the context of some For ad-hoc bindings, perhaps a more minimal approach would be to find a flexible enough representation to guide the transpiling to Rust native. F#/.NET has enough language features to be able to do that, IMO. The bindings can use a new attribute, or stick with I don't have all the details, I just know we need it, and it needs to make sense, in order to build a good community eco-system. See also #2779.
Yes, or perhaps we can do what seems to be usually done in Rust for generic programming and references, and implement traits for both Rc and non-Rc wrapped generic types. Not sure yet if that would be sufficient to make it work. |
Hmm I think I see where you are going with this... so off the top of my head you could define your input transforms IN f# rather than some proprietary string template.
This is really interesting because it also opens up a second transpile "mode" where you could perhaps work with rust low level abstractions 1-1 in F# (with the handicap of not having the borrow checker to guide you). This would give you far more control though. |
@alexswan10k Some code, for example, doesn't need precise dual mode, as we already support low-fidelity "native" dual mode which avoids Rc-wrapping and doesn't pass by reference (it's all or nothing, so not able to model more complicated APIs): module Performance =
type Duration =
abstract as_millis: unit -> uint64 // actually u128, may need custom type here
abstract as_secs_f64: unit -> float
type Instant =
abstract duration_since: Instant -> Duration
abstract elapsed: unit -> Duration
// not working, just an example
let now(): Instant = importMember "std::time::Instant"
// not working, just an example
[<Import("std::time::Instant::now")>]
let now2(): Instant = nativeOnly
// this already works, by intentionally losing the type and inlining
[<Emit("std::time::Instant::now")>]
let inline internal now(): obj = nativeOnly
// sample usage
let measureTime (f: unit -> 'T): 'T * float =
let t0 = Performance.now()
let res = f ()
let t1 = Performance.now()
// let elapsed = t1.duration_since(t0).as_secs_f64()
let elapsed = (t0 :?> Performance.Instant).elapsed().as_secs_f64()
res, elapsed * 1000.0 But we'll need the full-fidelity "dual" mode to describe more complicated APIs. |
Interesting. Ok @ncave here is another idea I wanted to throw at the wall. Maybe we can do this with types rather than a built in 'dual' mode.. although the effect would be largely the same. What if, we have a special low level type that can be used to work with real rust types.
What if, when the compiler sees a Raw wrapped type, it completely omits any operations by the ref count "manager" (so no implicit wrapping or cloning). It might even be worth going further and having 1-1 types for anything that is raw rust, where conversion functions could be supplied to transition between "raw" and "managed". An example:
The point here is the two conversion functions understand how to turn "dangerous" rust native types (things that can explode with the borrow checker but are invisible to the F# compiler) into implicit managed types that can be used without risk in F#. It may well be that there is also not a 1-1 mapping, and some degree of recursion required to unwrap etc. It could also perhaps take into account the state of global parallelism (are we using Rc or Arc etc). By using types to transition between "managed" and "unmanaged"... (we need new terms or something!) you basically have the same effect of twin mode compilation but without having to define exact boundaries to where this may or may not happen, which I imagine is nigh on impossible. This way you have far more control. Some random types off the top of my head
Maybe you don't need the extra raw wrapping, just trying to cover the scenario where anything can be ignored, simply by wrapping it in a Raw, and this allows this effect by composition. This allows you to model complex graphs of ignored structs etc. Some more Raw api experiments
experiments where any native Rust raw type is just added to the exclusion list and is unmanaged. Less noisy, caveat is all core types have to be known about by the compiler. Maybe we could enforce this with an attribute though.
Here is a fun problem I don't know how to solve though. How do we know "ABC" here should be unwrapped?
Maybe context (I am RAW) of the create function can be threaded through recursively? Not sure if this might create other problems. Alternatively, maybe string/number constants to raw rust are just special cases that need their own explicit transform hard-coded. |
It sounds like we might even be half way there then. Sorry my above post is a muddled consciousness dump! Let me go through your example but expand with this idea.
For completion sake let's quickly sketch out some built in types
The more I think about this, I believe the answer is for all raw rust types to have their own F# representation rather than using dotnet types to approximate them. This means that there is no way they can be accidentally confused, and a user is forced to convert between them somehow before use. This also safeguards any internal representation changes such as using a utf16 vs utf8 string, Rc/Arc etc. Using raw types is then generally not recommended unless doing interop because you are basically taking off the guard rails (a bit like unsafe), but it gives you full control. |
@alexswan10k I don't know, custom types that have to be converted when used adds a lot of code on the usage side, instead of letting the transpiler do its job. I understand it adds safety, but I'm still leaning towards exploring first a fine-grained type conversion in the import declaration, to see how far we can push it. In any case, an interesting option to evaluate, thanks! |
@alexswan10k Sorry to resurrect this one year old thread. Just picking your brain, as we still have the issue of not being able to implement out-of-crate traits for LRC-wrapped objects (see your comments above). Currently we can implement out-of-crate traits only on non-wrapped (value-type) structs, but not on LRC-wrapped ones. Any ideas how to overcome that? The usual work-around is to make either the trait or the type local to the crate. I don't think we can't use local smart ptr wrappers for each crate, as they would be incompatible. What about local subtraits that inherit from the out-of-crate ones (for, say, implementing
|
Hey @ncave , it’s not the most straight forward but do extension traits help? The wrapper is a good idea too. Any name works for me, although it might want to signify its local. I think GATs might potentially also be able to simplify our Lrc story but not sure, needs some thought. |
@alexswan10k Thanks for the suggestions, I'll take a look. |
I think you are right. Wrappers are probably the only option. Does implementing deref help at least on the consumption side? The good news is there is no runtime overhead. Maybe all that messy init code can be rolled up into the constructor. saying that, this gat eli5 example is interesting https://www.reddit.com/r/rust/comments/ynvm8a/could_someone_explain_the_gats_like_i_was_5/ so maybe it is possible to not need to reference a concrete lrc in a trait at all, thus circumventing the problem. |
@alexswan10k Yes, wrapper will implement Deref and other ops. As mentioned, CoerceUnsized is unstable, so we can't hide the construction neatly (the way Box/Rc/Arc construction hides it). It's a bit verbose but is needed for the implicit casting to dyn trait objects, so it's fine. I'll take a look at the GAT too, thanks! |
On the note of wrappers, what if every reference type came with its own locally defined wrapper? Anything needing it’s heap pointer would use this wrapper name instead of the lrc everywhere. This might get around some of the crate isolation problems of just having 1 wrapper for all cases. |
|
@alexswan10k On a side note, a nice read on some of the pains we're struggling with. |
Good read |
A little more progress on the seq module.
Not really sure what the best approach is for the fixed name ident problem I ran into here. It seems that "matchValue" is something generated by Fable. In some contexts (inner match) this was messing up the closure clone-on-capture mechanic as it did not exist. Happy to revisit if there are better ideas