-
Notifications
You must be signed in to change notification settings - Fork 13k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
std: Extract librustrt out of libstd #14638
Conversation
I'm curious what others think about the organization of librustrt. With the facade, we're at liberty to move anything pretty much anywhere, but it seems odd to me to have mutexes in librustrt, but not threads (for example). I suppose the question of "what is the runtime?" is a little hazy nowadays that there isn't really a clear line of what it is and what it isn't. |
You say "TLS" a few times, and context implies you're referring to "task local data" (rather than "thread local storage"). Maybe "TLD" would be clearer? The performance lost of stdin/out is a little sad, I assume the loss is entirely due to the TLD look-ups? (I imagine it will only get worse if more things are placed into TLD by a program.) (Also travis failed with a few todos.) |
Good point, I switched from saying TLS to TLD, and I'll try to get in the habit of doing so as well (it does get confusing having two...). Yes, the slowdown is from the TLD replacements. We have good ideas of how to make these faster, but it will take some time to implement. I would expect the cost of a TLD lookup or replacement to be on the order of accessing a RefCell. Thanks for pointing out those TODOs, I fixed up the documentation and pushed an update. |
Note that I'm going to leave this un-rebased for now until #14538 lands because none of the major components of it will change. I would still appreciate review, however. |
I feel very good about this. Do you have a crate-dag visualization after the change? r=me assuming no major objections appear. |
I do indeed! http://people.mozilla.org/~acrichton/out2.pdf |
The rtio interface is a thin low-level interface over the I/O subsystems, and the `Path` type is a little too high-level for this interface.
As part of the libstd facade efforts, this commit extracts the runtime interface out of the standard library into a standalone crate, librustrt. This crate will provide the following services: * Definition of the rtio interface * Definition of the Runtime interface * Implementation of the Task structure * Implementation of task-local-data * Implementation of task failure via unwinding via libunwind * Implementation of runtime initialization and shutdown * Implementation of thread-local-storage for the local rust Task Notably, this crate avoids the following services: * Thread creation and destruction. The crate does not require the knowledge of an OS threading system, and as a result it seemed best to leave out the `rt::thread` module from librustrt. The librustrt module does depend on mutexes, however. * Implementation of backtraces. There is no inherent requirement for the runtime to be able to generate backtraces. As will be discussed later, this functionality continues to live in libstd rather than librustrt. As usual, a number of architectural changes were required to make this crate possible. Users of "stable" functionality will not be impacted by this change, but users of the `std::rt` module will likely note the changes. A list of architectural changes made is: * The stdout/stderr handles no longer live directly inside of the `Task` structure. This is a consequence of librustrt not knowing about `std::io`. These two handles are now stored inside of task-local-data. The handles were originally stored inside of the `Task` for perf reasons, and TLD is not currently as fast as it could be. For comparison, 100k prints goes from 59ms to 68ms (a 15% slowdown). This appeared to me to be an acceptable perf loss for the successful extraction of a librustrt crate. * The `rtio` module was forced to duplicate more functionality of `std::io`. As the module no longer depends on `std::io`, `rtio` now defines structures such as socket addresses, addrinfo fiddly bits, etc. The primary change made was that `rtio` now defines its own `IoError` type. This type is distinct from `std::io::IoError` in that it does not have an enum for what error occurred, but rather a platform-specific error code. The native and green libraries will be updated in later commits for this change, and the bulk of this effort was put behind updating the two libraries for this change (with `rtio`). * Printing a message on task failure (along with the backtrace) continues to live in libstd, not in librustrt. This is a consequence of the above decision to move the stdout/stderr handles to TLD rather than inside the `Task` itself. The unwinding API now supports registration of global callback functions which will be invoked when a task fails, allowing for libstd to register a function to print a message and a backtrace. The API for registering a callback is experimental and unsafe, as the ramifications of running code on unwinding is pretty hairy. * The `std::unstable::mutex` module has moved to `std::rt::mutex`. * The `std::unstable::sync` module has been moved to `std::rt::exclusive` and the type has been rewritten to not internally have an Arc and to have an RAII guard structure when locking. Old code should stop using `Exclusive` in favor of the primitives in `libsync`, but if necessary, old code should port to `Arc<Exclusive<T>>`. * The local heap has been stripped down to have fewer debugging options. None of these were tested, and none of these have been used in a very long time. [breaking-change]
As part of the libstd facade efforts, this commit extracts the runtime interface out of the standard library into a standalone crate, librustrt. This crate will provide the following services: * Definition of the rtio interface * Definition of the Runtime interface * Implementation of the Task structure * Implementation of task-local-data * Implementation of task failure via unwinding via libunwind * Implementation of runtime initialization and shutdown * Implementation of thread-local-storage for the local rust Task Notably, this crate avoids the following services: * Thread creation and destruction. The crate does not require the knowledge of an OS threading system, and as a result it seemed best to leave out the `rt::thread` module from librustrt. The librustrt module does depend on mutexes, however. * Implementation of backtraces. There is no inherent requirement for the runtime to be able to generate backtraces. As will be discussed later, this functionality continues to live in libstd rather than librustrt. As usual, a number of architectural changes were required to make this crate possible. Users of "stable" functionality will not be impacted by this change, but users of the `std::rt` module will likely note the changes. A list of architectural changes made is: * The stdout/stderr handles no longer live directly inside of the `Task` structure. This is a consequence of librustrt not knowing about `std::io`. These two handles are now stored inside of task-local-data. The handles were originally stored inside of the `Task` for perf reasons, and TLD is not currently as fast as it could be. For comparison, 100k prints goes from 59ms to 68ms (a 15% slowdown). This appeared to me to be an acceptable perf loss for the successful extraction of a librustrt crate. * The `rtio` module was forced to duplicate more functionality of `std::io`. As the module no longer depends on `std::io`, `rtio` now defines structures such as socket addresses, addrinfo fiddly bits, etc. The primary change made was that `rtio` now defines its own `IoError` type. This type is distinct from `std::io::IoError` in that it does not have an enum for what error occurred, but rather a platform-specific error code. The native and green libraries will be updated in later commits for this change, and the bulk of this effort was put behind updating the two libraries for this change (with `rtio`). * Printing a message on task failure (along with the backtrace) continues to live in libstd, not in librustrt. This is a consequence of the above decision to move the stdout/stderr handles to TLD rather than inside the `Task` itself. The unwinding API now supports registration of global callback functions which will be invoked when a task fails, allowing for libstd to register a function to print a message and a backtrace. The API for registering a callback is experimental and unsafe, as the ramifications of running code on unwinding is pretty hairy. * The `std::unstable::mutex` module has moved to `std::rt::mutex`. * The `std::unstable::sync` module has been moved to `std::rt::exclusive` and the type has been rewritten to not internally have an Arc and to have an RAII guard structure when locking. Old code should stop using `Exclusive` in favor of the primitives in `libsync`, but if necessary, old code should port to `Arc<Exclusive<T>>`. * The local heap has been stripped down to have fewer debugging options. None of these were tested, and none of these have been used in a very long time.
As part of the libstd facade efforts, this commit extracts the runtime interface
out of the standard library into a standalone crate, librustrt. This crate will
provide the following services:
Notably, this crate avoids the following services:
an OS threading system, and as a result it seemed best to leave out the
rt::thread
module from librustrt. The librustrt module does depend onmutexes, however.
to be able to generate backtraces. As will be discussed later, this
functionality continues to live in libstd rather than librustrt.
As usual, a number of architectural changes were required to make this crate
possible. Users of "stable" functionality will not be impacted by this change,
but users of the
std::rt
module will likely note the changes. A list ofarchitectural changes made is:
The stdout/stderr handles no longer live directly inside of the
Task
structure. This is a consequence of librustrt not knowing about
std::io
.These two handles are now stored inside of task-local-data.
The handles were originally stored inside of the
Task
for perf reasons, andTLD is not currently as fast as it could be. For comparison, 100k prints goes
from 59ms to 68ms (a 15% slowdown). This appeared to me to be an acceptable
perf loss for the successful extraction of a librustrt crate.
The
rtio
module was forced to duplicate more functionality ofstd::io
. Asthe module no longer depends on
std::io
,rtio
now defines structures suchas socket addresses, addrinfo fiddly bits, etc. The primary change made was
that
rtio
now defines its ownIoError
type. This type is distinct fromstd::io::IoError
in that it does not have an enum for what error occurred,but rather a platform-specific error code.
The native and green libraries will be updated in later commits for this
change, and the bulk of this effort was put behind updating the two libraries
for this change (with
rtio
).Printing a message on task failure (along with the backtrace) continues to
live in libstd, not in librustrt. This is a consequence of the above decision
to move the stdout/stderr handles to TLD rather than inside the
Task
itself.The unwinding API now supports registration of global callback functions which
will be invoked when a task fails, allowing for libstd to register a function
to print a message and a backtrace.
The API for registering a callback is experimental and unsafe, as the
ramifications of running code on unwinding is pretty hairy.
The
std::unstable::mutex
module has moved tostd::rt::mutex
.The
std::unstable::sync
module has been moved tostd::rt::exclusive
andthe type has been rewritten to not internally have an Arc and to have an RAII
guard structure when locking. Old code should stop using
Exclusive
in favorof the primitives in
libsync
, but if necessary, old code should port toArc<Exclusive<T>>
.The local heap has been stripped down to have fewer debugging options. None of
these were tested, and none of these have been used in a very long time.