Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

std: Extract librustrt out of libstd #14638

Merged
merged 8 commits into from
Jun 7, 2014
Merged

Conversation

alexcrichton
Copy link
Member

As part of the libstd facade efforts, this commit extracts the runtime interface
out of the standard library into a standalone crate, librustrt. This crate will
provide the following services:

  • Definition of the rtio interface
  • Definition of the Runtime interface
  • Implementation of the Task structure
  • Implementation of task-local-data
  • Implementation of task failure via unwinding via libunwind
  • Implementation of runtime initialization and shutdown
  • Implementation of thread-local-storage for the local rust Task

Notably, this crate avoids the following services:

  • Thread creation and destruction. The crate does not require the knowledge of
    an OS threading system, and as a result it seemed best to leave out the
    rt::thread module from librustrt. The librustrt module does depend on
    mutexes, however.
  • Implementation of backtraces. There is no inherent requirement for the runtime
    to be able to generate backtraces. As will be discussed later, this
    functionality continues to live in libstd rather than librustrt.

As usual, a number of architectural changes were required to make this crate
possible. Users of "stable" functionality will not be impacted by this change,
but users of the std::rt module will likely note the changes. A list of
architectural changes made is:

  • The stdout/stderr handles no longer live directly inside of the Task
    structure. This is a consequence of librustrt not knowing about std::io.
    These two handles are now stored inside of task-local-data.

    The handles were originally stored inside of the Task for perf reasons, and
    TLD is not currently as fast as it could be. For comparison, 100k prints goes
    from 59ms to 68ms (a 15% slowdown). This appeared to me to be an acceptable
    perf loss for the successful extraction of a librustrt crate.

  • The rtio module was forced to duplicate more functionality of std::io. As
    the module no longer depends on std::io, rtio now defines structures such
    as socket addresses, addrinfo fiddly bits, etc. The primary change made was
    that rtio now defines its own IoError type. This type is distinct from
    std::io::IoError in that it does not have an enum for what error occurred,
    but rather a platform-specific error code.

    The native and green libraries will be updated in later commits for this
    change, and the bulk of this effort was put behind updating the two libraries
    for this change (with rtio).

  • Printing a message on task failure (along with the backtrace) continues to
    live in libstd, not in librustrt. This is a consequence of the above decision
    to move the stdout/stderr handles to TLD rather than inside the Task itself.
    The unwinding API now supports registration of global callback functions which
    will be invoked when a task fails, allowing for libstd to register a function
    to print a message and a backtrace.

    The API for registering a callback is experimental and unsafe, as the
    ramifications of running code on unwinding is pretty hairy.

  • The std::unstable::mutex module has moved to std::rt::mutex.

  • The std::unstable::sync module has been moved to std::rt::exclusive and
    the type has been rewritten to not internally have an Arc and to have an RAII
    guard structure when locking. Old code should stop using Exclusive in favor
    of the primitives in libsync, but if necessary, old code should port to
    Arc<Exclusive<T>>.

  • The local heap has been stripped down to have fewer debugging options. None of
    these were tested, and none of these have been used in a very long time.

@alexcrichton
Copy link
Member Author

I'm curious what others think about the organization of librustrt. With the facade, we're at liberty to move anything pretty much anywhere, but it seems odd to me to have mutexes in librustrt, but not threads (for example).

I suppose the question of "what is the runtime?" is a little hazy nowadays that there isn't really a clear line of what it is and what it isn't.

@huonw
Copy link
Member

huonw commented Jun 4, 2014

You say "TLS" a few times, and context implies you're referring to "task local data" (rather than "thread local storage"). Maybe "TLD" would be clearer?

The performance lost of stdin/out is a little sad, I assume the loss is entirely due to the TLD look-ups? (I imagine it will only get worse if more things are placed into TLD by a program.)

(Also travis failed with a few todos.)

@alexcrichton
Copy link
Member Author

Good point, I switched from saying TLS to TLD, and I'll try to get in the habit of doing so as well (it does get confusing having two...).

Yes, the slowdown is from the TLD replacements. We have good ideas of how to make these faster, but it will take some time to implement. I would expect the cost of a TLD lookup or replacement to be on the order of accessing a RefCell.

Thanks for pointing out those TODOs, I fixed up the documentation and pushed an update.

@alexcrichton
Copy link
Member Author

Note that I'm going to leave this un-rebased for now until #14538 lands because none of the major components of it will change. I would still appreciate review, however.

@brson
Copy link
Contributor

brson commented Jun 5, 2014

I feel very good about this. Do you have a crate-dag visualization after the change? r=me assuming no major objections appear.

@alexcrichton
Copy link
Member Author

@brson brson mentioned this pull request Jun 5, 2014
9 tasks
The rtio interface is a thin low-level interface over the I/O subsystems, and
the `Path` type is a little too high-level for this interface.
As part of the libstd facade efforts, this commit extracts the runtime interface
out of the standard library into a standalone crate, librustrt. This crate will
provide the following services:

* Definition of the rtio interface
* Definition of the Runtime interface
* Implementation of the Task structure
* Implementation of task-local-data
* Implementation of task failure via unwinding via libunwind
* Implementation of runtime initialization and shutdown
* Implementation of thread-local-storage for the local rust Task

Notably, this crate avoids the following services:

* Thread creation and destruction. The crate does not require the knowledge of
  an OS threading system, and as a result it seemed best to leave out the
  `rt::thread` module from librustrt. The librustrt module does depend on
  mutexes, however.
* Implementation of backtraces. There is no inherent requirement for the runtime
  to be able to generate backtraces. As will be discussed later, this
  functionality continues to live in libstd rather than librustrt.

As usual, a number of architectural changes were required to make this crate
possible. Users of "stable" functionality will not be impacted by this change,
but users of the `std::rt` module will likely note the changes. A list of
architectural changes made is:

* The stdout/stderr handles no longer live directly inside of the `Task`
  structure. This is a consequence of librustrt not knowing about `std::io`.
  These two handles are now stored inside of task-local-data.

  The handles were originally stored inside of the `Task` for perf reasons, and
  TLD is not currently as fast as it could be. For comparison, 100k prints goes
  from 59ms to 68ms (a 15% slowdown). This appeared to me to be an acceptable
  perf loss for the successful extraction of a librustrt crate.

* The `rtio` module was forced to duplicate more functionality of `std::io`. As
  the module no longer depends on `std::io`, `rtio` now defines structures such
  as socket addresses, addrinfo fiddly bits, etc. The primary change made was
  that `rtio` now defines its own `IoError` type. This type is distinct from
  `std::io::IoError` in that it does not have an enum for what error occurred,
  but rather a platform-specific error code.

  The native and green libraries will be updated in later commits for this
  change, and the bulk of this effort was put behind updating the two libraries
  for this change (with `rtio`).

* Printing a message on task failure (along with the backtrace) continues to
  live in libstd, not in librustrt. This is a consequence of the above decision
  to move the stdout/stderr handles to TLD rather than inside the `Task` itself.
  The unwinding API now supports registration of global callback functions which
  will be invoked when a task fails, allowing for libstd to register a function
  to print a message and a backtrace.

  The API for registering a callback is experimental and unsafe, as the
  ramifications of running code on unwinding is pretty hairy.

* The `std::unstable::mutex` module has moved to `std::rt::mutex`.

* The `std::unstable::sync` module has been moved to `std::rt::exclusive` and
  the type has been rewritten to not internally have an Arc and to have an RAII
  guard structure when locking. Old code should stop using `Exclusive` in favor
  of the primitives in `libsync`, but if necessary, old code should port to
  `Arc<Exclusive<T>>`.

* The local heap has been stripped down to have fewer debugging options. None of
  these were tested, and none of these have been used in a very long time.

[breaking-change]
bors added a commit that referenced this pull request Jun 7, 2014
As part of the libstd facade efforts, this commit extracts the runtime interface
out of the standard library into a standalone crate, librustrt. This crate will
provide the following services:

* Definition of the rtio interface
* Definition of the Runtime interface
* Implementation of the Task structure
* Implementation of task-local-data
* Implementation of task failure via unwinding via libunwind
* Implementation of runtime initialization and shutdown
* Implementation of thread-local-storage for the local rust Task

Notably, this crate avoids the following services:

* Thread creation and destruction. The crate does not require the knowledge of
  an OS threading system, and as a result it seemed best to leave out the
  `rt::thread` module from librustrt. The librustrt module does depend on
  mutexes, however.
* Implementation of backtraces. There is no inherent requirement for the runtime
  to be able to generate backtraces. As will be discussed later, this
  functionality continues to live in libstd rather than librustrt.

As usual, a number of architectural changes were required to make this crate
possible. Users of "stable" functionality will not be impacted by this change,
but users of the `std::rt` module will likely note the changes. A list of
architectural changes made is:

* The stdout/stderr handles no longer live directly inside of the `Task`
  structure. This is a consequence of librustrt not knowing about `std::io`.
  These two handles are now stored inside of task-local-data.

  The handles were originally stored inside of the `Task` for perf reasons, and
  TLD is not currently as fast as it could be. For comparison, 100k prints goes
  from 59ms to 68ms (a 15% slowdown). This appeared to me to be an acceptable
  perf loss for the successful extraction of a librustrt crate.

* The `rtio` module was forced to duplicate more functionality of `std::io`. As
  the module no longer depends on `std::io`, `rtio` now defines structures such
  as socket addresses, addrinfo fiddly bits, etc. The primary change made was
  that `rtio` now defines its own `IoError` type. This type is distinct from
  `std::io::IoError` in that it does not have an enum for what error occurred,
  but rather a platform-specific error code.

  The native and green libraries will be updated in later commits for this
  change, and the bulk of this effort was put behind updating the two libraries
  for this change (with `rtio`).

* Printing a message on task failure (along with the backtrace) continues to
  live in libstd, not in librustrt. This is a consequence of the above decision
  to move the stdout/stderr handles to TLD rather than inside the `Task` itself.
  The unwinding API now supports registration of global callback functions which
  will be invoked when a task fails, allowing for libstd to register a function
  to print a message and a backtrace.

  The API for registering a callback is experimental and unsafe, as the
  ramifications of running code on unwinding is pretty hairy.

* The `std::unstable::mutex` module has moved to `std::rt::mutex`.

* The `std::unstable::sync` module has been moved to `std::rt::exclusive` and
  the type has been rewritten to not internally have an Arc and to have an RAII
  guard structure when locking. Old code should stop using `Exclusive` in favor
  of the primitives in `libsync`, but if necessary, old code should port to
  `Arc<Exclusive<T>>`.

* The local heap has been stripped down to have fewer debugging options. None of
  these were tested, and none of these have been used in a very long time.
@bors bors closed this Jun 7, 2014
@bors bors merged commit 75014f7 into rust-lang:master Jun 7, 2014
@alexcrichton alexcrichton deleted the librustrt branch June 8, 2014 06:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants