Skip to content

Commit

Permalink
docs:Update rustc-dev-guide-book
Browse files Browse the repository at this point in the history
    Rename `StringReader` to `Lexer` and update links.

    The `StringReader` struct link is invalid since
    pr:Merge `TokenTreesReader` into `StringReader`.
  • Loading branch information
QingyaoLin committed Jan 17, 2025
1 parent 73c0ae6 commit 9b1f700
Show file tree
Hide file tree
Showing 4 changed files with 10 additions and 10 deletions.
2 changes: 1 addition & 1 deletion src/doc/rustc-dev-guide/src/appendix/code-index.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Item | Kind | Short description | Chapter |
`SourceFile` | struct | Part of the `SourceMap`. Maps AST nodes to their source code for a single source file. Was previously called FileMap | [The parser] | [compiler/rustc_span/src/lib.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_span/struct.SourceFile.html)
`SourceMap` | struct | Maps AST nodes to their source code. It is composed of `SourceFile`s. Was previously called CodeMap | [The parser] | [compiler/rustc_span/src/source_map.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_span/source_map/struct.SourceMap.html)
`Span` | struct | A location in the user's source code, used for error reporting primarily | [Emitting Diagnostics] | [compiler/rustc_span/src/span_encoding.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_span/struct.Span.html)
`StringReader` | struct | This is the lexer used during parsing. It consumes characters from the raw source code being compiled and produces a series of tokens for use by the rest of the parser | [The parser] | [compiler/rustc_parse/src/lexer/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_parse/lexer/struct.StringReader.html)
`Lexer` | struct | This is the lexer used during parsing. It consumes characters from the raw source code being compiled and produces a series of tokens for use by the rest of the parser | [The parser] | [compiler/rustc_parse/src/lexer/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_parse/lexer/struct.Lexer.html)
`rustc_ast::token_stream::TokenStream` | struct | An abstract sequence of tokens, organized into `TokenTree`s | [The parser], [Macro expansion] | [compiler/rustc_ast/src/tokenstream.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ast/tokenstream/struct.TokenStream.html)
`TraitDef` | struct | This struct contains a trait's definition with type information | [The `ty` modules] | [compiler/rustc_middle/src/ty/trait_def.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/trait_def/struct.TraitDef.html)
`TraitRef` | struct | The combination of a trait and its input types (e.g. `P0: Trait<P1...Pn>`) | [Trait Solving: Goals and Clauses] | [compiler/rustc_middle/src/ty/sty.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/type.TraitRef.html)
Expand Down
4 changes: 2 additions & 2 deletions src/doc/rustc-dev-guide/src/guides/editions.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ stability.
## Edition parsing

For the most part, the lexer is edition-agnostic.
Within [`StringReader`], tokens can be modified based on edition-specific behavior.
Within [`Lexer`], tokens can be modified based on edition-specific behavior.
For example, C-String literals like `c"foo"` are split into multiple tokens in editions before 2021.
This is also where things like reserved prefixes are handled for the 2021 edition.

Expand All @@ -114,7 +114,7 @@ For example, the deprecated `start...end` pattern syntax emits the
[`ellipsis_inclusive_range_patterns`] lint on editions before 2021, and in 2021 is an hard error via
the `emit_err` method.

[`StringReader`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_parse/lexer/struct.StringReader.html
[`Lexer`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_parse/lexer/struct.Lexer.html
[`ParseSess::edition`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_session/parse/struct.ParseSess.html#structfield.edition
[`ellipsis_inclusive_range_patterns`]: https://doc.rust-lang.org/nightly/rustc/lints/listing/warn-by-default.html#ellipsis-inclusive-range-patterns

Expand Down
6 changes: 3 additions & 3 deletions src/doc/rustc-dev-guide/src/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Unicode character encoding.

The token stream passes through a higher-level lexer located in
[`rustc_parse`] to prepare for the next stage of the compile process. The
[`StringReader`] `struct` is used at this stage to perform a set of validations
[`Lexer`] `struct` is used at this stage to perform a set of validations
and turn strings into interned symbols (_interning_ is discussed later).
[String interning] is a way of storing only one immutable
copy of each distinct string value.
Expand Down Expand Up @@ -153,7 +153,7 @@ the final binary.
[`rustc_parse::parser::Parser`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_parse/parser/struct.Parser.html
[`rustc_parse`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_parse/index.html
[`simplify_try`]: https://github.com/rust-lang/rust/pull/66282
[`StringReader`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_parse/lexer/struct.StringReader.html
[`Lexer`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_parse/lexer/struct.Lexer.html
[`Ty<'tcx>`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.Ty.html
[borrow checking]: borrow_check.md
[codegen]: backend/codegen.md
Expand Down Expand Up @@ -342,7 +342,7 @@ Compiler performance is a problem that we would like to improve on
(and are always working on). One aspect of that is parallelizing
`rustc` itself.

Currently, there is only one part of rustc that is parallel by default:
Currently, there is only one part of rustc that is parallel by default:
[code generation](./parallel-rustc.md#Codegen).

However, the rest of the compiler is still not yet parallel. There have been
Expand Down
8 changes: 4 additions & 4 deletions src/doc/rustc-dev-guide/src/the-parser.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ This happens in two stages: Lexing and Parsing.

2. _Parsing_ takes streams of tokens and turns them into a structured form
which is easier for the compiler to work with, usually called an [*Abstract
Syntax Tree* (AST)][ast] .
Syntax Tree* (AST)][ast] .

## The AST

Expand Down Expand Up @@ -52,7 +52,7 @@ the token stream, and then execute the parser to get a [`Crate`] (the root AST
node).

To minimize the amount of copying that is done,
both [`StringReader`] and [`Parser`] have lifetimes which bind them to the parent [`ParseSess`].
both [`Lexer`] and [`Parser`] have lifetimes which bind them to the parent [`ParseSess`].
This contains all the information needed while parsing, as well as the [`SourceMap`] itself.

Note that while parsing, we may encounter macro definitions or invocations.
Expand All @@ -67,7 +67,7 @@ Code for lexical analysis is split between two crates:
constituting tokens. Although it is popular to implement lexers as generated
finite state machines, the lexer in [`rustc_lexer`] is hand-written.

- [`StringReader`] integrates [`rustc_lexer`] with data structures specific to
- [`Lexer`] integrates [`rustc_lexer`] with data structures specific to
`rustc`. Specifically, it adds `Span` information to tokens returned by
[`rustc_lexer`] and interns identifiers.

Expand All @@ -76,7 +76,7 @@ Code for lexical analysis is split between two crates:
[`ParseSess`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_session/parse/struct.ParseSess.html
[`rustc_lexer`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lexer/index.html
[`SourceMap`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_span/source_map/struct.SourceMap.html
[`StringReader`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_parse/lexer/struct.StringReader.html
[`Lexer`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_parse/lexer/struct.Lexer.html
[ast module]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ast/ast/index.html
[ast]: ./ast-validation.md
[parser]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_parse/parser/index.html
Expand Down

0 comments on commit 9b1f700

Please sign in to comment.