cargo (for cargo script) and rustc disagee on shebangs

[epage]

Rustc has logic to detect and ignore shebangs
https://github.com/rust-lang/rust/blob/6171d944aea415a3023d4262e0895aa3b18c771f/compiler/rustc_lexer/src/lib.rs#L255-L278

Cargo needs something similar for knowing whether we should look for a frontmatter

cargo/src/cargo/util/toml/embedded.rs

Lines 195 to 215 in e3fa31e

	// See rust-lang/rust's compiler/rustc_lexer/src/lib.rs's `strip_shebang`
	// Shebang must start with `#!` literally, without any preceding whitespace.
	// For simplicity we consider any line starting with `#!` a shebang,
	// regardless of restrictions put on shebangs by specific platforms.
	if let Some(rest) = source.content.strip_prefix("#!") {
	// Ok, this is a shebang but if the next non-whitespace token is `[`,
	// then it may be valid Rust code, so consider it Rust code.
	if rest.trim_start().starts_with('[') {
	return Ok(source);
	}
	// No other choice than to consider this a shebang.
	let newline_end = source
	.content
	.find('\n')
	.map(|pos| pos + 1)
	.unwrap_or(source.content.len());
	let (shebang, content) = source.content.split_at(newline_end);
	source.shebang = Some(shebang);
	source.content = content;
	}

These differ and we need to decide how we want to reconcile these

Originally reported in #14857 (comment)

[epage]

The key problem is #! can either start a shebang or an attribute and we need to be able to tell which we are seeing on the first line.

Ours is implemented to the textual description (#! + whitespace + not [) but misses that "whitespace" can refer to space, tab, newlines (I think), line comments (including doc comments), and block comments (including doc comments). As we use trim_start, we do handle space, tab, and newlines. We are only missing comments.

Part of our intention with the frontmatter syntax is we don't have to have a full Rust parser, so I worry about having to deal with some of this extra syntax.

[epage]

An example of something that rustc considers to not have a shebang:

#!// Hello
[allow(dead_code)]

fn main() {
    println!("Hello, world!");
}

I lean towards

• Keeping what we have, treating the above as having a shebang
• Change logic to require one non-whitespace character between #! and \n
  • This works well for rustc allowing comments on following lines but not when rustc allows comments on the same line as #!
  • However, a user doing #! // or #! /* without it being a shebang is odd