Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

process: add execve #56496

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Conversation

ShogunPanda
Copy link
Contributor

@ShogunPanda ShogunPanda commented Jan 7, 2025

This PR adds a new process.execve method (absolutely willing to change the name if you want), which is a wrapper for the execve UNIX function.

The function will never return and will swap the current process with a new one.
All memory and system resources are automatically collected from execve, except for std{in,out,err}.

The primary use of this function is in shell scripts to allow to setup proper logics and then spawn another command.

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/startup

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. process Issues and PRs related to the process subsystem. labels Jan 7, 2025
@ShogunPanda ShogunPanda added the request-ci Add this label to start a Jenkins CI on a PR. label Jan 7, 2025
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jan 7, 2025
@nodejs-github-bot
Copy link
Collaborator

@ljharb
Copy link
Member

ljharb commented Jan 7, 2025

What does it do on non-Unix systems?

@ShogunPanda
Copy link
Contributor Author

@ljharb By non Unix we only mean Windows, isn't it? If that's the case, Windows also supports execve via _execve (doc here) so we should be good to go.

Once the CI ends I'll see which platform needs special assistance.

Copy link

codecov bot commented Jan 7, 2025

Codecov Report

Attention: Patch coverage is 96.15385% with 4 lines in your changes missing coverage. Please review.

Project coverage is 89.18%. Comparing base (afafee2) to head (57e8266).
Report is 50 commits behind head on main.

Files with missing lines Patch % Lines
src/node_process_methods.cc 91.66% 3 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #56496      +/-   ##
==========================================
+ Coverage   88.53%   89.18%   +0.64%     
==========================================
  Files         657      662       +5     
  Lines      190761   191855    +1094     
  Branches    36616    36895     +279     
==========================================
+ Hits       168899   171097    +2198     
+ Misses      15048    13629    -1419     
- Partials     6814     7129     +315     
Files with missing lines Coverage Δ
lib/internal/bootstrap/node.js 99.57% <100.00%> (+<0.01%) ⬆️
lib/internal/process/per_thread.js 99.39% <100.00%> (+0.07%) ⬆️
src/node_errors.h 88.00% <100.00%> (+3.00%) ⬆️
src/node_process_methods.cc 90.90% <91.66%> (+5.08%) ⬆️

... and 122 files with indirect coverage changes

@targos
Copy link
Member

targos commented Jan 7, 2025

On Windows:

D:\a\node\node\src\node_process_methods.cc(544,26): error C2065: 'F_GETFD': undeclared identifier [D:\a\node\node\libnode.vcxproj]
  (compiling source file '/src/node_process_methods.cc')
  
D:\a\node\node\src\node_process_methods.cc(544,17): error C3861: 'fcntl': identifier not found [D:\a\node\node\libnode.vcxproj]
  (compiling source file '/src/node_process_methods.cc')
  
D:\a\node\node\src\node_process_methods.cc(546,17): error C2065: 'FD_CLOEXEC': undeclared identifier [D:\a\node\node\libnode.vcxproj]
  (compiling source file '/src/node_process_methods.cc')
  
D:\a\node\node\src\node_process_methods.cc(547,16): error C2065: 'F_SETFD': undeclared identifier [D:\a\node\node\libnode.vcxproj]
  (compiling source file '/src/node_process_methods.cc')
  
D:\a\node\node\src\node_process_methods.cc(547,7): error C3861: 'fcntl': identifier not found [D:\a\node\node\libnode.vcxproj]
  (compiling source file '/src/node_process_methods.cc')

Copy link
Member

@targos targos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test/parallel/test-process-replace.js Outdated Show resolved Hide resolved
test/parallel/test-process-replace.js Outdated Show resolved Hide resolved
test/parallel/test-process-replace-fail.js Outdated Show resolved Hide resolved
test/parallel/test-process-replace-fail.js Outdated Show resolved Hide resolved
test/parallel/test-process-replace-socket.js Outdated Show resolved Hide resolved
test/parallel/test-process-replace-socket.js Outdated Show resolved Hide resolved
src/node_process_methods.cc Outdated Show resolved Hide resolved
src/node_process_methods.cc Outdated Show resolved Hide resolved
node::Utf8Value pathname_string(env->isolate(), args[0].As<String>());

argv = new char*[2];
argv[0] = strdup(*pathname_string);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need check not null also in here, maybe(?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good spot. Added.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should use std::memcpy or a similar modern C++ function other than this.

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ShogunPanda
Copy link
Contributor Author

@ljharb I was totally wrong. Despite of naming and similar signatures, the function _execve only creates a new process without replacing the old one.

I'll disable this function on Windows.

@ShogunPanda ShogunPanda added the request-ci Add this label to start a Jenkins CI on a PR. label Jan 7, 2025
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jan 7, 2025
@nodejs-github-bot
Copy link
Collaborator

Copy link
Member

@addaleax addaleax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is to be added, I'd absolutely recommend referring to it by a standard name such as execve(), or otherwise something that makes it clear that it spawns a new process.

This requires integration with the permissions API or is otherwise an immediate security hole.

Overall I'd recommend not adding this though, unless there's a concrete reason to believe that it fills a significant gap that the existing child_process API doesn't cover.

doc/api/errors.md Outdated Show resolved Hide resolved
resources from the current process are preserved, except for the standard input,
standard output and standard error file descriptor.
All other resources are discarded by system when the processes are swapped.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if this isn't the a desirable behavior in a given situation?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please expand this? What do you mean?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there's any point in adding execve(), it's the customizability that the method brings with it. Leaving file descriptors open and/or redirecting them intentionally can be part of that -- just like you could in a bash script do exec bash 0<&4 1>&4 to create a shell that reads and writes to e.g. a pre-opened socket or something along those lines.

(With a similar reasoning, you could also allow users to actually specify argv[0] with a value they control -- it doesn't need to equal the filename that's being executed)


THROW_ERR_PROCESS_REPLACE_FAILED(env, error_code);
}
#endif
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quite hard-to-follow C++ with lots of unnecessary explicit memory management that Node.js has been doing a lot of effort to move away from. I'd recommend looking a bit a how other parts of the Node.js code base handle strings and conversion between C++ and JS values.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm absolutely willing to. Since I'm not familiar with the C++ codebase, do you have any suggestion of places I can look to?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ShogunPanda Well, mostly anywhere else works. As a general rule, you'll want to get rid of new, new[], char*, memcpy() and strdup() as much as possible, and replace them with std::vector, std::string/Utf8Value as much as possible.

(You won't be entirely able to avoid something like std::vector<char*> because execve expects char** arguments, but the std::vector<char*>'s entries could point to the entries of a std::vector<std::string> or std::vector<Utf8Value> rather than having to manage memory manually).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@addaleax I vastly refactored the C++ part following your suggestions. It was great, thanks a lot for all the good hint.

I tried to use a single vector but when I instantiate a std::string or a Utf8Value inside a for loop it obviously went out of scope and garbage collected.
So I have to use two vectors for argv and two for envp. Is this the right approach or am I still missing something?

@ljharb
Copy link
Member

ljharb commented Jan 7, 2025

Why would we want to add a function that can't work on all tier 1 platforms?

@jasnell
Copy link
Member

jasnell commented Jan 7, 2025

Why would we want to add a function that can't work on all tier 1 platforms?

Well, we already have a number of such apis... process.getegid() for instance. There are actually quite a few already on process.

@ljharb
Copy link
Member

ljharb commented Jan 7, 2025

Gotcha, i wasn't aware of that.

@jasnell
Copy link
Member

jasnell commented Jan 7, 2025

I don't consider it to be ideal. I would have preferred a pattern like process.posix.getegid() similar to what we have with path.posix but these predate my involvement so they are what they are.

@jasnell
Copy link
Member

jasnell commented Jan 7, 2025

+1 on @addaleax's alternative name suggestion. I'm fine with adding this so long as the cleanup logic/expectations are clearly documented.

src/node_errors.h Outdated Show resolved Hide resolved
src/node_errors.h Outdated Show resolved Hide resolved
src/node_process_methods.cc Outdated Show resolved Hide resolved
src/node_process_methods.cc Outdated Show resolved Hide resolved
src/node_process_methods.cc Outdated Show resolved Hide resolved
src/node_process_methods.cc Outdated Show resolved Hide resolved
src/node_process_methods.cc Outdated Show resolved Hide resolved
src/node_process_methods.cc Outdated Show resolved Hide resolved
src/node_process_methods.cc Outdated Show resolved Hide resolved
src/node_process_methods.cc Outdated Show resolved Hide resolved
@ShogunPanda
Copy link
Contributor Author

Why would we want to add a function that can't work on all tier 1 platforms?

Well, we already have a number of such apis... process.getegid() for instance. There are actually quite a few already on process.

Thanks for involuntary hint. :)
I updated the code with #ifdef __POSIX__ and the docs to use the same wording already used there.

@ShogunPanda ShogunPanda changed the title process: add replace process: add execve Jan 8, 2025
@ShogunPanda
Copy link
Contributor Author

@addaleax

If this is to be added, I'd absolutely recommend referring to it by a standard name such as execve(), or otherwise something that makes it clear that it spawns a new process.

As requested, I've renamed it to execve. I didn't like the original name either :)

This requires integration with the permissions API or is otherwise an immediate security hole.

I just added integration with the permission API. It will require --allow-child-process.

Overall I'd recommend not adding this though, unless there's a concrete reason to believe that it fills a significant gap that the existing child_process API doesn't cover.

In the shell scripting context, there is no way to create a new process which would replace the current one. Think about using Node.js to build a complex command to run. After it, Node.js was not used anymore but since there was no way to swap the process, you would have to spawn a new process, manage it's stdin/stdout/stderr and so forth. That's why I added execve.

@ShogunPanda
Copy link
Contributor Author

@jasnell

+1 on @addaleax's alternative name suggestion. I'm fine with adding this so long as the cleanup logic/expectations are clearly documented.

I added two tests which check that:

  1. Test what happens if we leave a socket open and we try to reopen in the swapped process: the operation succeeds, which means the fd is destroyed. I also tried to install a on('close') but it is not invoked.

  2. Test what happens if there is a process.on('exit') on the original process. It does not get invoked, which means that the system call immediately swap the programs without running any cleanup logic.

I've added this to the documentation to reflect that. Does it look good now?

@addaleax
Copy link
Member

addaleax commented Jan 8, 2025

After it, Node.js was not used anymore but since there was no way to swap the process, you would have to spawn a new process, manage it's stdin/stdout/stderr and so forth. That's why I added execve.

I mean, to be clear, this is still a one-line operation (managing stdio literally just comes down to setting stdio: 'inherit') for the most part. I know where you're coming from but I wouldn't add this just for the sake of adding it.

@ShogunPanda
Copy link
Contributor Author

After it, Node.js was not used anymore but since there was no way to swap the process, you would have to spawn a new process, manage it's stdin/stdout/stderr and so forth. That's why I added execve.

I mean, to be clear, this is still a one-line operation (managing stdio literally just comes down to setting stdio: 'inherit') for the most part. I know where you're coming from but I wouldn't add this just for the sake of adding it.

I kinda agree.
You still would have to handle the exit code to ensure proper broadcasting to the shell.
Also, I'm not sure what would happen with "TUI" (ncurses and similar) programs and so forth.

Is your objection a full block or just an "intent"?

@addaleax
Copy link
Member

addaleax commented Jan 8, 2025

@ShogunPanda Just fyi, have you seen https://www.npmjs.com/package/foreground-child? That also works on Windows 🙂

My "request changes" marker only refers to the C++ memory management here, i.e. this comment specifically (and the integration with the permissions API, of course). I don't intend to block this feature, I've given my opinion but it's pretty clear overall that the Node.js project has a different stance from my own when it comes to 'what should go into core' 🙂

src/node_errors.h Outdated Show resolved Hide resolved
src/node_process_methods.cc Outdated Show resolved Hide resolved
int error_code = errno;

// Free up memory, then throw.
delete[] argv;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The abundance of manual memory management in this function really makes me nervous.

@ShogunPanda
Copy link
Contributor Author

ShogunPanda commented Jan 11, 2025

@anonrig @addaleax @jasnell I have vastly refactored the C++ to address Anna's and James' suggestions. Can you please review it again?

Apparently I'm more a C guy than a C++ guy. And last time I officially touched C was several years ago. Lol.

Thanks for the patience here!

Comment on lines +519 to +521
envp_strings[i] =
Utf8Value(isolate, envp_array->Get(context, i).ToLocalChecked())
.ToString();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For things like this, instead of using ToLocalChecked(), use a pattern like the following... this will avoid a process crash when a throw is appropriate:

Local<Value> str;
if (!envp_array->Get(context, i).ToLocal(&str)) {
  return;
}
envp_strings[i] = Utf8Value(str).ToString();


// Set stdin, stdout and stderr to be non-close-on-exec
// so that the new process will inherit it.
for (int i = 0; i < 3; i++) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rather than a using a loop here, consider something like:

if (persist_standard_stream(0) < 0 ||
    persist_standard_stream(1) < 0 ||
    persist_standard_stream(2) < 0) {
  THROW_ERR_PROCSS_EXECVE_FAILED(env, errno);
}

Or persist all three at once in one call to persist_standard_stream()

execve(*executable, argv.data(), envp.data());
int error_code = errno;

THROW_ERR_PROCESS_EXECVE_FAILED(env, error_code);
Copy link
Member

@jasnell jasnell Jan 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that you're running the RunAtExit(...) here, and given that the expectation is for this process to be replaced, I wonder if it would be more appropriate to crash the process here instead of just throwing (a throw can be caught and ignored and I'm not sure we want that here)

Karinza38

This comment was marked as spam.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. process Issues and PRs related to the process subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.