Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable tracing of thread pool tasks using NVTX #630

Open
wants to merge 9 commits into
base: branch-25.04
Choose a base branch
from

Conversation

kingcrimsontianyu
Copy link
Contributor

@kingcrimsontianyu kingcrimsontianyu commented Feb 9, 2025

This PR implements the basic feature outlined in #631.
The two good-to-haves are currently blocked.

@kingcrimsontianyu kingcrimsontianyu added feature request New feature or request non-breaking Introduces a non-breaking change c++ Affects the C++ API of KvikIO labels Feb 9, 2025
Copy link

copy-pr-bot bot commented Feb 9, 2025

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@kingcrimsontianyu kingcrimsontianyu changed the title [WIP] Enable tracing of thread pool tasks using NVTX Enable tracing of thread pool tasks using NVTX Feb 10, 2025
@kingcrimsontianyu
Copy link
Contributor Author

/ok to test

#define KVIKIO_NVTX_FUNC_RANGE_IMPL() NVTX3_FUNC_RANGE_IN(kvikio::libkvikio_domain)

// Implementation of KVIKIO_NVTX_SCOPED_RANGE(...)
#define KVIKIO_NVTX_SCOPED_RANGE_IMPL_3(message, payload_v, color) \
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why the variable has to be named payload_v. Otherwise payload would cause compile errors. Perhaps a name look-up related issue.

@@ -192,7 +194,8 @@ std::future<std::size_t> FileHandle::pread(void* buf,
std::size_t gds_threshold,
bool sync_default_stream)
{
KVIKIO_NVTX_MARKER("FileHandle::pread()", size);
auto& [nvtx_color, call_idx] = detail::get_next_color_and_call_idx();
KVIKIO_NVTX_SCOPED_RANGE("FileHandle::pread()", size, nvtx_color);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be consistent with RemoteHandle, the NVTX marker here is replaced with the scoped range.

@kingcrimsontianyu kingcrimsontianyu marked this pull request as ready for review February 10, 2025 20:16
@kingcrimsontianyu kingcrimsontianyu requested review from a team as code owners February 10, 2025 20:16
@kingcrimsontianyu
Copy link
Contributor Author

kingcrimsontianyu commented Feb 11, 2025

The following sample result shows the nsys profile generated using this PR. There are 4 worker threads in the thread pool, and I/O tasks of the same color come from the same pread()/pwrite() call. The payload of the task's time range is a global, incremental counter.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Affects the C++ API of KvikIO feature request New feature or request non-breaking Introduces a non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant