deploy: 643b291

_sources/olcf-workflow.rst.txt

@@ -541,7 +541,7 @@ Setup the environment:
 
 .. prompt:: bash
 
-   module load PrgEnv-cray
+   module load PrgEnv-gnu
    module load cray-mpich/8.1.27
    module load craype-accel-amd-gfx90a
    module load amd-mixed/5.6.0
@@ -559,16 +559,36 @@ Startup an interactive session:
 
    salloc -A ast106 -J mz -t 0:30:00 -p batch -N 1
 
-This will automatically log you onto the compute now.  Now set the following
-environment variables:
+This will automatically log you onto the compute now.
+
+.. note::
+
+   It's a good idea to do:
+
+   .. prompt:: bash
+
+      module restore
+
+   and then reload *the same* modules used for compiling in the interactive shell.
+
+Now set the following environment variables:
 
 .. prompt:: bash
 
    export HIP_ENABLE_DEFERRED_LOADING=0
-   export AMD_LOG_LEVEL=3
    export AMD_SERIALIZE_KERNEL=3
    export AMD_SERIALIZE_COPY=3
 
+.. note::
+
+   You can also set
+
+   .. prompt:: bash
+
+      export AMD_LOG_LEVEL=3
+
+   to get *a lot* of information about the GPU calls.
+
 Run the debugger:
 
 .. prompt:: bash

olcf-workflow.html

@@ -618,7 +618,7 @@ <h3>Job Chaining<a class="headerlink" href="#job-chaining" title="Link to this h
 <h3>Debugging<a class="headerlink" href="#debugging" title="Link to this heading">¶</a></h3>
 <p>Debugging is done with <code class="docutils literal notranslate"><span class="pre">rocgdb</span></code>.  Here’s a workflow that works:</p>
 <p>Setup the environment:</p>
-<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">module<span class="w"> </span>load<span class="w"> </span>PrgEnv-cray</span>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">module<span class="w"> </span>load<span class="w"> </span>PrgEnv-gnu</span>
 <span class="prompt1">module<span class="w"> </span>load<span class="w"> </span>cray-mpich/8.1.27</span>
 <span class="prompt1">module<span class="w"> </span>load<span class="w"> </span>craype-accel-amd-gfx90a</span>
 <span class="prompt1">module<span class="w"> </span>load<span class="w"> </span>amd-mixed/5.6.0</span>
@@ -627,13 +627,24 @@ <h3>Debugging<a class="headerlink" href="#debugging" title="Link to this heading
 <div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">make<span class="w"> </span><span class="nv">USE_HIP</span><span class="o">=</span>TRUE<span class="w"> </span><span class="nv">TEST</span><span class="o">=</span>TRUE<span class="w"> </span><span class="nv">USE_MPI</span><span class="o">=</span>FALSE<span class="w"> </span>-j<span class="w"> </span><span class="m">4</span></span>
 </pre></div></div><p>Startup an interactive session:</p>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">salloc<span class="w"> </span>-A<span class="w"> </span>ast106<span class="w"> </span>-J<span class="w"> </span>mz<span class="w"> </span>-t<span class="w"> </span><span class="m">0</span>:30:00<span class="w"> </span>-p<span class="w"> </span>batch<span class="w"> </span>-N<span class="w"> </span><span class="m">1</span></span>
-</pre></div></div><p>This will automatically log you onto the compute now.  Now set the following
-environment variables:</p>
+</pre></div></div><p>This will automatically log you onto the compute now.</p>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>It’s a good idea to do:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">module<span class="w"> </span>restore</span>
+</pre></div></div><p>and then reload <em>the same</em> modules used for compiling in the interactive shell.</p>
+</div>
+<p>Now set the following environment variables:</p>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1"><span class="nb">export</span><span class="w"> </span><span class="nv">HIP_ENABLE_DEFERRED_LOADING</span><span class="o">=</span><span class="m">0</span></span>
-<span class="prompt1"><span class="nb">export</span><span class="w"> </span><span class="nv">AMD_LOG_LEVEL</span><span class="o">=</span><span class="m">3</span></span>
 <span class="prompt1"><span class="nb">export</span><span class="w"> </span><span class="nv">AMD_SERIALIZE_KERNEL</span><span class="o">=</span><span class="m">3</span></span>
 <span class="prompt1"><span class="nb">export</span><span class="w"> </span><span class="nv">AMD_SERIALIZE_COPY</span><span class="o">=</span><span class="m">3</span></span>
-</pre></div></div><p>Run the debugger:</p>
+</pre></div></div><div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>You can also set</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1"><span class="nb">export</span><span class="w"> </span><span class="nv">AMD_LOG_LEVEL</span><span class="o">=</span><span class="m">3</span></span>
+</pre></div></div><p>to get <em>a lot</em> of information about the GPU calls.</p>
+</div>
+<p>Run the debugger:</p>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">rocgdb<span class="w"> </span>./Castro2d.hip.x86-trento.HIP.ex</span>
 </pre></div></div><p>Set the following inside of the debugger:</p>
 <div class="highlight-default notranslate"><div class="highlight"><pre><style type="text/css">