(base) [spadoalz@acn01.karolina test]$ mpirun -n 1 /scratch/project/open-28-44/smilei_GPU/Smilei/build/smilei input.py _ _ ___ _ | | _ \ \ Version : 5.0-58-gdeb419f-master / __| _ __ (_) | | ___ (_) | | \__ \ | ' \ _ | | / -_) _ | | |___/ |_|_|_| |_| |_| \___| |_| | | /_/ Reading the simulation parameters ------------------------------------------------------------------------------- HDF5 version 1.14.0 Python version 3.10.4 Parsing pyinit.py Parsing 5.0-58-gdeb419f-master Parsing pyprofiles.py Parsing input.py Parsing pycontrol.py Check for function preprocess() python preprocess function does not exist Calling python _smilei_check Calling python _prepare_checkpoint_dir Calling python _keep_python_running() : CAREFUL: Patches distribution: hilbertian Smilei will run on GPU devices [WARNING] src/Params/Params.cpp:1139 (compute) simulation_time has been redefined from 1024.000000 to 1023.996094 to match timestep. Geometry: 1Dcartesian ------------------------------------------------------------------------------- Interpolation order : 2 Maxwell solver : Yee simulation duration = 1023.996094, total number of iterations = 68985 timestep = 0.014844 = 0.950000 x CFL, time resolution = 67.368421 Grid length: 32 Cell length: 0.015625, 0, 0 Number of cells: 2048 Spatial resolution: 64 Electromagnetic boundary conditions ------------------------------------------------------------------------------- xmin periodic xmax periodic Vectorization: ------------------------------------------------------------------------------- Mode: off Calling python writeInfo Initializing MPI ------------------------------------------------------------------------------- applied topology for periodic BCs in x-direction MPI_THREAD_MULTIPLE enabled Number of MPI processes: 1 OpenMP disabled OpenMP task parallelization not activated Number of patches: 8 Number of cells in one patch: 256 Dynamic load balancing: every 100 iterations Initializing the restart environment ------------------------------------------------------------------------------- Initializing species ------------------------------------------------------------------------------- Creating Species #0: ion > Pusher: boris CAREFUL: For species 'ion' possible conflict between time-frozen & not cold initialization > Species frozen until time: 2048.000000 > Boundary conditions: periodic periodic > Density profile: 1D built-in profile `constant` (value: 1.000000) Creating Species #1: eon > Pusher: boris > Boundary conditions: periodic periodic > Density profile: 1D built-in profile `constant` (value: 1.000000) Initializing Patches ------------------------------------------------------------------------------- First patch created Approximately 10% of patches created Approximately 20% of patches created Approximately 30% of patches created Approximately 40% of patches created Approximately 50% of patches created Approximately 60% of patches created Approximately 70% of patches created All patches created Creating Diagnostics, antennas, and external fields ------------------------------------------------------------------------------- Created ParticleBinning #0: species eon Axis ekin from 0 to 0.000976562 in 32 steps Diagnostic Fields #0 : Ex Ey Rho_ion Rho_eon Finalize MPI environment ------------------------------------------------------------------------------- Done creating diagnostics, antennas, and external fields Minimum memory consumption (does not include all temporary buffers) ------------------------------------------------------------------------------- Particles: Master 2 MB; Max 2 MB; Global 0.00256 GB Fields: Master 0 MB; Max 0 MB; Global 0.000233 GB scalars.txt: Master 0 MB; Max 0 MB; Global 0 GB ParticleBinning0.h5: Master 0 MB; Max 0 MB; Global 2.38e-07 GB Fields0.h5: Master 0 MB; Max 0 MB; Global 0 GB Initial fields setup ------------------------------------------------------------------------------- Solving Poisson at time t = 0 Initializing E field through Poisson solver ------------------------------------------------------------------------------- Poisson solver converged at iteration: 1024, relative err is ctrl = 0.000000 x 1e-14 Poisson equation solved. Maximum err = 0.000000 at i= -1 Time in Poisson : 0.016941 Applying external fields at time t = 0 Applying prescribed fields at time t = 0 Applying antennas at time t = 0 GPU allocation and copy of the fields and particles ------------------------------------------------------------------------------- Open files & initialize diagnostics ------------------------------------------------------------------------------- Running diags at time t = 0 ------------------------------------------------------------------------------- line 447: cudaLaunchKernel returned status 98: invalid device function [acn01:103533:0:103533] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x10) ==== backtrace (tid: 103533) ==== 0 0x0000000000039a70 __cudaPopCallConfiguration() ???:0 1 0x000000000003d1c4 __cudaPopCallConfiguration() ???:0 2 0x00000000000237bc __cudaPopCallConfiguration() ???:0 3 0x00000000000253ab __cudaPopCallConfiguration() ???:0 4 0x00000000000260ca __pgi_uacc_cuda_unregister_fat_binary() /proj/build/22A/Linux_x86_64/rte/accel-uni/build/Linux_x86_64/../../src/cuda_init.c:661 5 0x0000000000026092 __pgi_uacc_cuda_unregister_fat_binaries() /proj/build/22A/Linux_x86_64/rte/accel-uni/build/Linux_x86_64/../../src/cuda_init.c:651 6 0x0000000000039ce9 __run_exit_handlers() :0 7 0x0000000000039d37 __GI_exit() :0 8 0x0000000000045750 __pgi_uacc_exit() /proj/build/22A/Linux_x86_64/rte/accel-uni/build/Linux_x86_64/../../src/init.c:777 9 0x000000000001f451 __pgi_cuda_error_handler() /proj/build/22A/Linux_x86_64/rte/accel-uni/build/Linux_x86_64/../../src/cuda_launch.c:71 10 0x000000000002140a __pgi_uacc_cuda_launchk3() /proj/build/22A/Linux_x86_64/rte/accel-uni/build/Linux_x86_64/../../src/cuda_launch.c:559 11 0x000000000002262e __pgi_uacc_cuda_launch2() /proj/build/22A/Linux_x86_64/rte/accel-uni/build/Linux_x86_64/../../src/cuda_launche.c:157 12 0x00000000000227af __pgi_uacc_cuda_launch() /proj/build/22A/Linux_x86_64/rte/accel-uni/build/Linux_x86_64/../../src/cuda_launche.c:190 13 0x0000000000046ac3 __pgi_uacc_launch() /proj/build/22A/Linux_x86_64/rte/accel-uni/build/Linux_x86_64/../../src/launch.c:55 14 0x000000000051a47f DiagnosticScalar::compute() /scratch/project/open-28-44/smilei_GPU/Smilei/src/Diagnostic/DiagnosticScalar.cpp:447 15 0x0000000000ad7fbb VectorPatch::runAllDiags() /scratch/project/open-28-44/smilei_GPU/Smilei/src/Patch/VectorPatch.cpp:65 16 0x0000000000ceb221 main() /scratch/project/open-28-44/smilei_GPU/Smilei/src/Smilei.cpp:429 17 0x0000000000022555 __libc_start_main() ???:0 18 0x000000000049c82f _start() ???:0 ================================= Stack trace (most recent call last): #19 Object "[0xffffffffffffffff]", at 0xffffffffffffffff, in #18 Object "/scratch/project/open-28-44/smilei_GPU/Smilei/build/smilei", at 0x49c82e, in #17 Object "/lib64/libc.so.6", at 0x2b249ef7f554, in __libc_start_main #16 Object "/scratch/project/open-28-44/smilei_GPU/Smilei/build/smilei", at 0xceb220, in main #15 Object "/scratch/project/open-28-44/smilei_GPU/Smilei/build/smilei", at 0xad7fba, in VectorPatch::runAllDiags(Params&, SmileiMPI*, unsigned int, Timers&, SimWindow*) #14 Object "/scratch/project/open-28-44/smilei_GPU/Smilei/build/smilei", at 0x51a47e, in DiagnosticScalar::compute(Patch*, int) #13 Object "/apps/all/NVHPC/22.7-CUDA-11.7.0/Linux_x86_64/22.7/compilers/lib/libacchost.so", at 0x2b249ce30ac2, in __pgi_uacc_launch #12 Object "/apps/all/NVHPC/22.7-CUDA-11.7.0/Linux_x86_64/22.7/compilers/lib/libacccuda.so", at 0x2b249b6b97ae, in __pgi_uacc_cuda_launch #11 Object "/apps/all/NVHPC/22.7-CUDA-11.7.0/Linux_x86_64/22.7/compilers/lib/libacccuda.so", at 0x2b249b6b962d, in #10 Object "/apps/all/NVHPC/22.7-CUDA-11.7.0/Linux_x86_64/22.7/compilers/lib/libacccuda.so", at 0x2b249b6b8409, in __pgi_uacc_cuda_launchk3 #9 Object "/apps/all/NVHPC/22.7-CUDA-11.7.0/Linux_x86_64/22.7/compilers/lib/libacccuda.so", at 0x2b249b6b6450, in #8 Object "/apps/all/NVHPC/22.7-CUDA-11.7.0/Linux_x86_64/22.7/compilers/lib/libacchost.so", at 0x2b249ce2f74f, in __pgi_uacc_exit #7 Object "/lib64/libc.so.6", at 0x2b249ef96d36, in exit #6 Object "/lib64/libc.so.6", at 0x2b249ef96ce8, in #5 Object "/apps/all/NVHPC/22.7-CUDA-11.7.0/Linux_x86_64/22.7/compilers/lib/libacccuda.so", at 0x2b249b6bd091, in #4 Object "/apps/all/NVHPC/22.7-CUDA-11.7.0/Linux_x86_64/22.7/compilers/lib/libacccuda.so", at 0x2b249b6bd0c9, in #3 Object "/apps/all/CUDA/11.7.0/lib/libcudart.so.11.0", at 0x2b249581d3aa, in #2 Object "/apps/all/CUDA/11.7.0/lib/libcudart.so.11.0", at 0x2b249581b7bb, in #1 Object "/apps/all/CUDA/11.7.0/lib/libcudart.so.11.0", at 0x2b24958351c3, in #0 Object "/apps/all/CUDA/11.7.0/lib/libcudart.so.11.0", at 0x2b2495831a70, in Segmentation fault (Signal sent by tkill() [0x1e730001946d]) watf? exit -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpirun detected that one or more processes exited with non-zero status, thus causing the job to be terminated. The first process to do so was: Process name: [[42069,1],0] Exit code: 1 --------------------------------------------------------------------------