Running program 6 on
Stampede2 - notes:
- If you name your wind components "U" and "V", plot3d will be able to plot
vorticity for you.
- Please send me PDFs, images, .doc files, .tar files,
.tar.gz files, or .zip files; I have had difficulty
reading .rar files in the past.
- Running on
Stampede2:
- Interactive use:
- Running faster on
Stampede: Compiling on
the front end login nodes is best, but running
long-wallclock jobs is discouraged. For long
runs, use idev
(interactive development tool) instead.. this is
like starting a batch job that remains attached to
your login session. When logged on Stampede,
you can type idev --help
for more information, or see
this
web page.
- When typed without arguments, idev starts up a 30-minute batch job
that gives you a full compute node (all 16 cores,
even if you only use one) for that time.
Since under idev
you are not sharing processors as you would on
front-end login nodes, things should run
quickly. Compiling should work under idev, though,
so just use it to run/test your compiled programs.
- Batch:
- Long running jobs should be run in batch.
An added advantage is you can start up a batch job
and logoff.
- See ~tg457444/502/Pgm6/[C or Fortran]/*.deck for
examples of job decks.
- More on running jobs (including batch) on
Stampede-2: see
here.
Other Stampede notes:
1) when calling the
putfield
routine, remember that it (like our plotting routines in
the past)
does not know about ghost
points. It assumes arrays are fully (nx,ny,nz).
If your array contains ghost points, make sure you
pass the full correct dimensions when calling putfield
(could be nx+2 instead of nx, say, or NXDIM instead of NX
in C).
2) that said, Stampede is unforgiving if you pass (to a
subroutine) or access an array which has not been
completely defined. For example, if you define an
array with ghost points and try to plot it (or pass it to
putfield) without having set those ghost point values, you
may get an error.
Remember, you can always First copy your array w/ghost
points to an array without ghost points, and pass that to
putfield. Or, in Fortran, call it with subscripting
(this is completely OK, but if you have compiled with
-check all, you will get warnings about "temporary
arrays"):
call
putfield("T",0.0,theta(1:nx,1:ny,1:nz),nx,ny,nz)
Note: the second argument to putfield is
expected to be a
real number;
use 0.0, not 0, as
the latter could be incorrectly interpreted as an integer
to pass to the routine, which causes unpredictable results
(i.e. this is bad!)
3) Stampede may not reliably initialize variables to zero
(common in Fortran on most machines), so if you do not
initialize a variable, it could contain something odd.
In Fortran, you can compile with the
-zero option to make
sure all variables are set to zero before you set (or use)
them. If you are careful about setting every
variable (or every array element) before accessing it,
this is unnecessary.
4) As a matter of habit, until you are running higher
resolution 3d problems where speed is essential, I
recommend compiling every routine with full debug options
to the compiler.
In Fortran, I suggest the following settings (set
everything after
ncargf90 as the OPTIONS in
your
makefile)
ncargf90 -g -O0 -traceback -check all -ftrapuv
-fpe0 -zero some_file.f90
-g ensures symbol information is
preserved, for debugging use.
-O0 (first character is an upper case "o", second is the
digit zero) specifies
no optimization.
-traceback
gives you line number/code info if the program fails.
-check all
looks for running off the end of arrays.
-ftrapuv fails
for any uninitialized variable (a variable was used
before it was set).
-fpe0
triggers floating point exceptions if your code
generates too-large or too-small numbers; the program
halts.
-zero sets
every variable to zero before your program runs
(supposedly. I'm not sure I trust it to do so, but
it does no harm).
Note that
only when you change such key
makefile OPTIONS will you need to "make
clean"
and re-compile everything for those changes
to take effect.
More on Fortran debugging options for various compilers
are
here.
In C, I recommend, as a minimum (yes, -traceback works on
Stampede):
ncargcc –g –debug extended -traceback
some_file.c
... or you can go farther:
ncargcc -g -debug extended
-traceback -check-uninit -Wuninitialized -Wcheck
some_file.c
... in the latter, you may get a huge number of warnings,
which you can suppress if you wish by
adding the '-w' option
(
before your
filename) to ignore (at your peril!) warning messages.
If your program stubbornly refuses to run and you
can't find the error
Beyond the suggestions above, try these:
- Run your program 6 code within the
debugger, e.g.
gdb
name-of-your-p6-program (gdb is the GNU debugger; info here)
run
... program prints out
extra info if it fails ... hopefully.
exit
(to leave the debugger)
- Run the valgrind
program program to check memory.
- On
Stampede you will first need to: module
load valgrind/3.8.1 (module
spider valgrid gives more info)
- More on running Valgrind: see this
page.