Spring, 2019
Jewett
 Computer Program #6 -- Run settings
 ATMS 502 / CSE 566
Numerical Fluid Dynamics
NOTE:  I recommend you first test your 3d code with the "Low resolution test" (150m grid spacing) below. 
If it all works - then do the high-resolution experiment below.
If the high resolution run works, turn in plots for that, instead of for the low-resolution run.  You don't need to do both,
   but the high-resolution run is for full credit, and the low-resolution is not worth full credit.

Run parameters for the final assignment, program 6
Low resolution run, dx=dy=dz=150m
Settings in table below are for PIECEWISE LINEAR -
If you are using Lax-Wendroff, make these changes to the table below:
* sound speed 40 m/s * u/v/w momentum diffusion = 75 * theta diffusion = 35 * time step 0.50 sec
The low-resolution case is the best to use for testing, and for partial credit if you cannot run the full resolution problem.
Physics Full physics, with piecewise-linear theta advection
Grid dimensions 105x105x30
Grid spacing
 150 meters in x,y,z
Sound speed
 50 meters/sec
Diffusion coefficients
 20 for U,V,W;  2.5 for potential temperature (theta)
Perturbation: U
 upertur = 4 m/s, random (+/- 2.0 m/s overall)
Perturbation: V
  -15 m/s (first theta [infinite line] perturbation on left/west side, flow north-to-south),
+15 m/s (second theta [infinite line] perturbation, on right/east side, flow south-to-north)
Perturbation: T
 1) -12.5 ˚C , center x=      75, y=7800, z=1425 meters, radii x=2500, y=999999, z=2500
2) -12.5 ˚C , center x=15675, y=7800, z=1425 meters, radii x=2500, y=999999, z=2500
Runtime
 750 seconds
Time Step
 0.75 s (If your code fails, you may need to decrease dt to 0.50s or even lower if using Lax-W)
Data save interval
 At least every 20 seconds (seconds, not time steps!)  More often is OK but note you can quickly
make large files; my RunHistory.dat file was 770 MB when I called putfield every 7.5 seconds.
See comments about calling putfield as often as you need it - at end of table below this one.

Full resolution run, dx=dx=dz= 50 m
My code is running this in the skx-dev queue in under 10 minutes, but you should plan on a long run time,
  e.g. over 2 hours, if your code is not parallelized or does not parallelize well.  Get the low-res case working, first!
You can use the skx-dev queue for up to 2 hours; beyond that, use skx-normal.
Physics Full physics, with piecewise-linear theta advection
Grid dimensions 300x300x75
Grid spacing
 50 meters in x,y,z
Sound speed Cs
 60 meters/sec (yes, it should be larger.  I'm trying to save you runtime.)
Diffusion coefficients
 25 m^2/sec for U,V,W;  5 K^2/sec for potential temperature (theta)
Perturbation: U
 upertur = 2 m/s, random (+/- 1 m/s overall)
Perturbation: V
  -20 m/s (first theta [infinite line] perturbation on left/west side, flow north-to-south),
+20 m/s (second theta [infinite line] perturbation, on right/east side, flow south-to-north)
Perturbation: T
 1) -9.5 ˚C , center x=     25, y=7475, z=1225 meters, radii x=2000, y=999999, z=2000
2) -9.5 ˚C, center x=14975, y=7475, z=1225 meters, radii x=2000, y=999999, z=2000
Runtime
 800 s (seconds, not time steps!)
Time Step
 0.16 s (If your code fails, you may need to decrease dt but this worked for me)
Data save interval
 I suggest plotting every 12 seconds (seconds, not time steps!)  More often is OK but this will slow down the plotting (with plot3d) after your run finishes.  Things work best if your data save_interval in seconds divides evenly by the time step dt.

I suggest you use a save interval (in seconds) to set nsave:
     Fortran:  nsave = nint( save_interval / dt )
     C:       nsave = round( save_interval / dt )  /* using math.h here */
and then use in the usual way:
   Fortran:  if ( mod(n,nsave) == 0 ) then .... (add lines to call putfield)
   C:            if ( n%nsave == 0) {  ....  (add lines to call putfield)


Don't have Piecewise-Linear working? 
  • You'll lose some points in grading (10% off the high-resolution grade)
  • You may have to turn your time step down or increase the damping coefficients for the run to work!
Out of time?  Suggestions -
  • Run the lower resolution/domain problem listed above; if this isn't working, run/plot some figures from the test cases
  • Please do make plot3d plots of your initial condition fields.  This is needed so I know your integration started out correctly
  • Note: you can plot your RunHistory.dat files with plot3d while the (batch or interactive) run is underway, but you will
    have to quit and restart it occasionally to have it recognize the latest data saved during your integration.

Submit for this program the following
(i.e. on compass - nothing need be printed)

NOTE: plot3d works fine (and often faster) from within an interactive development (idev) session. 
To use idev, connect as usual to Stampede2, type idev or idev -p skx-dev, and then run plot3d. 
  1. 'plot3d' plots listed below.
  2. One figure of 3D visualization of at least one isosurface (called a 3d "contour" in VisIt) using Paraview or VisIt
    (not just a 3d plot from plot3d).   You choose the plot field and isosurface value, viewing angle, etc.
  3. Code source files - not word or PDF, so I can run your code too. Use make archive.
  4. Makefile for your code
  5. Batch script ("job deck") files you used, if any
  6. Batch output files you produced, if any (e.g. jobname.o123456), so I can see you ran parallel
Plots from plot3d 
(Note: all times below such as T=300 refers to time in seconds, not number of time steps!)

Use plot3d to produce the following 13 plots from your run for full credit. 
Combine them, if possible, in a .zip or .tar file, or put them in a Word document to hand in. 
You do Not need to submit plots for both the low- and high-resolution cases.
Just make plots for the high-est resolution case you get completed.

Vorticity:  if U and V are named "U" and "V" when calling putfield, then plot3d will compute vorticity for you.  Type "vort" to plot it
just as you do the fields you saved.  Vorticity requires that you enter a correct value of the grid spacing when you start plot3d.

Plots to submit
For all of these, inside plot3d but before making any plots, type "set HL" and then "0" to turn off H/L labels
Field Type Contour interval location of plot section Time of plot
T' X-Z 1 degree (default) J=domain center initial condition T=0
V X-Z 2 m/s (default) J=domain center initial condition T=0
T'
 3-D
 -1 deg. surface
 (whole domain)
 any time > T=300, you choose
T' X-Y 1 degree (default) K=1, i.e. at ground any time > T=300, you choose
T' X-Z 1 degree J = ~ domain center any time > T=300, you choose
W X-Y 4 m/s Above ground: K=20 (full res run), K=5 (low res) any time > T=300, you choose
W X-Z 4 m/s J = ~ domain center any time > T=300, you choose
U X-Y 4 m/s (default) K=1, at ground any time > T=300, you choose
V X-Y 4 m/s (default) K=1, at ground any time > T=300, you choose
V X-Z 4 m/s J = ~ domain center any time > T=300, you choose
vort X-Y 1000
(200 for low res)
 Above ground, K=3 any time > T=300, you choose
vort 3-D 1500
(750 for low res)
 (whole domain) any time > T=300, you choose
vort X-Y 100 or 1000,
you choose		 (default) K=1, at ground summary figure for all times:
see explanation below.

     Plotting notes:

  • For all these plots
    • Inside plot3d, Before creating any plots, type "set HL" and then enter 0 to turn off H/L labels.

  • To plot a single time with plot3d
    • type: "set times" then, for example, type "0 0" to restrict the plot time to T=0
    • when plot is done, save the metacode file to a unique name that you can convert to GIF later, e.g.
                metagif  vortXY1.meta 
    • Remember to "set times" again to no-longer-restrict plot3d to T=0.

  • (The last) figure summarizing entire simulation (all times): 1 plot
    • Make sure you have the entire integrate time range selected (use set times if not).
    • Make sure you have the entire domain selected (use set window off if not). 
    • Type set HL to turn the High/Low labels off (option "0" does this).
    • Type set max to compute the max value at each grid point over the entire run (enter Y to confirm).
    • Then: plot vort (X-Y at ground K=1), 500 or 1000 contour interval (you choose).
    • For your own interest: try the above plot after doing "set color" and then a color range e.g. "0 8000" where 8000=red color.
FYI:

From your max-for-all-time vorticity plot, you should see evidence of merging Y-parallel lines of vorticity.  Localized vorticity maxima evolve as the vorticity sheets "roll up", with the largest values when the density currents collide.  If we had higher time resolution (of saved data, e.g. every few steps) this would look smoother.  With our U' perturbations, the leading edge of the density currents begin to roll up before colliding.

You might also be interested in the min-for-all-time plot (use set min in plot3d) of P (X-Y, K=1, interval 50; you can see the pressure minima accompanying vorticity maxima particularly in the high-resolution case), the max-for-all-time plot of P (X-Y, K=1, interval 50; hmm), and the max V (X-Y, K=1, interval 2),  but these are not required.