Example pages containing:tips resources functions/proceduresNCL: Plotting and working with shapefilesWebinar: NCL and ShapefilesShapefilesShapefiles are a supported dataformat in NCL, and hence can be readwith addfile. Unlike other supported formats,however, a "shapefile" refers to a collection of files, with ".shp",".shx", ".prj", ".dbf" and other possible extensions. In order toread shapefiles with NCL, you must have the ".shp", ".shx", and ".dbf"files in the same directory.From Wikipedia:The Esri* Shapefile or simplya shapefile is a popular geospatial vector data format forgeographic information systems software. It is developed and regulatedby Esri as a (mostly) open specification for data interoperabilityamong ESRI and other software products.[1] A "shapefile" commonlyrefers to a collection of files with ".shp", ".shx", ".prj", ".dbf",and other extensions on a common prefix name (e.g., "lakes.*"). Theactual shapefile relates specifically to files with the ".shp"extension, however this file alone is incomplete for distribution, asthese other supporting files are required.*Esri and the Esri Logo are licensed trademarks ofEnvironmental Systems Research Institute, Inc.Shapefiles describe a homogeneous set of geometrical features comprised of either points, polylines, or polygons. These, for example, could represent water wells, rivers, and lakes, respectively. Each feature may also have non-spatial attributes that describe it, such as the name or temperature.The Global Administrative Database(GADM) offers consistent administrative boundaries at multiple levels.The level 0 database (nations) is good to use for global or mesoscaleresults, level 1 is the first level of sub-national administration(typically states/provinces and territories) while level 2 offers thesecond level of administration and is potentially useful forhigh-resolution plots. Sometimes there are more levels (for example,France has 6) with even higher resolution. The global shapefiles arelarge, but it's possible to download individual countries separately.NOAA provides some useful AWIPSshapefile data.Shapefile plotting and masking routinesThese functions can be used to draw polygons, lines, and markers fromshapefile data:gsn_add_shapefile_polygonsgsn_add_shapefile_polylinesgsn_add_shapefile_polymarkersThe shapefile_utils.nclscript provides additional useful routines for working with shapefiles:print_shapefile_info - prints information about the shapefileplot_shapefile - creates a basic map plot with the shapefile outlines addedshapefile_mask_data - masks your own data against outlines in a shapefileImportant note: The shapefile_mask_data function is a powerful function used by many scripts on this page. In order for your datato be masked correctly using this function, your longitude values mustbe in the same range as the longitude values on the shapefile. Mostshapefile longitude values are in the range -180 to 180. You can usethe print_shapefile_info procedure above to print thisinformation.For a sample of the first two routines, try this simple 4-line NCL scriptwith your own shapefile data, ordownload sampleshapefiles here.load "./shapefile_utils.ncl"sname = "states.shp"print_shapefile_info(sname)plot_shapefile(sname)Shapefile detailsWhat follows are details of the contents of a shapefile. Severalfunctions (see above) have been added to hide some of this detail.Within NCL, a shapefile appears as a collection of 4 or 5 specifically named variables that encode the geometry of the features, along with some number of non-spatial variables. The number, names and types of the non-spatial variables depend upon the specific shapefile. The geometry of a feature is composed of one or more segments. Segments in turn are composed of an ordered-list of X, Y (and optionally Z) tuples. NCL uses the following variables to encode these relationships: geometry(num_features, 2) segments(num_segments, 2) x(num_points) y(num_points) z(num_points) ; 3D datasets only Each feature in the shapefile is represented by an entry in the geometry variable, along with corresponding entries in the non-spatial variables; i.e., the data for the i-th feature is found at the i-th entry of these variables. Each entry in geometry has two values: geometry(i, 0) specifies the index into variable segments of the 1st segment of the i-th feature, and geometry(i, 1) denotes the number of segments that make up the feature.Similarly, each entry in segments has two values: segments(j, 0) is the index of the 1st XY(Z) tuple of the j-th segment, and segments(j, 1) is the number of tuples belonging to the segment.With this encoding, any subsequent segments belonging to the i-th feature follow its first one in segments, and all the XY(Z) tuples belonging to the j-th segment follow the first in the x,y,(z) variables.NCL defines several global attributes for a shapefile: geom_segIndex = 0 geom_numSegs = 1 segs_xyzIndex = 0 segs_numPnts = 1 geometry_type = "point" "polyline" "polygon" layer_name = ; value derived from the shapefileThe first 4 attributes are intended as symbolic indices into the geometry and segments variable; see the examples below for how they should be used. shapefiles_1.ncl:Demonstrates how to read a shapefile and draw filled polygons over a map.Non-spatial variables from the associated database (i.e., the states.dbffile) are used to compute the polygon colors.To run this example, you must have the files "states.shp","states.dbf", and "states.shx". These can be obtained from the National Map. A Python version of this projection is available here.shapefiles_2.ncl:Demonstrates how to read a shapefile and draw selected informationbased upon a database query.In this case, the historical incidents of F5-class tornadoes inthe USA are plotted.To run this example, you must have the "states" shapefiles from theprevious example, along with "tornadx020.shp","tornadx020.dbf", and "tornadx020.shx". These can be obtained from the USGS's holdings on data.gov.mapoutlines_5.ncl: This script demonstrates the difference in resolution between thelow resolution and medium resolution map databases of NCL,and a world shapefile downloaded from cannot use the"HighRes" database with aglobal map, so the shapefile outlines are your best bet if you needmore resolution.mapoutlines_5_zoom.ncl: This script is similar to the mapoutlines_5.ncl one above, except itincludes the "HighRes"database, if installed. It produces two panel plots, so you cancompare the various map outlines.shapefiles_3.ncl:Demonstrates how to add shapefile outlines representing stream data inSouth America to an NCL map usinggsn_add_shapefile_polylines.To run this example, you must download the "HYDRO1k Streams data setfor South America as a tar file in shapefile format" (2.4 MB) from: _Data/Products_and_Data_Available/gtopo30/hydro/samericaNote: this dataset no longer appears to be available as of February 2014.shapefiles_4.ncl: Demonstratesusing "shapefile_mask_data" to mask an area in your data arrayusing a geographical outline. This scriptrequires shapefile_utils.ncl.See the description of thisfunction for important information.This particular example readsa shapefile to get an outline of theMississippi River Basin. You then have the option of masking out allareas inside or outside this outline.The "mrb.xxx" data files for this example can be found on theexample datasets page.The shapefiles_4_old.ncl isan older script that doesn't use a separate function to mask thedata. Look at this script if you are interested in seeing the "guts"of what it takes to mask data against a shapefile outline.shapefiles_5.ncl:Makes use of several shapefiles of differing resolutions and contents to mask data along county borders (Pakistan), and to draw and label selected boundaries and cities. Demonstrates querying the shapefiles' databases via non-spatial attributes to extract and draw specific geometry.Also provides an example of using table to create a custom map legend.The shapefiles for this example were obtained from DIVA-GIS. Search for administrative boundaries of Pakistan and download. The resultant zipfile contains four sets of shapefile files. shapefiles_6.ncl:This example uses a script very similar to example 3 above for South America to draw the canton outlines for Switzerland. The first frameshows the default map outline for Switzerland (admittedly not very good),and the second frame shows the data from the shapefile.The point is to show that shapefiles tend to have similar formats, andhence you can take a script and easily modify it to draw the outlinesyou're interested in.In this example, the outlines are drawn with polylines, and the placesof interest with text strings and polymarkers.With versions 6.1.2 or earlier of NCL, if you try to plot a lot ofindividual line segments or text strings using gsn_add_polyxxxxfunctions, then you may want to consider usinggsn_polyline instead ofgsn_add_polyline,gsn_polymarker instead ofgsn_add_polymarker, andgsn_text instead ofgsn_add_text.With version 6.2.0 of NCL, drawing polylines and polygons is much faster.Markers and text may still be slow.shapefiles_7.ncl:This script uses several shapefiles to draw river basins, points ofinterest, and indigenous areas in Australia. The shapefiles weredownloaded from several locations. See the comments in the script for details.The routines gsn_add_shapefile_polygons,gsn_add_shapefile_polylines, andgsn_add_shapefile_polymarkers are used to attach the shapefile information.shapefiles_8.ncl / shapefiles_8_panel.ncl:This script uses four shapefiles downloaded from draw various administrative areas for India.shapefiles_9.ncl:This uses the shapefile Example 4.This demonstrates calculating an areally weighted mean time seriesfor an irregularly shaped region. As in Example 4, an array containingonly the desired locations inside the shapefile is created. All othergrid points are set to _FillValue. Specifially, this computes theareal mean time series of monthly precipitation for the MississippiRiver Basin. The data is the monthly GPCP. mask_12.ncl: This example shows howto use a shapefile that contains polygon outlines to create a datamask for a variable with 1D coordinate arrays. The mask array is thenwritten to a copy of the input file. [This scriptrequires shapefile_utils.ncl.]In this case, the shapefile contains coastal outlines, which a landmask is created from. See the function "create_mask_from_shapefile"in the "mask_12.ncl" script. This function only works for data thatcontains coordinate arrays. You will need to modify it to work withcurvilinear or unstructured data.You should be able to use any shapefile that contains polygon data(point and polyline data won't work) to create the desired mask.The shapefile used in this example was part of a compressed file,"GSHHS_shp_2.2.0.zip", downloaded from: need to uncompress it with the "unzip" command. You can use anyof the other shapefiles that are included with this file, but they arepotentially a higher resolution, and hence creating the mask will takelonger.shapefiles_10.ncl /shapefiles_NM_10.ncl: Theseexamples show how to plot polygon data from a shapefile containinggeologic units and structural features in Colorado and New Mexico. The shapefiles weredownloaded from shapefile data came with a suggested lithologic color map: -color.php, which shows which colors to use for which rock type. This example draws the lithologic color map on a separate frame, using several labelbars.In order to use the suggested color map, thegsn_add_shapefile_polygonsprocedure was copied from the $NCARG_ROOT/lib/ncarg/nclscripts/csmdirectory, and then modified as needed.The New Mexico image additionally adds fault lines.shapefiles_11.ncl: Thisexample shows how to use a shapefile that contains an outline of theUnited States to create a land mask. The land mask is written to aNetCDF file so you can use it later for masking other variables on thesame grid.This script calls shapefile_mask_data, which is inthe shapefile_utils.nclscript. See the description of thisfunction for important information.Two USA shapefiles were tried with this example. The "gz_2010_us_020_00_5m.shp" shapefile is from and the "nationalp010g.shp" shapefile is from _scale/mld/1natlp.html. Thecensus one is smaller and hence the script runs much faster on thisone (200+ wall seconds versus 17 wall clock seconds). Which file youuse depends on how fine your original grid is, and how good of a maskyou need.shapefiles_12.ncl / shapefiles_12b.ncl: Thisexample shows how tomodify gsn_add_shapefile_polylinesto have it draw only a subset of the features in the given shapefile.See references to "feature_names", "vname", and "vlist" in the"gsn_add_shapefile_polylines_subset" function at the top of the NCLscript.The shapefile contains Interstate Highways, and only the primaryinterstate highways (I-5, I-82, and I-90) in Western United States aredrawn. (Special thanks to Dave Allured for his improvement of thisscript to correctly plot all highway segments with complex entries,like "I- 5, US 30".)The shapefile was downloaded from _13.ncl: Thisexample shows how to use shapefile data to collect lat/lon data pointsin each county of Georgia, and take an average of all the data valuesin that county. Based on each county average, a fill color is assignedand the county is filled usingmodify gsn_add_polygon.The gc_inout function is used to collect lat/lonpairs that fall in each county. Since this function can be slow ifyou are checking a lot of points, a number of "if" statementsare included to prevent unnecessary calls to this function.The county outlines for Georgia come from the USA_adm2.shp shapefile,which was downloaded from _14.ncl: Thisexample shows how to add shapefile outlines to an existing WRFcontour/map plot. In order to attach the shapefile outlines to the plot, you need to set"pltres@PanelPlot = True" sothat wrf_map_overlays doesn'tremove the contour plot. You can then usegsn_add_shapefile_polylines to add the shapefile outlines.The country outline for USA was downloaded from _14_mask.ncl: Thisexample is similar to the previous one, except it masks the WRF "hgt"variable against a select list of U.S. states outlines from theshapefile. This script calls shapefile_mask_data, which is inthe shapefile_utils.nclscript. See the description of thisfunction for important information.The outlines for the U.S. states comes from the USA_adm2.shpshapefile, which was downloaded from first frame is the unzoomed variable, and the second frame showshow to zoom in on the area of interest using the special "ZoomIn" WRFresource.Germany_coastal_counties_DEU_adm.ncl:This script, contributed by Karin Meier-Fleischerof Deutsche Klimarechenzentrum(DKRZ), shows how to use the DEU_adm3.shp shapefile downloadedfrom average data over coastal counties of Germany.You can customize what outlines are drawn by using command lineoptions. Examples:Coastal region ncl 'subregion="6.5,14.75,50.,55.5"' Germany_coastal_counties_DEU_adm.nclDraw "Schleswig-Holstein" (default) and plot only the sub-region ncl 'subregion="7.8,11.9,53.0,55.3"' Germany_coastal_counties_DEU_adm.nclDraw "Schleswig-Holstein" (default) but don't draw the border of all states ncl 'states_border=False' Germany_coastal_counties_DEU_adm.nclSelect the state "Hessen" but don't draw the borderline of Germany ncl 'state_name="Hessen"' 'country_border=False' Germany_coastal_counties_DEU_adm.nclFrance_1.ncl /France_2.ncl /France_3.ncl:The purpose of this example is to demonstrate the use of thegsSegments andgsColorsresources added in added in version 6.2.0,and how much routines like gsn_add_shapefile_polygons,gsn_add_shapefile_polylines,gsn_add_polyline,gsn_add_polygonhave been sped-up in version 6.2.0.If you run the France_1.ncl scriptwith the France administrative shapefiles downloadedfrom gadm.org/country, you willsee significant speed-up mostly for the "FRA_adm5.shp" file, which has36,612 features to draw:--------------------------NCL V6.1.2 timing results:--------------------------. . . SNIP . . .---> Drawing FRA_adm/FRA_adm5.shp... 207.032 CPU seconds--------------------------NCL V6.2.0 timing results:--------------------------. . . SNIP . . .---> Drawing FRA_adm/FRA_adm5.shp... 2.87449 CPU secondsNote: this scrip actually generates six images. The timingresults and the image displayed above are from running thisscript with the "do" loop changed to "do i=5,5".The France_2.ncl script producesidentical plots, only it usesgsn_add_polyline instead ofgsn_add_shapefile_polylines. Thisscript will only work with NCL V6.2.0, because it references thegsSegments resource directly.The France_3.ncl script drawsthe "FRA_adm1.shp" shapefile as filled polygons, usinggsColors to specify afill color for each segment.shapefiles_15.ncl: Thisexample is another one showing the difference between map outlines inNCL and map outlines added from a shapefile. The first plot in thepanel shows the counties of Colorado as plotted from NCL, and thesecond plot shows the counties as plotted from the USA_adm2 shapefiledownloaded from left image is from runningNCL V6.3.0 on thisscript. The right image is from runningNCL V6.4.0.In NCL versions 6.3.0 and earlier, the NCL outlines will not show thecounty of Broomfield or the updated counties around Denver. In NCLversion 6.4.0, the NCL map databases were updated to reflect thecurrent Colorado counties.shapefiles_16.ncl: Thisexample shows the effect that shapefile masking will have on both acoarse (32 x 64) and fine (64 x 128) grid.The gsn_coordinates procedure isused to draw the lat/lon grid as a set of markers.See example 21 below for an example that comparesmasking data using shapefile_mask_data, and maskingdata by drawing the undesired areas in white.shapefiles_17.ncl: Thisexample shows two ways to mask data based out outlines in ashapefile: one is to use the shapefile_mask_data functionmentioned in previous examples, and another is to simply fillin the undesired outlines in white over the originalcontour data, effectively masking it.The shapefile_mask_data function is inthe shapefile_utils.nclscript. See the description of thisfunction for important information.shapefiles_18.ncl: Thisexample uses a modified version of "shapefile_mask_data" tomask data based on a shapefile outline, and a "delta" distance(in km) outside the shapefile outline. See "delta_kilometer"in the script.This is useful if you want to keep some of your original data pointsthat fall just outside a shapefile outline.This script requires shapefile_mask_data_mod.ncl.shapefiles_19.ncl: This example shows how to either mask (retain) or remove data, based on a shapefileoutline. We get a lot of questions on this topic, so we decided to create a verybasic example that shows both types of masking.This example uses the shapefile_mask_data function foundshapefile_utils.ncl.See the description of thisfunction for important information.In order to mask data against outlines in a shapefile, you mustprovide the name of a variable on the shapefile that contains the list ofnames or values you want to mask against.For example, here, the "USA_adm1.shp" shapefile is being used, whichhas a "NAME_1" string array containing feature names that are simplythe U.S state names: "Alabama", "New Mexico", etc. You can referenceone or more of these strings to indicate you want to mask or keep datain these areas.The shapefile_mask_data function, in theshapefile_utils.nclscript, allows you to set the special "keep" option to False, whichwill cause data to be removed rather than retained.shapefiles_20.ncl: This example shows how to use the shapefile_mask_data function inshapefile_utils.nclto return a mask array, and then apply this mask array to a variablewith multiple time steps. The special option "return_mask" isset to True, telling the function to return the 0/1 mask array,instead of returning the masked data itself.The conform_dims function is used to conform themask array to a 3D array, and then mask is used toapply the mask.You can only do this kind of masking if your lat/lon grid remains thestationary across all time steps.See the description of shapefile_mask_datafor important information.shapefiles_21.ncl: This example demonstrates two ways to mask data based on shapefileoutlines of interest.The first plot shows the original data with no masking. The purplelines represent the shapefile areas that we want to apply the maskingto.The second plot shows the result of usingthe shapefile_mask_data function(in shapefile_utils.ncl) tomask the data based on an array of given areas to mask. (You can usestrings or numeric types for the areas you want to mask, but the arrayvalues should match whatever type the variable is on the shapefile.)You will see the blockiness near the shapefile outlines, due to thefact that NCL will not contour data that is bordered by missingvalues. The black dots represent the WRF data that was kept by thisfunction, and the red dots represent the data that was set to missing.The third plot shows how to mask the data by filling the undesiredareas in white. This is a bit of a pain, because you have to makesure you fill in all the undesired areas, and set various "draw order"resources to make sure things are drawn in the correct order. Thismethod is only useful when doing graphics, and not if you need to maskthe data for calculation purposes later.Note that the third plot shows some white areas inside the purple shapefileoutlines. This is due to the fact that the script set mpOceanFillColor to "white", whichcauses some shapefile areas to be covered up by the white ocean areas.shapefiles_22.ncl: Thisexample shows how to use two different shapefiles to mask data. Oneshapefile contains a single outline of the Caspian Sea, and the othercontains outlines of isles inside the Caspian Sea.If you want to keep the data inside the Caspian Sea, but not insidethe isles, then you need to mask the data twice: first by setting the"keep" attribute to True to keep the values inside the single outline,and second by setting "keep" to False to throw away values in the isleareas: opt = True opt@keep = True ; Keep values inside this shape data_mask_casp = shapefile_mask_data(data,caspian_shape_name,opt) opt@keep = False data_mask_isles = shapefile_mask_data(data_mask_casp,isles_shape_name,opt)Image #1: The original data before maskingImage #2: The data after masking against the Caspian Sea outline. Note that the little islesstill have data in them.Image #3: The data after masking the previously-masked data against the isle outlines. Notethat the isles no longer have data in them.Images #4-5: Zoomed in versions of images #2-3.shapefiles_23.ncl: This scriptshows how to use shapefile_mask_data to create a mask array formasking a WRF output variable against an outline in a shapefile. Itthen uses this mask array to mask select variables on the file.Creating a mask array from a shapefile can take a long time if youhave a big shapefile or lots of points to check. Once you have themask array, use it with the mask function to maskother variables of the same size. This goes MUCH faster thanusing shapefile_mask_data. You can also write thismask array to a NetCDF file for use by other scripts.Optionally, this script writes the masked variables to a new NetCDFfile, and/or plots the original data versus the masked data forcomparison.This script uses a Mississippi River Basin shapefile(mrb.xxx) for demonstrativepurposes. You should use your own shapefile data, but you can downloadthe mrb files if needed.shapefiles_24.ncl: This scriptuses a shapefile outline to mask some data, and then calculates theapproximate area (in km^2) of the valid data that was retainedinside the shapefile area.It does this by generating a series of quadrilaterals around everylat/lon point of valid data, and calling gc_qarea toget a sum of the area of all of them. This is only a roughapproximation of the actual area of the data, which you can see by thepink squares in the third image.For comparison purposes, the area (in km^2) of shapefile outline wasalso calculated, using area_poly_sphere.shapefiles_25.ncl: Read hourly METAR precipitation values for August 2018 at station locations.Calculate monthly total precipitation at each station. Use Barnes iterative correction (obj_anl_ic) to regrid thedata onto a user defined grid. Use a shape file to create grid mask to eliminatevalues outside the coterminus USA. Plot as 'raster' to illustrate the underlyinggrid resolution. The United States Census cartographic boundary shapefile, cb_2018_us_nation_5m.zip, was downloaded from: -files/2018/geo/carto-boundary-file.htmlshapefiles_26.ncl: Similar to shapefiles_25 except values from four different sources(METAR, HRR, GOES, Stage4) are used. The mask file created and saved in shapefiles_25 is used to mask values outside the coterminus USA. Plot as a panel.maponly_30.ncl: This scriptshows how to use the mpMaskOutlineSpecifiersto remove the outlines of India, while keeping outlines from surroundingcountries. In the third plot, meteorological subdivisions of Indiawere added using a shapefile downloaded off the web.
The Plot Against America Download.zipl
2ff7e9595c
Comments