OKM XDC IDPC
Oklahoma Mesonet
External Data Center
Instrument Data Processing Circuit
12/12/96
This data stream is currently collected on a daily basis. The
raw files are converted to NetCDF via the sgp05okm ingest. The raw
and NetCDF files are sent to the archive. For updated status check
out the XDC's SGP External
Surface Data Status Table .
Site: Oklahoma Mesonet
Contacts:
Input: none
Output: raw data files
Description:
The Oklahoma Mesonet (sponsored by the University of Oklahoma and
Oklahoma State) provide 05 and 15 minute surface data daily. There
are 111 automated observing stations located throughout the state.
Their names, latitudes and longitudes can be found at the following
URL address under "Site Description":
http://geowww.gc n.uoknor.edu/WWW/Mesonet/Mesonet.html
More information can be found in the Release Notice for Oklahoma
Mesonet Data at
http://www.xdc.arm.gov/docs/data_info/sgp/release_forms/sgpokm.release.html
Ftp script: ftpgetok05.csh - picks up files daily with a 1
day delay
Input: Raw data files, ascii format, 288 files per day
Output: One data file, ascii format, file name
sgp05okmX1.00.yymmdd.hhmmss
Status Reporting: Reports to user operx when number of files
received is less than 288.
Description:
Input: raw data
Ouput: Collection files are merged into 1 file per day
Activated by: Collections (see above)
Description:
File Check
Input: Current Date
Ouput: Daily File Check with a list of all files received in
the past 24 hours versus what is expected
Activated by: crontab scripts
Frequency: daily
Description: The files check is done on a nightly basis
to check expected number and sizes of incoming raw data files and
ingested files. This check is done nightly and is sent to operx via
e-mail and is updated on the Web.
QA in Collection
Input:
Ouput: e-mail sent to operx if number of small files does
not equal 288.
Activated by: collection script ∓ check script
Frequency: daily
Description: Message is sent as to the number of files
received if not 288. If the number of small files are less than
expected it may indicated that there is a problem with the the ftp
connection, or that a machine is down. Another script is also run
to check for more detail on missing data if a file is too small.
The information generated includes missing stations and missing
hours.
Input: sgp05okmX1.00 raw ascii data files
Ouput: sgp05okmX1.a1 NetCDF files
Activated by: ingest_xdata.csh script and sgp05okm ingest
Frequency: daily
Status Reporting: Messages are sent to the Zebra Logfile
created by the Eventer logger. The logfile is sent to operx
Description:
Input: Output:
Activated by:
Status Reporting:
Description : Currently only checks on data done by the
ingest are to flag missing with a value of -99999 and invalid data
with a value of -9999 within the NetCDf file.
Input: sgp05okmX1.00 and sgp05okmX1.a1
Output: none
Activated By: transfer.pl script which move files to the
transfer directory and site transfer which sends the data. Both
scripts are run via crontab on a daily basis.
Status Reporting: error messages sent via e-mail to operx
and root
Description :
Packing and shipping at the XDC basically is the process of
specifying the appropriate files to be sent on a daily basis to the
Archive and the Experiment Center.
Input: NetCDF Data
Output: NetCDF Data
Description :
See release notice at
ISM release NOTICE . Status of ISM is located in the XDC's SGP External Surface Data
Status Table
About this Document
The Oklahoma Mesonet IDPC was developed by Laurie Benedict and Alice Cialella
This document was written by Laurie Benedict on 12/12/96
For Questions/Comments you can e-mail the author at benedic@bnl.gov