[an error occurred while processing this directive]
[an error occurred while processing this directive]
WRF Portal: The Graphical Front End to WRF
CIRA magazine, Volume 25, Spring 2006
Jeff S. Smith
What is WRF?
WRF is the mesoscale Weather Research and Forecasting model designed for both operational forecasters and atmospheric researchers. It features multiple dynamical cores, a 3-dimensional variational (3DVAR) data assimilation system, and an extensible software architecture that supports parallel computing. WRF is available on the web at http://www.wrf-model.org and is currently in operational use at NCEP (National Centers for Environmental Prediction).
What is WRF Portal?
WRF portal is a NOAA/Earth System Research Laboratory/Global Systems Division (formerly FSL) funded project that is basically a Java webstart GUI front end for configuring and running both WRF cores: ARW and NMM. It simplifies the configuring and running of WRF models, the selection/localization of your domain, and the visualization of your model's output. WRF Portal runs on all platforms and can be accessed from a web page. It also includes a data locator for finding your data, a job manager that manages and executes the jobs, and a visualization tool for visualizing your WRF output. It does not include WRF itself--the WRF software must already be installed on a computer accessible by WRF Portal before you can use the GUI.
Why Would WRF Users Want To Use It?
WRF Portal saves user's time by automating tedious tasks and providing time saving features such as a domain wizard for the visual selection and preparation of a domain, a 'diff' tool for comparing different model and run configurations, a data locator for finding your input data, a robust job manager for running various WRF tasks, and a progress monitor for tracking the progress of your runs in a central monitor window. WRF Portal even allows you to graphically visualize your model output, and it stores all its information in a database so you can easily search and retrieve your information without the tedium of hunting through a myriad of files in numerous disk directories.
Running WRF without a GUI is Complicated
Configuring and running WRF is a very complicated process. The typical workflow entails editing multiple configuration files (namelists), setting environment variables, and running executables in the correct order. All of these files are scattered across many different directories. Moreover, editing the configuration files by hand in a text editor such as vi is tedious and mistake prone.Selecting a domain (geographic region) without visual feedback can be a frustrating exercise in trial and error. Here is a typical WRF work flow:
A Few Workflow Comparisons
WRF from command line
WRF using WRF Portal
Lats/Lons are looked up in an atlas, grid points are computed with a calculator, user manually types all this data into namelist text files
User draws a box around a region of the Earth on the screen, grid points are automatically calculated with minimal user input, namelists are automatically written
Namelists and scripts are scattered in multiple directories, manually edited in vi
Namelists and scripts are stored in database and edited in WRF Portal visual editors
New runs involve copying lots of files, editing them with vi, set environment vars, launching scripts manually.
New runs are launched from the New Run Config window. Most of the info is already filled out for you.
Runs are monitored by tailing certain log files in specific directories. Requires detailed knowledge of WRF.
Runs are monitored from a window which lists all tasks, how long they've been running and the est. time to finish
Comparing two different models involves viewing multiple namelists files for each model and hunting for the differences.
Comparing two different models is done from a window that highlights all differences between the models.
Finding an old run involves hunting through directories on different computers
Find an old run is done by typing in search criteria into a window
Runs are canceled by killing jobs from the command line
Runs are canceled with a mouse click
Visualization is done with external tools
Visualization is done within WRF Portal with a mouse click
WRF Portal Architecture
WRF Portal is written in Java so it runs on virtually all platforms (e.g. Linux, Windows, Unix) and can be launched from a web page.It stores most of the user's work and information in a mySQL database. Each site can have its own database instance, or multiple sites can share a single mySQL database. For example, ESRL might have its own database for ESRL WRF users, while NCAR might have a separate database for its users. When the user first starts up WRF Portal, he/she is prompted for a username and password; these credentials establish the user's identity for the duration of their WRF Portal session. Any configurations the user creates or runs and executes are associated with the user's username in the database, so, for example, when the user looks up a previous WRF run, the user only sees his/her own runs, not the runs of every other user in the system. By storing his/her data in a database instead of in multiple disk files, we've made it easy to back up the data (archive it) and query it based on multiple criteria. Furthermore, we've eliminated the clutter and complication of the user maintaining many files in multiple directories on potentially multiple file systems, and we've enabled the user to run WRF Portal and access most of his/her data from any computer, mounted to any file system.
After creating a model configuration in WRF Portal, the user can choose to run it on any computer previously configured in the WRF Portal software (and database). He/she can run one or more tasks in any order he/she chooses. For example, a user may choose to run GRIBPREP and HINTERP. When he/she selects the RUN option in WRF Portal, one of two things happens. If he/she has chosen to run it on the local computer, WRF Portal executes the tasks on the local computer in the order the tasks appeared in the run configuration workflow. If he/she has chosen to run it on a remote computer (perhaps a supercomputer), he/she is prompted for his/her SSH credentials. WRF Portal then establishes an SSH2 connection to the remote computer and executes all the tasks via this connection. Users also have the option of running WRF Portal 'locally' on a remote computer if they first SSH to the remote computer and then start up WRF Portal in an X Windows session.
The following diagram illustrates the architecture of the WRF Portal system.
WRF Portal Security
A WRF Portal user logs in with credentials that grants him/her the limited database access required to run WRF Portal. The database does not store any sensitive information such as usernames or passwords to other computers. All database traffic is transmitted via TCP/IP to port 3306 (or another port designated by the administrator), and the password is encrypted during all transfers. We are also considering the possibility of further strengthening database security by using SSH Port forwarding to encrypt all incoming and outgoing data.
WRF Portal uses SSH2 for all connections to remote computers, providing the same high level of security a WRF user would have if he/she SSH'd to a remote computer and then ran WRF from the command line.
WRF Portal Job Management
There are two ways to execute jobs (such as GRIBPREP, REAL, or WRF) with WRF Portal. One way is to use the Ruby Workflow Manager developed by Chris Harrop. This workflow manager must be installed and runs independently on the remote computer. It provides robust job management including job restart on failure, job monitoring, and running multiple tasks simultaneously.
The other way to execute jobs is to use the somewhat simpler (and less robust) internal Java Workflow Manager. This workflow manager is built into WRF Portal and thus runs on your desktop computer and can only monitor jobs while WRF Portal itself is running. If you shut down WRF Portal, the workflow manager is also shut down and hence cannot monitor and restart jobs that fail. The next time you restart WRF Portal, however, the Java Workflow Manager resumes monitoring and starting jobs from wherever it left off.
In summary, WRF Portal simplifies the configuration of domains, models, and runs, while providing many useful tools to make the WRF user's life easier. The software is still under development at ESRL and we are working with NCAR to make the software fully compatible with the upcoming release of WRF 3.0. We anticipate WRF 3.0 being released sometime in the latter part of this summer (2006), with WRF Portal's release following shortly thereafter.
WRF Portal Screenshots
This domain wizard enables a user to visually select and then localize a domain. The domain wizard currently supports the Lambert Conformal, Polar Stereographic, Mercator, and Long-Lat map projections, and will support nested domains.
This Run Configuration window is where a user selects the model, computer, tasks, and dates to run their WRF model.
The Monitor Runs window enables a user to follow the progress of his/her runs. For example, the user can find out how much longer a job has to run or whether it has errored out. The user can also view the output files.
The integrated visualization tool (based on IDV) enables users to visualize the output of their WRF runs. Users can also use this tool to visualize other meteorological data files.