AGU 2018 Demo

This notebook shows how to use birdy’s high-level interface to WPS processes.

Here we access a test server called Emu offering a dozen or so dummy processes.

The shell interface

[1]:

%%bash
export WPS_SERVICE="http://localhost:5000/wps?Service=WPS&Request=GetCapabilities&Version=1.0.0"
birdy -h

Usage: birdy [OPTIONS] COMMAND [ARGS]...

  Birdy is a command line client for Web Processing Services.

  Documentation is available on readthedocs:
  http://birdy.readthedocs.org/en/latest/

Options:
  --version             Show the version and exit.
  --cert TEXT           Client side certificate containing both certificate
                        and private key.

  -S, --send            Send client side certificate to WPS. Default: false
  -s, --sync            Execute process in sync mode. Default: async mode.
  -t, --token TEXT      Token to access the WPS service.
  -l, --language TEXT   Set the accepted language to send to the WPS service.
  -L, --show-languages  Show a list of accepted languages for the WPS service.
  -h, --help            Show this message and exit.

Commands:
  ultimate_question         Answer to the ultimate question: This process...
  sleep                     Sleep Process: Testing a long running process,...
  nap                       Afternoon Nap (supports sync calls only): This...
  bbox                      Bounding box in- and out: Give bounding box,...
  hello                     Say Hello: Just says a friendly Hello.Returns a...
  dummyprocess              Dummy Process: DummyProcess to check the WPS...
  wordcounter               Word Counter: Counts words in a given text.
  chomsky                   Chomsky text generator: Generates a random...
  inout                     In and Out: Testing all WPS input and output...
  binaryoperatorfornumbers  Binary Operator for Numbers: Performs operation...
  show_error                Show a WPS Error: This process will fail...
  multiple_outputs          Multiple Outputs: Produces multiple files and...
  esgf_demo                 ESGF Demo: Shows how to use WPS metadata for...
  output_formats            Return different output formats.: Dummy process...
  poly_centroid             Approximate centroid of a polygon.: Return the...
  ncmeta                    Return NetCDF Metadata: Return metadata from a...
  non.py-id                 Dummy process including non-pythonic...
  simple_dry_run            Simple Dry Run: A dummy download as simple...
  ncml                      Test NcML THREDDS capability: Return links to
                            an...

  translation               Translated process: Process including...

[2]:

%%bash
export WPS_SERVICE="http://localhost:5000/wps?Service=WPS&Request=GetCapabilities&Version=1.0.0"
birdy hello -h

Usage: birdy hello [OPTIONS]

  Say Hello: Just says a friendly Hello.Returns a literal string output with
  Hello plus the inputed name.

Options:
  --version                 Show the version and exit.
  --name TEXT               Your name
  --output_formats TEXT...  Modify output format (optional). Takes three
                            arguments, output name, as_reference (True, False,
                            or None for process default), and mimetype(None
                            for process default).

  -h, --help                Show this message and exit.

[3]:

%%bash
export WPS_SERVICE="http://localhost:5000/wps?Service=WPS&Request=GetCapabilities&Version=1.0.0"
birdy hello --name stranger

Output:
output=['Hello stranger']

The python interface

The WPSClient function creates a mock python module whose functions actually call a remote WPS process. The docstring and signature of the function are dynamically created from the remote’s process description. If you type wps. and then press Tab, you should see a drop-down list of available processes. Simply call help on each process of type ? after the process to print the docstring for that process.

[4]:

from birdy import WPSClient
url = "http://localhost:5000/wps?Service=WPS&Request=GetCapabilities&Version=1.0.0"
wps = WPSClient(url, verify=False)
help(wps.binaryoperatorfornumbers)

Help on method binaryoperatorfornumbers in module birdy.client.base:

binaryoperatorfornumbers(inputa=2.0, inputb=3.0, operator='add', output_formats=None) method of birdy.client.base.WPSClient instance
    Performs operation on two numbers and returns the answer. This example process is taken from Climate4Impact.

    Parameters
    ----------
    inputa : float
        Enter Input 1
    inputb : float
        Enter Input 2
    operator : {'add', 'substract', 'divide', 'multiply'}string
        Choose a binary Operator

    Returns
    -------
    output : float
        Binary operator result

Type wps. and the press Tab, you should see a drop-down list of available processes.

[5]:

# wps.

Process execution

Processes are executed by calling the function. Each process instantaneoulsy returns a WPSExecute object. The actual output values of the process are obtained by calling the get method. This get method returns a namedtuple storing the process outputs as native python objects.

[6]:

resp = wps.binaryoperatorfornumbers(1, 2, operator='add')
print(resp)
resp.get()

<birdy.client.utils.WPSExecution object at 0x108237d30>

[6]:

binaryoperatorfornumbersResponse(
    output=3.0
)

For instance, the inout function returns a wide variety of data types (float, integers, dates, etc) all of which are converted into a corresponding python type.

[7]:

wps.inout().get()

[7]:

inoutResponse(
    string='This is just a string',
    int=7,
    float=3.14,
    boolean=True,
    angle=90.0,
    time=datetime.time(12, 0),
    date=datetime.date(2012, 5, 1),
    datetime=datetime.datetime(2016, 9, 2, 12, 0, tzinfo=tzutc()),
    string_choice='scissor',
    string_multiple_choice='gentle albatros',
    int_range=1,
    any_value='any value',
    ref_value='Scotland',
    text='http://localhost:5000/outputs/e7700e9c-559c-11eb-bcba-784f435e8862/input.txt',
    dataset='http://localhost:5000/outputs/e7700e9c-559c-11eb-bcba-784f435e8862/input_pd0bgv88.txt',
    bbox=BoundingBox(minx='0.0', miny='0.0', maxx='10.0', maxy='10.0', crs=Crs(id='epsg:4326', naming_authority=None, category=None, type=None, authority='EPSG', version=None, code=4326, axisorder='yx', encoding='code'), dimensions=2)
)

Retrieving outputs by references

For ComplexData objects, WPS servers often return a reference to the output (an http link) instead of the actual data. This is useful if that output is to serve as an input to another process, so as to avoid passing back and forth large files for nothing.

With birdy, the outputs are by default return values are the references themselves, but it’s also possible to download these references in the background and convert them into python objects. To trigger this automatic conversion, set asobj to True when calling the get method. In the example below, we’re using a dummy process called output_formats, whose first output is a netCDF file, and second output is a json file. With asobj=True, the netCDF file is opened and returned as a netcdf4.Dataset instance, and the json file into a dictionary.

[8]:

# NBVAL_SKIP
# This cell is failing due to an unautheticated SSL certificate
out = wps.output_formats()
nc, json = out.get()
print(out.get())
ds, json = out.get(asobj=True)
print(json)
ds

output_formatsResponse(
    netcdf='http://localhost:5000/outputs/e78722ee-559c-11eb-8bc2-784f435e8862/dummy.nc',
    json='http://localhost:5000/outputs/e78722ee-559c-11eb-8bc2-784f435e8862/dummy.json'
)
{'testing': [1, 2]}

[8]:

<xarray.Dataset>
Dimensions:  (time: 1)
Coordinates:
  * time     (time) float64 42.0
Data variables:
    *empty*
Attributes:
    title:    Test dataset

Progress bar

It’s possible to display a progress bar when calling a process. The interface to do so at the moment goes like this. Note that the cancel button does not do much here, as the WPS server does not support interruption requests.

[9]:

wps = WPSClient(
    'http://localhost:5000/wps',
    progress=True)
resp = wps.sleep()