mapper Package

mapper Package

abstractmapper Module

This module contains the abstract class for any mapper. A mapper is used to read the content of a storage (file). It is designed to allow the mapping of this content to a datamodel (also refered as a feature), i.e. a class from cerbere.datamodel package.

It can also be used independently, providing the user with a single API to access any data format.

copyright:Copyright 2013 Ifremer / Cersat.
license:Released under GPL v3 license, see license.
class cerbere.mapper.abstractmapper.AbstractMapper(url=None, urlseries=None, mode='r', **kwargs)[source]

Bases: object

An abstract class for any mapper.

This does not actually open the corresponding file. Explicit opening must be done by calling the open() method.

Parameters:
  • url (str) – full path to the file.
  • mode (enum) – access mode (READ_ONLY, WRITE_NEW, READ_WRITE)
check_geolocation_dimensions(dimensions)[source]

Return True if the requested dimensions exist.

Used for internal purpose and should not be called directly.

check_geolocation_fields(fields)[source]

Return True if the requested geolocation fields exist.

Used for internal purpose and should not be called directly.

close()[source]

Close handler on storage

create_dim(dimname, size=None)[source]

Add a new dimension.

Parameters:
  • dimname (str) – name of the dimension.
  • size (int) – size of the dimension (unlimited if None)
create_field(field, dim_translation=None)[source]

Creates a new field in the mapper.

Creates the field structure but don’t write yet its values array.

Parameters:field (Field) – the field to be created.

See also

write_field() for writing the values array.

classmethod exists(url)[source]

tests if url is an existing resource

get_basename()[source]
get_bbox()[source]

Returns the bounding box of the feature, as a tuple.

Returns:bbox expressed as (lonmin, latmin, lonmax, latmax)
Return type:tuple
get_collection_id()[source]

return the identifier of the product collection

get_creation_date()[source]

return the date the product was generated (NOT the file date!)

get_cycle_number()[source]

In the case of a satellite orbit file, returns the cycle number.

Returns:the cycle number
Return type:int
get_dimensions(fieldname=None)[source]

Return the dimension’s standard names of a file or a field in the file.

Parameters:fieldname (str) – the name of the field from which to get the dimensions. For a geolocation field, use the cerbere standard name (time, lat, lon), though native field name will work too.
Returns:the standard dimensions of the field or file.
Return type:tuple<str>
get_dimsize(dimname)[source]

Return the size of a dimension.

Parameters:dimname (str) – name of the dimension.
Returns:size of the dimension.
Return type:int
get_end_time()[source]

Returns the maximum date of the file temporal coverage.

Returns:end time of the data in file.
Return type:datetime
get_field_handler(fieldname)[source]
get_fieldnames()[source]

Returns the list of geophysical fields stored for the feature.

The geolocation field names are excluded from this list.

Returns:list of field names
Return type:list<string>
get_full_dimensions(fieldname)[source]

Return the dimension names and sizes of a field.

Parameters:fieldname (str) – name of the field
Returns:ordered dictionary where keys are the dimension names and values are their respective size
Return type:OrderedDict
get_geolocation_field(fieldname)[source]

Return the equivalent field name in the file format for a standard geolocation field (lat, lon, time, z).

Used for internal purpose and should not be called directly.

Parameters:fieldname (str) – name of the standard geolocation field (lat, lon or time)
Returns:name of the corresponding field in the native file format. Returns None if no matching is found
Return type:str
get_handler()[source]

Return the handler to the physical file or resource from which data are to be read.

In case the mapper was initialised with a URLSeries, the handler to the reference time URL of the series is returned.

get_matching_dimname(dimname)[source]

Return the equivalent name in the native format for a standard dimension.

This is a translation of the standard names to native ones. It is used for internal purpose only and should not be called directly.

The standard dimension names are:

To be derived when creating an inherited data mapper class. This is mandatory for geolocation dimensions which must be standard.

Parameters:dimname (str) – standard dimension name.
Returns:return the native name for the dimension. Return dimname if the input dimension has no standard name.
Return type:str

See also

see get_standard_dimname() for the reverse operation

get_naming_authority()[source]
get_orbit_number()[source]

In the case of a satellite orbit file, returns the orbit number.

Returns:the orbit number
Return type:int
get_product_version()[source]

return the product version

get_size()[source]

return the product file size, in octets

get_spatial_resolution_in_deg()[source]

Returns the average spatial resolution in degrees.

Returns:resolution, in degrees.
Return type:float
get_standard_dimname(dimname)[source]

Returns the equivalent standard dimension name for a dimension in the native format.

This is a translation of the native names to standard ones. It is used for internal purpose and should not be called directly.

To be derived when creating an inherited data mapper class. This is mandatory for geolocation dimensions which must be standard.

Parameters:dimname (string) – native dimension name
Returns:the (translated) standard name for the dimension. Return dimname if the input dimension has no standard name.
Return type:str

See also

see get_matching_dimname() for the reverse operation

get_start_time()[source]

Returns the minimum date of the file temporal coverage.

Returns:start time of the data in file.
Return type:datetime
get_url()[source]
has_geolocation_field(fieldname)[source]

Return True if the field fieldname exists.

is_opened()[source]
is_readonly()[source]

Return True if the storage is opened in write mode

is_writable()[source]

Return True if the storage is opened in write mode

open(view=None, datamodel=None, datamodel_geolocation_dims=None)[source]

Open the file (or any other type of storage)

Parameters:
  • view (dict, optional) –

    a dictionary where keys are dimension names and values are slices. A view can be set on a file, meaning that only the subset defined by this view will be accessible. This view is expressed as any subset (see get_values()). For example:

    view = {‘time’:slice(0,0), ‘lat’:slice(200,300), ‘lon’:slice(200,300)}

  • datamodel (str) – type of feature read or written. Internal argument only used by the classes from datamodel package. Can be ‘Grid’, ‘Swath’, etc...
  • datamodel_geolocation_dims (list, optional) – list of the name of the geolocation dimensions defining the data model to be read in the file. Optional argument, only used by the datamodel classes, in case the mapper class can store different types of data models.
Returns:

a handler on the opened file

read_field(fieldname)[source]

Return the cerbere.field.Field object corresponding to the requested fieldname.

The cerbere.field.Field class contains all the metadata describing a field (equivalent to a variable in netCDF).

Parameters:fieldname (str) – name of the field
Returns:the corresponding field object
Return type:cerbere.field.Field
read_field_attributes(fieldname)[source]

Return the specific attributes of a field.

Parameters:fieldname (str) – name of the field.
Returns:a dictionary where keys are the attribute names.
Return type:dict<string, string or number or datetime>
read_fillvalue(fieldname)[source]

Read the fill value of a field.

Parameters:fieldname (str) – name of the field.
Returns:fill value of the field. The type is the as the type of the data in the field.
Return type:number or char or str
read_global_attribute(name)[source]

Returns the value of a global attribute.

Parameters:name (str) – name of the global attribute.
Returns:value of the corresponding attribute.
Return type:str, number or datetime
read_global_attributes()[source]

Returns the names of the global attributes.

Returns:the list of the attribute names.
Return type:list<str>
read_values(fieldname, slices=None, **kwargs)[source]

Read the data of a field.

Parameters:
  • fieldname (str) – name of the field which to read the data from
  • slices (list of slice, optional) – list of slices for the field if subsetting is requested. A slice must then be provided for each field dimension. The slices are relative to the opened view (see :func:open) if a view was set when opening the file.
Returns:

array of data read. Array type is the same as the storage type.

Return type:

MaskedArray

sync()[source]

force physical writing of the content on disk.

write_field(fieldname)[source]

Writes the field data on disk.

Parameters:fieldname (str) – name of the field to write.
write_global_attributes(attrs)[source]

Write the global attributes of the file.

Parameters:attrs (dict<string, string or number or datetime>) – a dictionary containing the attributes names and values to be written.
exception cerbere.mapper.abstractmapper.CorruptFileException[source]

Bases: exceptions.Exception

Exception class for corrupted or unreadable files.

class cerbere.mapper.abstractmapper.FieldHandler(mapper, fieldname, index=None, status=1)[source]

Bases: object

Storage pointer to the data of a particular field of one single feature (files can store several features).

is_saved()[source]
set_saved()[source]
set_unsaved()[source]

ascatifrncfile Module

ecmwfncfile Module

ghrsstncfile Module

cerbere.mapper.ghrsstncfile

Mapper class for GHRSST netcdf files

copyright:Copyright 2013 Ifremer / Cersat.
license:Released under GPL v3 license, see license.
class cerbere.mapper.ghrsstncfile.GHRSSTNCFile(url=None, **kwargs)[source]

Bases: cerbere.mapper.ncfile.NCFile

get_collection_id()[source]

return the identifier of the product collection

get_cycle_number()[source]
get_dimensions(fieldname=None)[source]
get_end_time()[source]

Returns the maximum date of the file temporal coverage

get_fieldnames()[source]

Returns the list of geophysical fields stored for the feature

Excludes the geolocation fields in the returned list

get_matching_dimname(dimname)[source]

Return the equivalent name in the native format for a standard dimension.

This is a translation of the standard names to native ones. It is used for internal purpose only and should not be called directly.

The standard dimension names are:

To be derived when creating an inherited data mapper class. This is mandatory for geolocation dimensions which must be standard.

Parameters:dimname (str) – standard dimension name.
Returns:return the native name for the dimension. Return dimname if the input dimension has no standard name.
Return type:str

See also

see get_standard_dimname() for the reverse operation

get_orbit_number()[source]
get_standard_dimname(dimname)[source]

Returns the equivalent standard dimension name for a dimension in the native format.

This is a translation of the native names to standard ones. It is used for internal purpose and should not be called directly.

To be derived when creating an inherited data mapper class. This is mandatory for geolocation dimensions which must be standard.

Parameters:dimname (string) – native dimension name
Returns:the (translated) standard name for the dimension. Return dimname if the input dimension has no standard name.
Return type:str

See also

see get_matching_dimname() for the reverse operation

get_start_time()[source]

Returns the minimum date of the file temporal coverage

is_l2()[source]

Return True if the product is a L2P

open(view=None, datamodel=None, datamodel_geolocation_dims=None)[source]

Open the netCDF file

Parameters:
  • view (dict, optional) –

    a dictionary where keys are dimension names and values are slices. A view can be set on a file, meaning that only the subset defined by this view will be accessible. This view is expressed as any subset (see get_values()). For example:

    view = {‘time’:slice(0,0), ‘lat’:slice(200,300), ‘lon’:slice(200,300)}

  • datamodel (str) – type of feature read or written. Internal argument only used by the classes from datamodel package. Can be ‘Grid’, ‘Swath’, etc...
  • datamodel_geolocation_dims (list, optional) – list of the name of the geolocation dimensions defining the data model to be read in the file. Optional argument, only used by the datamodel classes, in case the mapper class can store different types of data models.
Returns:

an handler on the opened file

read_field(fieldname)[source]

Return the field, without its values.

Actual values can be retrieved with read_values() method.

read_times(slices=None)[source]

Read time values of a file or file series.

Takes into account the fact that time information is splitted in two netcdf variables for a GHRSST file (time and sst_dtime).

read_values(fieldname, slices=None)[source]

Read the data of a field.

Parameters:
  • fieldname (str) – name of the field which to read the data from
  • slices (list of slice, optional) – list of slices for the field if subsetting is requested. A slice must then be provided for each field dimension. The slices are relative to the opened view (see :func:open) if a view was set when opening the file.
Returns:

array of data read. Array type is the same as the storage type.

Return type:

MaskedArray

gribfile Module

Mapper classs for grib format

copyright:Copyright 2013 Ifremer / Cersat.
license:Released under GPL v3 license, see license.
class cerbere.mapper.gribfile.GribFile(url=None, mode='r', **kwargs)[source]

Bases: cerbere.mapper.abstractmapper.AbstractMapper

Mapper class to read grib files

Initialize a Grib file mapper

close()[source]

Close file

create_dim(dimname, size=None)[source]
create_field(field, dim_translation=None)[source]
get_bbox()[source]

returns the bounding box of the feature

Return: tuple (lonmin, latmin, lonmax, latmax)

get_dimensions(fieldname=None)[source]
get_end_time()[source]
get_fieldnames()[source]
get_geolocation_field(fieldname)[source]
get_matching_dimname(dimname)[source]

Return the equivalent name in the current data mapper for a standard feature dimension, or None if not found.

To be derived when creating an inherited data mapper class.

get_product_version()[source]

return the product version

get_standard_dimname(dimname)[source]

Returns the equivalent standard dimension name for a dimension in the native format.

This is a translation of the native names to standard ones. It is used for internal purpose and should not be called directly.

To be derived when creating an inherited data mapper class. This is mandatory for geolocation dimensions which must be standard.

Parameters:dimname (string) – native dimension name
Returns:the (translated) standard name for the dimension. Return dimname if the input dimension has no standard name.
Return type:str

See: see get_matching_dimname() for the reverse operation

get_start_time()[source]

Returns the minimum date of the file temporal coverage

open(view=None, datamodel=None, datamodel_geolocation_dims=None)[source]

Open the file (or any other type of storage)

Parameters:
  • view (dict, optional) –

    a dictionary where keys are dimension names and values are slices. A view can be set on a file, meaning that only the subset defined by this view will be accessible. This view is expressed as any subset (see get_values()). For example:

    view = {‘time’:slice(0,0), ‘lat’:slice(200,300), ‘lon’:slice(200,300)}

  • datamodel (str) – type of feature read or written. Internal argument only used by the classes from :package:`cerbere.datamodel` package. Can be ‘Grid’, ‘Swath’, etc...
  • datamodel_geolocation_dims (list, optional) – list of the name of the geolocation dimensions defining the data model to be read in the file. Optional argument, only used by the datamodel classes, in case the mapper class can store different types of data models.
read_field(fieldname)[source]

Return the field, without its values.

Actual values can be retrieved with read_values() method.

read_field_attributes(fieldname)[source]
read_fillvalue(fieldname)[source]
read_global_attribute(name)[source]
read_global_attributes()[source]
read_values(fieldname, slices=None)[source]
write_field(fieldname)[source]
write_global_attributes(attrs)[source]

write the storage (file) global attributes

hdffile Module

nasaochdffile Module

ncfile Module

Class to read netCDF files using CF conventions.

copyright:Copyright 2013 Ifremer / Cersat.
license:Released under GPL v3 license, see license.
class cerbere.mapper.ncfile.NCFile(url=None, mode='r', ncformat='NETCDF4', geodim_matching=None, geofield_matching=None, is_reference=False, fillvalue=None, center_on_greenwhich=False, **kwargs)[source]

Bases: cerbere.mapper.abstractmapper.AbstractMapper

Initialize a CF compliant NetCDF file mapper

The file is not yet open (read mode) nor created (write mode), this is done explicitely by the caller when calling open()

Parameters:
  • ncformat (string) – specifies the required NetCDF format version: NETCDF3_CLASSIC, NETCDF4, NETCDF4_CLASSIC,....
  • geodim_matching (dict) – explicitly provides the matching between standard names for the geolocation dimensions (keys)and the corresponding native names (values) in the file. If not provided, the library will try to guess with respect to known conventions.
  • geofield_matching (dict) – explicitly provides the matching between standard names for the geolocation fields (keys)and the corresponding native names (values) in the file. If not provided, the library will try to guess with respect to known conventions.
  • is_reference (bool) – if set to True, then creates a virtual time field if missing otherwise any attempt to load the file would fail. Used for instance with climatology files with no time value provided. The time will be set to 0-1-1 by default (COARDS convention).
  • fillvalue (float) – additional fillvalue to use to mask some invalid data. Some fields don’t have a correct fill value set, and this value must be explicitly provided.
  • center_on_greenwhich (bool, optional) – if True, shift a grid so that it is centered on meridian 0 instead of meridian 180. Makes only sense for global grids in cylindrical projection. Default is False.
CONVENTIONS = 'CF-1.6, Unidata Observation Dataset v1.0'
STANDARD_ATTRIBUTES_VALUES = OrderedDict([('id', ''), ('naming_authority', 'fr.ifremer.cersat'), ('Metadata_Conventions', 'Unidata Dataset Discovery v1.0'), ('standard_name_vocabulary', 'NetCDF Climate and Forecast (CF) Metadata Convention'), ('institution', "Institut Francais de Recherche et d'Exploitation de la Mer/Centre de Recherche et d'Exploitation satellitaire"), ('institution_abbreviation', 'ifremer/cersat'), ('title', ''), ('summary', ''), ('cdm_data_type', ''), ('keywords', ''), ('keywords_vocabulary', 'NASA Global Change Master Directory (GCMD) Science Keywords'), ('scientific_project', ''), ('acknowledgement', ''), ('license', ''), ('format_version', ''), ('processing_software', 'Cersat/Cerbere 1.0'), ('product_version', ''), ('uuid', ''), ('processing_level', ''), ('history', ''), ('publisher_name', 'ifremer/cersat'), ('publisher_url', 'http://cersat.ifremer.fr'), ('publisher_email', 'cersat@ifremer.fr'), ('creator_name', ''), ('creator_url', ''), ('creator_email', ''), ('references', ''), ('metadata_link', ''), ('source', ''), ('source_version', ''), ('platform', ''), ('platform_type', ''), ('sensor', ''), ('sensor_type', ''), ('band', ''), ('spatial_resolution', ''), ('geospatial_lat_min', ''), ('geospatial_lat_max', ''), ('geospatial_lat_units', 'degrees'), ('geospatial_lat_resolution', ''), ('geospatial_lon_min', ''), ('geospatial_lon_max', ''), ('geospatial_lon_units', 'degrees'), ('geospatial_lon_resolution', ''), ('geospatial_vertical_min', ''), ('geospatial_vertical_max', ''), ('geospatial_vertical_units', 'meters above mean sea level'), ('geospatial_vertical_positive', 'up'), ('time_coverage_start', ''), ('time_coverage_end', ''), ('time_coverage_resolution', '')])
TIME_FORMAT = '%Y%m%dT%H%M%SZ'
close()[source]

Close file

create_dim(dimname, size=None)[source]

Add a new dimension.

Parameters:
  • dimname (str) – name of the dimension.
  • size (int) – size of the dimension (unlimited if None)
create_field(field, dim_translation=None, feature=None)[source]

Create netCDF variable for provided field

Parameters:
  • field (Field) – field to create.
  • dim_translation (dict, optional) – dictionary, alternative dimension names to be used.
  • feature (string, optional) – class name of the feature. Used to implement specific writing conventions for some features (ex: additional dummy time dimension for grids).
get_bbox()[source]

return the bounding box of the feature, as a tuple (lonmin, latmin, lonmax, latmax)

get_cycle_number()[source]
get_data_reference(time=None, proximity='exact')[source]

return a complete reference pointer to the requested data

time can be either a single time value or a time interval

get_dimensions(fieldname=None)[source]

Return the dimension’s standard names of a file or a field in the file.

Parameters:fieldname (str) – the name of the field from which to get the dimensions. For a geolocation field, use the cerbere standard name (time, lat, lon), though native field name will work too.
Returns:the standard dimensions of the field or file.
Return type:tuple<str>
get_dimsize(dimname)[source]

returns the size of a dimension

If a view was set when opening the file, the size of the view subset is returned.

Parameters:dimname (string) – name of the dimension for which the size is inquired. Can be provided as the native or standard dimension name.
Returns:size of the dimension.
Return type:int
get_end_time()[source]

Returns the maximum date of the file temporal coverage.

Returns:end time of the data in file.
Return type:datetime
get_fieldnames()[source]

Returns the list of geophysical fields stored for the feature

Excludes the geolocation fields in the returned list

get_full_dimensions(fieldname=None)[source]

return the name and size of a field’s dimensions as an ordered dictionary

Parameters:fieldname (string, optional) – name of the specific field (netCDF variable) for which the dimensions are requested. If not provided, all file dimensions are returned.
Returns:an ordered dict where keys are the dimension names and values are their size
Return type:OrderedDict
get_geolocation_field(fieldname)[source]

Return the equivalent field name in the file format for a standard geolocation field (lat, lon, time, z).

Used for internal purpose and should not be called directly.

Parameters:fieldname (str) – name of the standard geolocation field (lat, lon or time).
Returns:name of the corresponding field in the native file format. Returns None if no matching is found.
Return type:str
get_matching_dimname(geodimname)[source]

Return the equivalent name in the native format for a standard dimension.

This is a translation of the standard names to native ones. It is used for internal purpose only and should not be called directly.

The standard dimension names are:

To be derived when creating an inherited data mapper class. This is mandatory for geolocation dimensions which must be standard.

Parameters:dimname (str) – standard dimension name.
Returns:return the native name for the dimension. Return dimname if the input dimension has no standard name.
Return type:str

See also

see get_standard_dimname() for the reverse operation

classmethod get_netcdf_fillvalue(datatype)[source]
get_orbit_number()[source]
get_product_version()[source]

return the product version

get_spatial_resolution_in_deg()[source]

Returns the average spatial resolution in degrees

get_standard_dimname(geodimname)[source]

Returns the equivalent standard dimension name for a dimension in the native format.

This is a translation of the native names to standard ones. It is used for internal purpose and should not be called directly.

To be derived when creating an inherited data mapper class. This is mandatory for geolocation dimensions which must be standard.

Parameters:dimname (string) – native dimension name
Returns:the (translated) standard name for the dimension. Return dimname if the input dimension has no standard name.
Return type:str

See also

see get_matching_dimname() for the reverse operation

get_start_time()[source]

Returns the minimum date of the file temporal coverage.

Returns:start time of the data in file.
Return type:datetime
open(view=None, datamodel=None, datamodel_geolocation_dims=None)[source]

Open the netCDF file

Parameters:
  • view (dict, optional) –

    a dictionary where keys are dimension names and values are slices. A view can be set on a file, meaning that only the subset defined by this view will be accessible. This view is expressed as any subset (see get_values()). For example:

    view = {‘time’:slice(0,0), ‘lat’:slice(200,300), ‘lon’:slice(200,300)}

  • datamodel (str) – type of feature read or written. Internal argument only used by the classes from datamodel package. Can be ‘Grid’, ‘Swath’, etc...
  • datamodel_geolocation_dims (list, optional) – list of the name of the geolocation dimensions defining the data model to be read in the file. Optional argument, only used by the datamodel classes, in case the mapper class can store different types of data models.
Returns:

an handler on the opened file

read_field(fieldname)[source]

Return the cerbere.field.Field object corresponding to the requested fieldname.

The cerbere.field.Field class contains all the metadata describing a field (equivalent to a variable in netCDF).

Parameters:fieldname (str) – name of the field
Returns:the corresponding field object
Return type:cerbere.field.Field
read_field_attributes(fieldname)[source]

return the specific storage attributes of a field (such as _FillValue, scale_factor, add_offset,...)

Parameters:fieldname (str) – name of the field
Returns:a dictionary where keys are the attribute names.
Return type:Dict
read_fillvalue(fieldname)[source]
read_global_attribute(name)[source]
read_global_attributes()[source]

Returns the names of the global attributes.

Returns:the list of the attribute names.
Return type:list<str>
read_values(fieldname, slices=None)[source]

Read the data of a field.

Parameters:
  • fieldname (str) – name of the field which to read the data from
  • slices (tuple of slice, optional) – list of slices for the field if subsetting is requested. A slice must then be provided for each field dimension. The slices are relative to the opened view (see :func:open) if a view was set when opening the file.
Returns:

array of data read. Array type is the same as the storage type.

Return type:

MaskedArray

sync()[source]

force physical writing of the content on disk.

write_field(field)[source]

Write the data of a field in the netCDF file.

Parameters:field (Field) – the field which content has to be written in the file.

Warning

The field with all its metadata (attributes) must have been explicitely created by the mapper before, by calling the create_field() method of this class.

write_global_attributes(attrs)[source]

Write the global attributes of the file.

Includes a set of predefined standard attributes for CF compliant files. Default attributes will be filled automatically with default values unless you override them in attrs argument. You can also add additional attributes in attrs dictionary.

Parameters:attrs (dict<string, string or number or datetime>) – a dictionary containing the attributes names and values to be written.

qscathdffile Module

safeocnncfile Module

cerbere.mapper.safeocnncfile

Mapper class for ESA SAFE OCN NetCDF files (for Sentinel-1)

copyright:Copyright 2013 Ifremer / Cersat.
license:Released under GPL v3 license, see license.
class cerbere.mapper.safeocnncfile.SAFEOCNNCFile(url=None, mode='r', product='WIND', **kwargs)[source]

Bases: cerbere.mapper.ncfile.NCFile

Generic storage class for Sentinel-1 OCN files.

S-1 OCN files are actually netcdf files with specific conventions (they are not CF compliant) and additional metadata provided in joined XML files (in annotation folder)

Class constructor

Parameters:product (enum (WIND, WAVE, DOPPLER)) – OCN products contain different fiels that may have different resolution and therefore different attached lat/lon.

OCN products contain different fields at different resolution and therefore different attached lat/lon. Fields with different resolution are considered as different products and must be opened separately.

Reading product with all fields at the same resolution : >>> f = SAFEOCNNCFile(url=’asa-is2-ocn-hh-20051127t235932-20051128t002731-019580-000000-048.nc’,

product = WIND)

Reading a field at its native resolution : >>> f = SAFEOCNNCFile(url=’asa-is2-ocn-hh-20051127t235932-20051128t002731-019580-000000-048.nc’,

product = WAVE)
close()[source]
get_bbox()[source]
returns the bounding box of the feature, as a tuple
(lonmin, latmin, lonmax, latmax)
get_dimensions(fieldname=None)[source]

Return the dimension names of a file or a field in the file

Parameters:fieldname (str) – the field from which to get the dimension names. For a geolocation field, use the cerbere standard name (time, lat, lon), though native field name will work too.
Returns:the dimensions of the field or file.
Return type:tuple of strings
get_end_time()[source]

Returns the maximum date of the file temporal coverage

get_fieldnames()[source]

Returns the list of geophysical fields stored for the feature.

Fields that are not at the same resolution (and therefore have different lat/lon) than the opened product (ALL, WIND, WAVE, DOPPLER) are not returned. Only consistent fields with the product specified in open() call are returned.

Returns:list of field names
Return type:list<str>
get_geolocation_field(fieldname)[source]

return the equivalent field name in the file format for a standard geolocation field (lat, lon, time).

Used for internal purpose and should not be called directly.

Parameters:fieldname (str) – name of the standard geolocation field (lat, lon or time)
Return type:str or None
Returns:name of the corresponding field in the native file format. Returns None if no matching is found
get_matching_dimname(dimname)[source]

Return the equivalent name in the native format for a standard dimension.

This is a translation of the standard names to native ones. It is used for internal purpose only and should not be called directly.

To be derived when creating an inherited data mapper class. This is mandatory for geolocation dimensions which must be standard.

Parameters:dimname (str) – standard dimension name
Return type:str
Returns:return the native name for the dimension. return dimname if the input dimension has no standard name.

See

see get_standard_dimname() for the reverse operation

get_standard_dimname(dimname)[source]

Returns the equivalent standard dimension name for a dimension in the native format.

This is a translation of the native names to standard ones. It is used for internal purpose and should not be called directly.

To be derived when creating an inherited data mapper class. This is mandatory for geolocation dimensions which must be standard.

Parameters:dimname (str) – native dimension name
Return type:str
Returns:return the (translated) standard name for the dimension. return dimname if the input dimension has no standard name.

See

see get_matching_dimname() for the reverse operation

get_start_time()[source]

Returns the minimum date of the file temporal coverage

read_field(fieldname)[source]

Return the cerbere.field.Field object corresponding to the requested fieldname.

The cerbere.field.Field class contains all the metadata describing a field (equivalent to a variable in netCDF).

Parameters:fieldname (str) – name of the field
Returns:the corresponding field object
Return type:cerbere.field.Field
read_values(fieldname, slices=None)[source]

Read the values of a field.

slices is optional. When provided, give for each dimension the corresponding python slice object to subset this dimension. Only the dimensions to be subsetted need to be provided, by default the full dilmension length is read.

# extracting a subset of a field with slices
data = fd.read_values('owiWindSpeed', slices={'row':slice(10,20), 'cell':slice(30, 40)})
Parameters:
  • fieldname (str) – name of the field
  • slices (Dict<str, slice>) – dimensions slices, when reading a subset only for some dimensions

mod:safeolfile Module

mod:safeslfile Module

safegeotifffile Module

cerbere.mapper.safegeotifffile

Mapper class for ESA SAFE GeoTiff files (for Sentinel-1)

copyright:Copyright 2013 Ifremer / Cersat.
license:Released under GPL v3 license, see license.
class cerbere.mapper.safegeotifffile.SAFEGeoTiffFile(url=None, mode='r', **kwargs)[source]

Bases: cerbere.mapper.abstractmapper.AbstractMapper

Generic storage class for Sentinel GeoTiff files.

S-1 GeoTiff files are actually provided with additional metadata to be read in joined XML files (in annotation folder).

close()[source]

Close SAFE GeoTiff file.

create_dim(dimname, size=None)[source]
create_field(field, dim_translation=None)[source]
get_bbox()[source]

Return the bounding box of the feature, as a tuple (lonmin, latmin, lonmax, latmax).

get_dimensions(fieldname=None)[source]

Return the dimension names of a file or a field in the file.

Parameters:fieldname (str) – the field from which to get the dimension names. For a geolocation field, use the cerbere standard name (time, lat, lon), though native field name will work too.
Returns:the dimensions of the field or file.
Return type:tuple of strings
get_dimsize(dimname)[source]
get_end_time()[source]

Return the maximum date of the file temporal coverage.

get_fieldnames()[source]

Return the list of geophysical fields stored for the feature.

Returns:list of field names
Return type:list<str>
get_full_dimensions(fieldname)[source]

Return the dimension names and sizes of a field.

Parameters:fieldname (str) – name of the field
Return type:OrderedDict
Returns:Ordered dictionary where keys are the dimension names and values are their respective size
get_geolocation_field(fieldname)[source]

Return the equivalent field name in the file format for a standard geolocation field (lat, lon, time).

Used for internal purpose and should not be called directly.

Parameters:fieldname (str) – name of the standard geolocation field (lat, lon or time)
Return type:str or None
Returns:name of the corresponding field in the native file format. Returns None if no matching is found
get_matching_dimname(geodimname)[source]

Return the equivalent name in the native format for a standard dimension.

This is a translation of the standard names to native ones. It is used for internal purpose only and should not be called directly.

To be derived when creating an inherited data mapper class. This is mandatory for geolocation dimensions which must be standard.

Parameters:dimname (str) – standard dimension name
Return type:str
Returns:return the native name for the dimension. return dimname if the input dimension has no standard name.

See

see get_standard_dimname() for the reverse operation

get_spatial_resolution_in_deg()[source]

Return the average spatial resolution in degrees.

get_standard_dimname(geodimname)[source]

Return the equivalent standard dimension name for a dimension in the native format.

This is a translation of the native names to standard ones. It is used for internal purpose and should not be called directly.

To be derived when creating an inherited data mapper class. This is mandatory for geolocation dimensions which must be standard.

Parameters:dimname (str) – native dimension name
Return type:str
Returns:return the (translated) standard name for the dimension. return dimname if the input dimension has no standard name.

See

see get_matching_dimname() for the reverse operation

get_start_time()[source]

Return the minimum date of the file temporal coverage.

open(datamodel_geolocation_dims=None)[source]

Open SAFE GeoTiff file and read metadata.

read_field(fieldname)[source]

Return the cerbere.field.Field object corresponding to the requested fieldname.

The cerbere.field.Field class contains all the metadata describing a field (equivalent to a variable in netCDF).

Parameters:fieldname (str) – name of the field
Returns:the corresponding field object
Return type:cerbere.field.Field
read_field_attributes(fieldname)[source]

Return the specific storage attributes of a field (such as _FillValue, scale_factor, add_offset,...).

Parameters:fieldname (str) – name of the field
Returns:a dictionary where keys are the attribute names.
Return type:Dict
read_fillvalue(fieldname)[source]
read_global_attribute(attr)[source]
read_global_attributes()[source]
read_values(fieldname, slices=None, blocksize=None)[source]

Read the values of a field.

slices is optional. When provided, give for each dimension the corresponding python slice object to subset this dimension. Only the dimensions to be subsetted need to be provided, by default the full dimension length is read.

# extracting a subset of a field with slices
data = fd.read_values('sigma0', slices=[slice(10,20), slice(30,40)])
Parameters:
  • fieldname (str) – name of the field
  • slices (List<slice>) – dimensions slices, when reading a subset only for some dimensions
write_field(fieldname)[source]
write_global_attributes(attrs)[source]

Write the storage (file) global attributes.

saralncfile Module

class cerbere.mapper.saralncfile.SaralNCFile(url=None, mode='r', ncformat='NETCDF4', geodim_matching=None, geofield_matching=None, is_reference=False, fillvalue=None, center_on_greenwhich=False, **kwargs)[source]

Bases: cerbere.mapper.ncfile.NCFile

get_dimensions(fieldname=None)[source]
get_fieldnames()[source]

Returns the list of geophysical fields stored for the feature

sopranoncfile Module

cerbere.mapper.sopranoncfile

Mapper class for SOPRANO netcdf files by CLS

copyright:Copyright 2013 Ifremer / Cersat.
license:Released under GPL v3 license, see license.
class cerbere.mapper.sopranoncfile.SopranoNCFile(url=None, mode='r', ncformat='NETCDF4', geodim_matching=None, geofield_matching=None, is_reference=False, fillvalue=None, center_on_greenwhich=False, **kwargs)[source]

Bases: cerbere.mapper.ncfile.NCFile

Mapper class for the Soprano files from CLS.

The Soprano files are in netCDF format but don’t have a time variable (time is in a global attribute). This class is inherited from the NCFile mapper but extended to mimic the presence of a time variable.

get_bbox()[source]

return the bounding box of the feature, as a tuple (lonmin, latmin, lonmax, latmax)

get_dimsize(dimname)[source]
get_end_time()[source]

Return end of temporal coverage

get_geolocation_field(fieldname)[source]
get_matching_dimname(geodimname)[source]
get_standard_dimname(geodimname)[source]
get_start_time()[source]

Return start of temporal coverage

read_field(fieldname)[source]

Return the field, without its values.

Actual values can be retrieved with read_values() method.

read_field_attributes(fieldname)[source]
read_times(slices=None)[source]

Read time values of a file

read_values(fieldname, slices=None)[source]

urlseries Module

class cerbere.mapper.urlseries.URLSeries(urlpattern=None, timepattern=None, periodicity=None, time_reference=None, **kwargs)[source]

Bases: object

implement a chronologically ordered series of URLs (files,...)

Create a chronologically ordered series of URLs (files,...) from which a feature (PointTimeSeries, GridTimeSeries,Trajectory) overlapping several files can be instantiated from.

A time series can be defined in different ways: * providing a regular expression - or pattern

SUPPRIMER TIME PATTERN ?

Parameters:
  • urlpattern – pattern of the files, including the time information (use python datetime directives) for the feature start or reference time.
  • time_increment
get_example_url()[source]

Return the reference URL of a URL series. This reference is used to extract the feature information (dimensions, variables, etc...). It can be any relevant file from the file series.

If the reference time was not provided at the URL series creation, the latest time in the series is used

get_timesteps_in_interval(start, end, forbid_overlap=True)[source]
get_url_for_time(time, proximity='exact', forbid_overlap=True, delta=relativedelta(day=1))[source]

:keyword forbid_overlap:forbids having several overlapping files for the same time step (if True raise exception). In such case, a liste of files is sent rather than a single file. proximity: -before: echeance before given time -after: echeance after the given time -exact: given time correspond to an echeance -closest: look for the closest @return: datetime

get_urllist_in_interval(start, end, times=None, forbid_overlap=True, unique_result=False)[source]

return the list of files (or URLs) found in the start/end interval.

If times is not None, it will contains the corresponding list of reference time associated with each found URL.