mth5 package

Submodules

mth5.calibration module

Created on Thu May 21 17:33:00 2020

@author: jpeacock

mth5.groups module

Containers to hold the various groups Station, channel, Channel

Created on Fri May 29 15:09:48 2020

copyright

Jared Peacock (jpeacock@usgs.gov)

license

MIT

class mth5.groups.AuxiliaryDataset(group, **kwargs)[source]

Bases: mth5.groups.ChannelDataset

Holds a channel dataset. This is a simple container for the data to make sure that the user has the flexibility to turn the channel into an object they want to deal with.

For now all the numpy type slicing can be used on hdf5_dataset

Parameters
Raises

MTH5Error – If the dataset is not of the correct type

Utilities will be written to create some common objects like:

  • xarray.DataArray

  • pandas.DataFrame

  • zarr

  • dask.Array

The benefit of these other objects is that they can be indexed by time, and they have much more buit-in funcionality.

>>> from mth5 import mth5
>>> mth5_obj = mth5.MTH5()
>>> mth5_obj.open_mth5(r"/test.mth5", mode='a')
>>> run = mth5_obj.stations_group.get_station('MT001').get_run('MT001a')
>>> channel = run.get_channel('Ex')
>>> channel
Channel Electric:
-------------------
            component:        Ey
    data type:        electric
    data format:      float32
    data shape:       (4096,)
    start:            1980-01-01T00:00:00+00:00
    end:              1980-01-01T00:00:01+00:00
    sample rate:      4096
class mth5.groups.BaseGroup(group, group_metadata=None, **kwargs)[source]

Bases: object

Generic object that will have functionality for reading/writing groups, including attributes. To access the hdf5 group directly use the BaseGroup.hdf5_group property.

>>> base = BaseGroup(hdf5_group)
>>> base.hdf5_group.ref
<HDF5 Group Reference>

Note

All attributes should be input into the metadata object, that way all input will be validated against the metadata standards. If you change attributes in metadata object, you should run the BaseGroup.write_metadata method. This is a temporary solution working on an automatic updater if metadata is changed.

>>> base.metadata.existing_attribute = 'update_existing_attribute'
>>> base.write_metadata()

If you want to add a new attribute this should be done using the metadata.add_base_attribute method.

>>> base.metadata.add_base_attribute('new_attribute',
...                                  'new_attribute_value',
...                                  {'type':str,
...                                   'required':True,
...                                   'style':'free form',
...                                   'description': 'new attribute desc.',
...                                   'units':None,
...                                   'options':[],
...                                   'alias':[],
...                                   'example':'new attribute'})

Includes intializing functions that makes a summary table and writes metadata.

property dataset_options
property groups_list
initialize_group()[source]

Initialize group by making a summary table and writing metadata

initialize_summary_table()[source]

Initialize summary table as a dataset based on default values to

/Group/summary

The initial size is 0, but is extentable to self._defaults_summary_attrs[max_shape]

read_metadata()[source]

read metadata from the HDF5 group into metadata object

property summary_table
write_metadata()[source]

Write HDF5 metadata from metadata object.

class mth5.groups.ChannelDataset(dataset, dataset_metadata=None, **kwargs)[source]

Bases: object

Holds a channel dataset. This is a simple container for the data to make sure that the user has the flexibility to turn the channel into an object they want to deal with.

For now all the numpy type slicing can be used on hdf5_dataset

Parameters
Raises

MTH5Error – If the dataset is not of the correct type

Utilities will be written to create some common objects like:

  • xarray.DataArray

  • pandas.DataFrame

  • zarr

  • dask.Array

The benefit of these other objects is that they can be indexed by time, and they have much more buit-in funcionality.

>>> from mth5 import mth5
>>> mth5_obj = mth5.MTH5()
>>> mth5_obj.open_mth5(r"/test.mth5", mode='a')
>>> run = mth5_obj.stations_group.get_station('MT001').get_run('MT001a')
>>> channel = run.get_channel('Ex')
>>> channel
Channel Electric:
-------------------
            component:        Ey
    data type:        electric
    data format:      float32
    data shape:       (4096,)
    start:            1980-01-01T00:00:00+00:00
    end:              1980-01-01T00:00:01+00:00
    sample rate:      4096
property channel_entry

channel entry that will go into a full channel summary of the entire survey

property end

return end time based on the data

extend_dataset(new_data_array, start_time, sample_rate, fill=None, max_gap_seconds=1, fill_window=10)[source]

Append data according to how the start time aligns with existing data. If the start time is before existing start time the data is prepended, similarly if the start time is near the end data will be appended.

If the start time is within the existing time range, existing data will be replace with the new data.

If there is a gap between start or end time of the new data with the existing data you can either fill the data with a constant value or an error will be raise depending on the value of fill.

Parameters
  • new_data_array (numpy.ndarray) – new data array with shape (npts, )

  • start_time (string or mth5.utils.mttime.MTime) – start time of the new data array in UTC

  • sample_rate (float) – Sample rate of the new data array, must match existing sample rate

  • fill (string, None, float, integer) – If there is a data gap how do you want to fill the gap * None: will raise an mth5.utils.exceptions.MTH5Error * ‘mean’: will fill with the mean of each data set within the fill window * ‘median’: will fill with the median of each data set within the fill window * value: can be an integer or float to fill the gap * ‘nan’: will fill the gap with NaN

  • max_gap_seconds (float or integer) – sets a maximum number of seconds the gap can be. Anything over this number will raise a mth5.utils.exceptions.MTH5Error.

  • fill_window (integer) – number of points from the end of each data set to estimate fill value from.

Raises

mth5.utils.excptions.MTH5Error if sample rate is not the same, or fill value is not understood,

Rubric

>>> ex = mth5_obj.get_channel('MT001', 'MT001a', 'Ex')
>>> ex.n_samples
4096
>>> ex.end
2015-01-08T19:32:09.500000+00:00
>>> t = timeseries.ChannelTS('electric',
...                     data=2*np.cos(4 * np.pi * .05 *         ...                                   np.linspace(0,4096l num=4096) *
...                                   .01),
...                     channel_metadata={'electric':{
...                        'component': 'ex',
...                        'sample_rate': 8,
...                        'time_period.start':(ex.end+(1)).iso_str}})
>>> ex.extend_dataset(t.ts, t.start, t.sample_rate, fill='median',
...                   max_gap_seconds=2)
2020-07-02T18:02:47 - mth5.groups.Electric.extend_dataset - INFO -
filling data gap with 1.0385180759767025
>>> ex.n_samples
8200
>>> ex.end
2015-01-08T19:40:42.500000+00:00
from_channel_ts(channel_ts_obj, how='replace', fill=None, max_gap_seconds=1, fill_window=10)[source]

fill data set from a mth5.timeseries.ChannelTS object.

Will check for time alignement, and metadata.

Parameters
  • channel_ts_obj (mth5.timeseries.ChannelTS) – time series object

  • how

    how the new array will be input to the existing dataset:

    • ’replace’ -> replace the entire dataset nothing is left over.

    • ’extend’ -> add onto the existing dataset, any overlapping values will be rewritten, if there are gaps between data sets those will be handled depending on the value of fill.

    param fill

    If there is a data gap how do you want to fill the gap:

    • None -> will raise an mth5.utils.exceptions.MTH5Error

    • ’mean’-> will fill with the mean of each data set within the fill window

    • ’median’ -> will fill with the median of each data set within the fill window

    • value -> can be an integer or float to fill the gap

    • ’nan’ -> will fill the gap with NaN

  • max_gap_seconds (float or integer) – sets a maximum number of seconds the gap can be. Anything over this number will raise a mth5.utils.exceptions.MTH5Error.

  • fill_window (integer) – number of points from the end of each data set to estimate fill value from.

from_xarray(data_array, how='replace', fill=None, max_gap_seconds=1, fill_window=10)[source]

fill data set from a xarray.DataArray object.

Will check for time alignement, and metadata.

Parameters
  • data_array_obj – Xarray data array

  • how

    how the new array will be input to the existing dataset:

    • ’replace’ -> replace the entire dataset nothing is left over.

    • ’extend’ -> add onto the existing dataset, any overlapping

    values will be rewritten, if there are gaps between data sets those will be handled depending on the value of fill.

    param fill

    If there is a data gap how do you want to fill the gap:

    • None -> will raise an mth5.utils.exceptions.MTH5Error

    • ’mean’-> will fill with the mean of each data set within

      the fill window

    • ’median’ -> will fill with the median of each data set

      within the fill window

    • value -> can be an integer or float to fill the gap

    • ’nan’ -> will fill the gap with NaN

  • max_gap_seconds (float or integer) – sets a maximum number of seconds the gap can be. Anything over this number will raise a mth5.utils.exceptions.MTH5Error.

  • fill_window (integer) – number of points from the end of each data set to estimate fill value from.

get_index_from_time(given_time)[source]

get the appropriate index for a given time.

Parameters

given_time (TYPE) – DESCRIPTION

Returns

DESCRIPTION

Return type

TYPE

property master_station_group

shortcut to master station group

property n_samples
read_metadata()[source]

Read metadata from the HDF5 file into the metadata container, that way it can be validated.

replace_dataset(new_data_array)[source]

replace the entire dataset with a new one, nothing left behind

Parameters

new_data_array (numpy.ndarray) – new data array shape (npts, )

property run_group

shortcut to run group

property sample_rate
property start
property station_group

shortcut to station group

property table_entry

Creat a table entry to put into the run summary table.

property time_index

time index given parameters in metadata :rtype: pandas.DatetimeIndex

Type

return

time_slice(start_time, end_time=None, n_samples=None, return_type='channel_ts')[source]

Get a time slice from the channel and return the appropriate type

  • numpy array with metadata

  • pandas.Dataframe with metadata

  • xarray.DataFrame with metadata

  • mth5.timeseries.ChannelTS ‘default’

  • dask.DataFrame with metadata ‘not yet’

Parameters
Returns

the correct container for the time series.

Return type

[ xarray.DataArray | pandas.DataFrame | mth5.timeseries.ChannelTS | numpy.ndarray ]

Raises

ValueError if both end_time and n_samples are None or given.

Example with number of samples

>>> ex = mth5_obj.get_channel('FL001', 'FL001a', 'Ex')
>>> ex_slice = ex.time_slice("2015-01-08T19:49:15", n_samples=4096)
>>> ex_slice
<xarray.DataArray (time: 4096)>
array([0.93115046, 0.14233688, 0.87917119, ..., 0.26073634, 0.7137319 ,
       0.88154395])
Coordinates:
  * time     (time) datetime64[ns] 2015-01-08T19:49:15 ... 2015-01-08T19:57:46.875000
Attributes:
    ac.end:                      None
    ac.start:                    None
    ...

>>> type(ex_slice)
mth5.timeseries.ChannelTS

# plot the time series
>>> ex_slice.ts.plot()
Example with start and end time

>>> ex_slice = ex.time_slice("2015-01-08T19:49:15",
...                          end_time="2015-01-09T19:49:15")
Raises Example

>>> ex_slice = ex.time_slice("2015-01-08T19:49:15",
...                          end_time="2015-01-09T19:49:15",
...                          n_samples=4096)
ValueError: Must input either end_time or n_samples, not both.
to_channel_ts()[source]
Returns

a Timeseries with the appropriate time index and metadata

Return type

mth5.timeseries.ChannelTS

loads from memory (nearly half the size of xarray alone, not sure why)

to_dataframe()[source]
Returns

a dataframe where data is stored in the ‘data’ column and attributes are stored in the experimental attrs attribute

Return type

pandas.DataFrame

Note

that metadta will not be validated if changed in an xarray.

loads into RAM

to_numpy()[source]
Returns

a numpy structured array with 2 columns (time, channel_data)

Return type

numpy.core.records

data is a builtin to numpy and cannot be used as a name

loads into RAM

to_xarray()[source]
Returns

an xarray DataArray with appropriate metadata and the appropriate time index.

Return type

xarray.DataArray

Note

that metadta will not be validated if changed in an xarray.

loads from memory

write_metadata()[source]

Write metadata from the metadata container to the HDF5 attrs dictionary.

class mth5.groups.ElectricDataset(group, **kwargs)[source]

Bases: mth5.groups.ChannelDataset

Holds a channel dataset. This is a simple container for the data to make sure that the user has the flexibility to turn the channel into an object they want to deal with.

For now all the numpy type slicing can be used on hdf5_dataset

Parameters
Raises

MTH5Error – If the dataset is not of the correct type

Utilities will be written to create some common objects like:

  • xarray.DataArray

  • pandas.DataFrame

  • zarr

  • dask.Array

The benefit of these other objects is that they can be indexed by time, and they have much more buit-in funcionality.

>>> from mth5 import mth5
>>> mth5_obj = mth5.MTH5()
>>> mth5_obj.open_mth5(r"/test.mth5", mode='a')
>>> run = mth5_obj.stations_group.get_station('MT001').get_run('MT001a')
>>> channel = run.get_channel('Ex')
>>> channel
Channel Electric:
-------------------
            component:        Ey
    data type:        electric
    data format:      float32
    data shape:       (4096,)
    start:            1980-01-01T00:00:00+00:00
    end:              1980-01-01T00:00:01+00:00
    sample rate:      4096
class mth5.groups.FilterDataset(dataset, dataset_metadata=None, **kwargs)[source]

Bases: object

Holds a filter dataset. This is a simple container for the filter to make sure that the user has the flexibility to turn the channel into an object they want to deal with.

For now all the numpy type slicing can be used on hdf5_dataset

Parameters
  • dataset (h5py.Dataset) – dataset object for the filter

  • dataset_metadata (mth5.metadata.Filter, optional) – metadata container, defaults to None

Raises

MTH5Error – If the dataset is not of the correct type

Utilities will be written to create some common objects like:

  • xarray.DataArray

  • pandas.DataFrame

  • zarr

  • dask.Array

The benefit of these other objects is that they can be indexed by time, and they have much more buit-in funcionality.

>>> from mth5 import mth5
>>> mth5_obj = mth5.MTH5()
>>> mth5_obj.open_mth5(r"/test.mth5", mode='a')
>>> f = mth5_obj.filter_group.add_filter("table")
property filter_type

filter type

property frequency
property imaginary
property name

filter name

property poles

convenience to poles if there are any

read_metadata()[source]

read metadata from the HDF5 group into metadata object

property real
to_filter_object()[source]

convert to a mth5.filters.Filter object

Returns

DESCRIPTION

Return type

TYPE

property units_in

units in

property units_out

units out

write_metadata()[source]

Write HDF5 metadata from metadata object.

property zeros

convenience to zeros if there are any

property zpk_gain
class mth5.groups.FiltersGroup(group, **kwargs)[source]

Bases: mth5.groups.BaseGroup

Not implemented yet

add_filter(filter_name, filter_type, values=None, filter_metadata=None)[source]

Add a filter dataset based on type

current types are:
  • zpk –> zeros, poles, gain

  • table –> frequency look up table

Parameters
  • filter_name (TYPE) – DESCRIPTION

  • filter_type (TYPE) – DESCRIPTION

  • values (TYPE, optional) – DESCRIPTION, defaults to None

  • metadata (TYPE, optional) – DESCRIPTION, defaults to None

Returns

DESCRIPTION

Return type

TYPE

class mth5.groups.MTH5Table(hdf5_dataset)[source]

Bases: object

Use the underlying NumPy basics, there are simple actions in this table, if a user wants to use something more sophisticated for querying they should try using a pandas table. In this case entries in the table are more difficult to change and datatypes need to be kept track of.

add_row(row, index=None)[source]

Add a row to the table.

row must be of the same data type as the table

Parameters
  • row (TYPE) – row entry for the table

  • index (integer, if None is given then the row is added to the end of the array) – index of row to add

Returns

index of the row added

Return type

integer

check_dtypes(other_dtype)[source]

Check to make sure datatypes match

property dtype
locate(column, value, test='eq')[source]

locate index where column is equal to value :param column: DESCRIPTION :type column: TYPE :param value: DESCRIPTION :type value: TYPE :type test: type of test to try * ‘eq’: equals * ‘lt’: less than * ‘le’: less than or equal to * ‘gt’: greater than * ‘ge’: greater than or equal to. * ‘be’: between or equal to * ‘bt’: between

If be or bt input value as a list of 2 values

Returns

DESCRIPTION

Return type

TYPE

property nrows
remove_row(index)[source]

Remove a row

Note

that there is not index value within the array, so the indexing is on the fly. A user should use the HDF5 reference instead of index number that is the safest and most robust method.

Parameters

index (TYPE) – DESCRIPTION

Returns

DESCRIPTION

Return type

TYPE

This isn’t as easy as just deleteing an element. Need to delete the element from the weakly referenced array and then set the summary table dataset to the new array.

So set to a null array for now until a more clever option is found.

property shape
to_dataframe()[source]

Convert the table into a pandas.DataFrame object.

Returns

convert table into a pandas.DataFrame with the appropriate data types.

Return type

pandas.DataFrame

class mth5.groups.MagneticDataset(group, **kwargs)[source]

Bases: mth5.groups.ChannelDataset

Holds a channel dataset. This is a simple container for the data to make sure that the user has the flexibility to turn the channel into an object they want to deal with.

For now all the numpy type slicing can be used on hdf5_dataset

Parameters
Raises

MTH5Error – If the dataset is not of the correct type

Utilities will be written to create some common objects like:

  • xarray.DataArray

  • pandas.DataFrame

  • zarr

  • dask.Array

The benefit of these other objects is that they can be indexed by time, and they have much more buit-in funcionality.

>>> from mth5 import mth5
>>> mth5_obj = mth5.MTH5()
>>> mth5_obj.open_mth5(r"/test.mth5", mode='a')
>>> run = mth5_obj.stations_group.get_station('MT001').get_run('MT001a')
>>> channel = run.get_channel('Ex')
>>> channel
Channel Electric:
-------------------
            component:        Ey
    data type:        electric
    data format:      float32
    data shape:       (4096,)
    start:            1980-01-01T00:00:00+00:00
    end:              1980-01-01T00:00:01+00:00
    sample rate:      4096
class mth5.groups.MasterStationGroup(group, **kwargs)[source]

Bases: mth5.groups.BaseGroup

Utility class to holds information about the stations within a survey and accompanying metadata. This class is next level down from Survey for stations /Survey/Stations. This class provides methods to add and get stations. A summary table of all existing stations is also provided as a convenience look up table to make searching easier.

To access MasterStationGroup from an open MTH5 file:

>>> from mth5 import mth5
>>> mth5_obj = mth5.MTH5()
>>> mth5_obj.open_mth5(r"/test.mth5", mode='a')
>>> stations = mth5_obj.stations_group

To check what stations exist

>>> stations.group_list
['summary', 'MT001', 'MT002', 'MT003']

To access the hdf5 group directly use SurveyGroup.hdf5_group.

>>> stations.hdf5_group.ref
<HDF5 Group Reference>

Note

All attributes should be input into the metadata object, that way all input will be validated against the metadata standards. If you change attributes in metadata object, you should run the SurveyGroup.write_metadata() method. This is a temporary solution, working on an automatic updater if metadata is changed.

>>> stations.metadata.existing_attribute = 'update_existing_attribute'
>>> stations.write_metadata()

If you want to add a new attribute this should be done using the metadata.add_base_attribute method.

>>> stations.metadata.add_base_attribute('new_attribute',
>>> ...                                'new_attribute_value',
>>> ...                                {'type':str,
>>> ...                                 'required':True,
>>> ...                                 'style':'free form',
>>> ...                                 'description': 'new attribute desc.',
>>> ...                                 'units':None,
>>> ...                                 'options':[],
>>> ...                                 'alias':[],
>>> ...                                 'example':'new attribute

To add a station:

>>> new_station = stations.add_station('new_station')
>>> stations
/Survey/Stations:
====================
    --> Dataset: summary
    ......................
    |- Group: new_station
    ---------------------
        --> Dataset: summary
        ......................

Add a station with metadata:

>>> from mth5.metadata import Station
>>> station_metadata = Station()
>>> station_metadata.id = 'MT004'
>>> station_metadata.time_period.start = '2020-01-01T12:30:00'
>>> station_metadata.location.latitude = 40.000
>>> station_metadata.location.longitude = -120.000
>>> new_station = stations.add_station('Test_01', station_metadata)
>>> # to look at the metadata
>>> new_station.metadata
{
    "station": {
        "acquired_by.author": null,
        "acquired_by.comments": null,
        "id": "MT004",
        ...
        }
}

See also

mth5.metadata for details on how to add metadata from various files and python objects.

To remove a station:

>>> stations.remove_station('new_station')
>>> stations
/Survey/Stations:
====================
    --> Dataset: summary
    ......................

Note

Deleting a station is not as simple as del(station). In HDF5 this does not free up memory, it simply removes the reference to that station. The common way to get around this is to copy what you want into a new file, or overwrite the station.

To get a station:

>>> existing_station = stations.get_station('existing_station_name')
>>> existing_station
/Survey/Stations/existing_station_name:
=======================================
    --> Dataset: summary
    ......................
    |- Group: run_01
    ----------------
        --> Dataset: summary
        ......................
        --> Dataset: Ex
        ......................
        --> Dataset: Ey
        ......................
        --> Dataset: Hx
        ......................
        --> Dataset: Hy
        ......................
        --> Dataset: Hz
        ......................

A summary table is provided to make searching easier. The table summarized all stations within a survey. To see what names are in the summary table:

>>> stations.summary_table.dtype.descr
[('id', ('|S5', {'h5py_encoding': 'ascii'})),
 ('start', ('|S32', {'h5py_encoding': 'ascii'})),
 ('end', ('|S32', {'h5py_encoding': 'ascii'})),
 ('components', ('|S100', {'h5py_encoding': 'ascii'})),
 ('measurement_type', ('|S12', {'h5py_encoding': 'ascii'})),
 ('sample_rate', '<f8')]

Note

When a station is added an entry is added to the summary table, where the information is pulled from the metadata.

>>> stations.summary_table
index |   id    |            start             |             end
 | components | measurement_type | sample_rate
 -------------------------------------------------------------------------
 --------------------------------------------------
 0   |  Test_01   |  1980-01-01T00:00:00+00:00 |  1980-01-01T00:00:00+00:00
 |  Ex,Ey,Hx,Hy,Hz   |  BBMT   | 100
add_station(station_name, station_metadata=None)[source]
Add a station with metadata if given with the path:

/Survey/Stations/station_name

If the station already exists, will return that station and nothing is added.

Parameters
  • station_name (string) – Name of the station, should be the same as metadata.id

  • station_metadata (mth5.metadata.Station, optional) – Station metadata container, defaults to None

Returns

A convenience class for the added station

Return type

mth5_groups.StationGroup

Example
>>> from mth5 import mth5
>>> mth5_obj = mth5.MTH5()
>>> mth5_obj.open_mth5(r"/test.mth5", mode='a')
>>> # one option
>>> stations = mth5_obj.stations_group
>>> new_station = stations.add_station('MT001')
>>> # another option
>>> new_staiton = mth5_obj.stations_group.add_station('MT001')
get_station(station_name)[source]

Get a station with the same name as station_name

Parameters

station_name (string) – existing station name

Returns

convenience station class

Return type

mth5.mth5_groups.StationGroup

Raises

MTH5Error – if the station name is not found.

Example

>>> from mth5 import mth5
>>> mth5_obj = mth5.MTH5()
>>> mth5_obj.open_mth5(r"/test.mth5", mode='a')
>>> # one option
>>> stations = mth5_obj.stations_group
>>> existing_station = stations.get_station('MT001')
>>> # another option
>>> existing_staiton = mth5_obj.stations_group.get_station('MT001')
MTH5Error: MT001 does not exist, check station_list for existing names
remove_station(station_name)[source]

Remove a station from the file.

Note

Deleting a station is not as simple as del(station). In HDF5 this does not free up memory, it simply removes the reference to that station. The common way to get around this is to copy what you want into a new file, or overwrite the station.

Parameters

station_name (string) – existing station name

Example
>>> from mth5 import mth5
>>> mth5_obj = mth5.MTH5()
>>> mth5_obj.open_mth5(r"/test.mth5", mode='a')
>>> # one option
>>> stations = mth5_obj.stations_group
>>> stations.remove_station('MT001')
>>> # another option
>>> mth5_obj.stations_group.remove_station('MT001')
class mth5.groups.ReportsGroup(group, **kwargs)[source]

Bases: mth5.groups.BaseGroup

Not sure how to handle this yet

add_report(report_name, report_metadata=None, report_data=None)[source]
Parameters
  • report_name (TYPE) – DESCRIPTION

  • report_metadata (TYPE, optional) – DESCRIPTION, defaults to None

  • report_data (TYPE, optional) – DESCRIPTION, defaults to None

Returns

DESCRIPTION

Return type

TYPE

class mth5.groups.RunGroup(group, run_metadata=None, **kwargs)[source]

Bases: mth5.groups.BaseGroup

RunGroup is a utility class to hold information about a single run and accompanying metadata. This class is the next level down from Stations –> /Survey/Stations/station/station{a-z}.

This class provides methods to add and get channels. A summary table of all existing channels in the run is also provided as a convenience look up table to make searching easier.

Parameters
  • group (h5py.Group) – HDF5 group for a station, should have a path /Survey/Stations/station_name/run_name

  • station_metadata (mth5.metadata.Station, optional) – metadata container, defaults to None

Access RunGroup from an open MTH5 file

>>> from mth5 import mth5
>>> mth5_obj = mth5.MTH5()
>>> mth5_obj.open_mth5(r"/test.mth5", mode='a')
>>> run = mth5_obj.stations_group.get_station('MT001').get_run('MT001a')
Check what channels exist

>>> station.group_list
['Ex', 'Ey', 'Hx', 'Hy']

To access the hdf5 group directly use RunGroup.hdf5_group

>>> station.hdf5_group.ref
<HDF5 Group Reference>

Note

All attributes should be input into the metadata object, that way all input will be validated against the metadata standards. If you change attributes in metadata object, you should run the SurveyGroup.write_metadata() method. This is a temporary solution, working on an automatic updater if metadata is changed.

>>> run.metadata.existing_attribute = 'update_existing_attribute'
>>> run.write_metadata()

If you want to add a new attribute this should be done using the metadata.add_base_attribute method.

>>> station.metadata.add_base_attribute('new_attribute',
>>> ...                                 'new_attribute_value',
>>> ...                                 {'type':str,
>>> ...                                  'required':True,
>>> ...                                  'style':'free form',
>>> ...                                  'description': 'new attribute desc.',
>>> ...                                  'units':None,
>>> ...                                  'options':[],
>>> ...                                  'alias':[],
>>> ...                                  'example':'new attribute
Add a channel

>>> new_channel = run.add_channel('Ex', 'electric',
>>> ...                            data=numpy.random.rand(4096))
>>> new_run
/Survey/Stations/MT001/MT001a:
=======================================
    --> Dataset: summary
    ......................
    --> Dataset: Ex
    ......................
    --> Dataset: Ey
    ......................
    --> Dataset: Hx
    ......................
    --> Dataset: Hy
    ......................
Add a channel with metadata

>>> from mth5.metadata import Electric
>>> ex_metadata = Electric()
>>> ex_metadata.time_period.start = '2020-01-01T12:30:00'
>>> ex_metadata.time_period.end = '2020-01-03T16:30:00'
>>> new_ex = run.add_channel('Ex', 'electric',
>>> ...                       channel_metadata=ex_metadata)
>>> # to look at the metadata
>>> new_ex.metadata
{
     "electric": {
        "ac.end": 1.2,
        "ac.start": 2.3,
        ...
        }
}

See also

mth5.metadata for details on how to add metadata from various files and python objects.

Remove a channel

>>> run.remove_channel('Ex')
>>> station
/Survey/Stations/MT001/MT001a:
=======================================
    --> Dataset: summary
    ......................
    --> Dataset: Ey
    ......................
    --> Dataset: Hx
    ......................
    --> Dataset: Hy
    ......................

Note

Deleting a station is not as simple as del(station). In HDF5 this does not free up memory, it simply removes the reference to that station. The common way to get around this is to copy what you want into a new file, or overwrite the station.

Get a channel

>>> existing_ex = stations.get_channel('Ex')
>>> existing_ex
Channel Electric:
-------------------
    data type:        Ex
    data type:        electric
    data format:      float32
    data shape:       (4096,)
    start:            1980-01-01T00:00:00+00:00
    end:              1980-01-01T00:32:+08:00
    sample rate:      8
Summary Table

A summary table is provided to make searching easier. The table summarized all stations within a survey. To see what names are in the summary table:

>>> run.summary_table.dtype.descr
[('component', ('|S5', {'h5py_encoding': 'ascii'})),
 ('start', ('|S32', {'h5py_encoding': 'ascii'})),
 ('end', ('|S32', {'h5py_encoding': 'ascii'})),
 ('n_samples', '<i4'),
 ('measurement_type', ('|S12', {'h5py_encoding': 'ascii'})),
 ('units', ('|S25', {'h5py_encoding': 'ascii'})),
 ('hdf5_reference', ('|O', {'ref': h5py.h5r.Reference}))]

Note

When a run is added an entry is added to the summary table, where the information is pulled from the metadata.

>>> new_run.summary_table
index | component | start | end | n_samples | measurement_type | units |
hdf5_reference
--------------------------------------------------------------------------
-------------
add_channel(channel_name, channel_type, data, channel_dtype='f', max_shape=(None), chunks=True, channel_metadata=None, **kwargs)[source]

add a channel to the run

Parameters
Raises

MTH5Error – If channel type is not correct

Returns

Channel container

Return type

[ mth5.mth5_groups.ElectricDatset | mth5.mth5_groups.MagneticDatset | mth5.mth5_groups.AuxiliaryDatset ]

>>> new_channel = run.add_channel('Ex', 'electric', None)
>>> new_channel
Channel Electric:
-------------------
                component:        None
        data type:        electric
        data format:      float32
        data shape:       (1,)
        start:            1980-01-01T00:00:00+00:00
        end:              1980-01-01T00:00:00+00:00
        sample rate:      None
from_channel_ts(channel_ts_obj)[source]

create a channel data set from a mth5.timeseries.ChannelTS object and update metadata.

Parameters

channel_ts_obj (mth5.timeseries.ChannelTS) – a single time series object

Returns

new channel dataset

Return type

:class:`mth5.groups.ChannelDataset

from_runts(run_ts_obj)[source]

create channel datasets from a mth5.timeseries.RunTS object and update metadata.

:parameter mth5.timeseries.RunTS run_ts_obj: Run object with all the appropriate channels and metadata.

Will create a run group and appropriate channel datasets.

get_channel(channel_name)[source]

Get a channel from an existing name. Returns the appropriate container.

Parameters

channel_name (string) – name of the channel

Returns

Channel container

Return type

[ mth5.mth5_groups.ElectricDatset | mth5.mth5_groups.MagneticDatset | mth5.mth5_groups.AuxiliaryDatset ]

Raises

MTH5Error – If no channel is found

Example

>>> existing_channel = run.get_channel('Ex')
MTH5Error: Ex does not exist, check groups_list for existing names'
>>> run.group_list
['Ey', 'Hx', 'Hz']
>>> existing_channel = run.get_channel('Ey')
>>> existing_channel
Channel Electric:
-------------------
                component:        Ey
        data type:        electric
        data format:      float32
        data shape:       (4096,)
        start:            1980-01-01T00:00:00+00:00
        end:              1980-01-01T00:00:01+00:00
        sample rate:      4096
property master_station_group

shortcut to master station group

remove_channel(channel_name)[source]

Remove a run from the station.

Note

Deleting a channel is not as simple as del(channel). In HDF5 this does not free up memory, it simply removes the reference to that channel. The common way to get around this is to copy what you want into a new file, or overwrite the channel.

Parameters

station_name (string) – existing station name

Example

>>> from mth5 import mth5
>>> mth5_obj = mth5.MTH5()
>>> mth5_obj.open_mth5(r"/test.mth5", mode='a')
>>> run = mth5_obj.stations_group.get_station('MT001').get_run('MT001a')
>>> run.remove_channel('Ex')
property station_group

shortcut to station group

property table_entry

Get a run table entry

Returns

a properly formatted run table entry

Return type

numpy.ndarray with dtype:

>>> dtype([('id', 'S20'),
         ('start', 'S32'),
         ('end', 'S32'),
         ('components', 'S100'),
         ('measurement_type', 'S12'),
         ('sample_rate', np.float),
         ('hdf5_reference', h5py.ref_dtype)])
to_runts()[source]

create a mth5.timeseries.RunTS object from channels of the run

Returns

DESCRIPTION

Return type

TYPE

validate_run_metadata()[source]

Update metadata and table entries to ensure consistency

Returns

DESCRIPTION

Return type

TYPE

class mth5.groups.StandardsGroup(group, **kwargs)[source]

Bases: mth5.groups.BaseGroup

The StandardsGroup is a convenience group that stores the metadata standards that were used to make the current file. This is to help a user understand the metadata directly from the file and not have to look up documentation that might not be updated.

The metadata standards are stored in the summary table /Survey/Standards/summary

>>> standards = mth5_obj.standards_group
>>> standards.summary_table
index | attribute | type | required | style | units | description |
options  |  alias |  example
--------------------------------------------------------------------------
get_attribute_information(attribute_name)[source]

get information about an attribute

The attribute name should be in the summary table.

Parameters

attribute_name (string) – attribute name

Returns

prints a description of the attribute

Raises

MTH5TableError – if attribute is not found

>>> standars = mth5_obj.standards_group
>>> standards.get_attribute_information('survey.release_license')
survey.release_license
--------------------------
        type          : string
        required      : True
        style         : controlled vocabulary
        units         :
        description   : How the data can be used. The options are based on
                 Creative Commons licenses. For details visit
                 https://creativecommons.org/licenses/
        options       : CC-0,CC-BY,CC-BY-SA,CC-BY-ND,CC-BY-NC-SA,CC-BY-NC-ND
        alias         :
        example       : CC-0
initialize_group()[source]

Initialize the group by making a summary table that summarizes the metadata standards used to describe the data.

Also, write generic metadata information.

summary_table_from_dict(summary_dict)[source]

Fill summary table from a dictionary that summarizes the metadata for the entire survey.

Parameters

summary_dict (dictionary) – Flattened dictionary of all metadata standards within the survey.

class mth5.groups.StationGroup(group, station_metadata=None, **kwargs)[source]

Bases: mth5.groups.BaseGroup

StationGroup is a utility class to hold information about a single station and accompanying metadata. This class is the next level down from Stations –> /Survey/Stations/station_name.

This class provides methods to add and get runs. A summary table of all existing runs in the station is also provided as a convenience look up table to make searching easier.

Parameters
  • group (h5py.Group) – HDF5 group for a station, should have a path /Survey/Stations/station_name

  • station_metadata (mth5.metadata.Station, optional) – metadata container, defaults to None

Usage

Access StationGroup from an open MTH5 file

>>> from mth5 import mth5
>>> mth5_obj = mth5.MTH5()
>>> mth5_obj.open_mth5(r"/test.mth5", mode='a')
>>> station = mth5_obj.stations_group.get_station('MT001')
Check what runs exist

>>> station.group_list
['MT001a', 'MT001b', 'MT001c', 'MT001d']

To access the hdf5 group directly use StationGroup.hdf5_group.

>>> station.hdf5_group.ref
<HDF5 Group Reference>

Note

All attributes should be input into the metadata object, that way all input will be validated against the metadata standards. If you change attributes in metadata object, you should run the SurveyGroup.write_metadata() method. This is a temporary solution, working on an automatic updater if metadata is changed.

>>> station.metadata.existing_attribute = 'update_existing_attribute'
>>> station.write_metadata()

If you want to add a new attribute this should be done using the metadata.add_base_attribute method.

>>> station.metadata.add_base_attribute('new_attribute',
>>> ...                                 'new_attribute_value',
>>> ...                                 {'type':str,
>>> ...                                  'required':True,
>>> ...                                  'style':'free form',
>>> ...                                  'description': 'new attribute desc.',
>>> ...                                  'units':None,
>>> ...                                  'options':[],
>>> ...                                  'alias':[],
>>> ...                                  'example':'new attribute
To add a run

>>> new_run = stations.add_run('MT001e')
>>> new_run
/Survey/Stations/Test_01:
=========================
    |- Group: MT001e
    -----------------
        --> Dataset: summary
        ......................
    --> Dataset: summary
    ......................
Add a run with metadata

>>> from mth5.metadata import Run
>>> run_metadata = Run()
>>> run_metadata.time_period.start = '2020-01-01T12:30:00'
>>> run_metadata.time_period.end = '2020-01-03T16:30:00'
>>> run_metadata.location.latitude = 40.000
>>> run_metadata.location.longitude = -120.000
>>> new_run = runs.add_run('Test_01', run_metadata)
>>> # to look at the metadata
>>> new_run.metadata
{
    "run": {
        "acquired_by.author": "new_user",
        "acquired_by.comments": "First time",
        "channels_recorded_auxiliary": ['T'],
        ...
        }
}

See also

mth5.metadata for details on how to add metadata from various files and python objects.

Remove a run

>>> station.remove_run('new_run')
>>> station
/Survey/Stations/Test_01:
=========================
    --> Dataset: summary
    ......................

Note

Deleting a station is not as simple as del(station). In HDF5 this does not free up memory, it simply removes the reference to that station. The common way to get around this is to copy what you want into a new file, or overwrite the station.

Get a run

>>> existing_run = stations.get_station('existing_run')
>>> existing_run
/Survey/Stations/MT001/MT001a:
=======================================
    --> Dataset: summary
    ......................
    --> Dataset: Ex
    ......................
    --> Dataset: Ey
    ......................
    --> Dataset: Hx
    ......................
    --> Dataset: Hy
    ......................
    --> Dataset: Hz
    ......................
Summary Table

A summary table is provided to make searching easier. The table summarized all stations within a survey. To see what names are in the summary table:

>>> new_run.summary_table.dtype.descr
[('id', ('|S20', {'h5py_encoding': 'ascii'})),
 ('start', ('|S32', {'h5py_encoding': 'ascii'})),
 ('end', ('|S32', {'h5py_encoding': 'ascii'})),
 ('components', ('|S100', {'h5py_encoding': 'ascii'})),
 ('measurement_type', ('|S12', {'h5py_encoding': 'ascii'})),
 ('sample_rate', '<f8'),
 ('hdf5_reference', ('|O', {'ref': h5py.h5r.Reference}))]

Note

When a run is added an entry is added to the summary table, where the information is pulled from the metadata.

>>> station.summary_table
index | id | start | end | components | measurement_type | sample_rate |
hdf5_reference
--------------------------------------------------------------------------
-------------
add_run(run_name, run_metadata=None)[source]

Add a run to a station.

Parameters
  • run_name (string) – run name, should be id{a-z}

  • metadata (mth5.metadata.Station, optional) – metadata container, defaults to None

need to be able to fill an entry in the summary table.

get_run(run_name)[source]

get a run from run name

Parameters

run_name (string) – existing run name

Returns

Run object

Return type

mth5.mth5_groups.RunGroup

>>> existing_run = station.get_run('MT001')
locate_run(sample_rate, start)[source]

Locate a run based on sample rate and start time from the summary table

Parameters
Returns

appropriate run name, None if not found

Return type

string or None

make_run_name()[source]

Make a run name that will be the next alphabet letter extracted from the run list. Expects that all runs are labled as id{a-z}.

Returns

metadata.id + next letter

Return type

string

>>> station.metadata.id = 'MT001'
>>> station.make_run_name()
'MT001a'
property master_station_group

shortcut to master station group

property name
remove_run(run_name)[source]

Remove a run from the station.

Note

Deleting a station is not as simple as del(station). In HDF5 this does not free up memory, it simply removes the reference to that station. The common way to get around this is to copy what you want into a new file, or overwrite the station.

Parameters

station_name (string) – existing station name

Example
>>> from mth5 import mth5
>>> mth5_obj = mth5.MTH5()
>>> mth5_obj.open_mth5(r"/test.mth5", mode='a')
>>> # one option
>>> stations = mth5_obj.stations_group
>>> stations.remove_station('MT001')
>>> # another option
>>> mth5_obj.stations_group.remove_station('MT001')
property table_entry

make table entry

validate_station_metadata()[source]

Check metadata from the runs and make sure it matches the station metadata

Returns

DESCRIPTION

Return type

TYPE

class mth5.groups.SurveyGroup(group, **kwargs)[source]

Bases: mth5.groups.BaseGroup

Utility class to holds general information about the survey and accompanying metadata for an MT survey.

To access the hdf5 group directly use SurveyGroup.hdf5_group.

>>> survey = SurveyGroup(hdf5_group)
>>> survey.hdf5_group.ref
<HDF5 Group Reference>

Note

All attributes should be input into the metadata object, that way all input will be validated against the metadata standards. If you change attributes in metadata object, you should run the SurveyGroup.write_metadata() method. This is a temporary solution, working on an automatic updater if metadata is changed.

>>> survey.metadata.existing_attribute = 'update_existing_attribute'
>>> survey.write_metadata()

If you want to add a new attribute this should be done using the metadata.add_base_attribute method.

>>> survey.metadata.add_base_attribute('new_attribute',
>>> ...                                'new_attribute_value',
>>> ...                                {'type':str,
>>> ...                                 'required':True,
>>> ...                                 'style':'free form',
>>> ...                                 'description': 'new attribute desc.',
>>> ...                                 'units':None,
>>> ...                                 'options':[],
>>> ...                                 'alias':[],
>>> ...                                 'example':'new attribute

Tip

If you want ot add stations, reports, etc to the survey this should be done from the MTH5 object. This is to avoid duplication, at least for now.

To look at what the structure of /Survey looks like:

>>> survey
/Survey:
====================
    |- Group: Filters
    -----------------
        --> Dataset: summary
    -----------------
    |- Group: Reports
    -----------------
        --> Dataset: summary
        -----------------
    |- Group: Standards
    -------------------
        --> Dataset: summary
        -----------------
    |- Group: Stations
    ------------------
        --> Dataset: summary
        -----------------
property stations_group
update_survey_metadata()[source]

update start end dates and location corners from stations_group.summary_table

mth5.helpers module

Helper functions for HDF5

Created on Tue Jun 2 12:37:50 2020

copyright

Jared Peacock (jpeacock@usgs.gov)

license

MIT

mth5.helpers.close_open_files()[source]
mth5.helpers.get_tree(parent)[source]

Simple function to recursively print the contents of an hdf5 group :param parent: HDF5 (sub-)tree to print :type parent: h5py.Group

mth5.helpers.recursive_hdf5_tree(group, lines=[])[source]
mth5.helpers.validate_compression(compression, level)[source]

validate that the input compression is supported.

Parameters
  • compression (string, [ 'lzf' | 'gzip' | 'szip' | None ]) – type of lossless compression

  • level (string for 'szip' or int for 'gzip') – compression level if supported

Returns

compression type

Return type

string

Returns

compressiong level

Return type

string for ‘szip’ or int for ‘gzip’

Raises

ValueError if comporession or level are not supported

Raises

TypeError if compression level is not a string

mth5.metadata module

metadata

This module deals with metadata as defined by the MT metadata standards. metadata documentation.

There are multiple containers for each type of metadata, named appropriately.

Each container will be able to read and write:
  • dictionary

  • json

  • xml

  • csv?

  • pandas.Series

  • anything else?

Because a lot of the name words in the metadata are split by ‘.’ there are some issues we need to deal with. I wrote in get and set attribute functions to handle these types of names so the user shouldn’t have to work about splitting the names themselves.

These containers will be the building blocks for the metadata and how they are interchanged between the HDF5 file and the user. A lot of the metadata you can get directly from the raw time series files, but the user will need to input a decent amount on their own. Dictionaries are the most fundamental type we should be dealing with.

Each container has an attribute called _attr_dict which dictates if the attribute is included in output objects, the data type, whether it is a required parameter, and the style of output. This should help down the road with validation and keeping the data types consistent. And if things change you should only have to changes these dictionaries.

self._attr_dict = {‘nameword’:{‘type’: str, ‘required’: True, ‘style’: ‘name’}}

Created on Sun Apr 24 20:50:41 2020

copyright

Jared Peacock (jpeacock@usgs.gov)

license

MIT

class mth5.metadata.Auxiliary(**kwargs)[source]

Bases: mth5.metadata.Channel

Metadata Key

Description

Example

channel_number

Required: True

Units: None

Type: integer

Style: number

channel number on the data logger

1

comments

Required: False

Units: None

Type: string

Style: free form

any comments about the channel

ambient air temperature

component

Required: True

Units: None

Type: string

Style: controlled vocabulary

name of the component measured

T

measurement_azimuth

Required: True

Units: degrees

Type: float

Style: number

azimuth of channel in measurement coordinate system

0

measurement_tilt

Required: True

Units: degrees

Type: float

Style: number

tilt of channel in measurement coordinate system

0

sample_rate

Required: True

Units: samples per second

Type: float

Style: number

sample rate

8

translated_azimuth

Required: False

Units: degrees

Type: float

Style: number

azimuth of channel in translated coordinate system

0

translated_tilt

Required: False

Units: degrees

Type: float

Style: number

tilt of channel in translated coordinate system

0

type

Required: True

Units: None

Type: string

Style: free form

data type for the channel

temperature

units

Required: True

Units: None

Type: string

Style: controlled vocabulary

units of the data

celsius

data_quality.warnings

Required: False

Units: None

Type: string

Style: free form

any warnings about the data that should be noted

periodic pipeline noise

data_quality.rating.author

Required: False

Units: None

Type: string

Style: free form

author of who rated the data

gradstudet ace

data_quality.rating.method

Required: False

Units: None

Type: string

Style: free form

the method used to rate the data

standard deviation

data_quality.rating.value

Required: True

Units: None

Type: integer

Style: number

a rating from 1-5 where 1 is bad and 5 is good and 0 if unrated

4

filter.name

Required: True

Units: None

Type: string

Style: name list

name of filter applied or to be applies. If more than one filter input as a comma separated list

“[counts2mv, lo wpass_magnetic] “

filter.applied

Required: True

Units: None

Type: boolean

Style: name list

boolean if filter has been applied or not. If more than one filter input as a comma separated list. Needs to be the same length as name or if only one entry is given it is assumed to apply to all filters listed.

“[True, False]”

filter.comments

Required: False

Units: None

Type: string

Style: name

any comments on filters

low pass is not calibrated

time_period.end

Required: True

Units: None

Type: string

Style: time

end date and time of collection in UTC

2020-02-04T16:2 3:45.453670+00: 00

time_period.start

Required: True

Units: None

Type: string

Style: time

start date and time of collection in UTC

2020-02-01T09:2 3:45.453670+00: 00

sensor.id

Required: True

Units: None

Type: string

Style: free form

instrument ID number can be serial number or a designated ID

mt01

sensor.manufacturer

Required: True

Units: None

Type: string

Style: free form

who manufactured the instrument

mt gurus

sensor.type

Required: True

Units: None

Type: string

Style: free form

instrument type

broadband 32-bit

sensor.model

Required: False

Units: None

Type: string

Style: free form

model version of the instrument

falcon5

fdsn.id

Required: False

Units: None

Type: string

Style: alpha numeric

Given FDSN archive ID name

MT001

fdsn.network

Required: False

Units: None

Type: string

Style: alpha numeric

Given two character FDSN archive network code

EM

fdsn.channel_code

Required: False

Units: None

Type: string

Style: alpha numeric

Three character FDSN channel code

LQN

fdsn.new_epoch

Required: False

Units: None

Type: boolean

Style: name

Boolean telling if a new epoch needs to be created or not

False

location.latitude

Required: True

Units: degrees

Type: float

Style: number

latitude of location in datum specified at survey level

23.134

location.longitude

Required: True

Units: degrees

Type: float

Style: number

longitude of location in datum specified at survey level

14.23

location.elevation

Required: True

Units: degrees

Type: float

Style: number

elevation of location in datum specified at survey level

123.4

class mth5.metadata.Base(attr_dict={}, **kwargs)[source]

Bases: object

A Base class that is common to most of the Metadata objects

Includes:

  • to_json

  • from_json

  • to_dict

  • from_dict

  • to_series

  • from_series

add_base_attribute(name, value, value_dict)[source]

Add an attribute to _attr_dict so it will be included in the output dictionary

Parameters
  • name (string) – name of attribute

  • value (described in value_dict) – value of the new attribute

  • value_dict

    dictionary describing the attribute, must have keys [‘type’, ‘required’, ‘style’, ‘units’, ‘alias’, ‘description’,

    ’options’, ‘example’]

  • type –> the data type [ str | int | float | bool ]

  • required –> required in the standards [ True | False ]

  • style –> style of the string

  • units –> units of the attribute, must be a string

  • alias –> other possible names for the attribute

  • options –> if only a few options are accepted, separated by | or comma.b [ option_01 | option_02 | other ]. ‘other’ means other options available but not yet defined.

  • example –> an example of the attribute

Example

>>> extra = {'type': str,
>>> ...      'style': 'controlled vocabulary',
>>> ...      'required': False,
>>> ...      'units': celsius,
>>> ...      'description': 'local temperature',
>>> ...      'alias': ['temp'],
>>> ...      'options': [ 'ambient', 'air', 'other'],
>>> ...      'example': 'ambient'}
>>> r = Run()
>>> r.add_base_attribute('temperature', 'ambient', extra)
attribute_information(name=None)[source]

return a descriptive string of the attribute if none returns for all

Parameters

key (TYPE, optional) – DESCRIPTION, defaults to None

Returns

DESCRIPTION

Return type

TYPE

from_dict(meta_dict)[source]

fill attributes from a dictionary

Parameters

meta_dict (dictionary) – dictionary with keys equal to metadata.

from_json(json_str)[source]

read in a json string and update attributes of an object

Parameters

json_str (string) – json string

from_series(pd_series)[source]

Fill attributes from a Pandas series

Note

Currently, the series must be single layered with key names separated by dots. (location.latitude)

Parameters

pd_series (pandas.Series) – Series containing metadata information

..todo:: Force types in series

from_xml(xml_element)[source]
Parameters

xml_element (etree.Element) – XML element

Returns

Fills attributes accordingly

get_attr_from_name(name)[source]

Access attribute from the given name.

The name can contain the name of an object which must be separated by a ‘.’ for e.g. {object_name}.{name} –> location.latitude

..note:: this is a helper function for names with ‘.’ in the name for

easier getting when reading from dictionary.

Parameters

name (string) – name of attribute to get.

Returns

attribute value

Return type

type is defined by the attribute name

Example

>>> b = Base(**{'category.test_attr':10})
>>> b.get_attr_from_name('category.test_attr')
10
get_attribute_list()[source]

return a list of the attributes

set_attr_from_name(name, value)[source]

Helper function to set attribute from the given name.

The name can contain the name of an object which must be separated by a ‘.’ for e.g. {object_name}.{name} –> location.latitude

..note:: this is a helper function for names with ‘.’ in the name for

easier getting when reading from dictionary.

Parameters
  • name (string) – name of attribute to get.

  • value (type is defined by the attribute name) – attribute value

Example

>>> b = Base(**{'category.test_attr':10})
>>> b.set_attr_from_name('category.test_attr', '10')
>>> print(b.category.test_attr)
'10'
to_dict(nested=False, single=False, required=True)[source]

make a dictionary from attributes, makes dictionary from _attr_list.

Parameters
  • nested ([ True | False ] , default is False) – make the returned dictionary nested

  • single ([ True | False ], default is False) – return just metadata dictionary -> meta_dict[class_name]

  • required – return just the required elements and any elements with non-None values

to_json(nested=False, indent='    ', required=True)[source]

Write a json string from a given object, taking into account other class objects contained within the given object.

Parameters

nested ([ True | False ] , default is False) – make the returned json nested

to_series(required=True)[source]

Convert attribute list to a pandas.Series

Note

this is a flattened version of the metadata

Returns

pandas.Series

Return type

pandas.Series

to_xml(string=False, required=True)[source]

make an xml element for the attribute that will add types and units.

Parameters

string ([ True | False ], default is False) – output a string instead of an XML element

Returns

XML element or string

class mth5.metadata.Battery(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

type

Required: False

Units: None

Type: string

Style: name

battery type

pb-acid gel cell

id

Required: False

Units: None

Type: string

Style: name

battery id

battery01

voltage.start

Required: False

Units: volts

Type: float

Style: number

starting voltage

14.3

voltage.end

Required: False

Units: volts

Type: float

Style: number

end voltage

12.1

comments

Required: False

Units: None

Type: string

Style: name

any comment about the battery

this is a comment

class mth5.metadata.Channel(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

channel_number

Required: True

Units: None

Type: integer

Style: number

channel number on the data logger

1

comments

Required: False

Units: None

Type: string

Style: free form

any comments about the channel

ambient air temperature

component

Required: True

Units: None

Type: string

Style: controlled vocabulary

name of the component measured

T

measurement_azimuth

Required: True

Units: degrees

Type: float

Style: number

azimuth of channel in measurement coordinate system

0

measurement_tilt

Required: True

Units: degrees

Type: float

Style: number

tilt of channel in measurement coordinate system

0

sample_rate

Required: True

Units: samples per second

Type: float

Style: number

sample rate

8

translated_azimuth

Required: False

Units: degrees

Type: float

Style: number

azimuth of channel in translated coordinate system

0

translated_tilt

Required: False

Units: degrees

Type: float

Style: number

tilt of channel in translated coordinate system

0

type

Required: True

Units: None

Type: string

Style: free form

data type for the channel

temperature

units

Required: True

Units: None

Type: string

Style: controlled vocabulary

units of the data

celsius

data_quality.warnings

Required: False

Units: None

Type: string

Style: free form

any warnings about the data that should be noted

periodic pipeline noise

data_quality.rating.author

Required: False

Units: None

Type: string

Style: free form

author of who rated the data

gradstudet ace

data_quality.rating.method

Required: False

Units: None

Type: string

Style: free form

the method used to rate the data

standard deviation

data_quality.rating.value

Required: True

Units: None

Type: integer

Style: number

a rating from 1-5 where 1 is bad and 5 is good and 0 if unrated

4

filter.name

Required: True

Units: None

Type: string

Style: name list

name of filter applied or to be applies. If more than one filter input as a comma separated list

“[counts2mv, lo wpass_magnetic] “

filter.applied

Required: True

Units: None

Type: boolean

Style: name list

boolean if filter has been applied or not. If more than one filter input as a comma separated list. Needs to be the same length as name or if only one entry is given it is assumed to apply to all filters listed.

“[True, False]”

filter.comments

Required: False

Units: None

Type: string

Style: name

any comments on filters

low pass is not calibrated

time_period.end

Required: True

Units: None

Type: string

Style: time

end date and time of collection in UTC

2020-02-04T16:2 3:45.453670+00: 00

time_period.start

Required: True

Units: None

Type: string

Style: time

start date and time of collection in UTC

2020-02-01T09:2 3:45.453670+00: 00

sensor.id

Required: True

Units: None

Type: string

Style: free form

instrument ID number can be serial number or a designated ID

mt01

sensor.manufacturer

Required: True

Units: None

Type: string

Style: free form

who manufactured the instrument

mt gurus

sensor.type

Required: True

Units: None

Type: string

Style: free form

instrument type

broadband 32-bit

sensor.model

Required: False

Units: None

Type: string

Style: free form

model version of the instrument

falcon5

fdsn.id

Required: False

Units: None

Type: string

Style: alpha numeric

Given FDSN archive ID name

MT001

fdsn.network

Required: False

Units: None

Type: string

Style: alpha numeric

Given two character FDSN archive network code

EM

fdsn.channel_code

Required: False

Units: None

Type: string

Style: alpha numeric

Three character FDSN channel code

LQN

fdsn.new_epoch

Required: False

Units: None

Type: boolean

Style: name

Boolean telling if a new epoch needs to be created or not

False

location.latitude

Required: True

Units: degrees

Type: float

Style: number

latitude of location in datum specified at survey level

23.134

location.longitude

Required: True

Units: degrees

Type: float

Style: number

longitude of location in datum specified at survey level

14.23

location.elevation

Required: True

Units: degrees

Type: float

Style: number

elevation of location in datum specified at survey level

123.4

property component
class mth5.metadata.Citation(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

doi

Required: True

Units: None

Type: string

Style: url

full url of the doi number

http://doi.###

class mth5.metadata.Copyright(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

release_license

Required: True

Units: None

Type: string

Style: controlled vocabulary

How the data can be used. The options are based on Creative Commons licenses. For details visit https://creativecommons.org/licenses/

CC-0

class mth5.metadata.DataLogger(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

id

Required: True

Units: None

Type: string

Style: free form

instrument ID number can be serial number or a designated ID

mt01

manufacturer

Required: True

Units: None

Type: string

Style: free form

who manufactured the instrument

mt gurus

type

Required: True

Units: None

Type: string

Style: free form

instrument type

broadband 32-bit

model

Required: False

Units: None

Type: string

Style: free form

model version of the instrument

falcon5

timing_system.comments

Required: False

Units: None

Type: string

Style: free form

any comment on timing system

GPS locked with internal quartz clock

timing_system.drift

Required: True

Units: seconds

Type: float

Style: number

estimated drift of the timing system

0.001

timing_system.type

Required: True

Units: None

Type: string

Style: free form

type of timing system

GPS

timing_system.uncertainty

Required: True

Units: seconds

Type: float

Style: number

estimated uncertainty of the timing system

0.0002

firmware.author

Required: True

Units: None

Type: string

Style: free form

author of the software

nerd alert

firmware.version

Required: True

Units: None

Type: string

Style: free form

software version

12.01a

firmware.name

Required: True

Units: None

Type: string

Style: free form

software name

mtrules

power_source.type

Required: False

Units: None

Type: string

Style: name

battery type

pb-acid gel cell

power_source.id

Required: False

Units: None

Type: string

Style: name

battery id

battery01

power_source.voltage.start

Required: False

Units: volts

Type: float

Style: number

starting voltage

14.3

power_source.voltage.end

Required: False

Units: volts

Type: float

Style: number

end voltage

12.1

power_source.comments

Required: False

Units: None

Type: string

Style: name

any comment about the battery

this is a comment

class mth5.metadata.DataQuality(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

warnings

Required: False

Units: None

Type: string

Style: free form

any warnings about the data that should be noted

periodic pipeline noise

rating.author

Required: False

Units: None

Type: string

Style: free form

author of who rated the data

gradstudet ace

rating.method

Required: False

Units: None

Type: string

Style: free form

the method used to rate the data

standard deviation

rating.value

Required: True

Units: None

Type: integer

Style: number

a rating from 1-5 where 1 is bad and 5 is good and 0 if unrated

4

class mth5.metadata.Declination(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

comments

Required: False

Units: None

Type: string

Style: free form

any comments on declination

estimated from WMM 2016

model

Required: True

Units: None

Type: string

Style: controlled vocabulary

geomagnetic reference model used to calculate declination

WMM

value

Required: True

Units: degrees

Type: float

Style: number

declination angle relative to geographic north positive clockwise

12.3

class mth5.metadata.Diagnostic(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

end

Required: False

Units: None

Type: float

Style: number

end value of a diagnostic measurement

10

start

Required: False

Units: None

Type: float

Style: number

start value of a diagnostic measurement

12.3

class mth5.metadata.Electric(**kwargs)[source]

Bases: mth5.metadata.Channel

Metadata Key

Description

Example

dipole_length

Required: True

Units: meters

Type: float

Style: number

length of the dipole

55.25

contact_resistance.start

Required: False

Units: ohms

Type: float

Style: number list

starting contact resistance; if more than one measurement input as a list of number [1 2 3 …]

“[1.2, 1.4]”

contact_resistance.end

Required: False

Units: ohms

Type: float

Style: number list

starting contact resistance; if more than one measurement input as a list of number [1 2 3 …]

“[1.5, 1.8]”

ac.start

Required: False

Units: volts

Type: float

Style: number

starting AC value; if more than one measurement input as a list of number [1 2 3 …]

“[52.1, 55.8]”

ac.end

Required: False

Units: volts

Type: float

Style: number

ending AC value; if more than one measurement input as a list of number [1 2 3 …]

“[45.3, 49.5]”

dc.start

Required: False

Units: volts

Type: float

Style: number

starting DC value; if more than one measurement input as a list of number [1 2 3 …]

1.1

dc.end

Required: False

Units: volts

Type: float

Style: number

ending DC value; if more than one measurement input as a list of number [1 2 3 …]

1.5

channel_number

Required: True

Units: None

Type: integer

Style: number

channel number on the data logger

1

comments

Required: False

Units: None

Type: string

Style: free form

any comments about the channel

ambient air temperature

component

Required: True

Units: None

Type: string

Style: controlled vocabulary

name of the component measured

T

measurement_azimuth

Required: True

Units: degrees

Type: float

Style: number

azimuth of channel in measurement coordinate system

0

measurement_tilt

Required: True

Units: degrees

Type: float

Style: number

tilt of channel in measurement coordinate system

0

sample_rate

Required: True

Units: samples per second

Type: float

Style: number

sample rate

8

translated_azimuth

Required: False

Units: degrees

Type: float

Style: number

azimuth of channel in translated coordinate system

0

translated_tilt

Required: False

Units: degrees

Type: float

Style: number

tilt of channel in translated coordinate system

0

type

Required: True

Units: None

Type: string

Style: free form

data type for the channel

temperature

units

Required: True

Units: None

Type: string

Style: controlled vocabulary

units of the data

celsius

data_quality.warnings

Required: False

Units: None

Type: string

Style: free form

any warnings about the data that should be noted

periodic pipeline noise

data_quality.rating.author

Required: False

Units: None

Type: string

Style: free form

author of who rated the data

gradstudet ace

data_quality.rating.method

Required: False

Units: None

Type: string

Style: free form

the method used to rate the data

standard deviation

data_quality.rating.value

Required: True

Units: None

Type: integer

Style: number

a rating from 1-5 where 1 is bad and 5 is good and 0 if unrated

4

filter.name

Required: True

Units: None

Type: string

Style: name list

name of filter applied or to be applies. If more than one filter input as a comma separated list

“[counts2mv, lo wpass_magnetic] “

filter.applied

Required: True

Units: None

Type: boolean

Style: name list

boolean if filter has been applied or not. If more than one filter input as a comma separated list. Needs to be the same length as name or if only one entry is given it is assumed to apply to all filters listed.

“[True, False]”

filter.comments

Required: False

Units: None

Type: string

Style: name

any comments on filters

low pass is not calibrated

positive.id

Required: True

Units: None

Type: string

Style: free form

instrument ID number can be serial number or a designated ID

mt01

positive.manufacturer

Required: True

Units: None

Type: string

Style: free form

who manufactured the instrument

mt gurus

positive.type

Required: True

Units: None

Type: string

Style: free form

instrument type

broadband 32-bit

positive.model

Required: False

Units: None

Type: string

Style: free form

model version of the instrument

falcon5

positive.latitude

Required: True

Units: degrees

Type: float

Style: number

latitude of location in datum specified at survey level

23.134

positive.longitude

Required: True

Units: degrees

Type: float

Style: number

longitude of location in datum specified at survey level

14.23

positive.elevation

Required: True

Units: degrees

Type: float

Style: number

elevation of location in datum specified at survey level

123.4

negative.id

Required: True

Units: None

Type: string

Style: free form

instrument ID number can be serial number or a designated ID

mt01

negative.manufacturer

Required: True

Units: None

Type: string

Style: free form

who manufactured the instrument

mt gurus

negative.type

Required: True

Units: None

Type: string

Style: free form

instrument type

broadband 32-bit

negative.model

Required: False

Units: None

Type: string

Style: free form

model version of the instrument

falcon5

negative.latitude

Required: True

Units: degrees

Type: float

Style: number

latitude of location in datum specified at survey level

23.134

negative.longitude

Required: True

Units: degrees

Type: float

Style: number

longitude of location in datum specified at survey level

14.23

negative.elevation

Required: True

Units: degrees

Type: float

Style: number

elevation of location in datum specified at survey level

123.4

time_period.end

Required: True

Units: None

Type: string

Style: time

end date and time of collection in UTC

2020-02-04T16:2 3:45.453670+00: 00

time_period.start

Required: True

Units: None

Type: string

Style: time

start date and time of collection in UTC

2020-02-01T09:2 3:45.453670+00: 00

class mth5.metadata.Electrode(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

id

Required: True

Units: None

Type: string

Style: free form

instrument ID number can be serial number or a designated ID

mt01

manufacturer

Required: True

Units: None

Type: string

Style: free form

who manufactured the instrument

mt gurus

type

Required: True

Units: None

Type: string

Style: free form

instrument type

broadband 32-bit

model

Required: False

Units: None

Type: string

Style: free form

model version of the instrument

falcon5

latitude

Required: True

Units: degrees

Type: float

Style: number

latitude of location in datum specified at survey level

23.134

longitude

Required: True

Units: degrees

Type: float

Style: number

longitude of location in datum specified at survey level

14.23

elevation

Required: True

Units: degrees

Type: float

Style: number

elevation of location in datum specified at survey level

123.4

class mth5.metadata.Fdsn(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

id

Required: False

Units: None

Type: string

Style: alpha numeric

Given FDSN archive ID name

MT001

network

Required: False

Units: None

Type: string

Style: alpha numeric

Given two character FDSN archive network code

EM

channel_code

Required: False

Units: None

Type: string

Style: alpha numeric

Three character FDSN channel code

LQN

new_epoch

Required: False

Units: None

Type: boolean

Style: name

Boolean telling if a new epoch needs to be created or not

False

class mth5.metadata.Filter(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

name

Required: True

Units: None

Type: string

Style: name list

name of filter applied or to be applies. If more than one filter input as a comma separated list

“[counts2mv, lo wpass_magnetic] “

type

Required: True

Units: None

Type: string

Style: controlled vocabulary

type of filter

table

units_in

Required: True

Units: None

Type: string

Style: alpha numeric

name of the input units to the filter. Should be all lowercase and separated with an underscore

count

units_out

Required: True

Units: None

Type: string

Style: alpha numeric

name of the output units. Should be alpha numeric.

millivolt

calibration_date

Required: False

Units: None

Type: string

Style: date

latest date of filter calibration

2/1/2020

normalization_factor

Required: False

Units: None

Type: float

Style: number

normalization factor for a poles and zeros filter

2.34

normalization_frequency

Required: False

Units: None

Type: float

Style: number

normalization frequency for a poles and zeros filter

1

cutoff

Required: False

Units: None

Type: float

Style: number

cutoff frequency for filter

1

operation

Required: False

Units: None

Type: float

Style: controlled vocabulary

how the filter should be applied

multiply

n_poles

Required: False

Units: None

Type: integer

Style: number

how many poles are in the filter

4

n_zeros

Required: False

Units: None

Type: integer

Style: number

how many zeros are in the filter

2

conversion_factor

Required: False

Units: None

Type: float

Style: number

unit conversion factor

100

property calibration_date
class mth5.metadata.Filtered(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

name

Required: True

Units: None

Type: string

Style: name list

name of filter applied or to be applies. If more than one filter input as a comma separated list

“[counts2mv, lo wpass_magnetic] “

applied

Required: True

Units: None

Type: boolean

Style: name list

boolean if filter has been applied or not. If more than one filter input as a comma separated list. Needs to be the same length as name or if only one entry is given it is assumed to apply to all filters listed.

“[True, False]”

comments

Required: False

Units: None

Type: string

Style: name

any comments on filters

low pass is not calibrated

property applied
property name
class mth5.metadata.Instrument(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

id

Required: True

Units: None

Type: string

Style: free form

instrument ID number can be serial number or a designated ID

mt01

manufacturer

Required: True

Units: None

Type: string

Style: free form

who manufactured the instrument

mt gurus

type

Required: True

Units: None

Type: string

Style: free form

instrument type

broadband 32-bit

model

Required: False

Units: None

Type: string

Style: free form

model version of the instrument

falcon5

class mth5.metadata.Location(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

latitude

Required: True

Units: degrees

Type: float

Style: number

latitude of location in datum specified at survey level

23.134

longitude

Required: True

Units: degrees

Type: float

Style: number

longitude of location in datum specified at survey level

14.23

elevation

Required: True

Units: degrees

Type: float

Style: number

elevation of location in datum specified at survey level

123.4

declination.comments

Required: False

Units: None

Type: string

Style: free form

any comments on declination

estimated from WMM 2016

declination.model

Required: True

Units: None

Type: string

Style: controlled vocabulary

geomagnetic reference model used to calculate declination

WMM

declination.value

Required: True

Units: degrees

Type: float

Style: number

declination angle relative to geographic north positive clockwise

12.3

property elevation
property latitude
property longitude
class mth5.metadata.Magnetic(**kwargs)[source]

Bases: mth5.metadata.Channel

Metadata Key

Description

Example

h_field_min.start

Required: False

Units: nt

Type: float

Style: number

minimum magnetic field strength at beginning of measurement

40345.1

h_field_min.end

Required: False

Units: nt

Type: float

Style: number

minimum magnetic field strength at end of measurement

50453.2

h_field_max.start

Required: False

Units: nt

Type: float

Style: number

maximum magnetic field strength at beginning of measurement

34565.2

h_field_max.end

Required: False

Units: nt

Type: float

Style: number

maximum magnetic field strength at end of measurement

34526.1

channel_number

Required: True

Units: None

Type: integer

Style: number

channel number on the data logger

1

comments

Required: False

Units: None

Type: string

Style: free form

any comments about the channel

ambient air temperature

component

Required: True

Units: None

Type: string

Style: controlled vocabulary

name of the component measured

T

measurement_azimuth

Required: True

Units: degrees

Type: float

Style: number

azimuth of channel in measurement coordinate system

0

measurement_tilt

Required: True

Units: degrees

Type: float

Style: number

tilt of channel in measurement coordinate system

0

sample_rate

Required: True

Units: samples per second

Type: float

Style: number

sample rate

8

translated_azimuth

Required: False

Units: degrees

Type: float

Style: number

azimuth of channel in translated coordinate system

0

translated_tilt

Required: False

Units: degrees

Type: float

Style: number

tilt of channel in translated coordinate system

0

type

Required: True

Units: None

Type: string

Style: free form

data type for the channel

temperature

units

Required: True

Units: None

Type: string

Style: controlled vocabulary

units of the data

celsius

data_quality.warnings

Required: False

Units: None

Type: string

Style: free form

any warnings about the data that should be noted

periodic pipeline noise

data_quality.rating.author

Required: False

Units: None

Type: string

Style: free form

author of who rated the data

gradstudet ace

data_quality.rating.method

Required: False

Units: None

Type: string

Style: free form

the method used to rate the data

standard deviation

data_quality.rating.value

Required: True

Units: None

Type: integer

Style: number

a rating from 1-5 where 1 is bad and 5 is good and 0 if unrated

4

filter.name

Required: True

Units: None

Type: string

Style: name list

name of filter applied or to be applies. If more than one filter input as a comma separated list

“[counts2mv, lo wpass_magnetic] “

filter.applied

Required: True

Units: None

Type: boolean

Style: name list

boolean if filter has been applied or not. If more than one filter input as a comma separated list. Needs to be the same length as name or if only one entry is given it is assumed to apply to all filters listed.

“[True, False]”

filter.comments

Required: False

Units: None

Type: string

Style: name

any comments on filters

low pass is not calibrated

time_period.end

Required: True

Units: None

Type: string

Style: time

end date and time of collection in UTC

2020-02-04T16:2 3:45.453670+00: 00

time_period.start

Required: True

Units: None

Type: string

Style: time

start date and time of collection in UTC

2020-02-01T09:2 3:45.453670+00: 00

sensor.id

Required: True

Units: None

Type: string

Style: free form

instrument ID number can be serial number or a designated ID

mt01

sensor.manufacturer

Required: True

Units: None

Type: string

Style: free form

who manufactured the instrument

mt gurus

sensor.type

Required: True

Units: None

Type: string

Style: free form

instrument type

broadband 32-bit

sensor.model

Required: False

Units: None

Type: string

Style: free form

model version of the instrument

falcon5

fdsn.id

Required: False

Units: None

Type: string

Style: alpha numeric

Given FDSN archive ID name

MT001

fdsn.network

Required: False

Units: None

Type: string

Style: alpha numeric

Given two character FDSN archive network code

EM

fdsn.channel_code

Required: False

Units: None

Type: string

Style: alpha numeric

Three character FDSN channel code

LQN

fdsn.new_epoch

Required: False

Units: None

Type: boolean

Style: name

Boolean telling if a new epoch needs to be created or not

False

location.latitude

Required: True

Units: degrees

Type: float

Style: number

latitude of location in datum specified at survey level

23.134

location.longitude

Required: True

Units: degrees

Type: float

Style: number

longitude of location in datum specified at survey level

14.23

location.elevation

Required: True

Units: degrees

Type: float

Style: number

elevation of location in datum specified at survey level

123.4

class mth5.metadata.Orientation(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

method

Required: True

Units: None

Type: string

Style: controlled vocabulary

method for orienting station layout

compass

reference_frame

Required: True

Units: None

Type: string

Style: controlled vocabulary

“Reference frame for station layout. There are only 2 options geographic and geomagnetic. Both assume a right-handed coordinate system with North=0 E=90 and vertical positive downward”

geomagnetic

class mth5.metadata.Person(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

author

Required: True

Units: None

Type: string

Style: free form

author name

person name

organization

Required: True

Units: None

Type: string

Style: free form

organization name

mt gurus

email

Required: True

Units: None

Type: string

Style: email

email of the contact person

mt.guru@em.org

comments

Required: False

Units: None

Type: string

Style: email

email of the contact person

expert digger

class mth5.metadata.Provenance(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

creation_time

Required: True

Units: None

Type: string

Style: date time

date and time the file was created

2020-02-08T12:2 3:40.324600+00: 00

comments

Required: False

Units: None

Type: string

Style: free form

any comments on provenance of the data

all good

log

Required: False

Units: None

Type: string

Style: free form

a history of changes made to the data

2020-02-10T14:2 4:45+00:00 updated metadata

software.author

Required: True

Units: None

Type: string

Style: free form

author of the software

nerd alert

software.version

Required: True

Units: None

Type: string

Style: free form

software version

12.01a

software.name

Required: True

Units: None

Type: string

Style: free form

software name

mtrules

person.author

Required: True

Units: None

Type: string

Style: free form

author name

person name

person.organization

Required: True

Units: None

Type: string

Style: free form

organization name

mt gurus

person.email

Required: True

Units: None

Type: string

Style: email

email of the contact person

mt.guru@em.org

person.comments

Required: False

Units: None

Type: string

Style: email

email of the contact person

expert digger

property creation_time
class mth5.metadata.Rating(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

author

Required: False

Units: None

Type: string

Style: free form

author of who rated the data

gradstudet ace

method

Required: False

Units: None

Type: string

Style: free form

the method used to rate the data

standard deviation

value

Required: True

Units: None

Type: integer

Style: number

a rating from 1-5 where 1 is bad and 5 is good and 0 if unrated

4

class mth5.metadata.Run(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

channels_recorded_auxiliary

Required: True

Units: None

Type: string

Style: name list

list of auxiliary channels recorded

[T]

channels_recorded_electric

Required: True

Units: None

Type: string

Style: name list

list of electric channels recorded

“[Ex | Ey]”

channels_recorded_magnetic

Required: True

Units: None

Type: string

Style: name list

list of magnetic channels recorded

“[Hx | Hy | Hz]”

comments

Required: False

Units: None

Type: string

Style: free form

any comments on the run

cows chewed cables

data_type

Required: True

Units: None

Type: string

Style: controlled vocabulary

type of data recoreded for this run

BBMT

id

Required: True

Units: None

Type: string

Style: alpha numeric

run ID should be station name followed by an alphabet letter for the run

mt02b

sample_rate

Required: True

Units: samples per second

Type: float

Style: number

rate of sampling renureded for this run

100

fdsn.id

Required: False

Units: None

Type: string

Style: alpha numeric

Given FDSN archive ID name

MT001

fdsn.network

Required: False

Units: None

Type: string

Style: alpha numeric

Given two character FDSN archive network code

EM

fdsn.channel_code

Required: False

Units: None

Type: string

Style: alpha numeric

Three character FDSN channel code

LQN

fdsn.new_epoch

Required: False

Units: None

Type: boolean

Style: name

Boolean telling if a new epoch needs to be created or not

False

data_logger.id

Required: True

Units: None

Type: string

Style: free form

instrument ID number can be serial number or a designated ID

mt01

data_logger.manufacturer

Required: True

Units: None

Type: string

Style: free form

who manufactured the instrument

mt gurus

data_logger.type

Required: True

Units: None

Type: string

Style: free form

instrument type

broadband 32-bit

data_logger.model

Required: False

Units: None

Type: string

Style: free form

model version of the instrument

falcon5

data_logger.timing_system.comments

Required: False

Units: None

Type: string

Style: free form

any comment on timing system

GPS locked with internal quartz clock

data_logger.timing_system.drift

Required: True

Units: seconds

Type: float

Style: number

estimated drift of the timing system

0.001

data_logger.timing_system.type

Required: True

Units: None

Type: string

Style: free form

type of timing system

GPS

data_logger.timing_system.uncertainty

Required: True

Units: seconds

Type: float

Style: number

estimated uncertainty of the timing system

0.0002

data_logger.firmware.author

Required: True

Units: None

Type: string

Style: free form

author of the software

nerd alert

data_logger.firmware.version

Required: True

Units: None

Type: string

Style: free form

software version

12.01a

data_logger.firmware.name

Required: True

Units: None

Type: string

Style: free form

software name

mtrules

data_logger.power_source.type

Required: False

Units: None

Type: string

Style: name

battery type

pb-acid gel cell

data_logger.power_source.id

Required: False

Units: None

Type: string

Style: name

battery id

battery01

data_logger.power_source.voltage.start

Required: False

Units: volts

Type: float

Style: number

starting voltage

14.3

data_logger.power_source.voltage.end

Required: False

Units: volts

Type: float

Style: number

end voltage

12.1

data_logger.power_source.comments

Required: False

Units: None

Type: string

Style: name

any comment about the battery

this is a comment

time_period.end

Required: True

Units: None

Type: string

Style: time

end date and time of collection in UTC

2020-02-04T16:2 3:45.453670+00: 00

time_period.start

Required: True

Units: None

Type: string

Style: time

start date and time of collection in UTC

2020-02-01T09:2 3:45.453670+00: 00

acquired_by.author

Required: True

Units: None

Type: string

Style: free form

author name

person name

acquired_by.comments

Required: False

Units: None

Type: string

Style: email

email of the contact person

expert digger

metadata_by.author

Required: True

Units: None

Type: string

Style: free form

author name

person name

metadata_by.comments

Required: False

Units: None

Type: string

Style: email

email of the contact person

expert digger

provenance.comments

Required: False

Units: None

Type: string

Style: free form

any comments on provenance of the data

all good

provenance.log

Required: False

Units: None

Type: string

Style: free form

a history of changes made to the data

2020-02-10T14:2 4:45+00:00 updated metadata

property channels_recorded_all

a list of all channels recorded :rtype: TYPE

Type

return

property n_channels
class mth5.metadata.Software(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

author

Required: True

Units: None

Type: string

Style: free form

author of the software

nerd alert

version

Required: True

Units: None

Type: string

Style: free form

software version

12.01a

name

Required: True

Units: None

Type: string

Style: free form

software name

mtrules

property author
class mth5.metadata.Station(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

channel_layout

Required: False

Units: None

Type: string

Style: controlled vocabulary

how the station was laid out

x

channels_recorded

Required: True

Units: None

Type: string

Style: name list

list of components recorded by the station. Should be a summary of all channels recorded dropped channels will be recorded in Run metadata.

“[ Ex, Ey, Hx, Hy, Hz, T]”

comments

Required: False

Units: None

Type: string

Style: free form

any comments on the station

5 runs

data_type

Required: True

Units: None

Type: string

Style: controlled vocabulary

type of data recorded. If multiple types input as a comma separated list

BBMT

geographic_name

Required: True

Units: None

Type: string

Style: free form

closest geographic name to the station

“Whitehorse, YK”

id

Required: True

Units: None

Type: string

Style: alpha numeric

Station ID name. This should be an alpha numeric name that is typically 5-6 characters long. Commonly the project name in 2 or 3 letters and the station number.

MT001

fdsn.id

Required: False

Units: None

Type: string

Style: alpha numeric

Given FDSN archive ID name

MT001

fdsn.network

Required: False

Units: None

Type: string

Style: alpha numeric

Given two character FDSN archive network code

EM

fdsn.channel_code

Required: False

Units: None

Type: string

Style: alpha numeric

Three character FDSN channel code

LQN

fdsn.new_epoch

Required: False

Units: None

Type: boolean

Style: name

Boolean telling if a new epoch needs to be created or not

False

location.latitude

Required: True

Units: degrees

Type: float

Style: number

latitude of location in datum specified at survey level

23.134

location.longitude

Required: True

Units: degrees

Type: float

Style: number

longitude of location in datum specified at survey level

14.23

location.elevation

Required: True

Units: degrees

Type: float

Style: number

elevation of location in datum specified at survey level

123.4

location.declination.comments

Required: False

Units: None

Type: string

Style: free form

any comments on declination

estimated from WMM 2016

location.declination.model

Required: True

Units: None

Type: string

Style: controlled vocabulary

geomagnetic reference model used to calculate declination

WMM

location.declination.value

Required: True

Units: degrees

Type: float

Style: number

declination angle relative to geographic north positive clockwise

12.3

acquired_by.author

Required: True

Units: None

Type: string

Style: free form

author name

person name

acquired_by.comments

Required: False

Units: None

Type: string

Style: email

email of the contact person

expert digger

orientation.method

Required: True

Units: None

Type: string

Style: controlled vocabulary

method for orienting station layout

compass

orientation.reference_frame

Required: True

Units: None

Type: string

Style: controlled vocabulary

“Reference frame for station layout. There are only 2 options geographic and geomagnetic. Both assume a right-handed coordinate system with North=0 E=90 and vertical positive downward”

geomagnetic

provenance.creation_time

Required: True

Units: None

Type: string

Style: date time

date and time the file was created

2020-02-08T12:2 3:40.324600+00: 00

provenance.comments

Required: False

Units: None

Type: string

Style: free form

any comments on provenance of the data

all good

provenance.log

Required: False

Units: None

Type: string

Style: free form

a history of changes made to the data

2020-02-10T14:2 4:45+00:00 updated metadata

provenance.software.author

Required: True

Units: None

Type: string

Style: free form

author of the software

nerd alert

provenance.software.version

Required: True

Units: None

Type: string

Style: free form

software version

12.01a

provenance.software.name

Required: True

Units: None

Type: string

Style: free form

software name

mtrules

provenance.submitter.author

Required: True

Units: None

Type: string

Style: free form

author name

person name

provenance.submitter.organization

Required: True

Units: None

Type: string

Style: free form

organization name

mt gurus

provenance.submitter.email

Required: True

Units: None

Type: string

Style: email

email of the contact person

mt.guru@em.org

time_period.end

Required: True

Units: None

Type: string

Style: time

end date and time of collection in UTC

2020-02-04T16:2 3:45.453670+00: 00

time_period.start

Required: True

Units: None

Type: string

Style: time

start date and time of collection in UTC

2020-02-01T09:2 3:45.453670+00: 00

class mth5.metadata.Survey(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

survey_id

Required: True

Units: None

Type: string

Style: alpha numeric

Alphanumeric survey ID that will be specific to the archive

EMT20

comments

Required: False

Units: None

Type: string

Style: free form

any comments about the survey

long survey

country

Required: True

Units: None

Type: string

Style: free form

country(s) countries survey located in. If multiple input as comma separated names

“[USA, Canada]”

datum

Required: True

Units: None

Type: string

Style: controlled vocabulary

datum of latitude and longitude coordinates. Should be a well-known datum [ WGS84 ] and will be the reference datum for all location

WGS84

geographic_name

Required: True

Units: None

Type: string

Style: free form

closest geographic reference to survey

Yukon

name

Required: True

Units: None

Type: string

Style: free form

descriptive name of the survey

MT Characteriza tion of Yukon Terrane

project

Required: True

Units: None

Type: string

Style: free form

alphanumeric name for the project e.g USGS- GEOMAG

YUTOO

summary

Required: True

Units: None

Type: string

Style: free form

summary paragraph of survey including the purpose; difficulties; data quality; summary of outcomes if the data have been processed and modeled

long project of characterizing mineral resources in Yukon

time_period.end_date

Required: True

Units: None

Type: string

Style: date

end date of the survey in UTC

1/2/1995

time_period.start_date

Required: True

Units: None

Type: string

Style: date

start date of the survey in UTC

1/2/2020

fdsn.id

Required: False

Units: None

Type: string

Style: alpha numeric

Given FDSN archive ID name

MT001

fdsn.network

Required: False

Units: None

Type: string

Style: alpha numeric

Given two character FDSN archive network code

EM

fdsn.channel_code

Required: False

Units: None

Type: string

Style: alpha numeric

Three character FDSN channel code

LQN

fdsn.new_epoch

Required: False

Units: None

Type: boolean

Style: name

Boolean telling if a new epoch needs to be created or not

False

acquired_by.author

Required: True

Units: None

Type: string

Style: free form

author name

person name

acquired_by.comments

Required: False

Units: None

Type: string

Style: email

email of the contact person

expert digger

citation_dataset.doi

Required: True

Units: None

Type: string

Style: url

full url of the doi number

http://doi.###

citation_journal.doi

Required: True

Units: None

Type: string

Style: url

full url of the doi number

http://doi.###

northwest_corner.latitude

Required: True

Units: degrees

Type: float

Style: number

latitude of location in datum specified at survey level

23.134

northwest_corner.longitude

Required: True

Units: degrees

Type: float

Style: number

longitude of location in datum specified at survey level

14.23

southeast_corner.latitude

Required: True

Units: degrees

Type: float

Style: number

latitude of location in datum specified at survey level

23.134

southeast_corner.longitude

Required: True

Units: degrees

Type: float

Style: number

longitude of location in datum specified at survey level

14.23

project_lead.author

Required: True

Units: None

Type: string

Style: free form

author name

person name

project_lead.organization

Required: True

Units: None

Type: string

Style: free form

organization name

mt gurus

project_lead.email

Required: True

Units: None

Type: string

Style: email

email of the contact person

mt.guru@em.org

release_license

Required: True

Units: None

Type: string

Style: controlled vocabulary

How the data can be used. The options are based on Creative Commons licenses. For details visit https://creativecommons.org/licenses/

CC-0

class mth5.metadata.TimePeriod(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

end

Required: True

Units: None

Type: string

Style: time

end date and time of collection in UTC

2020-02-04T16:2 3:45.453670+00: 00

start

Required: True

Units: None

Type: string

Style: time

start date and time of collection in UTC

2020-02-01T09:2 3:45.453670+00: 00

property end
property end_date
property start
property start_date
class mth5.metadata.TimingSystem(**kwargs)[source]

Bases: mth5.metadata.Base

Metadata Key

Description

Example

comments

Required: False

Units: None

Type: string

Style: free form

any comment on timing system

GPS locked with internal quartz clock

drift

Required: True

Units: seconds

Type: float

Style: number

estimated drift of the timing system

0.001

type

Required: True

Units: None

Type: string

Style: free form

type of timing system

GPS

uncertainty

Required: True

Units: seconds

Type: float

Style: number

estimated uncertainty of the timing system

0.0002

mth5.metadata.wrap_description(description, column_width)[source]

split a description into separate lines

mth5.metadata.write_lines(attr_dict, c1=45, c2=45, c3=15)[source]

write table lines

Parameters

lines_list – DESCRIPTION

mth5.mth5 module

MTH5

MTH5 deals with reading and writing an MTH5 file, which are HDF5 files developed for magnetotelluric (MT) data. The code is based on h5py and therefor numpy. This is the simplest and we are not really dealing with large tables of data to warrant using pytables.

Created on Sun Dec 9 20:50:41 2018

copyright

Jared Peacock (jpeacock@usgs.gov)

license

MIT

class mth5.mth5.MTH5(filename=None, compression='gzip', compression_opts=3, shuffle=True, fletcher32=True, data_level=1)[source]

Bases: object

MTH5 is the main container for the HDF5 file format developed for MT data

It uses the metadata standards developled by the IRIS PASSCAL software group and defined in the metadata documentation.

MTH5 is built with h5py and therefore numpy. The structure follows the different levels of MT data collection: Survey

|_Reports |_Standards |_Filters |_Station

|_Run

|_Channel

All timeseries data are stored as individual channels with the appropriate metadata defined for the given channel, i.e. electric, magnetic, auxiliary.

Each level is represented as a mth5 group class object which has methods to add, remove, and get a group from the level below. Each group has a metadata attribute that is the approprate metadata class object. For instance the SurveyGroup has an attribute metadata that is a mth5.metadata.Survey object. Metadata is stored in the HDF5 group attributes as (key, value) pairs.

All groups are represented by their structure tree and can be shown at any time from the command line.

Each level has a summary array of the contents of the levels below to hopefully make searching easier.

Parameters
  • filename (string or pathlib.Path) – name of the to be or existing file

  • compression

    compression type. Supported lossless compressions are * ‘lzf’ - Available with every installation of h5py

    (C source code also available). Low to moderate compression, very fast. No options.

    • ’gzip’ - Available with every installation of HDF5,

      so it’s best where portability is required. Good compression, moderate speed. compression_opts sets the compression level and may be an integer from 0 to 9, default is 3.

    • ’szip’ - Patent-encumbered filter used in the NASA

      community. Not available with all installations of HDF5 due to legal reasons. Consult the HDF5 docs for filter options.

  • compression_opts (string or int depending on compression type) – compression options, see above

  • shuffle (boolean) – Block-oriented compressors like GZIP or LZF work better when presented with runs of similar values. Enabling the shuffle filter rearranges the bytes in the chunk and may improve compression ratio. No significant speed penalty, lossless.

  • fletcher32 (boolean) – Adds a checksum to each chunk to detect data corruption. Attempts to read corrupted chunks will fail with an error. No significant speed penalty. Obviously shouldn’t be used with lossy compression filters.

  • data_level (integer, defaults to 1) –

    level the data are stored following levels defined by NASA ESDS

    • 0 - Raw data

    • 1 - Raw data with response information and full metadata

    • 2 - Derived product, raw data has been manipulated

Usage

  • Open a new file and show initialized file

>>> from mth5 import mth5
>>> mth5_obj = mth5.MTH5()
>>> # Have a look at the dataset options
>>> mth5.dataset_options
{'compression': 'gzip',
 'compression_opts': 3,
 'shuffle': True,
 'fletcher32': True}
>>> mth5_obj.open_mth5(r"/home/mtdata/mt01.mth5", 'w')
>>> mth5_obj
/:
====================
    |- Group: Survey
    ----------------
        |- Group: Filters
        -----------------
            --> Dataset: summary
            ......................
        |- Group: Reports
        -----------------
            --> Dataset: summary
            ......................
        |- Group: Standards
        -------------------
            --> Dataset: summary
            ......................
        |- Group: Stations
        ------------------
            --> Dataset: summary
            ......................
  • Add metadata for survey from a dictionary

>>> survey_dict = {'survey':{'acquired_by': 'me', 'archive_id': 'MTCND'}}
>>> survey = mth5_obj.survey_group
>>> survey.metadata.from_dict(survey_dict)
>>> survey.metadata
{
"survey": {
    "acquired_by.author": "me",
    "acquired_by.comments": null,
    "archive_id": "MTCND"
    ...}
}
  • Add a station from the convenience function

>>> station = mth5_obj.add_station('MT001')
>>> mth5_obj
/:
====================
    |- Group: Survey
    ----------------
        |- Group: Filters
        -----------------
            --> Dataset: summary
            ......................
        |- Group: Reports
        -----------------
            --> Dataset: summary
            ......................
        |- Group: Standards
        -------------------
            --> Dataset: summary
            ......................
        |- Group: Stations
        ------------------
            |- Group: MT001
            ---------------
                --> Dataset: summary
                ......................
            --> Dataset: summary
            ......................
>>> station
/Survey/Stations/MT001:
====================
    --> Dataset: summary
    ......................
>>> data.schedule_01.ex[0:10] = np.nan
>>> data.calibration_hx[...] = np.logspace(-4, 4, 20)

Note

if replacing an entire array with a new one you need to use […] otherwise the data will not be updated.

Warning

You can only replace entire arrays with arrays of the same size. Otherwise you need to delete the existing data and make a new dataset.

add_channel(station_name, run_name, channel_name, channel_type, data, channel_metadata=None)[source]

Convenience function to add a channel using mth5.stations_group.get_station().get_run().add_channel()

add a channel to a given run for a given station

Parameters
  • station_name (string) – existing station name

  • run_name (string) – existing run name

  • channel_name (string) – name of the channel

  • channel_type (string) – [ electric | magnetic | auxiliary ]

  • channel_metadata ([ mth5.metadata.Electric | mth5.metadata.Magnetic | mth5.metadata.Auxiliary ], optional) – metadata container, defaults to None

Raises

MTH5Error – If channel type is not correct

Returns

Channel container

Return type

[ mth5.mth5_groups.ElectricDatset | mth5.mth5_groups.MagneticDatset | mth5.mth5_groups.AuxiliaryDatset ]

Example

>>> new_channel = mth5_obj.add_channel('MT001', 'MT001a''Ex',
>>> ...                                'electric', None)
>>> new_channel
Channel Electric:
-------------------
                component:        None
        data type:        electric
        data format:      float32
        data shape:       (1,)
        start:            1980-01-01T00:00:00+00:00
        end:              1980-01-01T00:00:00+00:00
        sample rate:      None
add_run(station_name, run_name, run_metadata=None)[source]

Convenience function to add a run using mth5.stations_group.get_station(station_name).add_run()

Add a run to a given station.

Parameters
  • run_name (string) – run name, should be archive_id{a-z}

  • metadata (mth5.metadata.Station, optional) – metadata container, defaults to None

Example

>>> new_run = mth5_obj.add_run('MT001', 'MT001a')
add_station(name, station_metadata=None)[source]

Convenience function to add a station using mth5.stations_group.add_station

Add a station with metadata if given with the path:

/Survey/Stations/station_name

If the station already exists, will return that station and nothing is added.

Parameters
  • station_name (string) – Name of the station, should be the same as metadata.archive_id

  • station_metadata (mth5.metadata.Station, optional) – Station metadata container, defaults to None

Returns

A convenience class for the added station

Return type

mth5_groups.StationGroup

Example

>>> new_staiton = mth5_obj.add_station('MT001')
close_mth5()[source]

close mth5 file to make sure everything is flushed to the file

property data_level

data level

property dataset_options

summary of dataset options

property file_type

File Type should be MTH5

property file_version

mth5 file version

property filename

file name of the hdf5 file

property filters_group

Convenience property for /Survey/Filters group

from_reference(h5_reference)[source]

Get an HDF5 group, dataset, etc from a reference

Parameters

h5_reference (TYPE) – DESCRIPTION

Returns

DESCRIPTION

Return type

TYPE

get_channel(station_name, run_name, channel_name)[source]

Convenience function to get a channel using mth5.stations_group.get_station().get_run().get_channel()

Get a channel from an existing name. Returns the appropriate container.

Parameters
  • station_name (string) – existing station name

  • run_name (string) – existing run name

  • channel_name (string) – name of the channel

Returns

Channel container

Return type

[ mth5.mth5_groups.ElectricDatset | mth5.mth5_groups.MagneticDatset | mth5.mth5_groups.AuxiliaryDatset ]

Raises

MTH5Error – If no channel is found

Example

>>> existing_channel = mth5_obj.get_channel(station_name,
>>> ...                                     run_name,
>>> ...                                     channel_name)
>>> existing_channel
Channel Electric:
-------------------
                component:        Ex
        data type:        electric
        data format:      float32
        data shape:       (4096,)
        start:            1980-01-01T00:00:00+00:00
        end:              1980-01-01T00:00:01+00:00
        sample rate:      4096
get_run(station_name, run_name)[source]

Convenience function to get a run using mth5.stations_group.get_station(station_name).get_run()

get a run from run name for a given station

Parameters
  • station_name (string) – existing station name

  • run_name (string) – existing run name

Returns

Run object

Return type

mth5.mth5_groups.RunGroup

Example

>>> existing_run = mth5_obj.get_run('MT001', 'MT001a')
get_station(station_name)[source]

Convenience function to get a station using mth5.stations_group.get_station

Get a station with the same name as station_name

Parameters

station_name (string) – existing station name

Returns

convenience station class

Return type

mth5.mth5_groups.StationGroup

Raises

MTH5Error – if the station name is not found.

Example

>>> existing_staiton = mth5_obj.get_station('MT001')
MTH5Error: MT001 does not exist, check station_list for existing names
h5_is_write()[source]

check to see if the hdf5 file is open and writeable

open_mth5(filename=None, mode='a')[source]

open an mth5 file

Returns

Survey Group

Type

groups.SurveyGroup

Example

>>> from mth5 import mth5
>>> mth5_object = mth5.MTH5()
>>> survey_object = mth5_object.open_mth5('Test.mth5', 'w')
remove_channel(station_name, run_name, channel_name)[source]

Convenience function to remove a channel using mth5.stations_group.get_station().get_run().remove_channel()

Remove a channel from a given run and station.

Note

Deleting a channel is not as simple as del(channel). In HDF5 this does not free up memory, it simply removes the reference to that channel. The common way to get around this is to copy what you want into a new file, or overwrite the channel.

Parameters
  • station_name (string) – existing station name

  • run_name (string) – existing run name

  • channel_name (string) – existing station name

Example

>>> mth5_obj.remove_channel('MT001', 'MT001a', 'Ex')
remove_run(station_name, run_name)[source]

Convenience function to add a run using mth5.stations_group.get_station(station_name).remove_run()

Remove a run from the station.

Note

Deleting a run is not as simple as del(run). In HDF5 this does not free up memory, it simply removes the reference to that station. The common way to get around this is to copy what you want into a new file, or overwrite the run.

Parameters
  • station_name (string) – existing station name

  • run_name (string) – existing run name

Example

>>> mth5_obj.remove_station('MT001', 'MT001a')
remove_station(station_name)[source]

Convenience function to remove a station using mth5.stations_group.remove_station

Remove a station from the file.

Note

Deleting a station is not as simple as del(station). In HDF5 this does not free up memory, it simply removes the reference to that station. The common way to get around this is to copy what you want into a new file, or overwrite the station.

Parameters

station_name (string) – existing station name

Example

>>> mth5_obj.remove_station('MT001')
property reports_group

Convenience property for /Survey/Reports group

property software_name

software name that wrote the file

property standards_group

Convenience property for /Survey/Standards group

property station_list

list of existing stations names

property stations_group

Convenience property for /Survey/Stations group

property survey_group

Convenience property for /Survey group

validate_file()[source]

Validate an open mth5 file

will test the attribute values and group names

Returns

Boolean [ True = valid, False = not valid]

Return type

Boolean

mth5.mth5_tables module

Created on Mon Apr 27 11:15:27 2020

@author: jpeacock

class mth5.mth5_tables.MTH5(**kwargs)[source]

Bases: object

container for MT data in HDF5 format

close_mth5_file()[source]

Close file

new_mth5_file(mth5_fn, title='MTH5 File')[source]

Make new MTH5 file

Parameters

mth5_fn (TYPE) – DESCRIPTION.

Returns

Return type

None.

open_mth5_file(mth5_fn)[source]
Parameters

mth5_fn (TYPE) – DESCRIPTION.

Returns

Return type

None.

mth5.timeseries module

lists and arrays that goes on, seems easiest to convert all lists to strings and then convert them back if read in.

copyright

Jared Peacock (jpeacock@usgs.gov)

license

MIT

class mth5.timeseries.ChannelTS(channel_type='auxiliary', data=None, channel_metadata=None, station_metadata=None, run_metadata=None, **kwargs)[source]

Bases: object

Note

Assumes equally spaced samples from the start time.

The time series is stored in an xarray.Dataset that has coordinates of time and is a 1-D array labeled ‘data’. The xarray.Dataset can be accessed and set from the :attribute:`ts`. The data is stored in :attribute:’ts.data’ and the time index is a coordinate of :attribute:`ts`.

The time coordinate is made from the start time, sample rate and number of samples. Currently, End time is a derived property and cannot be set.

Channel time series object is based on xarray and mth5.metadata therefore any type of interpolation, resampling, groupby, etc can be done using xarray methods.

There are 3 metadata classes that hold important metadata

belongs to. * :class`mth5.metadata.Channel` holds information specific to the channel.

This way a single channel will hold all information needed to represent the channel.

Example

>>> from mth5.timeseries import ChannelTS
>>> ts_obj = ChannelTS('auxiliary')
>>> ts_obj.sample_rate = 8
>>> ts_obj.start = '2020-01-01T12:00:00+00:00'
>>> ts_obj.ts = range(4096)
>>> ts_obj.station_metadata.id = 'MT001'
>>> ts_obj.run_metadata.id = 'MT001a'
>>> ts_obj.component = 'temperature'
>>> print(ts_obj)
        Station      = MT001
        Run          = MT001a
        Channel Type = auxiliary
    Component    = temperature
        Sample Rate  = 8.0
        Start        = 2020-01-01T12:00:00+00:00
        End          = 2020-01-01T12:08:31.875000+00:00
        N Samples    = 4096
>>> p = ts_obj.ts.plot()
property channel_type

Channel Type

property component
property end

MTime object

from_obspy_trace(obspy_trace)[source]

Fill data from an obspy.core.Trace

Parameters

obspy_trace (obspy.core.trace) – Obspy trace object

get_slice(start, end)[source]

Get a slice from the time series given a start and end time.

Looks for >= start & <= end

Uses loc to be exact with milliseconds

Parameters
  • start (TYPE) – DESCRIPTION

  • end (TYPE) – DESCRIPTION

Returns

DESCRIPTION

Return type

TYPE

property has_data

check to see if there is an index in the time series

property n_samples

number of samples

resample(dec_factor=1, inplace=False)[source]

decimate the data by using scipy.signal.decimate

Parameters

dec_factor (int) – decimation factor

  • refills ts.data with decimated data and replaces sample_rate

property sample_rate

sample rate in samples/second

property start

MTime object

to_obspy_trace()[source]

Convert the time series to an obspy.core.trace.Trace object. This will be helpful for converting between data pulled from IRIS and data going into IRIS.

Returns

DESCRIPTION

Return type

TYPE

property ts
update_xarray_metadata()[source]

Update xarray attrs dictionary with metadata. Here we are assuming that self.metadata is the parent and attrs in xarray are children because all metadata will be validated by mth5.metadata class objects.

Eventually there should be a way that this is automatic, but I’m not that clever yet.

This should be mainly used internally but gives the user a way to update metadata.

class mth5.timeseries.RunTS(array_list=None, run_metadata=None, station_metadata=None)[source]

Bases: object

holds all run ts in one aligned array

components –> {‘ex’: ex_xarray, ‘ey’: ey_xarray}

add_channel(channel)[source]

Add a channel to the dataset, can be an xarray.DataArray or mth5.timeseries.ChannelTS object.

Need to be sure that the coordinates and dimensions are the same as the existing dataset, namely coordinates are time, and dimensions are the same, if the dimesions are larger than the existing dataset then the added channel will be clipped to the dimensions of the existing dataset.

If the start time is not the same nan’s will be placed at locations where the timing does not match the current start time. This is a feature of xarray.

Parameters

channel (xarray.DataArray or mth5.timeseries.ChannelTS) – a channel xarray or ChannelTS to add to the run

property channels
property dataset
property end
from_obspy_stream(obspy_stream)[source]

Get a run from an obspy.core.stream which is a list of obspy.core.Trace objects.

Parameters

obspy_stream (obspy.core.Stream) – Obspy Stream object

property has_data

check to see if there is data

plot()[source]

plot the time series probably slow for large data sets

Returns

DESCRIPTION

Return type

TYPE

property sample_rate
set_dataset(array_list, align_type='outer')[source]
Parameters
  • array_list (list of mth5.timeseries.ChannelTS objects) – list of xarrays

  • align_type (string) – how the different times will be aligned * ’outer’: use the union of object indexes * ’inner’: use the intersection of object indexes * ’left’: use indexes from the first object with each dimension * ’right’: use indexes from the last object with each dimension * ’exact’: instead of aligning, raise ValueError when indexes to be aligned are not equal * ’override’: if indexes are of same size, rewrite indexes to be those of the first object with that dimension. Indexes for the same dimension must have the same size in all objects.

property start
property summarize_metadata

Get a summary of all the metadata

Returns

DESCRIPTION

Return type

TYPE

to_obspy_stream()[source]

convert time series to an obspy.core.Stream which is like a list of obspy.core.Trace objects.

Returns

An Obspy Stream object from the time series data

Return type

obspy.core.Stream

validate_metadata()[source]

Check to make sure that the metadata matches what is in the data set.

updates metadata from the data.

Check the start and end times, channels recorded :return: DESCRIPTION :rtype: TYPE

mth5.timeseries.make_dt_coordinates(start_time, sample_rate, n_samples, logger)[source]

get the date time index from the data

Parameters
  • start_time (string) – start time in time format

  • sample_rate (float) – sample rate in samples per seconds

  • n_samples (int) – number of samples in time series

:param logging.logger logger: logger class object

Returns

date-time index

mth5.to_stationxml module

Map an MT object to seismic StationXML as close as we can.

Add extra fields where needed.

Created on Tue May 12 14:09:28 2020

@author: jpeacock

mth5.to_stationxml.to_inventory_channel(mt_channel)[source]

convert an MT channel to a Obspy inventory.Channel

Parameters

mt_channel – An mt metadata.Channel object

Module contents

Top-level package for MTH5.