mth5.timeseries package
Submodules
mth5.timeseries.channel_ts module
lists and arrays that goes on, seems easiest to convert all lists to strings and then convert them back if read in.
- copyright
Jared Peacock (jpeacock@usgs.gov)
- license
MIT
- class mth5.timeseries.channel_ts.ChannelTS(channel_type='auxiliary', data=None, channel_metadata=None, station_metadata=None, run_metadata=None, survey_metadata=None, **kwargs)[source]
Bases:
object
Note
Assumes equally spaced samples from the start time.
The time series is stored in an
xarray.Dataset
that has coordinates of time and is a 1-D array labeled ‘data’. Thexarray.Dataset
can be accessed and set from the ts. The data is stored in ‘ts.data’ and the time index is a coordinate of ts.The time coordinate is made from the start time, sample rate and number of samples. Currently, End time is a derived property and cannot be set.
Channel time series object is based on xarray and
mth5.metadata
therefore any type of interpolation, resampling, groupby, etc can be done using xarray methods.There are 3 metadata classes that hold important metadata
mth5.metadata.Station
holds information about the stationmth5.metadata.Run
holds information about the run the channel
belongs to. * :class`mth5.metadata.Channel` holds information specific to the channel.
This way a single channel will hold all information needed to represent the channel.
- Rubric
>>> from mth5.timeseries import ChannelTS >>> ts_obj = ChannelTS('auxiliary') >>> ts_obj.sample_rate = 8 >>> ts_obj.start = '2020-01-01T12:00:00+00:00' >>> ts_obj.ts = range(4096) >>> ts_obj.station_metadata.id = 'MT001' >>> ts_obj.run_metadata.id = 'MT001a' >>> ts_obj.component = 'temperature' >>> print(ts_obj) Station = MT001 Run = MT001a Channel Type = auxiliary Component = temperature Sample Rate = 8.0 Start = 2020-01-01T12:00:00+00:00 End = 2020-01-01T12:08:31.875000+00:00 N Samples = 4096 >>> p = ts_obj.ts.plot()
- property channel_metadata
station metadata
- property channel_response_filter
Full channel response filter
- Returns
DESCRIPTION
- Return type
TYPE
- property channel_type
Channel Type
- property component
- property end
MTime object
- from_obspy_trace(obspy_trace)[source]
Fill data from an
obspy.core.Trace
- Parameters
obspy_trace (obspy.core.trace) – Obspy trace object
- get_slice(start, end=None, n_samples=None)[source]
Get a slice from the time series given a start and end time.
Looks for >= start & <= end
Uses loc to be exact with milliseconds
- Parameters
start (TYPE) – DESCRIPTION
end (TYPE) – DESCRIPTION
- Returns
DESCRIPTION
- Return type
TYPE
- property has_data
check to see if there is an index in the time series
- property n_samples
number of samples
- remove_instrument_response(**kwargs)[source]
Remove instrument response from the given channel response filter
The order of operations is important (if applied):
detrend
zero mean
zero pad
time window
frequency window
remove response
undo time window
bandpass
kwargs
- Parameters
plot (boolean, default True) – to plot the calibration process [ False | True ]
detrend (boolean, default True) – Remove linar trend of the time series
zero_mean (boolean, default True) – Remove the mean of the time series
zero_pad (boolean, default True) – pad the time series to the next power of 2 for efficiency
t_window (string, default None) – Time domain windown name see scipy.signal.windows for options
t_window_params – Time domain window parameters, parameters can be
found in scipy.signal.windows :type t_window_params: dictionary :param f_window: Frequency domain windown name see scipy.signal.windows for options :type f_window: string, defualt None :param f_window_params: Frequency window parameters, parameters can be found in scipy.signal.windows :type f_window_params: dictionary :param bandpass: bandpass freequency and order {“low”:, “high”:, “order”:,} :type bandpass: dictionary
- resample(dec_factor=1, inplace=False)[source]
decimate the data by using scipy.signal.decimate
- Parameters
dec_factor (int) – decimation factor
refills ts.data with decimated data and replaces sample_rate
- property run_metadata
station metadata
- property sample_interval
Sample interval = 1 / sample_rate
- Returns
DESCRIPTION
- Return type
TYPE
- property sample_rate
sample rate in samples/second
- property start
MTime object
- property station_metadata
station metadata
- property survey_metadata
survey metadata
- property time_index
time index as a numpy array dtype np.datetime[ns]
- Returns
array of the time index
- Return type
np.ndarray(dtype=np.datetime[ns])
- to_obspy_trace()[source]
Convert the time series to an
obspy.core.trace.Trace
object. This will be helpful for converting between data pulled from IRIS and data going into IRIS.- Returns
DESCRIPTION
- Return type
TYPE
- to_xarray()[source]
Returns a
xarray.DataArray
object of the channel timeseries this way metadata from the metadata class is updated upon return.- Returns
Returns a
xarray.DataArray
object of the channel timeseries
this way metadata from the metadata class is updated upon return. :rtype:
xarray.DataArray
>>> import numpy as np >>> from mth5.timeseries import ChannelTS >>> ex = ChannelTS("electric") >>> ex.start = "2020-01-01T12:00:00" >>> ex.sample_rate = 16 >>> ex.ts = np.random.rand(4096)
- property ts
- mth5.timeseries.channel_ts.make_dt_coordinates(start_time, sample_rate, n_samples, logger)[source]
get the date time index from the data
- Parameters
start_time (string) – start time in time format
sample_rate (float) – sample rate in samples per seconds
n_samples (int) – number of samples in time series
logger (”
logging.logger
) – logger class object
- Returns
date-time index
mth5.timeseries.run_ts module
lists and arrays that goes on, seems easiest to convert all lists to strings and then convert them back if read in.
- copyright
Jared Peacock (jpeacock@usgs.gov)
- license
MIT
- class mth5.timeseries.run_ts.RunTS(array_list=None, run_metadata=None, station_metadata=None, survey_metadata=None)[source]
Bases:
object
holds all run ts in one aligned array
components –> {‘ex’: ex_xarray, ‘ey’: ey_xarray}
ToDo, have a single Survey object under the hood and properties to other metadata objects for get/set.
- add_channel(channel)[source]
Add a channel to the dataset, can be an
xarray.DataArray
ormth5.timeseries.ChannelTS
object.Need to be sure that the coordinates and dimensions are the same as the existing dataset, namely coordinates are time, and dimensions are the same, if the dimesions are larger than the existing dataset then the added channel will be clipped to the dimensions of the existing dataset.
If the start time is not the same nan’s will be placed at locations where the timing does not match the current start time. This is a feature of xarray.
- Parameters
channel (
xarray.DataArray
ormth5.timeseries.ChannelTS
) – a channel xarray or ChannelTS to add to the run
- calibrate(**kwargs)[source]
Calibrate the data according to the filters in each channel.
- Returns
calibrated run
- Return type
- property channels
List of channel names in dataset
- property dataset
xarray.Dataset
- property end
End time UTC
- property filters
Dictionary of filters used by the channels
- from_obspy_stream(obspy_stream, run_metadata=None)[source]
Get a run from an
obspy.core.stream
which is a list ofobspy.core.Trace
objects.- Parameters
obspy_stream (
obspy.core.Stream
) – Obspy Stream object
- get_slice(start, end=None, n_samples=None)[source]
- Parameters
start (TYPE) – DESCRIPTION
end (TYPE, optional) – DESCRIPTION, defaults to None
n_samples (TYPE, optional) – DESCRIPTION, defaults to None
- Raises
ValueError – DESCRIPTION
- Returns
DESCRIPTION
- Return type
TYPE
- plot(color_map={'ex': (1, 0.2, 0.2), 'ey': (1, 0.5, 0), 'hx': (0, 0.5, 1), 'hy': (0.5, 0.2, 1), 'hz': (0.2, 1, 1)}, channel_order=None)[source]
plot the time series probably slow for large data sets
- property run_metadata
station metadata
- property sample_interval
Sample interval = 1 / sample_rate :return: DESCRIPTION :rtype: TYPE
- property sample_rate
Sample rate, this is estimated by the mdeian difference between samples in time, if data is present. Otherwise return the metadata sample rate.
- set_dataset(array_list, align_type='outer')[source]
- Parameters
array_list (list of
mth5.timeseries.ChannelTS
objects) – list of xarraysalign_type (string) – how the different times will be aligned * ’outer’: use the union of object indexes * ’inner’: use the intersection of object indexes * ’left’: use indexes from the first object with each dimension * ’right’: use indexes from the last object with each dimension * ’exact’: instead of aligning, raise ValueError when indexes to be aligned are not equal * ’override’: if indexes are of same size, rewrite indexes to be those of the first object with that dimension. Indexes for the same dimension must have the same size in all objects.
- property start
Start time UTC
- property station_metadata
station metadata
- property summarize_metadata
Get a summary of all the metadata
- Returns
A summary of all channel metadata in one place
- Return type
dictionary
- property survey_metadata
survey metadata
mth5.timeseries.ts_filters module
time series filters
- class mth5.timeseries.ts_filters.RemoveInstrumentResponse(ts, time_array, sample_interval, channel_response_filter, **kwargs)[source]
Bases:
object
Remove instrument response from the given channel response filter
The order of operations is important (if applied):
detrend
zero mean
zero pad
time window
frequency window
remove response
undo time window
bandpass
- Parameters
ts (np.ndarray((N,) , dtype=float)) – time series data to remove response from
time_array (np.ndarray((N,) , dtype=np.datetime[ns])) – time index that corresponds to the time series
sample_interval (float) – seconds per sample (time interval between samples)
channel_response_filter – Channel response filter with all filters
included to convert from counts to physical units :type channel_response_filter: class:mt_metadata.timeseries.filters.ChannelResponseFilter`
kwargs
- Parameters
plot (boolean, default True) – to plot the calibration process [ False | True ]
detrend (boolean, default True) – Remove linar trend of the time series
zero_mean (boolean, default True) – Remove the mean of the time series
zero_pad (boolean, default True) – pad the time series to the next power of 2 for efficiency
t_window (string, default None) – Time domain windown name see scipy.signal.windows for options
t_window_params – Time domain window parameters, parameters can be
found in scipy.signal.windows :type t_window_params: dictionary :param f_window: Frequency domain windown name see scipy.signal.windows for options :type f_window: string, defualt None :param f_window_params: Frequency window parameters, parameters can be found in scipy.signal.windows :type f_window_params: dictionary :param bandpass: bandpass freequency and order {“low”:, “high”:, “order”:,} :type bandpass: dictionary
- apply_bandpass(ts)[source]
apply a bandpass filter to the calibrated data
- Parameters
ts (np.ndarray) – calibrated time series
- Returns
bandpassed time series
- Return type
np.ndarray
- apply_detrend(ts)[source]
Detrend time series using scipy.detrend(‘linear’)
- Parameters
ts (np.ndarray) – input time series
- Returns
detrended time series
- Return type
np.ndarray
- apply_f_window(data)[source]
Apply a frequency domain window. Get the available windows from scipy.signal.windows
Need to create a window twice the size of the input because we are only taking the rfft which gives just half the spectra and then take only half the window
- Parameters
data (np.ndarray) – input spectra
- Returns
windowed spectra
- Return type
np.ndarray
- apply_t_window(ts)[source]
Apply a window in the time domain. Get the available windows from scipy.signal.windows
- Parameters
ts (np.ndarray) – input time series
- Returns
windowed time series
- Return type
np.ndarray
- apply_zero_mean(ts)[source]
Remove the mean from the time series
- Parameters
ts (np.ndarray) – input time series
- Returns
zero mean time series
- Return type
np.ndarray
- apply_zero_pad(ts)[source]
zero pad to power of 2, at the end of the time series to make the FFT more efficient
- Parameters
ts (np.ndarray) – input time series
- Returns
zero padded time series
- Return type
np.ndarray
- mth5.timeseries.ts_filters.adaptive_notch_filter(bx, df=100, notches=[50, 100], notchradius=0.5, freqrad=0.9, rp=0.1, dbstop_limit=5.0)[source]
- Parameters
bx (np.ndarray) – time series to filter
df (float, optional) – sample rate in samples per second, defaults to 100
notches (list, optional) – list of frequencies to locate notches at in Hz, defaults to [50, 100]
notchradius (float, optional) – notch radius, defaults to 0.5
freqrad (float, optional) – radius to search for a peak at the notch frequency, defaults to 0.9
rp (float, optional) – ripple of Chebyshev type 1 filter, lower numbers means less ripples, defaults to 0.1
dbstop_limit (float, optional) – limits the difference between the peak at the notch and surrounding spectra. Any difference above dbstop_limit will be filtered, anything less will not, defaults to 5.0
- Returns
notch filtered data
- Return type
np.ndarray
- Returns
list of notch frequencies
- Return type
list
Example
>>> import RemovePeriodicNoise_Kate as rmp >>> # make a variable for the file to load in >>> fn = r"/home/MT/mt01_20130101_000000.BX" >>> # load in file, if the time series is not an ascii file >>> # might need to add keywords to np.loadtxt or use another >>> # method to read in the file >>> bx = np.loadtxt(fn) >>> # create a list of frequencies to filter out >>> freq_notches = [50, 150, 200] >>> # filter data >>> bx_filt, filt_lst = rmp.adaptiveNotchFilter(bx, df=100. >>> ... notches=freq_notches) >>> #save the filtered data into a file >>> np.savetxt(r"/home/MT/Filtered/mt01_20130101_000000.BX", bx_filt)
Notes
Most of the time the default parameters work well, the only thing you need to change is the notches and perhaps the radius. I would test it out with a few time series to find the optimum parameters. Then make a loop over all you time series data. Something like
>>> import os >>> dirpath = r"/home/MT" >>> #make a director to save filtered time series >>> save_path = r"/home/MT/Filtered" >>> if not os.path.exists(save_path): >>> os.mkdir(save_path) >>> for fn in os.listdir(dirpath): >>> bx = np.loadtxt(os.path.join(dirpath, fn) >>> bx_filt, filt_lst = rmp.adaptiveNotchFilter(bx, df=100. >>> ... notches=freq_notches) >>> np.savetxt(os.path.join(save_path, fn), bx_filt)
- mth5.timeseries.ts_filters.butter_bandpass(lowcut, highcut, fs, order=5)[source]
Butterworth bandpass filter using scipy.signal
- Parameters
lowcut (float) – low cut frequency in Hz
highcut (float) – high cut frequency in Hz
fs (float) – Sample rate
order (int, optional) – Butterworth order, defaults to 5
- Returns
SOS scipy.signal format
- Return type
scipy.signal.SOS?
- mth5.timeseries.ts_filters.butter_bandpass_filter(data, lowcut, highcut, fs, order=5)[source]
- Parameters
data (np.ndarray) – 1D time series data
lowcut (float) – low cut frequency in Hz
highcut (float) – high cut frequency in Hz
fs (float) – Sample rate
order (int, optional) – Butterworth order, defaults to 5
- Returns
filtered data
- Return type
np.ndarray
- mth5.timeseries.ts_filters.low_pass(data, low_pass_freq, cutoff_freq, sampling_rate)[source]
- Parameters
data (np.ndarray) – 1D time series data
low_pass_freq (float) – low pass frequency in Hz
cutoff_freq (float) – cut off frequency in Hz
sampling_rate (float) – Sample rate in samples per second
- Returns
lowpass filtered data
- Return type
np.ndarray
- mth5.timeseries.ts_filters.zero_pad(input_array, power=2, pad_fill=0)[source]
- Parameters
input_array (np.ndarray) – 1D array
power – base power to used to pad to, defaults to 2 which is optimal
for the FFT :type power: int, optional :param pad_fill: fill value for padded values, defaults to 0 :type pad_fill: float, optional :return: zero padded array :rtype: np.ndarray
Module contents
- class mth5.timeseries.ChannelTS(channel_type='auxiliary', data=None, channel_metadata=None, station_metadata=None, run_metadata=None, survey_metadata=None, **kwargs)[source]
Bases:
object
Note
Assumes equally spaced samples from the start time.
The time series is stored in an
xarray.Dataset
that has coordinates of time and is a 1-D array labeled ‘data’. Thexarray.Dataset
can be accessed and set from the ts. The data is stored in ‘ts.data’ and the time index is a coordinate of ts.The time coordinate is made from the start time, sample rate and number of samples. Currently, End time is a derived property and cannot be set.
Channel time series object is based on xarray and
mth5.metadata
therefore any type of interpolation, resampling, groupby, etc can be done using xarray methods.There are 3 metadata classes that hold important metadata
mth5.metadata.Station
holds information about the stationmth5.metadata.Run
holds information about the run the channel
belongs to. * :class`mth5.metadata.Channel` holds information specific to the channel.
This way a single channel will hold all information needed to represent the channel.
- Rubric
>>> from mth5.timeseries import ChannelTS >>> ts_obj = ChannelTS('auxiliary') >>> ts_obj.sample_rate = 8 >>> ts_obj.start = '2020-01-01T12:00:00+00:00' >>> ts_obj.ts = range(4096) >>> ts_obj.station_metadata.id = 'MT001' >>> ts_obj.run_metadata.id = 'MT001a' >>> ts_obj.component = 'temperature' >>> print(ts_obj) Station = MT001 Run = MT001a Channel Type = auxiliary Component = temperature Sample Rate = 8.0 Start = 2020-01-01T12:00:00+00:00 End = 2020-01-01T12:08:31.875000+00:00 N Samples = 4096 >>> p = ts_obj.ts.plot()
- property channel_metadata
station metadata
- property channel_response_filter
Full channel response filter
- Returns
DESCRIPTION
- Return type
TYPE
- property channel_type
Channel Type
- property component
- property end
MTime object
- from_obspy_trace(obspy_trace)[source]
Fill data from an
obspy.core.Trace
- Parameters
obspy_trace (obspy.core.trace) – Obspy trace object
- get_slice(start, end=None, n_samples=None)[source]
Get a slice from the time series given a start and end time.
Looks for >= start & <= end
Uses loc to be exact with milliseconds
- Parameters
start (TYPE) – DESCRIPTION
end (TYPE) – DESCRIPTION
- Returns
DESCRIPTION
- Return type
TYPE
- property has_data
check to see if there is an index in the time series
- property n_samples
number of samples
- remove_instrument_response(**kwargs)[source]
Remove instrument response from the given channel response filter
The order of operations is important (if applied):
detrend
zero mean
zero pad
time window
frequency window
remove response
undo time window
bandpass
kwargs
- Parameters
plot (boolean, default True) – to plot the calibration process [ False | True ]
detrend (boolean, default True) – Remove linar trend of the time series
zero_mean (boolean, default True) – Remove the mean of the time series
zero_pad (boolean, default True) – pad the time series to the next power of 2 for efficiency
t_window (string, default None) – Time domain windown name see scipy.signal.windows for options
t_window_params – Time domain window parameters, parameters can be
found in scipy.signal.windows :type t_window_params: dictionary :param f_window: Frequency domain windown name see scipy.signal.windows for options :type f_window: string, defualt None :param f_window_params: Frequency window parameters, parameters can be found in scipy.signal.windows :type f_window_params: dictionary :param bandpass: bandpass freequency and order {“low”:, “high”:, “order”:,} :type bandpass: dictionary
- resample(dec_factor=1, inplace=False)[source]
decimate the data by using scipy.signal.decimate
- Parameters
dec_factor (int) – decimation factor
refills ts.data with decimated data and replaces sample_rate
- property run_metadata
station metadata
- property sample_interval
Sample interval = 1 / sample_rate
- Returns
DESCRIPTION
- Return type
TYPE
- property sample_rate
sample rate in samples/second
- property start
MTime object
- property station_metadata
station metadata
- property survey_metadata
survey metadata
- property time_index
time index as a numpy array dtype np.datetime[ns]
- Returns
array of the time index
- Return type
np.ndarray(dtype=np.datetime[ns])
- to_obspy_trace()[source]
Convert the time series to an
obspy.core.trace.Trace
object. This will be helpful for converting between data pulled from IRIS and data going into IRIS.- Returns
DESCRIPTION
- Return type
TYPE
- to_xarray()[source]
Returns a
xarray.DataArray
object of the channel timeseries this way metadata from the metadata class is updated upon return.- Returns
Returns a
xarray.DataArray
object of the channel timeseries
this way metadata from the metadata class is updated upon return. :rtype:
xarray.DataArray
>>> import numpy as np >>> from mth5.timeseries import ChannelTS >>> ex = ChannelTS("electric") >>> ex.start = "2020-01-01T12:00:00" >>> ex.sample_rate = 16 >>> ex.ts = np.random.rand(4096)
- property ts
- class mth5.timeseries.RunTS(array_list=None, run_metadata=None, station_metadata=None, survey_metadata=None)[source]
Bases:
object
holds all run ts in one aligned array
components –> {‘ex’: ex_xarray, ‘ey’: ey_xarray}
ToDo, have a single Survey object under the hood and properties to other metadata objects for get/set.
- add_channel(channel)[source]
Add a channel to the dataset, can be an
xarray.DataArray
ormth5.timeseries.ChannelTS
object.Need to be sure that the coordinates and dimensions are the same as the existing dataset, namely coordinates are time, and dimensions are the same, if the dimesions are larger than the existing dataset then the added channel will be clipped to the dimensions of the existing dataset.
If the start time is not the same nan’s will be placed at locations where the timing does not match the current start time. This is a feature of xarray.
- Parameters
channel (
xarray.DataArray
ormth5.timeseries.ChannelTS
) – a channel xarray or ChannelTS to add to the run
- calibrate(**kwargs)[source]
Calibrate the data according to the filters in each channel.
- Returns
calibrated run
- Return type
- property channels
List of channel names in dataset
- property dataset
xarray.Dataset
- property end
End time UTC
- property filters
Dictionary of filters used by the channels
- from_obspy_stream(obspy_stream, run_metadata=None)[source]
Get a run from an
obspy.core.stream
which is a list ofobspy.core.Trace
objects.- Parameters
obspy_stream (
obspy.core.Stream
) – Obspy Stream object
- get_slice(start, end=None, n_samples=None)[source]
- Parameters
start (TYPE) – DESCRIPTION
end (TYPE, optional) – DESCRIPTION, defaults to None
n_samples (TYPE, optional) – DESCRIPTION, defaults to None
- Raises
ValueError – DESCRIPTION
- Returns
DESCRIPTION
- Return type
TYPE
- plot(color_map={'ex': (1, 0.2, 0.2), 'ey': (1, 0.5, 0), 'hx': (0, 0.5, 1), 'hy': (0.5, 0.2, 1), 'hz': (0.2, 1, 1)}, channel_order=None)[source]
plot the time series probably slow for large data sets
- property run_metadata
station metadata
- property sample_interval
Sample interval = 1 / sample_rate :return: DESCRIPTION :rtype: TYPE
- property sample_rate
Sample rate, this is estimated by the mdeian difference between samples in time, if data is present. Otherwise return the metadata sample rate.
- set_dataset(array_list, align_type='outer')[source]
- Parameters
array_list (list of
mth5.timeseries.ChannelTS
objects) – list of xarraysalign_type (string) – how the different times will be aligned * ’outer’: use the union of object indexes * ’inner’: use the intersection of object indexes * ’left’: use indexes from the first object with each dimension * ’right’: use indexes from the last object with each dimension * ’exact’: instead of aligning, raise ValueError when indexes to be aligned are not equal * ’override’: if indexes are of same size, rewrite indexes to be those of the first object with that dimension. Indexes for the same dimension must have the same size in all objects.
- property start
Start time UTC
- property station_metadata
station metadata
- property summarize_metadata
Get a summary of all the metadata
- Returns
A summary of all channel metadata in one place
- Return type
dictionary
- property survey_metadata
survey metadata