tsad.tasks package

Submodules

tsad.tasks.deep_learning_anomaly_detection module

tsad.tasks.deep_learning_forecasting module

tsad.tasks.eda module

class tsad.tasks.eda.EquipmentDowntimeResult

Bases: TaskResult

The result of the EquipmentDowntimeTask task.

Attributes:

equipment_downtimespd.DataFrame

The table with all equipment downtimes.

Methods

show() -> None

Displays the results of the EquipmentDowntimeTask task.

equipment_downtimes: DataFrame

show() → None

Displays the results of the EquipmentDowntimeTask task.

class tsad.tasks.eda.EquipmentDowntimeTask(name: str | None = None)

Bases: Task

Class of exploratory data analysis problem in the field of equipment downtime estimation. Analyzes, displays and saves information (in EquipmentDowntimeResult) about equipment downtimes in the dataset.

Notes

When the fit method is called, the following information is stored in EquipmentDowntimeResult:

equipment_downtimes : The table with all equipment downtimes.

Methods

fit_predict(df)	Fit the EquipmentDowntimeTask.
predict(df, result)	Predict by EquipmentDowntimeTask.

fit_predict(df: DataFrame) → tuple[pandas.core.frame.DataFrame, tsad.tasks.eda.EquipmentDowntimeResult]

Fit the EquipmentDowntimeTask.

Parameters:

dfpd.DataFrame

The input dataset.

Returns:

tuple[pd.DataFrame, EquipmentDowntimeResult]

The input dataset and the result of the analysis.

predict(df: DataFrame, result: EquipmentDowntimeResult) → tuple[pandas.core.frame.DataFrame, tsad.tasks.eda.EquipmentDowntimeResult]

Predict by EquipmentDowntimeTask. This method does nothing. It is needed for implementing high-level pipelines.

Parameters:

dfpd.DataFrame

The input dataset.

resultEquipmentDowntimeResult

The result of the analysis.

Returns:

tuple[pd.DataFrame, EquipmentDowntimeResult]

The output dataset and the result of the analysis.

class tsad.tasks.eda.FindNaNResult

Bases: TaskResult

The results of the FindNaNResult task.

Attributes:

mask_nanpd.DataFrame

The mask of NaN values in the original dataset.

full_nan_col_nameslist[str]

The list of columns that contain only NaN values.

full_nan_col_numbersint

The number of columns that contain only NaN values.

full_nan_col_percentfloat

The percentage of columns that contain only NaN values.

full_nan_row_nameslist

The list of rows that contain only NaN values.

full_nan_row_numbersint

The number of rows that contain only NaN values.

full_nan_row_percentfloat

The percentage of rows that contain only NaN values.

total_nan_numberint

The total number of NaN values in the dataset.

total_nan_percentfloat

The percentage of NaN values in the dataset.

matrix_nanNone

The matrix of pairwise intersections of NaN values between columns.

sum_nan_by_colpd.Series

The total number of NaN values per column.

nan_by_colpd.DataFrame

The table with distribution of NaN values per column.

Methods

show() -> None

Displays the results of the FindNaNTask task.

dataset_analysis_result: HighLevelDatasetAnalysisResult

full_nan_col_names: list

full_nan_col_numbers: int

full_nan_col_percent: float

full_nan_row_names: list

full_nan_row_numbers: int

full_nan_row_percent: float

mask_nan: DataFrame

matrix_nan: None

nan_by_col: DataFrame

show() → None

Displays the results of the FindNaNTask task.

sum_nan_by_col: Series

total_nan_number: int

total_nan_percent: float

class tsad.tasks.eda.FindNaNTask(name: str | None = None)

Bases: Task

Class of exploratory data analysis problem in the field of gap estimation. It is recommended to perform this after clearing duplicates and bringing the dataset to a single sampling rate. Analyzes, displays and saves information (in FindNaNResult) about gaps in the dataset.

Notes

When the fit method is called, the following information is stored in FindNaNResult:

mask_nan : The mask of NaN values in the original dataset. full_nan_col_names : The list of columns that contain only NaN values. full_nan_col_numbers : The number of columns that contain only NaN values. full_nan_col_percent : The percentage of columns that contain only NaN values. full_nan_row_names : The list of rows that contain only NaN values. full_nan_row_numbers : The number of rows that contain only NaN values. full_nan_row_percent : The percentage of rows that contain only NaN values. total_nan_number : The total number of NaN values in the dataset. total_nan_percent : The percentage of NaN values in the dataset. matrix_nan : The matrix of pairwise intersections of NaN values between columns. sum_nan_by_col : The total number of NaN values per column. nan_by_col : The table with distribution of NaN values per column.

Methods

fit_predict(df)	Fit the FindNaNTask.
predict(df, result)	Predict by FindNaNTask.

fit_predict(df: DataFrame) → tuple[pandas.core.frame.DataFrame, tsad.tasks.eda.FindNaNResult]

Fit the FindNaNTask.

Parameters:

dfpd.DataFrame

The input dataset.

Returns:

tuple[pd.DataFrame, FindNaNResult]

The output dataset and the result of the analysis.

Notes

When the fit method is called, the following information is stored in FindNaNResult:

mask_nan : The mask of NaN values in the original dataset. full_nan_col_names : The list of columns that contain only NaN values. full_nan_col_numbers : The number of columns that contain only NaN values. full_nan_col_percent : The percentage of columns that contain only NaN values. full_nan_row_names : The list of rows that contain only NaN values. full_nan_row_numbers : The number of rows that contain only NaN values. full_nan_row_percent : The percentage of rows that contain only NaN values. total_nan_number : The total number of NaN values in the dataset. total_nan_percent : The percentage of NaN values in the dataset. matrix_nan : The matrix of pairwise intersections of NaN values between columns. sum_nan_by_col : The total number of NaN values per column. nan_by_col : The table with distribution of NaN values per column.

predict(df: DataFrame, result: FindNaNResult) → tuple[pandas.core.frame.DataFrame, tsad.tasks.eda.FindNaNResult]

Predict by FindNaNTask. This method does nothing. It is needed for implementing high-level pipelines.

Parameters:

dfpd.DataFrame

The input dataset.

resultFindNaNResult

The result of the analysis.

Returns:

tuple[pd.DataFrame, FindNaNResult]

The output dataset and the result of the analysis.

class tsad.tasks.eda.HighLevelDatasetAnalysisResult

Bases: TaskResult

This is the result of the HighLevelDatasetAnalysisTask:

Attributes:

start_timepd.DatetimeIndex

The first time index in the source dataset

end_timepd.DatetimeIndex

The last time index in the source dataset.

durationpd.Timedelta

The overall duration in the source dataset.

lengthint

The number of samples in the source dataset.

columns_numint

The number of columns in the source dataset.

columnslist[str]

The list of columns in the source dataset.

typespd.Series

The table of data types of columns.

Methods

show()

Displays the result of the HighLevelDatasetAnalysisTask

save

columns = None

duration = None

end_time: DatetimeIndex

length = None

show() → None

Displays the result of the HighLevelDatasetAnalysisTask

start_time: DatetimeIndex

types: Series

class tsad.tasks.eda.HighLevelDatasetAnalysisTask(name: str | None = None)

Bases: Task

Class for exploratory data analysis task to evaluate general information about the dataset.

Methods

fit_predict(df)	Fit the HighLevelDatasetAnalysisTask.
predict(df, result)	Predict the HighLevelDatasetAnalysisTask.

Class for exploratory data analysis task to evaluate general information about the dataset. Performs analysis, output, and saving of high-level information about the dataset. Saving is done through HighLevelDatasetAnalysisResult for using the obtained information in subsequent tasks demanded within the high-level pipeline.

Notes

When the fit method is called, the following information is saved in HighLevelDatasetAnalysisResult:

start_time : The first time index in the source dataset end_time : The last time index in the source dataset. duration : The time span in the source dataset. length : The number of samples in the source dataset. columns_num : The number of columns in the source dataset. columns : The list of columns in the source dataset. types : The table of data types of columns

Methods

fit_predict(df)	Fit the HighLevelDatasetAnalysisTask.
predict(df, result)	Predict the HighLevelDatasetAnalysisTask.

fit_predict(df: DataFrame) → tuple[pandas.core.frame.DataFrame, tsad.tasks.eda.HighLevelDatasetAnalysisResult]

Fit the HighLevelDatasetAnalysisTask.

Parameters:

dfpd.DataFrame

The input dataset.

Returns:

tuple[pd.DataFrame, HighLevelDatasetAnalysisResult]

The output dataset and the result of the analysis.

Notes

In this case, the method saves the following information in HighLevelDatasetAnalysisResult:

start_time : The first time index in the source dataset end_time : The last time index in the source dataset. duration : The time span in the source dataset. length : The number of samples in the source dataset. columns_num : The number of columns in the source dataset. columns : The list of columns in the source dataset. types : The table of data types of columns

predict(df: DataFrame, result: HighLevelDatasetAnalysisResult) → tuple[pandas.core.frame.DataFrame, tsad.tasks.eda.HighLevelDatasetAnalysisResult]

Predict the HighLevelDatasetAnalysisTask. Nothing happens in this method. Needed to implement top-level pipelines.

Parameters:

dfpd.DataFrame

The input dataset.

resultHighLevelDatasetAnalysisResult

The result of the analysis.

Returns:

tuple[pd.DataFrame, HighLevelDatasetAnalysisResult]

The output dataset and the result of the analysis.

class tsad.tasks.eda.TimeDiscretizationResult

Bases: TaskResult

The result of the TimeDiscretizationTask:

Attributes:

freq_tobepd.Timedelta | str

The computed discretization frequency that is optimal for the given dataset. See TimeDiscretizationTask for more details.

frequence_of_diff_interval: pd.Series

The table where the index is the range of possible periods between samples, the column is the number of cases in the dataset with such a range of periods.

index_freq_climed: pd.Timedelta | str | None

The unified designated period of the source dataset, if one exists.

min_diffpd.Timedelta | str

The minimum period between samples in the dataset.

max_diffpd.Timedelta | str

The maximum period between samples in the dataset.

most_frequent_diff_valuepd.Timedelta | str

The most frequently period.

most_frequent_diff_amount_casesint

The number of cases with the most frequently period.

most_frequent_diff_amount_cases_percentfloat

The proportion of cases with the most frequently occurring period.

amount_unique_diffint

The number of unique periods.

amount_unique_diff_percentfloat

The proportion of unique periods among the total number of samples.

Methods

show()

Prints the results of the TimeDiscretizationTask.

save

amount_unique_diff: int

amount_unique_diff_percent: float

dataset_analysis_result: HighLevelDatasetAnalysisResult

freq_tobe: str

frequence_of_diff_interval: Series

index_freq_climed = None

max_diff = None

min_diff = None

most_frequent_diff_amount_cases: int

most_frequent_diff_amount_cases_percent: float

most_frequent_diff_value = None

show() → None

Prints the results of the TimeDiscretizationTask.

class tsad.tasks.eda.TimeDiscretizationTask(name: str | None = None, freq_tobe=None, freq_tobe_approach: str = 'auto')

Bases: Task

A class of exploratory data analysis task for analyzing and printing information about the frequency of time index discretization and saving this information to TimeDiscretizationResult.

Parameters:

freq_tobepd.Timedelta | str, default None

The user-defined discretization frequency. If not None, the search for the optimal frequency is not performed, and the parameter freq_tobe_approach becomes “custom”.

freq_tobe_approachstr, default ‘auto’, {‘custom’, ‘min_period’,

‘most_frequent’, ‘auto’} The method of forming the optimal discretization frequency, which may be required for further processing.

• If ‘custom’, the frequency from the freq_tobe parameter is taken as the optimal frequency.

• If ‘min_period’, the frequency corresponding to the minimum period between samples is taken as the optimal frequency.

• If ‘most_frequent’, the frequency corresponding to the most frequently occurring period between samples is taken as the optimal frequency.

• If ‘auto’, the frequency that is found in a complex way based on rounding a larger number of periods is taken as the optimal frequency.

See the code for more details.

Notes

When the fit method is called, the following information is saved in TimeDiscretizationResult:

freq_tobe: the computed discretization frequency frequence_of_diff_interval: the distribution of periods between samples index_freq_climed: the unified designated period of the source dataset, if one exists min_diff: the minimum period between samples in the dataset max_diff: the maximum period between samples in the dataset most_frequent_diff_value: the most frequently occurring period most_frequent_diff_amount_cases: the number of cases with the most frequently occurring period most_frequent_diff_amount_cases_percent: the proportion of cases with the most frequently occurring period amount_unique_diff: the number of unique periods amount_unique_diff_percent: the proportion of unique periods among the total number of samples

Methods

fit_predict(df)	Fit the TimeDiscretizationTask.
predict(df, result)	Predict by TimeDiscretizationTask.

fit_predict(df: DataFrame) → tuple[pandas.core.frame.DataFrame, tsad.tasks.eda.TimeDiscretizationResult]

Fit the TimeDiscretizationTask.

Parameters:

dfpd.DataFrame

The input dataset.

Returns:

tuple[pd.DataFrame, TimeDiscretizationResult]

The output dataset and the result of the analysis.

Notes

When the fit method is called, the following information is saved in TimeDiscretizationResult:

freq_tobe: the computed discretization frequency frequence_of_diff_interval: the distribution of periods between samples index_freq_climed: the unified designated period of the source dataset, if one exists min_diff: the minimum period between samples in the dataset max_diff: the maximum period between samples in the dataset most_frequent_diff_value: the most frequently occurring period most_frequent_diff_amount_cases: the number of cases with the most frequently occurring period most_frequent_diff_amount_cases_percent: the proportion of cases with the most frequently occurring period amount_unique_diff: the number of unique periods amount_unique_diff_percent: the proportion of unique periods among the total number of samples

predict(df: DataFrame, result: TimeDiscretizationResult) → tuple[pandas.core.frame.DataFrame, tsad.tasks.eda.TimeDiscretizationResult]

Predict by TimeDiscretizationTask. This method does nothing. It is needed for implementing high-level pipelines.

Parameters:

dfpd.DataFrame

The input dataset.

resultTimeDiscretizationResult

The result of the analysis.

Returns:

tuple[pd.DataFrame, TimeDiscretizationResult]

The output dataset and the result of the analysis.

tsad.tasks.feature_generation module

tsad.tasks.feature_selection module

tsad.tasks.preprocess module

class tsad.tasks.preprocess.PrepareSeqSamplesResult

Bases: TaskResult

The results of the PrepareSeqSamplesTask task: Nothing is saved or displayed as a result of this task.

Methods

save
show

show() → None

class tsad.tasks.preprocess.PrepareSeqSamplesTask(name: str | None = None, len_seq=10, points_ahead=1, gap=0, step=1, intersection=True, test_size=None, train_size=None, random_state=None, what_to_shuffle='train')

Bases: Task

Class of the data preprocessing task in the part of preparing samples in the form of a sequence for sequence processing algorithms and train test spitting execution.

Parameters:

name(str | None)

Optional name for the task.

Next parameters is used in ts_train_test_split function of utils subpackage.

https://tsad.readthedocs.io/en/latest/tsad.utils.html#tsad.utils.trainTestSplitting.ts_train_test_split

len_seqint, default=10

Length of the sequence, which is used to predict the next point/points.

points_aheadint, default=0

How many points ahead we predict, reflected in y

gapint, default=0

The gap between last point of sequence, which we used as input for prediction and first point of potential model output sequence (prediction).If the last point of input sequence is t, then the first point of the output sequence is t + gap +1. The parameter is designed to be able to predict sequence after a additional time interval.

stepint, default=1.

Sample generation step. If the first point was t for the 1st sample (sequence) of the train, then for the 2nd sample (sequence) of the train it will be t + step if intersection=True, otherwise the same but without intersections of the series values.

intersectionbool, default=True

The presence of one point in time in different samples (sequences) for the train set and and separately for the test test. If True, the train and the test never have common time points.

test_sizefloat or int or timestamp for df, or list of timestamps, default=0.25.

The size of the test set. - If float, should be between 0.0 and 1.0 and represent the proportion

of the dataset to include in the test split.

• If int, represents the absolute number of test samples. If None, the value is set to the

  complement of the train size.

• If 0, then it will return the X,y values in X_train, y_train.

• If timestamp, for X_test we will use set from df[t:]

• If list of timestamps [t1,t2], for X_test we will use set from df[t1:t2]

• If train_size is None, it will be set to 0.25. *

train_sizefloat or int, default=None.

The size of the train set. - If float, should be between 0.0 and 1.0 and represent the

proportion of the dataset to include in the train split.

• If int, represents the absolute number of train samples.

• If 0, then it will return the X,y values in X_test, y_test.

• If timestamp for df, for X_train we will use set for train from df[:t]

• If list of timestamps [t1,t2], for X_train we will use set for train from df[t1:t2]

• If None,the value is automatically set to the complement of the test size.

what_to_shuffle: {‘nothing’, ‘all’,’train’}, str. Default = ‘train’.

• If ‘train’ we random shuffle only X_train, and y_train.

  Test samples are unused for the shuffle. Any sample from X_test is later than any sample from X_train. This is also true for respectively

• If ‘all’ in analogy with sklearn.model_selection.train_test_split

• If ‘nothing’ shuffle is not performed.

random_stateint, RandomState instance or None, default=None

Controls the shuffling applied to the data before applying the split. Pass an int for reproducible output across multiple function calls. See Glossary.*

Methods

fit_predict(df)	Fit the PrepareSeqSamplesTask.
predict(df, result)	Predict by PrepareSeqSamplesTask.

fit_predict(df: DataFrame | list[pandas.core.frame.DataFrame]) → tuple[pandas.core.frame.DataFrame | list[pandas.core.frame.DataFrame], tsad.tasks.preprocess.PrepareSeqSamplesResult]

Fit the PrepareSeqSamplesTask.

Parameters:

dfspd.DataFrame | list[pd.DataFrame]

The input dataset / input datasets

Returns:

dfspd.DataFrame | list[pd.DataFrame] | list[list[pd.DataFrame]]

The output datasets of sequences

resultPrepareSeqSamplesResult

The object that stores the results of the PrepareSeqSamplesTask task

predict(df: DataFrame, result: PrepareSeqSamplesResult) → tuple[pandas.core.frame.DataFrame, tsad.tasks.preprocess.PrepareSeqSamplesResult]

Predict by PrepareSeqSamplesTask.

Parameters:

dfpd.DataFrame | list[pd.DataFrame]

The input dataset / input datasets

resultPrepareSeqSamplesResult

The object that stores the results of the PrepareSeqSamplesTask task

Returns:

dfpd.DataFrame | list[pd.DataFrame] | list[list[pd.DataFrame]]

The output datasets of sequences

resultPrepareSeqSamplesResult

The object that stores the results of the PrepareSeqSamplesTask task

class tsad.tasks.preprocess.ResampleProcessingResult

Bases: TaskResult

Methods

save
show

freq_tobe: str

show() → None

class tsad.tasks.preprocess.ResampleProcessingTask(name: str | None = None, freq_tobe: str | None = None)

Bases: Task

Class of the data preprocessing task in the field of resampling. This is a task that represents a class for resampling the frequency of the time index with some meta-information stored in ResampleProcessingResult.

Parameters:

freq_tobepd.Timedelta | str, default None

The desired frequency of the time index. If None, the frequency is taken from the results of the TimeDiscretizationTask.

Methods

fit_predict(df, time_result)	Fit the ResampleProcessingTask.
predict(df, rp_result)	Predict using the ResampleProcessingTask.

fit_predict(df: DataFrame, time_result: TimeDiscretizationResult) → tuple[pandas.core.frame.DataFrame, tsad.tasks.preprocess.ResampleProcessingResult]

Fit the ResampleProcessingTask. The method resamples the input dataframe according to freq_tobe or the frequency found in the TimeDiscretizationTask.

Parameters:

dfpd.DataFrame

Input dataset

time_resultTimeDiscretizationResult

Result of the TimeDiscretizationTask

Returns:

tuple[pd.DataFrame, ResampleProcessingResult]

Resampled dataframe and ResampleProcessingResult

predict(df: DataFrame, rp_result: ResampleProcessingResult) → tuple[pandas.core.frame.DataFrame, tsad.base.task.TaskResult]

Predict using the ResampleProcessingTask. The method resamples the input dataframe according to the frequency found in the ResampleProcessingResult.

Parameters:

dfpd.DataFrame

Input dataset

rp_resultResampleProcessingResult

Result of the ResampleProcessingTask

Returns:

tuple[pd.DataFrame, TaskResult]

Resampled dataframe and ResampleProcessingResult

class tsad.tasks.preprocess.SplitByNaNResult

Bases: TaskResult

The results of the SplitByNaNTask task: Nothing is saved or displayed as a result of this task.

Methods

save
show

show() → None

class tsad.tasks.preprocess.SplitByNaNTask(name: str | None = None)

Bases: Task

Class for preprocessing data by splitting the original dataset into separate datasets based on the continuity of the data.

A class of data preprocessing task in terms of dividing the original sample into separate minidatasets according to the principle of data continuity. That is, the original df is divided into samples of the maximum length, by the gaps. These gaps, due to which the dataset is broken, are completely will be removed, that is, the continuous part to the left and to the right of the gap there give us 2 minidatasets that were obtained splitting by this gap. Needed for working with sequences due to the requirement: the absence of gaps and the same sampling frequency.

Parameters:

namestr | None

The name of the task.

Methods

fit_predict(df, time_result)	Fits the SplitByNaNTask.
predict(df, result, time_result)	Predicts by SplitByNaNTask.

fit_predict(df: DataFrame, time_result: ResampleProcessingResult) → tuple[pandas.core.frame.DataFrame, tsad.tasks.preprocess.SplitByNaNResult]

Fits the SplitByNaNTask.

Parameters:

dfpd.DataFrame

The input dataset.

time_result: ResampleProcessingResult

The result of the ResampleProcessingTask, mainly due to the freq_tobe argument, which is needed for splitting.

Returns:

dfslist[pd.DataFrame]

The output datasets that satisfy the continuity condition.

resultSplitByNaNResult

The object that stores the results of the SplitByNaNTask.

predict(df: DataFrame, result: SplitByNaNResult, time_result: ResampleProcessingResult) → tuple[pandas.core.frame.DataFrame, tsad.tasks.preprocess.SplitByNaNResult]

Predicts by SplitByNaNTask.

Parameters:

dfpd.DataFrame

The input dataset.

time_result: ResampleProcessingResult

Need freq_tobe parametr for splitting, which is stored in the ResampleProcessingTask.

Returns:

dfslist[pd.DataFrame]

The output datasets that satisfy the continuity condition.

resultSplitByNaNResult

The object that stores the results of the SplitByNaNTask.

class tsad.tasks.preprocess.ValueRangeProcessingResult

Bases: TaskResult

Methods

save
show

max_values: Series

min_values: Series

show() → None

class tsad.tasks.preprocess.ValueRangeProcessingTask(name: str | None = None, method: str | None = 'auto', show: bool | None = True)

Bases: Task

Class of the data preprocessing task in the field of value range processing.

Methods

fit_predict(df)	Fit the ValueRangeProcessingTask.
predict(df, vrp_result)	Predict using the ValueRangeProcessingTask.

This is a task that represents a class for value range processing with some meta-information stored in ValueRangeProcessingResult.

Parameters:

methodstr, default ‘auto’

The method of value range processing. If ‘auto’, then the method is set to ‘MinMax’.

showbool, default True

Whether to show the warning message if some values are out of the allowed range.

Methods

fit_predict(df)	Fit the ValueRangeProcessingTask.
predict(df, vrp_result)	Predict using the ValueRangeProcessingTask.

fit_predict(df: DataFrame) → tuple[pandas.core.frame.DataFrame, tsad.base.task.TaskResult]

Fit the ValueRangeProcessingTask. The method finds the minimum and maximum values of the input dataframe and replaces the values outside the allowed range with NaN.

Parameters:

dfpd.DataFrame

Input dataset

Returns:

tuple[pd.DataFrame, TaskResult]

Dataframe with values outside the allowed range replaced with NaN and ValueRangeProcessingResult

predict(df: DataFrame, vrp_result: ValueRangeProcessingResult) → tuple[pandas.core.frame.DataFrame, tsad.base.task.TaskResult]

Predict using the ValueRangeProcessingTask. The method replaces the values outside the allowed range with NaN.

Parameters:

dfpd.DataFrame

Input dataset

vrp_resultValueRangeProcessingResult

Result of the ValueRangeProcessingTask

Returns:

tuple[pd.DataFrame, TaskResult]

Dataframe with values outside the allowed range replaced with NaN and ValueRangeProcessingResult

tsad.tasks.visualization module

Module contents