reference documentation for LSTM objects

doc/DataSeries.rst

@@ -0,0 +1,160 @@
+:digest: A set of data series associated with identifiers.
+:species: data
+:sc-categories: UGens>FluidManipulation
+:sc-related: Classes/Dictionary
+:see-also: LabelSet, DataSet, DTW
+:max-seealso: dict
+:description: FluidDataSeries is a container associating series of data points with identifiers
+:control name:
+
+   The name of the FluidDataSeries. This is unique between all FluidDataSeries.
+
+:message addFrame:
+
+   :arg identifier: The identifier for the series to add to.
+
+   :arg buffer: A |buffer| containing the data for the frame (only the first channel is used).
+
+   Add a new frame to the end of a series, creates the series if it does not exist. Sets the dimensionality of the DataSeries if it is the first frame added, otherwise if the buffer is too short an error will be reported.
+
+:message addSeries:
+
+   :arg identifier: The identifier for the series to add.
+
+   :arg buffer: A |buffer| containing the data for the series (each channel is a distinct time frame).
+
+   Add a new series from a buffer. Sets the dimensionality of the DataSeries if it is the first series added, otherwise if the buffer is too short an error will be reported. If the identifier already exists an error will be reported.
+
+:message getFrame:
+
+   :arg identifier: The identifier for the series to get from.
+
+   :arg time: which time frame to get.
+
+   :arg buffer: A |buffer| to write the frame to (only the first channel is used, will be resized).
+
+   Get a frame from a series. Negative indexing starts from the last frame. If the identifier doesn't exist or if that series doesnt have a frame for that time point an error will be reported.
+
+:message getSeries:
+
+   :arg identifier: The identifier for the series to get.
+
+   :arg buffer: A |buffer| to write the series to (each channel is a distinct time frame, will be resized).
+
+   Get a series. If the identifier doesn't exist an error will be reported.
+
+:message setFrame:
+
+   :arg identifier: The identifier for the series to set a frame in.
+
+   :arg time: which time frame to set.
+
+   :arg buffer: A |buffer| containing the data for the frame (only the first channel is used).
+
+   Updates a time frame in a series, or adds it to the end if there is no frame at that time point. Negative indexing starts from the last frame. Sets the dimensionality of the DataSeries if it is the first frame added, otherwise if the buffer is too short an error will be reported.
+
+:message setSeries:
+
+   :arg identifier: The identifier for the series to set.
+
+   :arg buffer: A |buffer| containing the data for the series (each channel is a distinct time frame).
+
+   Updates a time series, or adds it if it doesn't exist. Sets the dimensionality of the DataSeries if it is the first series added, otherwise if the buffer is too short an error will be reported.
+
+:message updateFrame:
+
+   :arg identifier: The identifier for the series to update a frame in.
+
+   :arg time: which time frame to update.
+
+   :arg buffer: A |buffer| containing the data for the frame (only the first channel is used).
+
+   Updates an existing frame. Negative indexing starts from the last frame. If the buffer is too short an error will be reported. If the identifier doesn't exist or if that series doesnt have a frame for that time point an error will be reported.
+
+:message updateSeries:
+
+   :arg identifier: The identifier for the series to update.
+
+   :arg buffer: A |buffer| containing the data for the series (each channel is a distinct time frame).
+
+   Updates a new series. If the buffer is too short an error will be reported. If the identifier doesn't exist an error will be reported.
+
+:message deleteFrame:
+
+   :arg identifier: The identifier for the series to delete a frame from.
+
+   :arg time: which time frame to remove.
+
+   Delete a frame from a series, deletes the series if it is the only frame. Negative indexing starts from the last frame. If the identifier doesn't exist or if that series doesnt have a frame for that time point an error will be reported.
+
+:message deleteSeries:
+
+   :arg identifier: The identifier for the series to delete.
+
+   Delete a series. If the identifier doesn't exist an error will be reported.
+
+:message getDataSet:
+
+   :arg time: which time frame to extract.
+
+   :arg dataSet: The Dataset to write the slice to. Will overwrite and resize.
+
+   Get a dataset with the `time`th frame of every series, i.e. can create a :fluid-obj:`DataSet` with every Nth frame of every series. Negative indexing starts from the last frame. If an identifier doesn't have enough frames it is merely not added to the output dataset.
+
+:message clear:
+
+   Empty the data series of all series and frames.
+
+:message getIds:
+
+   :arg labelSet: The FluidLabelSet to export to. Its content will be replaced.
+
+   Export the dataseries identifiers to a :fluid-obj:`LabelSet`.
+
+:message merge:
+
+   :arg sourceDataSeries: The source DataSeries to be merged.
+
+   :arg overwrite: A flag to allow overwrite points with the same identifier.
+
+   Merge sourceDataSeries in the current DataSeries. It will replace the value of points with the same identifier if overwrite is set to 1.
+
+:message kNearest:
+
+   :arg buffer: A |buffer| containing a data point to match against.
+
+   :arg k: The number of nearest neighbours to return.
+
+   Returns the identifiers of the ``k`` points nearest to the one passed in distance order (closest first). Note that this is a brute force distance measure, and inefficient for repeated queries against large dataseries.
+
+:message kNearestDist:
+
+   :arg buffer: A |buffer| containing a data point to match against. The number of frames in the buffer must match the dimensionality of the DataSet.
+
+   :arg k: The number of nearest neighbours to return. The identifiers will be sorted, beginning with the nearest.
+
+   Returns the distances to the ``k`` points nearest to the one passed in descending order. Note that this is a brute force distance measure, and inefficient for repeated queries against large dataseries.
+
+:message print:
+
+   Post an abbreviated content of the DataSeries in the window by default, but you can supply a custom action instead. 
+
+:message read:
+
+   :arg filename: optional, filename to save to
+
+   Read a saved object in JSON format from disk, will prompt for file location if not filename not provided
+
+:message write:
+
+   :arg filename: optional, filename to save to
+
+   Save the contents of the object to a JSON file on disk to the file specified, will prompt for file location if not filename not provided
+
+:message load:
+
+   Load the state of this object from a Dictionary.
+
+:message dump:
+
+   Dump the state of this object as a Dictionary.

doc/LSTMClassifier.rst

@@ -0,0 +1,66 @@
+:digest: Series Classification with an LSTM
+:species: data
+:sc-categories: Machine learning, Classification, LSTM
+:sc-related: 
+:see-also: LSTMRegressor, LSTMForecaster, DataSeries, LabelSet
+:description: 
+
+  Perform classification between a :fluid-obj:`DataSeries` and a :fluid-obj:`LabelSet` using a long-short term memory recurrent neural network (LSTM)
+
+:discussion:  
+
+   For a thorough explanation of how this object works and more information on the parameters, visit the page on **Recurrent Neural Networks** (https://learn.flucoma.org/learn/recurrent-networks).
+
+   Also visit the classification tutorial, this is for the :fluid-obj:`MLPRegressor`, but it is good to understand regression conceptually: (https://learn.flucoma.org/learn/classification-neural-network/)
+
+   Conceptually equivalent to the :fluid-obj:`MLPClassifier`, but where that maps a :fluid-obj:`DataSet` to a :fluid-obj:`LabelSet`, recurrent networks can encode time-based patterns and learn those much more efficiently, so map a :fluid-obj:`DataSeries` to a :fluid-obj:`LabelSet`
+
+:control hiddenLayers:
+
+   An array of numbers that specifies the internal structure of the neural network. Each number in the list represents one hidden layer of the neural network, the value of which is the number of neurons in that layer. Changing this will reset the neural network, clearing any learning that has happened.
+
+:control maxIter:
+
+   The number of epochs to train for when ``fit`` is called on the object. An epoch consists of training on all the data points one time. Note every frame is processed so the the number of epochs will be much lower here than with the MLP objects (try around 5)
+
+:control learnRate:
+
+   A scalar for indicating how much the neural network should adjust its internal parameters during training. This is the most important parameter to adjust while training a neural network. 
+
+:control momentum:
+
+   A scalar that applies a portion of previous adjustments to a current adjustment being made by the neural network during training.
+
+:control batchSize:
+
+   The number of data series to use in between adjustments of the LSTM's internal parameters during training.
+
+:control validation:
+
+   A percentage (represented as a decimal) of the data points to randomly select, set aside, and not use for training (this "validation set" is reselected on each ``fit``). These points will be used after each epoch to check how the neural network is performing. If it is found to be no longer improving, training will stop, even if a ``fit`` has not reached its ``maxIter`` number of epochs.
+
+:message fit:
+
+   :arg sourceDataSeries: Source data
+
+   :arg targetLabelSet: Target labels
+
+   Train the network to map between a source :fluid-obj:`DataSeries` and target :fluid-obj:`LabelSet`
+
+:message predict:
+
+   :arg sourceDataSeries: Input data
+
+   :arg targetLabelSet: :fluid-obj:`LabelSet` to write the predicted labels into
+
+   Predict labels for a :fluid-obj:`DataSeries` (given a trained network)
+
+:message predictSeries:
+
+   :arg sourceBuffer: Input series
+
+   Predict a label for a single data series in a |buffer|
+
+:message clear:
+
+   This will erase all the learning done in the neural network.

doc/LSTMForecaster.rst

@@ -0,0 +1,68 @@
+:digest: Series Forecasting with an LSTM
+:species: data
+:sc-categories: Machine Learning, LSTM, Prediction
+:sc-related: 
+:see-also: LSTMClassifier, LSTMRegressor, DataSeries
+:description: 
+
+   Predict/forecast a continuation to an input :fluid-obj:`DataSeries` using a long-short term memory recurrent neural network (LSTM)
+
+:discussion:
+
+   For a thorough explanation of how this object works and more information on the parameters, visit the page on **Recurrent Neural Networks** (https://learn.flucoma.org/learn/recurrent-networks).
+
+   This object is not like anything that has been seen yet - it takes a single :fluid-obj:`DataSeries` and learns to predict continuations to the series with the same 'style'. It is currently rather limited in its ability, but will recieve improvements in predicting ability in the future!
+
+:control hiddenLayers:
+
+   An array of numbers that specifies the internal structure of the neural network. Each number in the list represents one hidden layer of the neural network, the value of which is the number of neurons in that layer. Changing this will reset the neural network, clearing any learning that has happened.
+
+:control maxIter:
+
+   The number of epochs to train for when ``fit`` is called on the object. An epoch consists of training on all the data points one time. Note every frame is processed so the the number of epochs will be much lower here than with the MLP objects (try around 5)
+
+:control learnRate:
+
+   A scalar for indicating how much the neural network should adjust its internal parameters during training. This is the most important parameter to adjust while training a neural network. 
+
+:control momentum:
+
+   A scalar that applies a portion of previous adjustments to a current adjustment being made by the neural network during training.
+
+:control batchSize:
+
+   The number of data series to use in between adjustments of the LSTM's internal parameters during training.
+
+:control validation:
+
+   A percentage (represented as a decimal) of the data points to randomly select, set aside, and not use for training (this "validation set" is reselected on each ``fit``). These points will be used after each epoch to check how the neural network is performing. If it is found to be no longer improving, training will stop, even if a ``fit`` has not reached its ``maxIter`` number of epochs.
+
+:message fit:
+
+   :arg sourceDataSeries: Source data
+
+   Train the network to learn to continue a :fluid-obj:`DataSeries`
+
+:message predict:
+
+   :arg sourceDataSeries: Input data
+
+   :arg targetDataSeries: Where to output the forecasted data
+
+   :arg forecastLength: how many frames to predict into the future, if left blank it will return the same number of frames provided for each series
+
+   Predict continuations for a :fluid-obj:`DataSeries` (given a trained network)
+
+:message predictSeries:
+
+   :arg sourceBuffer: Input series
+
+   :arg targetBuffer: Where to output the forecasted data
+
+   :arg forecastLength: how many frames to predict into the future, if left blank it will return the same number of frames provided
+
+   Predict a continuation to the data in a |buffer| (given a trained network)
+
+:message clear:
+
+   This will erase all the learning done in the neural network.

doc/LSTMRegressor.rst

@@ -0,0 +1,68 @@
+:digest: Series Regression with an LSTM
+:species: data
+:sc-categories: Machine learning, Regression, LSTM
+:sc-related: 
+:see-also: LSTMClassifier, LSTMForecaster, DataSeries, DataSet
+:description: 
+
+   Perform regression between a :fluid-obj:`DataSeries` and a :fluid-obj:`DataSet` using a long-short term memory recurrent neural network (LSTM)
+
+:discussion:  
+
+   For a thorough explanation of how this object works and more information on the parameters, visit the page on **Recurrent Neural Networks** (https://learn.flucoma.org/learn/recurrent-networks).
+
+   Also visit the regression tutorial, this is for the :fluid-obj:`MLPRegressor`, but it is good to understand regression conceptually: (https://learn.flucoma.org/learn/regression-neural-network/)
+
+   Conceptually equivalent to the :fluid-obj:`MLPRegressor`, but where that maps a :fluid-obj:`DataSet` to another, recurrent networks can encode time-based patterns and learn those much more efficiently.
+
+:control hiddenLayers:
+
+   An array of numbers that specifies the internal structure of the neural network. Each number in the list represents one hidden layer of the neural network, the value of which is the number of neurons in that layer. Changing this will reset the neural network, clearing any learning that has happened.
+
+:control maxIter:
+
+   The number of epochs to train for when ``fit`` is called on the object. An epoch consists of training on all the data points one time. Note the the number of epochs will be much lower here than with the MLP objects (try around 5) as every frame in every series is processed.
+
+:control learnRate:
+
+   A scalar for indicating how much the neural network should adjust its internal parameters during training. This is the most important parameter to adjust while training a neural network. 
+
+:control momentum:
+
+   A scalar that applies a portion of previous adjustments to a current adjustment being made by the neural network during training.
+
+:control batchSize:
+
+   The number of data series to use in between adjustments of the LSTM's internal parameters during training.
+
+:control validation:
+
+   A percentage (represented as a decimal) of the data points to randomly select, set aside, and not use for training (this "validation set" is reselected on each ``fit``). These points will be used after each epoch to check how the neural network is performing. If it is found to be no longer improving, training will stop, even if a ``fit`` has not reached its ``maxIter`` number of epochs.
+
+:message fit:
+
+   :arg sourceDataSeries: Source data
+
+   :arg targetDataSet: Target data
+
+   Train the network to map between a source :fluid-obj:`DataSeries` and target :fluid-obj:`DataSet`
+
+:message predict:
+
+   :arg sourceDataSeries: Input data
+
+   :arg targetDataSet: Output data
+
+   Apply the learned mapping to a :fluid-obj:`DataSet` (given a trained network)
+
+:message predictSeries:
+
+   :arg sourceBuffer: Input series
+
+   :arg targetBuffer: Output point
+
+   Predict a point for a single data series in a |buffer|
+
+:message clear:
+
+   This will erase all the learning done in the neural network.

example-code/sc/DataSeries.scd

@@ -0,0 +1,5 @@
+code::
+
+//soon
+
+::

example-code/sc/LSTMClassifier.scd

@@ -0,0 +1,5 @@
+code::
+
+//soon
+
+::

example-code/sc/LSTMForecaster.scd

@@ -0,0 +1,5 @@
+code::
+
+//soon
+
+::

example-code/sc/LSTMRegressor.scd

@@ -0,0 +1,5 @@
+code::
+
+//soon
+
+::