Input:?
Learning:?
Output: any format with purpose similar to BNF.
Supervisor: function gen_character generates characters valid to language analyzed to pass into review_character, review_character is a state machine that based on the BNF either accepts and saves (and changes state) or rejects input. The input saved by review_character gets later checked against a compiler for syntax errors.
Performance estimate: the line number where last syntax error was. Might want to ignore specific types of errors… For starters, let's ignore all non-syntax errors.
Alternative supervisor and estimate: instead of random characters use known valid source code that wasn't analyzed. Performance then measured by going further into the code without rejecting it.
- take word w
- remember distance to other words in distance d
- if the word already happened, check for patterns both in regard to prev. word and the new one.
- ↑ repeat for the next w.
Preliminary idea: when a word repetition found, check if prev. similar words have same distances to words… which ones?
Shall I may be set in advance possible patterns? Like "any word", "closing word", etc.
Learns input specifics with the purpose of restoring the input from broken or noisy samples.
Applications: denoising images, restoring images, coloring black-and-white images (which is also sort of restoring).
Full example: eyes from sequential photos fed to autoencoder layer for denoising, and then to LSTM to detect a motion sign over the series. Been used to control stuff with eyes motion.
How can I use it: to restore code structure for user is typing, so it's in invalid state. But it is better to restore patterns instead of text because it's more semantic and probably cheaper. So autoencoder should get noisy patterns as input, and to output restored ones. Hence patterns in the first place should be learned by other kind of layer.
Typically have output nodes representing classes, and, given input, assigns probability of each class to the output.
How can I use it: it's typically supervised learning, and have predefined classes. Unless I gotta find how to use it unsupervisingly, and to make it to create output nodes itself, it's useless for the task.
A usual perceptron working with sequential data. The name have nothing to do with the ANN, it's because the data delayed before feeding to the ANN, so that data from different time is analized together, like if the data happened simulationously.
Useless for me for reasons mentioned in "Probabilistic" ann; useless for anyone because LSTM should do the same without artificial delays.
Uses math convolution operation. Don't know much details yet, but it don't seem to be used anywhere except of a work with images.
???
Assign points to clusters in n-dimensional data. Requires I think predefined outputs.
Contras mentioned in "Probabilistic" holds.
???
???
A useless RNN with sparse connections, where only output layer neurons can change.
Odd technique of using two RNNs, one processing inputs left-to-right, and the other right-to-left; and then summing the results. Used for labeling and prediction. It's said to be very useful with LSTM.
For my purpose the complain about predefined outputs holds.
???
???
??? used for pattern recognition.
Any ANN varying its topology and size.
???
Probably akin to LSTM.
Both calculate "distance" of a word to every other. Except word2vec is probability-based, and GloVe is count-based. Also that GloVe is more easily paralleled since it takes whole matrix at once, whereas word2vec ANN-based, and uses many passes.
Acc. to internet, final results are more-or-less identical.