API

AdaGram is written almost entirely in Julia (some performance-critical parts are implemented in C with Julia wrappers provided). This page describes how one can use a trained AgaGram model from Julia.

Types

This section describes types defined in the package. Normally, you don't have to create instances of these types by yourself, only load/save them by predefined functions (unless you want to train a model from Julia, in this case please refer to train.jl)

Dictionary

type Dictionary
	word2id::Dict{String, Int32}
	id2word::Array{String}
end

• word2id maps a String representation of a word to its Int32 identifier
• id2word is a reverse mapping

VectorModel

type VectorModel
	frequencies::DenseArray{Int64}
	code::DenseArray{Int8, 2}
	path::DenseArray{Int32, 2}
	In::DenseArray{Float32, 3}
	Out::DenseArray{Float32, 2}
	alpha::Float64
	d::Float64
	counts::DenseArray{Float32, 2}
end

This type represents an AdaGram model. DenseArrays are usually SharedArrays.

• frequencies - number of occurrences for each word
• code[:, w] - binary code in hierarchical softmax (AdaGram uses Huffman tree similarly to SkipGram) for the word w
• path[:, w] - path (sequence of node numbers) in hierarchical softmax for the word w
• In[:, k, w] - input representation for the prototype k of word w, this is what you normally want as word features
• Out[:, n] - output representation for the node n in hierarchical softmax
• alpha - concentration parameter for Dirichlet Process (assumed to be the same for all words)
• d - parameter for Pitman-Yor process
• counts[k, w] - number of occurrences assigned to the prototype k of word w and hence the posterior parameter. sum(counts, 1) = frequencies

Loading and saving models

load_model(path::String) → (vm::VectorModel, dict::Dictionary)

Loads and returns saved VectorModel and Dictionary in a tuple.

save_model(path::String, vm::VectorModel, dict::Dictionary, min_prob=1e-5)

Saves VectorModel and Dictionary at path ignoring prototypes with prior probability less than min_prob in order to save disk space.

Working with a model

expected_pi{Tw <: Integer}(vm::VectorModel, w::Tw, min_prob=1e-3) → pi::Array{Float64}

Returns prior probability of each word prototype, optionally setting probabilities below min_prob to zero.

disambiguate{Tw <: Integer}(vm::VectorModel, x::Tw, context::AbstractArray{Tw, 1}, use_prior::Bool=true, min_prob::Float64=1e-3) → pi::Array{Float64}

Returns posterior probability of each word prototype given context words. By setting use_prior to false, uniform prior will be used instead of expected_pi (to which min_prob is passed).

log_skip_gram{T1 <: Integer, T2 <: Integer}(vm::VectorModel, w::T1, s::T2, v::T1) → Float32

Returns log probability of a context word v given input word w and its prototype s