Skip to content
naiaden
SLM
Public

About

SLM is a Bayesian skipgram language model based on hierarchical Pitman-Yor processes. It facilitates multiple backoff strategies, interpolation strategies, and offers tools to train and evaluate language models, and rescore nbest lists

Topics

bayesianlanguage-modelpitman-yorskipgram

Resources

Readme
GPL-3.0 license
Activity

Stars

3 stars

Watchers

1 watching

Forks

0 forks
Report repository
BranchesTags

Latest commit

 

History

158 Commits

Folders and files

NameName
Last commit message
Last commit date
cpyp
cpyp
 
 
tools
tools
 
 
train
train
 
 
BackoffStrategies.cpp
BackoffStrategies.cpp
 
 
BackoffStrategies.h
BackoffStrategies.h
 
 
BackoffStrategy.cpp
BackoffStrategy.cpp
 
 
BackoffStrategy.h
BackoffStrategy.h
 
 
CountInterpolationStrategy.cpp
CountInterpolationStrategy.cpp
 
 
CountInterpolationStrategy.h
CountInterpolationStrategy.h
 
 
EntropyInterpolationStrategy.cpp
EntropyInterpolationStrategy.cpp
 
 
EntropyInterpolationStrategy.h
EntropyInterpolationStrategy.h
 
 
FullBackoffStrategy.cpp
FullBackoffStrategy.cpp
 
 
FullBackoffStrategy.h
FullBackoffStrategy.h
 
 
InterpolationStrategy.cpp
InterpolationStrategy.cpp
 
 
InterpolationStrategy.h
InterpolationStrategy.h
 
 
LICENSE
LICENSE
 
 
LanguageModel.cpp
LanguageModel.cpp
 
 
LanguageModel.h
LanguageModel.h
 
 
LimitedBackoffStrategy.cpp
LimitedBackoffStrategy.cpp
 
 
LimitedBackoffStrategy.h
LimitedBackoffStrategy.h
 
 
Logging.cpp
Logging.cpp
 
 
Logging.h
Logging.h
 
 
MLEInterpolationStrategy.cpp
MLEInterpolationStrategy.cpp
 
 
MLEInterpolationStrategy.h
MLEInterpolationStrategy.h
 
 
Makefile
Makefile
 
 
NBestList.cpp
NBestList.cpp
 
 
NBestList.h
NBestList.h
 
 
NgramBackoffStrategy.cpp
NgramBackoffStrategy.cpp
 
 
NgramBackoffStrategy.h
NgramBackoffStrategy.h
 
 
NprefInterpolationStrategy.cpp
NprefInterpolationStrategy.cpp
 
 
NprefInterpolationStrategy.h
NprefInterpolationStrategy.h
 
 
PerplexityInterpolationStrategy.cpp
PerplexityInterpolationStrategy.cpp
 
 
PerplexityInterpolationStrategy.h
PerplexityInterpolationStrategy.h
 
 
ProgramOptions.cpp
ProgramOptions.cpp
 
 
ProgramOptions.h
ProgramOptions.h
 
 
README.md
README.md
 
 
RandomInterpolationStrategy.cpp
RandomInterpolationStrategy.cpp
 
 
RandomInterpolationStrategy.h
RandomInterpolationStrategy.h
 
 
RescoreModule.cpp
RescoreModule.cpp
 
 
RescoreModule.h
RescoreModule.h
 
 
SLM-ppl.cpp
SLM-ppl.cpp
 
 
SLM-rescore.cpp
SLM-rescore.cpp
 
 
SLM-search.cpp
SLM-search.cpp
 
 
SLM-server.cpp
SLM-server.cpp
 
 
TextPreprocessor.cpp
TextPreprocessor.cpp
 
 
TextPreprocessor.h
TextPreprocessor.h
 
 
Timer.cpp
Timer.cpp
 
 
Timer.h
Timer.h
 
 
UniformInterpolationStrategy.cpp
UniformInterpolationStrategy.cpp
 
 
UniformInterpolationStrategy.h
UniformInterpolationStrategy.h
 
 
Utils.cpp
Utils.cpp
 
 
Utils.h
Utils.h
 
 
ValueInterpolationStrategy.cpp
ValueInterpolationStrategy.cpp
 
 
ValueInterpolationStrategy.h
ValueInterpolationStrategy.h
 
 
cmdline.h
cmdline.h
 
 
uniform_vocab.h
uniform_vocab.h
 
 
uvector.h
uvector.h
 
 

Repository files navigation

About

SLM is a Bayesian skipgram language model based on hierarchical Pitman-Yor processes. It facilitates multiple backoff strategies, interpolation strategies, and offers tools to train and evaluate language models, and rescore nbest lists

Topics

bayesianlanguage-modelpitman-yorskipgram

Resources

Readme
GPL-3.0 license
Activity

Stars

3 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages