evaluation
Class SingleMethodEvaluation

java.lang.Object
  evaluation.SingleMethodEvaluation

public class SingleMethodEvaluation
extends java.lang.Object

Evaluation of single method

Author:

Mateusz Kopec

Field Summary

java.lang.String	methodId Description of method evaluated
int	segmentsTaggedAsUnknown Segment tagged as unknown sense by the method

Constructor Summary

SingleMethodEvaluation(java.lang.String methodName, corpusapi.tei.TEISenseInventory dict)
Constructor

Method Summary

void	addCorrectAnnotation(java.lang.String lexeme, java.lang.String senseId) Adds correct annotation to evaluation
void	addIncorrectAnnotation(java.lang.String lexeme, java.lang.String senseId) Adds incorrect annotation to evaluation
java.lang.Double	getEqualDistLexemeAccuracy(java.lang.String lexeme) Returns accuracy for lexeme if the sense distribution would be changed to equal
java.lang.Double	getLexemeAccuracy(java.lang.String lexeme) Returns accuracy for a lexeme
int	getLexemeCount(java.lang.String lexeme) Gets number of annotations for lexeme
double	getLexemeImprovement(AnnotationStats goldenStandard, java.lang.String lexeme) Returns the improvement agains mfs baseline for given lexeme
java.util.Set<java.lang.String>	getLexemeSet() Returns set of all annotated lexemes
java.lang.String	getMethodId() Gets method description
int	getNumberOfImprovements(java.lang.String pos, AnnotationStats goldenStandard) Calculates the number of lexemes for which this method was better than mfs (and which are of given part of speech)
java.lang.Double	getSubsenseAccuracy(java.lang.String lexeme, java.lang.String subsenseId) Returns accuracy for a subsense
java.util.Set<java.lang.String>	getSubsenseSet(java.lang.String lexeme) Returns set of annotated subsenses of given lexeme
java.lang.Double	getTotalAccuracy() Returns total accuracy of the method evaluated
double	getTotalAccuracy(java.lang.String pos, AnnotationStats goldenStandard) Gets total accuracy for given part of speech
double	getWeightedRank(java.lang.String pos, AnnotationStats goldenStandard) Gets RARE result of method for given part of speech
void	printStats() Prints statistics about evaluated method.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

methodId

public java.lang.String methodId

Description of method evaluated

segmentsTaggedAsUnknown

public int segmentsTaggedAsUnknown

Segment tagged as unknown sense by the method

Constructor Detail

SingleMethodEvaluation

public SingleMethodEvaluation(java.lang.String methodName,
                              corpusapi.tei.TEISenseInventory dict)

Constructor

Parameters:

methodName - method id

dict - dictionary of senses

Method Detail

printStats

public void printStats()

Prints statistics about evaluated method.

getTotalAccuracy

public java.lang.Double getTotalAccuracy()

Returns total accuracy of the method evaluated

Returns:

result

getLexemeImprovement

public double getLexemeImprovement(AnnotationStats goldenStandard,
                                   java.lang.String lexeme)

Returns the improvement agains mfs baseline for given lexeme

Parameters:

goldenStandard -

lexeme -

Returns:

result

getLexemeCount

public int getLexemeCount(java.lang.String lexeme)

Gets number of annotations for lexeme

Parameters:

lexeme -

Returns:

result

getEqualDistLexemeAccuracy

public java.lang.Double getEqualDistLexemeAccuracy(java.lang.String lexeme)

Returns accuracy for lexeme if the sense distribution would be changed to equal

Parameters:

lexeme -

Returns:

result

getLexemeAccuracy

public java.lang.Double getLexemeAccuracy(java.lang.String lexeme)

Returns accuracy for a lexeme

Parameters:

lexeme -

Returns:

result

getSubsenseAccuracy

public java.lang.Double getSubsenseAccuracy(java.lang.String lexeme,
                                            java.lang.String subsenseId)

Returns accuracy for a subsense

Parameters:

lexeme -

subsenseId -

Returns:

result

getMethodId

public java.lang.String getMethodId()

Gets method description

Returns:

description

getNumberOfImprovements

public int getNumberOfImprovements(java.lang.String pos,
                                   AnnotationStats goldenStandard)

Calculates the number of lexemes for which this method was better than mfs (and which are of given part of speech)

Parameters:

pos - part of speech

goldenStandard - golden standard for mfs calculation

Returns:

result

getWeightedRank

public double getWeightedRank(java.lang.String pos,
                              AnnotationStats goldenStandard)

Gets RARE result of method for given part of speech

Parameters:

pos - part of speech

goldenStandard -

Returns:

result

addCorrectAnnotation

public void addCorrectAnnotation(java.lang.String lexeme,
                                 java.lang.String senseId)

Adds correct annotation to evaluation

Parameters:

lexeme -

senseId -

addIncorrectAnnotation

public void addIncorrectAnnotation(java.lang.String lexeme,
                                   java.lang.String senseId)

Adds incorrect annotation to evaluation

Parameters:

lexeme -

senseId -

getLexemeSet

public java.util.Set<java.lang.String> getLexemeSet()

Returns set of all annotated lexemes

Returns:

set

getSubsenseSet

public java.util.Set<java.lang.String> getSubsenseSet(java.lang.String lexeme)

Returns set of annotated subsenses of given lexeme

Parameters:

lexeme -

Returns:

set

getTotalAccuracy

public double getTotalAccuracy(java.lang.String pos,
                               AnnotationStats goldenStandard)

Gets total accuracy for given part of speech

Parameters:

pos - part of speech

goldenStandard -

Returns:

result