-

© . !. $ # # ( # $ ( # $ '' '" ) # & $ & $ $ '"+ 02 + '

, $ ' ( $ ) # % +' ( ), +) $ - 1' ( ,

, + # ) % . # 1'

# ' # '" )# # '"

, +) , $ ' - #) ) ) +) , $ # ' '" 0 - #

( - .

) $ . # $ ".

# ( $ # $ - '0( $ - ' , ' $ )3 ( )# +)

$ $ % & ( #.

# '0( ) # ) 1 , (

TextTiling, & # % 0 ( # .

# #

% '"+ 2

# , $ # '" ) %

-' +) 3 , 6&& # ),

" $ - $ - ' & # % ' '" $ ' ) ,

XVII DAMDID/RCDL’2015 « !" », # 13-16

2 # . -. & # % ' , 6 , - )2 ' - 02

1 & , ( ( 30]. . $.

' + $

' # ( & # %

#

, $ - # $ )# ' '0( )#

$ ' - #

( 0 ) ) # # '" ' , $ $

' 1- #

# & # % ( ,

# $ - #) + % ' * #

# $ + '

+# 2 # # 0 + $ $ # – - % # . + $

#, ' " ' 2 % ' ' ) + ( 0 ( )#

' 1 # + #, $ -", ( # $ '"+ [e-mail]

[e-mail] $ ) < # ' (1 – 2 #

$ # $ )

, ( ,' , $ , ' . 6

( ( ) ' 0 $ +),

" '"3 , +) -' [16, 26,

$ & ),

, " -' -), ( )# #, ( .

- ' )# -'

$ # , # + '" $ '"+ # $ ) ( # , - # 3 , + - ( (

# '" $ ' ( ) 1, '

%) '0( ), " % '" #

+ % + - ( $ " , # ' ) # # '

+ # 1# # , ).

# ' + - # - ' * + 1# 3 #

# ) '

$$ W ; 0 +# - '" 0 , 1

# ' $$ 3 s # s=1,

0

W-

W- - (W*(k + - (W*(k+2))$ ' % ; " # 1- , ( W*k), 0 cos 1 , (1) $

#.

, ), ' $

$ ' ' #) ( #., $ , ' 1

, ( # , [ 7, 14, 21 ]).

# - '" ' ( - # # # #, $ ( # - $ -$ ),

' +) .

'" ),, + ( ) ' , '0(

0 + ) # # # # ) ' #, $ $ 0 $ '"3 ,

+ +) + - ( [ 3, 24 ].

, $ ' ) ( ) ' " ,, *&& (

. $ ' ( ' # + , # + % '"3 + $ - ' ), # 1 02 , , $ ), # - ,

-' ) #

( - # $$) ),, $

# ' ( " + ' , + ' # " # , , ( ) "

. !' TextTiling, # ) ,

, ( # ' ( +) 0 ' $

' ;

E+) & ( % + - ' ( . #) [ 21 ], ( $ ) # + % ( ), , – " ) + '

) $ $ 3' $ " +) % . I # & +) # '" ), $ $ % )

! ' " + # , +) , ( - # &

, . D , (

' ' ) '. 1. ' '' '" ) & # ) # ' ($ +. 1, 3),

- ($ +. 2, 4). ' # - ( - # $ + ' ( #

#) 0,4 0,2

0 3 # 3.1 # & # % + ( $

+) $ & $ -, + ), ' # ' $ # 2"0

# ' + 1)' & & # ( # ' % 2.

+ ( N # %

. R $ - ' . ' ) 3 # % [28]: ' $ '"+

-' ' " % ' ( + + ) $ $ ' ## ' ## ' ##

' ' ' ## $ - '0( + % + % 2 + % 2 + % + - ' + ( + ( + ( 3 '" ),

( '" ),, $ ' ( ( ' $ - '

+# & +#

'" ), +# + + + ), (

0

P * $ . 1.

%

z, ' $

# ( ) ' – ( ", R

TP TP

FN , FP – $

' ( $ 2 % #

' 1 ), + ( ), + ( ,

.

* ' +# # 1

-' ), & % # # - # ) , % $ ' " + " $ # #

+ #. ! ' + # ) (1) + #

z. R ( # ' " # (H).

$ ' ( ), $ ' '" $

, $ ' '" ) #

1 )#, $

R cos , # # # '" ( TP)

'" # #) («$ + % ' '

$ + % -' $ + ' ' +, (

$' $) $ , - $ '" ) «$ (FP), «$ $ - ' $ z " #

0 % $

& # ( $ ' 1 ' ( # #, ( #

( $ % ,

+ 2 # ,

')» ')» ' (3

' $ ( 1 - # # # – # " # ' 4 & 4.1 ( , #. - ' ( # "3 ' #), . .

- ' "

' " A H)

')») - , ' '" ) #) 2 , + ' , ( $ # )

% , , $ $ $ # # '" ) z=0.1…0.15. - $ ' '" ), $ - $ + ( F-# + ( , z (

P

R ( 1

$ + ( , + (

) . 3).

& 4.3

3 # # ( $ 1 0,8 0,6 0,4 0,2 0 - ' 02 # $ $ #

# %) " $ z.

" P, . . $ $ ' R, . .

. # " $ ( 0 , ,

, $ $

'. 4. $ ' 1 + (

, , ' ' ' ! ' +

' %) $ # +) $ '"+ $ -+ $' , + - 1 + $ # # ( - '

6 + '" $ # 2"0 # 1- + % # ( . 1).

$' 3 )#

(' ( - ' , # 1, + )

0 - ' + (

N+ 0.21 0.17 0.44 0.22 0.60 , (

+ # (

' ) ' # )#» 1 ), - $ ' )3 .

$ # ) ( 3 , ( 2012. [27] !

+ $ -,

% $' . 8. [28] . .

#$"0 [29] .!.

. ., ! # $ -- 1 -' - # . ., ), $ '"+ , . .

' + & - . ! #

( + 1 & ' ' . 140–150. [32] . . & # . C

( : ' % .– [33] .!. + . #$"0

+) . ( +) , # .

( . .: '

'" , 2004.

This paper considers how to apply topic segmentation algorithms to real scientific texts. To study it we used monographs on the same subject written in three languages. The corpus includes several fragments both in the original and in professional translation.

The research is based on the TextTiling algorithm that analyses how tightly adjoining parts of a text cohere. We examined how some parameters (the cutoff rate, the size of moving window and of the shift from one block to the next one) influence the segmentation quality. The optimum combinations of these parameters are defined for several languages. The studies on the Russian language argue that external lexical resources (stop-lists, classifiers, ontologies) notably upgrade the quality of segmentation.

References

[1] Beeferman , Douglas, Adam Berger , and John Lafferty . Statistical models of text segmentation . Machine Learning , 34 ( 1-3 ), February 1999 .

[2] David M. Blei , Andrew Y. Ng , and Michael I. Jordan . 2003 . Latent Dirichlet allocation . Journal of Machine Learning Research , 3 , p. 993 - 1022 .

[3] Anja Habacha Chaibi , Marwa Naili, Samia Sammoud . Topic segmentation for textual document written in Arabic language . 18th International Conference on Knowledge-Based and Intelligent Information & Engineering Systems - KES2014. Procedia Computer Science 35 ( 2014 ), p. 437 - 446

[4] Choi , F.Y.Y. 2000 . Advances in domain independent linear text segmentation . In Proceedings of the 1st Meeting of the North American Chapter of the Association for Computational Linguistics , p. 26 - 33 .

[5] G. Dias , E. Alves , J.G.P. Lopes . Topic Segmentation Algorithms for Text Summarization and Passage Retrieval: An Exhaustive Evaluation . AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 2 . p. 1334 - 1339

[6] Lan Du , Wray Buntine, and Mark Johnson . 2013 . Topic Segmentation with a Structured Topic Model . In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , p. 190 - 200 , Atlanta, Georgia.

[7] Jacob Eisenstein / Hierarchical Text Segmentation from Multi-Scale Lexical Cohesion . NAACL '09 Proceedings of Human Language Technologies : The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics , p. 353 - 361

[8] Jacob Eisenstein and Regina Barzilay . 2008 . Bayesian Unsupervised Topic Segmentation . In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing , p. 334 - 343 , Honolulu, Hawaii.

[9] Dominik Flejter , KarolWieloch, Witold Abramowicz. Unsupervised Methods of Topical Text Segmentation for Polish . Balto-Slavonic Natural Language Processing 2007 , June 29, 2007 , p. 51 - 58 . Prague, June 2007. Association for Computational Linguistics .

[10] M Georgescul , A Clark , and S Armstrong . An analysis of quantitative aspects in the evaluation of thematic segmentation algorithms . SigDIAL'06 Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue , Jan. 2009 .

[11] Marti A. Hearst . TextTiling: Segmenting text into multi-paragraph subtopic passages . Computational Linguistics , 23 ( 1 ), p. 33 - 64 , March 1997 .

[12] Marti A. Hearst and Christian Plaunt . 1993 . Subtopic structuring for full-length document access . In SIGIR '93: Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval , p. 59 - 68 , New York, NY, USA. ACM Press.

[13] Anna Kazantseva and Stan Szpakowicz . 2011 . Linear Text Segmentation Using Affinity Propagation . In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing , p. 284 - 293 , Edinburgh, Scotland.

[14] Anna Kazantseva , Stan Szpakowicz . Hierarchical Topical Segmentation with Affinity Propagation . In Proceedings of COLING 2014 , the 25th International Conference on Computational Linguistics: Technical Papers , p. 37 - 47 , Dublin, Ireland.

[15] S. Lamprier , T. Amghar , and B. Levrat . 2008 . On evaluation methodologies for text segmentation algorithms . 19th IEEE International Conference on Tools with Artificial Intelligence - Vol. 2 , Jan .

[16] David YW Lee. Genres , Registers, Text Types, Domains, and Styles: Clarifying the Concepts and Navigating a Path through the BNC Jungle . Language Learning & Technology . Vol. 5 , No. 3 , p. 37 - 72 , September 2001 .

[17] Hemant Misra , Franc`ois Yvon, Olivier Capp , and Joemon M. Jose . 2011 . Text segmentation: A topic modeling perspective . Information Processing and Management , 47 ( 4 ), p. 528 - 544 .

[18] POS Tagging Open Xerox . https://open.xerox.com/Services/fst-nlptools/Consume/Part%20of% 20Speech %20Taggi ng% 20 %28Standard% 29 - 178 / 2 - 14 . 03 .15

[19] Ponte , Jay and Bruce Croft . 1997 . Text segmentation by topic . In Proceedings of the First European Conference on Research and Advanced Technology for Digitial Libraries.

[20] Jeffrey C. Reynar . 1994 . An automatic method of finding topic boundaries . In Proceedings of the 32nd annual meeting on Association for Computational Linguistics , p. 331 - 333 , Morristown, NJ, USA. Association for Computational Linguistics.

[21] Martin Riedl , Chris Biemann . Text Segmentation with Topic Models . JLCL 2012 - Band 27 ( 1 ), p. 47 - 69 .

[22] M. Scaiano , D. Inkpen . Getting More from Segmentation Evaluation. 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , p. 362 - 366 , Montreqal, Canada, June 3-8, 2012 .

[23] Stark , Heather A. , 1988 . What do paragraph markings do? Discourse Processes 11 , p. 275 - 303 .

[24] Xiaojun Wan . On the effectiveness of subwords for lexical cohesion based story segmentation of Chinese broadcast news . Information Sciences 177 ( 2007 ), p. 3718 - 3730 .

[25] Na Ye , Jingbo Zhu, Huizhen Wang, Matthew Y. Ma, Bin Zhang. An Improved Model of Dotplotting for Text Segmentation . Journal of Chinese Language and Computing 17 ( 1 ), p. 27 - 40 .

[26] ! . . + ( # ' ( " M. Black “Metaphor”) // . C . - 2013. )$. 4 ( 24 ). -