首页 诗词 字典 板报 句子 名言 友答 励志 学校 网站地图
当前位置: 首页 > 教程频道 > 数据库 > 其他数据库 >

【摘引】Oracle全文检索方面的研究(全7)

2012-08-26 
【引用】Oracle全文检索方面的研究(全7)3.7 Stoplist 属性Stoplist 允许屏蔽某些常用的词,比如is,a,this,对

【引用】Oracle全文检索方面的研究(全7)

3.7 Stoplist 属性

Stoplist 允许屏蔽某些常用的词,比如is,a,this,对这些词进行索引用处不大,系统

默认会使用和数据库语言相对应的停用词库(原文:Stoplists identify the words in your

language that are not to be indexed. In English, you can also identify stopthemes that are not to be indexed. By default, the system indexes text using the system-supplied stoplist that corresponds to your database language.),Oracle text 提供最常用的停用词库语言包括English, French, German,Spanish, Chinese, Dutch, and Danish

分别有basic_stoplist,empty_stoplist,default_stoplist,multi_stoplist 几种类型

?

3.7.1 Basic_stoplist

建立用户自定义的停用词库,文档中关于stoplist 的介绍相当少,只有寥寥的数行

例子:

Create table my_stop (id number, docs varchar2(1000));

Insert into my_stop values (1, 'Stoplists identify the words in your language that are not

to be indexed.');

Insert into my_stop values (2, 'ou can also identify stopthemes that are not to be indexed');

Commit;

/

--建立basic stoplist

Begin

Ctx_ddl.create_stoplist('test_stoplist', 'basic_stoplist');

End;

Create index ind_m_stop on my_stop(docs) indextype is ctxsys.context

parameters ('stoplist test_stoplist');

Select * from my_stop where contains(docs, 'words') > 0;

Begin

Ctx_ddl.add_stopword('test_stoplist', 'language'); --添加停用词

ctx_ddl.sync_index('ind_m_stop', '2m'); --同步索引

End;

Select * from my_stop where contains(docs, 'language') > 0; --添加停用词,同步索引后发现还是

能查到,需要重新建立索引才能生效

Drop index ind_m_stop;

Create index ind_m_stop on my_stop(docs) indextype is ctxsys.context

parameters ('stoplist test_stoplist');

Select * from my_stop where contains(docs, 'language') > 0; --停用词生效

添加停用词,同步索引后发现还是能查到,需要重新建立索引才能生效。

?

3.7.2 Empty_stoplist

停用词库没有任何停用词,适用于不需要过滤的查询中,如不需要过滤is this,a 等

?

3.7.3 Default_stoplist

建立basic_stoplist 后,里面不包含任何的停用词,而default_stoplist 在basic_stoplist 的基础

上增加了预定义的默认停用词,对于不同的语言,默认的停用词库数据也不一样

例子:

Create table my_stop (id number, docs varchar2(1000));

Insert into my_stop values (1, 'Stoplists identify the words in your language that are not

to be indexed.');

Insert into my_stop values (2, 'ou can also identify stopthemes that are not to be indexed');

Commit;

/

--建立lexer,不同lexer 属性会默认不同的停用词库

Begin

ctx_ddl.create_preference('test_b_lexer', 'basic_lexer');

End;

drop index ind_m_word;

--建立默认停用词default_stoplist

Create index ind_m_word on my_stop(docs) indextype is ctxsys.context

Parameters ('lexer test_b_lexer stoplist ctxsys.default_stoplist');

--检查默认词库中是否存在

Select * from my_stop where contains(docs, 'the') > 0;

Select * from my_stop where contains(docs, 'stopthemes') > 0;

--往默认词库中添加停用词

conn ctxsys/ctxsys;

Begin

ctx_ddl.add_stopword('default_stoplist', 'stopthemes'); --增加停用词

ctx_ddl.add_stopword('default_stoplist', 'words');

ctx_ddl.remove_stopword('default_stoplist', 'words');--删除停用词

End;

--添加后需重新建立索引才能生效

conn oratext/oratext;

drop index ind_m_word;

Create index ind_m_word on my_stop(docs) indextype is ctxsys.context

Parameters ('lexer test_b_lexer stoplist ctxsys.default_stoplist');

Select * from my_stop where contains(docs, 'words') > 0;

Select * from my_stop where contains(docs, 'stopthemes') > 0;

--相关数据字典

Select * from ctx_preferences where pre_name = 'DEFAULT_LEXER';

Select * from ctx_stopwords where spw_stoplist = 'DEFAULT_STOPLIST';

?

3.7.4 multi_stoplist

多语言停用词,适用在文档中包含不同的语言(A multi-language stoplist is useful when you use

the MULTI_LEXER to index a table that contains documents in different languages, such as

English, German, and Japanese)

增加停用词时,可以为停用词指定某种语言,只对指定的语言生效,默认情况下停用词对任

何语言都是生效的。

--建立multi_stoplist

begin

ctx_ddl.create_stoplist('multistop1', 'MULTI_STOPLIST');

ctx_ddl.add_stopword('multistop1', 'Die', 'german');

ctx_ddl.add_stopword('multistop1', 'Or', 'english');

end;

添加停用词,同步索引后发现还是能查到,需要重新建立索引才能生效。

?

3.7.5 参考脚本

--建立stoplist:

Begin

Ctx_ddl.create_stoplist('test_stoplist', 'basic_stoplist');

End;

--删除stoplist:

begin

ctx_ddl.drop_stoplist(' test_stoplist ');

end;

--增加停用词

ctx_ddl.add_stopword('default_stoplist', 'stopthemes'); --增加停用词

--删除停用词

??????????????? ctx_ddl.remove_stopword('default_stoplist', 'words');--删除停用词

?

3.8 Theme 主题查询

主题查询的概念是根据文档的含义,而不仅仅是根据某个词的匹配程度来返回查询结果

的。比如查询about(’US politics’)可能会返回‘US presidential elections’ 和 ‘US foreign

policy’之类的结果(原文:An ABOUT query is a query on a document theme. A document theme

is a concept that is sufficiently developed in the text. For example, an ABOUT query on US politics

might return documents containing information about US presidential elections and US foreign

policy. Documents need not contain the exact phrase US politics to be returned.)

10g 只支持两种主题查询语言:English,French

例子:

--在context 中启用主题查询

BEGIN

CTX_DDL.CREATE_PREFERENCE('TEST_ABOUT', 'BASIC_LEXER');

CTX_DDL.SET_ATTRIBUTE('TEST_ABOUT', 'INDEX_THEMES', 'YES');

CTX_DDL.SET_ATTRIBUTE('TEST_ABOUT', 'INDEX_TEXT', 'YES');

END;

CREATE INDEX IND_m_about ON my_about(DOCS) INDEXTYPE IS CTXSYS.CONTEXT

PARAMETERS ('LEXER CTXSYS.TEST_ABOUT');

--查询

SELECT * FROM my_about WHERE CONTAINS(DOCS, 'ABOUT(US politics)') > 0;

???????????????

热点排行