python xapian存储结构
在项目中为了支持搜索服务,我们使用xapian作为后端的搜索引擎.其因性能良好以及易用受到大家欢迎.下面是基本代码:
from @brass_table.cc/* A B-tree comprises (a) a base file, containing essential information (Block size, number of the B-tree root block etc), (b) a bitmap, the Nth bit of the bitmap being set if the Nth block of the B-tree file is in use, and (c) a file DB containing the B-tree proper. The DB file is divided into a sequence of equal sized blocks, numbered 0, 1, 2 ... some of which are free, some in use. Those in use are arranged in a tree. Each block, b, has a structure like this: R L M T D o1 o2 o3 ... oN <gap> [item] .. [item] .. [item] ... <---------- D ----------> <-M-> And then, R = REVISION(b) is the revision number the B-tree had when the block was written into the DB file. L = GET_LEVEL(b) is the level of the block, which is the number of levels towards the root of the B-tree structure. So leaf blocks have level 0 and the one root block has the highest level equal to the number of levels in the B-tree. M = MAX_FREE(b) is the size of the gap between the end of the directory and the first item of data. (It is not necessarily the maximum size among the bits of space that are free, but I can't think of a better name.) T = TOTAL_FREE(b)is the total amount of free space left in b. D = DIR_END(b) gives the offset to the end of the directory. o1, o2 ... oN are a directory of offsets to the N items held in the block. The items are key-tag pairs, and as they occur in the directory are ordered by the keys. An item has this form: I K key x C tag <--K--> <------I------> A long tag presented through the API is split up into C tags small enough to be accommodated in the blocks of the B-tree. The key is extended to include a counter, x, which runs from 1 to C. The key is preceded by a length, K, and the whole item with a length, I, as depicted above.