xinb
/
ip2region_private
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

xdb_maker and usage docs

This commit is contained in:
lion 2022-06-17 15:35:58 +08:00
parent 0873aa7d69
commit bae17f007b
2 changed files with 29 additions and 27 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

36
maker/golang/ReadMe.md
View File

@ -1,27 +1,28 @@
# ip2region xdb golang 生成实现 # ip2region xdb golang 生成实现
# 程序编译 # 程序编译
通过如下方式编译得到 dbmaker 可执行程序:
通过如下方式编译得到 xdb_maker 可执行程序:
``` ```
# 切换到golang maker 根目录 # 切换到golang maker 根目录
go build go build -o xdb_maker
``` ```
编译成功后会在当前目录生成一个 dbmaker 的可执行文件 编译成功后会在当前目录生成一个 xdb_maker 的可执行文件
# 数据生成 # 数据生成
通过 `dbmaker gen` 命令生成 ip2region.xdb 二进制文件: 通过 `xdb_maker gen` 命令生成 ip2region.xdb 二进制文件:
``` ```
➜ golang git:(v2.0_xdb) ✗ ./dbmaker gen ➜ golang git:(v2.0_xdb) ✗ ./xdb_maker gen
dbmaker gen [command options] xdb_maker gen [command options]
options: options:
--src string source ip text file path --src string source ip text file path
--dst string destination binary db file path --dst string destination binary xdb file path
``` ```
例如，使用默认的 data/ip.merge.txt 作为源数据，生成一个 ip2region.xdb 到当前目录： 例如，使用默认的 data/ip.merge.txt 作为源数据，生成一个 ip2region.xdb 到当前目录：
```bash ```bash
./dbmaker gen --src=../../data/ip.merge.txt --dst=./ip2region.xdb ➜ golang git:(v2.0_xdb) ✗ ./xdb_maker gen --src=../../data/ip.merge.txt --dst=./ip2region.xdb
# 会看到一堆输出，最终会看到类似如下输出表示运行结束 # 会看到一堆输出，最终会看到类似如下输出表示运行结束
... ...
2022/06/16 16:38:48 maker.go:317: write done, with 13804 data blocks and (683591, 720221) index blocks 2022/06/16 16:38:48 maker.go:317: write done, with 13804 data blocks and (683591, 720221) index blocks
@ -30,17 +31,17 @@ options:
# 数据查询 # 数据查询
通过 `dbmaker search` 命令来测试查询输入的 ip： 通过 `xdb_maker search` 命令来测试查询输入的 ip：
``` ```
dbmaker test [command options] xdb_maker search [command options]
options: options:
--db string ip2region binary db file path --db string ip2region binary xdb file path
``` ```
例如，使用自带的 xdb 文件来运行查询测试： 例如，使用自带的 xdb 文件来运行查询测试：
```bash ```bash
./dbmaker search --db=../../data/ip2region.xdb ➜ golang git:(v2.0_xdb) ✗ ./xdb_maker search --db=../../data/ip2region.xdb
ip2region 2.0 test program, commands: ip2region xdb search test program, commands:
loadIndex : load the vector index for search speedup. loadIndex : load the vector index for search speedup.
clearIndex: clear the vector index. clearIndex: clear the vector index.
quit : exit the test program quit : exit the test program
@ -57,10 +58,10 @@ ip2region>>
# bench 测试 # bench 测试
如果你自主生成了 xdb 文件，请确保运行如下的 `dbmaker bench` 命令来确保制定的 xdb 文件的正确性： 如果你自主生成了 xdb 文件，请确保运行如下的 `xdb_maker bench` 命令来确保生成的的 xdb 文件的正确性：
``` ```
➜ golang git:(v2.0_xdb) ✗ ./dbmaker bench ➜ golang git:(v2.0_xdb) ✗ ./xdb_maker bench
dbmaker bench [command options] xdb_maker bench [command options]
options: options:
--db string ip2region binary xdb file path --db string ip2region binary xdb file path
--src string source ip text file path --src string source ip text file path
@ -69,7 +70,7 @@ options:
例如：使用 data/ip.merge.txt 源文件来 bench 测试 data/ip2region.xdb 这个 xdb 文件： 例如：使用 data/ip.merge.txt 源文件来 bench 测试 data/ip2region.xdb 这个 xdb 文件：
```bash ```bash
./dbmaker bench --db=../../data/ip2region.xdb --src=../../data/ip.merge.txt ➜ golang git:(v2.0_xdb) ✗ ./xdb_maker bench --db=../../data/ip2region.xdb --src=../../data/ip.merge.txt
# 会看到一堆输出，看到类似如下的数据表示 bench 测试通过了，否则就会报错 # 会看到一堆输出，看到类似如下的数据表示 bench 测试通过了，否则就会报错
... ...
try to bench segment: `224.0.0.0|255.255.255.255|0|0|0|内网IP|内网IP` try to bench segment: `224.0.0.0|255.255.255.255|0|0|0|内网IP|内网IP`
@ -80,4 +81,5 @@ try to bench segment: `224.0.0.0|255.255.255.255|0|0|0|内网IP|内网IP`
|-try to bench ip '255.255.255.255' ... --[Ok] |-try to bench ip '255.255.255.255' ... --[Ok]
Bench finished, {count: 3417955, failed: 0, took: 52.200116397s} Bench finished, {count: 3417955, failed: 0, took: 52.200116397s}
``` ```
*请注意 bench 测试使用的 `src` 文件需要是对应的生成 ip2region.xdb 的源文件*。
如果运行过程中有错误会立马停止运行，也可以执行 --ignore-error=true 参数来忽略错误，在最后看 failed 的统计结果。 如果运行过程中有错误会立马停止运行，也可以执行 --ignore-error=true 参数来忽略错误，在最后看 failed 的统计结果。

20
maker/golang/main.go
View File

@ -15,12 +15,12 @@ import (
) )
func printHelp() { func printHelp() {
fmt.Printf("ip2region dbmaker 2.0\n") fmt.Printf("ip2region xdb maker\n")
fmt.Printf("dbmaker [command] [command options]\n") fmt.Printf("%s [command] [command options]\n", os.Args[0])
fmt.Printf("Command: \n") fmt.Printf("Command: \n")
fmt.Printf(" gen generate the binary db file\n") fmt.Printf(" gen generate the binary db file\n")
fmt.Printf(" search binary db search test\n") fmt.Printf(" search binary xdb search test\n")
fmt.Printf(" bench binary db bench test\n") fmt.Printf(" bench binary xdb bench test\n")
} }
func genDb() { func genDb() {
@ -58,10 +58,10 @@ func genDb() {
} }
if srcFile == "" || dstFile == "" { if srcFile == "" || dstFile == "" {
fmt.Printf("dbmaker gen [command options]\n") fmt.Printf("%s gen [command options]\n", os.Args[0])
fmt.Printf("options:\n") fmt.Printf("options:\n")
fmt.Printf(" --src string source ip text file path\n") fmt.Printf(" --src string source ip text file path\n")
fmt.Printf(" --dst string destination binary db file path\n") fmt.Printf(" --dst string destination binary xdb file path\n")
return return
} }
@ -116,9 +116,9 @@ func testSearch() {
} }
if dbFile == "" { if dbFile == "" {
fmt.Printf("dbmaker test [command options]\n") fmt.Printf("%s search [command options]\n", os.Args[0])
fmt.Printf("options:\n") fmt.Printf("options:\n")
fmt.Printf(" --db string ip2region binary db file path\n") fmt.Printf(" --db string ip2region binary xdb file path\n")
return return
} }
@ -131,7 +131,7 @@ func testSearch() {
fmt.Printf("test program exited, thanks for trying\n") fmt.Printf("test program exited, thanks for trying\n")
}() }()
fmt.Println(`ip2region 2.0 test program, commands: fmt.Println(`ip2region xdb search test program, commands:
loadIndex : load the vector index for search speedup. loadIndex : load the vector index for search speedup.
clearIndex: clear the vector index. clearIndex: clear the vector index.
quit : exit the test program`) quit : exit the test program`)
@ -219,7 +219,7 @@ func testBench() {
} }
if dbFile == "" || srcFile == "" { if dbFile == "" || srcFile == "" {
fmt.Printf("dbmaker bench [command options]\n") fmt.Printf("%s bench [command options]\n", os.Args[0])
fmt.Printf("options:\n") fmt.Printf("options:\n")
fmt.Printf(" --db string ip2region binary xdb file path\n") fmt.Printf(" --db string ip2region binary xdb file path\n")
fmt.Printf(" --src string source ip text file path\n") fmt.Printf(" --src string source ip text file path\n")