hbase语法及一些问题的解决(其中有失误的地方,往体谅)
hbase 基础命令
(以下部分摘抄网络)
============================创建blogtable表=========================
create 'blogtable', 'info','text','comment_title','comment_author','comment_text'
<!--info,text等代表的是列簇,不是具体的列,可以在插入数据是任意的添加具体的列簇-->
============================插入概要信息=========================
put 'blogtable', '1', 'info:title', 'this is doc title'
put 'blogtable', '1', 'info:author', 'javabloger'
put 'blogtable', '1', 'info:url', 'http://www.javabloger.com/index.php'
put 'blogtable', '2', 'info:title', 'this is doc title2'
put 'blogtable', '2', 'info:author', 'H.E.'
put 'blogtable', '2', 'info:url', 'http://www.javabloger.com/index.html'
============================插入正文信息=========================
put 'blogtable', '1', 'text:', 'what is this doc context ?'
put 'blogtable', '2', 'text:', 'what is this doc context2?'
<---其中的1,2代表的是id号,只要是string就可以
==========================插入评论信息===============================
put 'blogtable', '1', 'comment_title:', 'this is doc comment_title '
put 'blogtable', '1', 'comment_author:', 'javabloger'
put 'blogtable', '1', 'comment_text:', 'this is nice doc'
put 'blogtable', '2', 'comment_title:', 'this is blog comment_title '
put 'blogtable', '2', 'comment_author:', 'H.E.'
put 'blogtable', '2', 'comment_text:', 'this is nice blog'
==========================搜索方式================================
scan 'blogtable' ,{COLUMNS => ['text:','info:title'] } —> 列出 文章的内容和标题
scan 'blogtable' , {COLUMNS => 'info:url' , STARTROW => '2'} —> 根据范围列出 文章的内容和标题
===>get与scan的区别,get方式的执行最终也是转换成scan的方式,get一般要带id号,否则查询内容?
get 'blogtable','1' —> 列出 文章id 等于1的数据
get 'blogtable','1', {COLUMN => 'info'} —> 列出 文章id 等于1 的 info 的头(Head)内容
get 'blogtable','1', {COLUMN => 'text'} —> 列出 文章id 等于1 的 text 的具体(Body)内容
get 'blogtable','1', {COLUMN => ['text','info:author']} —> 列出 文章id 等于1 的内容和作者(Body/Author)内容
(以上部分摘抄网络)
=========================查询所有的数据库名称=======================
list
=========================查看某表结构===============================
describe ‘表名’
=========================统计表中的记录数===========================
count '表名' 该统计的是所有的行记录。同一行名称的只能算是1条记录
名称 命令表达式
创建表 create '表名称', '列名称1','列名称2','列名称N'
添加记录 put '表名称', '行名称', '列名称:', '值'
查看记录 get '表名称', '行名称'
查看表中的记录总数 count '表名称'
删除记录 delete '表名' ,'行名称' , '列名称'
删除一张表 先要屏蔽该表,才能对该表进行删除,第一步 disable '表名称' 第二步 drop '表名称'
查看所有记录 scan "表名称"
某个表某个列中所有数据 scan "表名称" , ['列名称:']
更新记录 就是重写一遍进行覆盖
------------------------------------简单的程序使用----------------------------------
示例程序:
/**
*
*/
import java.io.IOException;
import java.util.List;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.KeyValue;
import org.apache.hadoop.hbase.MasterNotRunningException;
import org.apache.hadoop.hbase.ZooKeeperConnectionException;
import org.apache.hadoop.hbase.client.Get;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Result;
public class HBaseTest {
static HBaseConfiguration cfg = null;
static {
Configuration HBASE_CONFIG = new Configuration();
HBASE_CONFIG.set("hbase.zookeeper.quorum", "xxx.xxx.xxx.193,xxx.xxx.xxx.194,xxx.xxx.xxx.221");
HBASE_CONFIG.set("hbase.zookeeper.property.clientPort", "2181");
cfg = new HBaseConfiguration();
// cfg = new HBaseConfiguration();
}
public void createTables() throws IOException{
HBaseAdmin admin = new HBaseAdmin(cfg);
if(!admin.tableExists("tableForTest")){
HTableDescriptor tableDesc = new HTableDescriptor("tableForTest");
//增加列簇
tableDesc.addFamily(new HColumnDescriptor("name"));
tableDesc.addFamily(new HColumnDescriptor("salary"));
tableDesc.addFamily(new HColumnDescriptor("department"));
tableDesc.addFamily(new HColumnDescriptor("age"));
tableDesc.addFamily(new HColumnDescriptor("sex"));
admin.createTable(tableDesc);
}
}
public void addRecords() throws IOException{
HTable table = new HTable(cfg,"tableForTest");
Put row = new Put(String.format("row%09d", 1).getBytes());
row.add("name".getBytes(), null, "name1".getBytes());
row.add("salary".getBytes(),null,"$1000".getBytes());
row.add("department".getBytes(),null,"aramy".getBytes());
row.add("age".getBytes(),null,"24".getBytes());
row.add("sex".getBytes(),null,"male".getBytes());
Put row1 = new Put(String.format("row%09d", 2).getBytes());
row1.add("name".getBytes(), null, "name2".getBytes());
row1.add("salary".getBytes(),null,"$2000".getBytes());
row1.add("department".getBytes(),null,"aramy".getBytes());
row1.add("age".getBytes(),null,"28".getBytes());
row1.add("sex".getBytes(),null,"female".getBytes());
Put row2 = new Put("3".getBytes());
row2.add("name".getBytes(), null, "name3".getBytes());
row2.add("salary".getBytes(),null,"$3000".getBytes());
row2.add("department".getBytes(),null,"aramy".getBytes());
row2.add("age".getBytes(),null,"26".getBytes());
row2.add("sex".getBytes(),null,"female".getBytes());
table.put(row);
table.put(row1);
table.put(row3);
}
public void getRow() throws IOException{
HTable table = new HTable(cfg,"tableForTest");
Get row = new Get("1".getBytes());
row.addColumn("name".getBytes());
Result result = table.get(row);
List<KeyValue> list = result.list();
for(KeyValue kv:list){
System.out.println(new String(kv.getKey())+" "+new String(kv.getValue()));
}
}
public void delete() throws IOException{
//删除表
HBaseAdmin hAdmin = new HBaseAdmin(cfg);
if (hAdmin.tableExists("tableForTest")) {
hAdmin.disableTable("tableForTest");
hAdmin.deleteTable("tableForTest");
}
}
public static void main(String[] args) throws IOException {
HBaseTest test = new HBaseTest();
test.addRecords();
test.getRow();
}
}
问题:(windows环境,172网段)
12/08/07 17:34:59 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=xxx.xxx.xxx.221:2181,xxx.xxx.xxx.194:2181,xxx.xxx.xxx.193:2181 sessionTimeout=180000 watcher=hconnection
12/08/07 17:34:59 INFO zookeeper.ClientCnxn: Opening socket connection to server /xxx.xxx.xxx.193:2181
12/08/07 17:35:08 INFO zookeeper.ClientCnxn: Socket connection established to xxx.xxx.xxx.193/xxx.xxx.xxx.193:2181, initiating session
12/08/07 17:35:08 INFO zookeeper.ClientCnxn: Session establishment complete on server xxx.xxx.xxx.193/xxx.xxx.xxx.193:2181, sessionid = 0x139006921560002, negotiated timeout = 180000
12/08/07 17:35:22 INFO ipc.HbaseRPC: Server at xxx.xxx.xxx.205/xxx.xxx.xxx.205:60020 could not be reached after 1 tries, giving up.
心得:
总是遇到Server at xxx.xxx.xxx.205/xxx.xxx.xxx.205:60020 could not be reached after 1 tries,当时怀疑的是我的机器连不到192的ip(后来验证,的确是如此,本机是172的网段,服务器是192的网段,首先要连接192网段的时候,解析的xxx.xxx.xxx.205,是没有hbase服务的)。
后来发现又或许是由于user.name不是root(这也是个问题,即使在同一网段下,没有root权限访问受限的),访问不到192上root权限的hbase。移植程序到192的机器上,这次应该ok了吧?发现还是有问题。。。
(xxx.xxx.xxx.221,linux环境)
12/08/07 18:03:32 INFO zookeeper.ZooKeeper: Client environment:user.name=root
12/08/07 18:03:32 INFO zookeeper.ZooKeeper: Client environment:user.home=/root
12/08/07 18:03:32 INFO zookeeper.ZooKeeper: Client environment:user.dir=/data/HBaseTest
12/08/07 18:03:32 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection
12/08/07 18:03:32 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/xxx.xxx.xxx.205:2181
12/08/07 18:03:33 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
12/08/07 18:03:34 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/xxx.xxx.xxx.205:2181
悲剧了,这下该如何解决?为什么读到的server是localhost?(因为连接到本机会默认的识别本机的hostname),又为什么session for server 是null?查了半天资料也没看到网上有什么解决办法,说是classpath的问题。没办法继续找找吧(这个问题困扰了一个下午,后来结合了网上的结论和阅读了一些相关资料,发现查找的时候会查找本机的hostname)
问题解决了,不是由于classpath的问题,和/etc/hosts有关,通过不断的调整测试,发现
原来的/etc/hosts文件是
xxx.xxx.xxx.221 5.xxx.com
xxx.xxx.xxx.194 pp.xxx.com
xxx.xxx.xxx.193 pp2.xxx.com
127.0.0.1 localhost.domain localhost
现在被我更改为
此时我的/etc/hosts文件是
xxx.xxx.xxx.221 5.xxx.com
xxx.xxx.xxx.194 pp.xxx.com
xxx.xxx.xxx.193 pp2.xxx.com
127.0.0.1 localhost
解决!