There is an overlap in the region chain

ERROR: (regions day_hotstatic,860010-2355010000_20140417_12_entry_00000000321,1398674475358.0dc205736ec1e890bd2d37a2e3220acc. and day_hotstatic,860010-2368000000_20140413_14_visit_00000001964,1400060700465.a590268ef714ef76779486a62fe837a3.) There is an overlap in the region chain.
14/05/15 16:38:37 WARN util.HBaseFsck: reached end of problem group: 860010-2368010000_20140417_14_exit_00000000390
ERROR: Found inconsistency in table day_hotstatic

解决方法:

     hbase hbck -fixHdfsOverlaps day_hotstatic

参考下文:

Region Overlap Repairs

Table integrity problems can require repairs that deal with overlaps. This is a riskier operation because it requires modifications to the file system, requires some decision making, and may require some manual steps. For these repairs it is best to analyze the output of a hbck -details run so that you isolate repairs attempts only upon problems the checks identify. Because this is riskier, there are safeguard that should be used to limit the scope of the repairs. WARNING: This is a relatively new and have only been tested on online but idle HBase instances (no reads/writes). Use at your own risk in an active production environment! The options for repairing table integrity violations include:

  • -fixHdfsOrphans option for “adopting” a region directory that is missing a region metadata file (the .regioninfo file).
  • -fixHdfsOverlaps ability for fixing overlapping regions

When repairing overlapping regions, a region’s data can be modified on the file system in two ways: 1) by merging regions into a larger region or 2) by sidelining regions by moving data to “sideline” directory where data could be restored later. Merging a large number of regions is technically correct but could result in an extremely large region that requires series of costly compactions and splitting operations. In these cases, it is probably better to sideline the regions that overlap with the most other regions (likely the largest ranges) so that merges can happen on a more reasonable scale. Since these sidelined regions are already laid out in HBase’s native directory and HFile format, they can be restored by using HBase’s bulk load mechanism. The default safeguard thresholds are conservative. These options let you override the default thresholds and to enable the large region sidelining feature.

  • -maxMerge <n> maximum number of overlapping regions to merge
  • -sidelineBigOverlaps if more than maxMerge regions are overlapping, sideline attempt to sideline the regions overlapping with the most other regions.
  • -maxOverlapsToSideline <n> if sidelining large overlapping regions, sideline at most n regions.

Since often times you would just want to get the tables repaired, you can use this option to turn on all repair options:

  • -repair includes all the region consistency options and only the hole repairing table integrity options.

Finally, there are safeguards to limit repairs to only specific tables. For example the following command would only attempt to check and repair table TableFoo and TableBar.

$ ./bin/hbase hbck -repair TableFoo TableBar

B.4.1. Special cases: Meta is not properly assigned

There are a few special cases that hbck can handle as well. Sometimes the meta table’s only region is inconsistently assigned or deployed. In this case there is a special -fixMetaOnly option that can try to fix meta assignments.

$ ./bin/hbase hbck -fixMetaOnly -fixAssignments

B.4.2. Special cases: HBase version file is missing

HBase’s data on the file system requires a version file in order to start. If this flie is missing, you can use the -fixVersionFile option to fabricating a new HBase version file. This assumes that the version of hbck you are running is the appropriate version for the HBase cluster.

B.4.3. Special case: Root and META are corrupt.

The most drastic corruption scenario is the case where the ROOT or META is corrupted and HBase will not start. In this case you can use the OfflineMetaRepair tool create new ROOT and META regions and tables. This tool assumes that HBase is offline. It then marches through the existing HBase home directory, loads as much information from region metadata files (.regioninfo files) as possible from the file system. If the region metadata has proper table integrity, it sidelines the original root and meta table directories, and builds new ones with pointers to the region directories and their data.

$ ./bin/hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair

NOTE: This tool is not as clever as uberhbck but can be used to bootstrap repairs that uberhbck can complete. If the tool succeeds you should be able to start hbase and run online repairs if necessary.

B.4.4. Special cases: Offline split parent

Once a region is split, the offline parent will be cleaned up automatically. Sometimes, daughter regions are split again before their parents are cleaned up. HBase can clean up parents in the right order. However, there could be some lingering offline split parents sometimes. They are in META, in HDFS, and not deployed. But HBase can't clean them up. In this case, you can use the -fixSplitParents option to reset them in META to be online and not split. Therefore, hbck can merge them with other regions if fixing overlapping regions option is used.

This option should not normally be used, and it is not in -fixAll.

时间: 2024-09-20 21:27:17

There is an overlap in the region chain的相关文章

There is an overlap in the region chain修复

ERROR: (region day_hotstatic,860010-2355010000_20140417_12_entry_00000000321,1400060700465.fda3b0aca340570aeb64410c97e3cb73.) Multiple regions have the same startkey: 860010-2355010000_20140417_12_entry_00000000321ERROR: (region day_hotstatic,860010-

(转)Markov Chain Monte Carlo

Nice R Code Punning code better since 2013 RSS Blog Archives Guides Modules About Markov Chain Monte Carlo 10 JUNE 2013 This topic doesn't have much to do with nicer code, but there is probably some overlap in interest. However, some of the topics th

android.graphics.Region.Op

android 的文档里不知道为什么没有写 android.graphics.Region.Op 这个枚举的详细内容 在网上搜到一篇文章,记下来备忘: //INTERSECT 取两者交集,默认的方式 //DIFFERENCE 第一次不同于第二次的部分显示出来 //REPLACE 显示第二次的 //REVERSE_DIFFERENCE 第二次不同于第一次的部分显示 //UNION 取全集 //XOR 补集,就是全集的减去交集的剩余部分显示

HBase中如何解决Region Server Compact过程占用大量网络出口带宽的问题

HBase 0.92版本之后,Region Server的Compact过程根据待合并的文件大小分为small compaction和large compaction两种,由此可能导致在集群写入量大的时候Compact占用过多的网络出口带宽.本文将详细描述集群使用过程中遇到这一问题的排查过程及其解决方法. 1. 发现问题 HBase集群(版本为0.94.0)运行过程中,发现5台Region Server的网络出口带宽经常维持在100MB/s以上,接近到网卡的极限:同时Region Server的

buffer cache实验5-latch:cache buffers chain

1.CBC latch产生的原理: 一次逻辑读时CBC latch锁及Buffer pin锁的获取和释放过程如下: 1.加Latch X 2.进入hash chain,在相应的BH上加Buffer pin S (0-->1) 3.释放Latch X 4.进行逻辑读--也就是通过BH中的buffer adderss找到数据块在内存中真实位置  ---假如读了1MS 5.加Latch X 6.释放Buffer pin S (1-->0)  0:没锁  1:共享锁  2:独占锁 7.释放Latch

UVA 442:Matrix Chain Multiplication 数据结构专题

题目链接:http://uva.onlinejudge.org/index.php?option=com_onlinejudge&Itemid=8&category=103&page=show_problem&problem=383 题目类型: 数据结构, 链表 样例输入: 9 A 50 10 B 10 20 C 20 5 D 30 35 E 35 15 F 15 5 G 5 10 H 10 20 I 20 25 A B C (AA) (AB) (AC) (A(BC)) (

设计模式学习笔记(十九)—Chain of Responsibility职责链模式

由于本人水平有限,写出来的东西也许对初学者有所帮助.如果不小心哪位大侠看了不要见笑,哪里有不正确的地方还请批评指正.好了不说废话了. Chain of Responsibility模式定义: 为了避免请求的发送者和接收者之间的耦合关系,使多个接受对象都有机会处理请求.将这些对象连成一条链,并沿着这条链传递该请求,直到有一个对象处理它为止. 我的理解: 在不止一个对象可以处理客户端请求的时候,为了使每个对象都有处理请求的机会,把这些对象顺序地串联起来形成一个链,每个被串联的对象都有一个指向下一个对

界面开发(三)--- 设置窗体的Region

经过上面两篇的界面开发,我们使用Windows的Hook技术,成功的将系统中的窗体的消息截 取,并添加了自己的事件处理程序,这篇文章就是通过我们对这些消息的处理的第一步,设置 窗体的样式和窗体的Region. Region对于窗体来说是一个非常重要的概念,他就是Windows分配给窗体显示的区域,所以 ,我们第一步就是根据我们的需要设计我们的窗体区域.而对于Windows窗体区域的设置, WndProc中的STYLECHANGED,SHOWWINDOW,SIZE,WINDOWPOSCHANGED

算法:ZOJ 3659 Conquer a New Region(并查集)

[题目大意] 有N个城市,N-1条路把这些城市连起来(刚好是一个树).相邻的两个城市有一个 运输容量C(i, j),而城市x到城市y的那条路的运输能力S取决与这条路上所经过的所有路中最小的那个容量 . 以那一个城市为中心,到其他N-1个城市的运输能力总和最大? [思路] 用神奇的并查集 ,把路按照权值从大到小排序,然后用类似Kruskal的方法不断的加入边. 对于要加入的一条路,这条路连 接这城市x和y,x所在的集合为A, y所在的集合为B, 可以确定A,B集合内的所有路都比当前这条路的权值 大