Linux问题情报分享（3）：CentOS 7上最新kernel-debuginfo包与当前内核版本不匹配

CentOS 7上最新的kernel-debuginfo包，是kernel-debuginfo-4.x.x-x.el7，而当前内核是kernel-3.10.0。因此，如果安装或者升级到了最新的kernel-debuginfo包，会导致类似SystemTap这样需要内核头文件和调试符号的工具执行出错。这是CentOS 7的bug。

以SystemTap为例，其报错大致如下

[root@pusf ~]# rpm -qa | grep kernel-debuginfo
kernel-debuginfo-4.9.31-203.el7.centos.x86_64
kernel-debuginfo-common-x86_64-4.9.31-203.el7.centos.x86_64[root@pusf ~]
[root@pusf ~]# stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}'
Pass 1: parsed user script and 119 library scripts using 117224virt/33688res/3120shr/30824data kb, in 240usr/10sys/292real ms.
semantic error: while resolving probe point: identifier 'kernel' at /usr/share/systemtap/tapset/linux/vfs.stp:882:18
        source: probe vfs.read = kernel.function("vfs_read")
                      ^
semantic error: missing x86_64 kernel/module debuginfo [man warning::debuginfo] under '/lib/modules/3.10.0-514.26.2.el7.x86_64/build'
semantic error: while resolving probe point: identifier 'vfs' at <input>:1:7
        source: probe vfs.read {printf("read performed\n"); exit()}
                      ^
semantic error: no match

Pass 2: analyzed script: 0 probes, 0 functions, 0 embeds, 0 globals using 120568virt/37092res/5108shr/32068data kb, in 100usr/140sys/443real ms.
Missing separate debuginfos, use: debuginfo-install kernel-3.10.0-514.26.2.el7.x86_64
Pass 2: analysis failed.  [man error::pass2]
[root@pusf ~]#

因此，出现这种情况时，需要卸载kernel-debuginfo-4.x.x-x.el7和kernel-debuginfo-common-4.x.x-x.el7的包，重新按照当前内核版本安装kernel-debuginfo即可

rpm -qa | grep -E '^kernel-' | grep -v 3.10.0 | xargs yum -y remove
debuginfo-install -y kernel-$(uname -r)

再测试下SystemTap的脚本，会发现问题已经解决了

[root@pusf ~]# rpm -qa | grep kernel-debuginfo
kernel-debuginfo-common-x86_64-3.10.0-514.26.2.el7.x86_64
kernel-debuginfo-3.10.0-514.26.2.el7.x86_64
[root@pusf ~]# stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}'

Pass 1: parsed user script and 119 library scripts using 117224virt/33684res/3120shr/30824data kb, in 230usr/10sys/293real ms.

Pass 2: analyzed script: 1 probe, 1 function, 4 embeds, 0 globals using 248216virt/165932res/4336shr/161816data kb, in 1260usr/370sys/1809real ms.

Pass 3: translated to C into "/tmp/stapI5iwL4/stap_1aa47f863c3f13e51da3e80cc92942be_1682_src.c" using 248216virt/166236res/4640shr/161816data kb, in 20usr/40sys/57real ms. 

Pass 4: compiled C into "stap_1aa47f863c3f13e51da3e80cc92942be_1682.ko" in 5550usr/1240sys/7187real ms.

Pass 5: starting run.
read performed
Pass 5: run completed in 0usr/60sys/378real ms.
[root@pusf ~]#

在CentOS官方修正bug前，可以在/etc/yum.conf中加入如下配置，先排除问题包

exclude=kernel-debuginfo*

这样，升级时不会再次安装了问题包。当然，内核升级时，需要额外调整下配置。

顺便说下，在Ubuntu上使用SystemTap，需要额外配置和步骤，请参考SystemTap on Ubuntu。

参考

1. SystemTap
2. SystemTap on CentOS
3. SystemTap on Ubuntu
4. Where to find the kernel-debuginfo package