背景：
想停rac环境，之前10g都是用crs_stop -all 来停止，所以，这次也这样做的，没想到报了如下错误：
[grid@rac2 ~]$ crs_stop -all
CRS-2500: Cannot stop resource 'ora.gsd' as it is not running
CRS-2500: Cannot stop resource 'ora.oc4j' as it is not running
Attempting to stop `ora.ons` on member `rac2`
Attempting to stop `ora.LISTENER.lsnr` on member `rac2`
Attempting to stop `ora.eons` on member `rac2`
CRS-2789: Cannot stop resource 'ora.gsd' as it is not running on server 'rac2'
Attempting to stop `ora.eons` on member `rac1`
Attempting to stop `ora.LISTENER.lsnr` on member `rac1`
Attempting to stop `ora.DATA.dg` on member `rac2`
Attempting to stop `ora.FRA.dg` on member `rac2`
Attempting to stop `ora.OCR_VOTE.dg` on member `rac2`
Attempting to stop `ora.registry.acfs` on member `rac2`
Attempting to stop `ora.test.db` on member `rac2`
CRS-2789: Cannot stop resource 'ora.gsd' as it is not running on server 'rac1'
Attempting to stop `ora.DATA.dg` on member `rac1`
Attempting to stop `ora.FRA.dg` on member `rac1`
Attempting to stop `ora.OCR_VOTE.dg` on member `rac1`
Attempting to stop `ora.registry.acfs` on member `rac1`
Attempting to stop `ora.test.db` on member `rac1`
Stop of `ora.LISTENER.lsnr` on member `rac2` succeeded.
CRS-2789: Cannot stop resource 'ora.LISTENER.lsnr' as it is not running on server 'rac2'
Attempting to stop `ora.rac2.vip` on member `rac2`
Stop of `ora.rac2.vip` on member `rac2` succeeded.
Stop of `ora.ons` on member `rac2` succeeded.
CRS-2789: Cannot stop resource 'ora.ons' as it is not running on server 'rac2'
Attempting to stop `ora.net1.network` on member `rac2`
Stop of `ora.net1.network` on member `rac2` succeeded.
Stop of `ora.LISTENER.lsnr` on member `rac1` succeeded.
Attempting to stop `ora.ons` on member `rac1`
Attempting to stop `ora.rac1.vip` on member `rac1`
Attempting to stop `ora.LISTENER_SCAN1.lsnr` on member `rac1`
Stop of `ora.registry.acfs` on member `rac2` succeeded.
Stop of `ora.rac1.vip` on member `rac1` succeeded.
Stop of `ora.LISTENER_SCAN1.lsnr` on member `rac1` succeeded.
Attempting to stop `ora.scan1.vip` on member `rac1`
Stop of `ora.scan1.vip` on member `rac1` succeeded.
Stop of `ora.ons` on member `rac1` succeeded.
CRS-2789: Cannot stop resource 'ora.ons' as it is not running on server 'rac1'
Attempting to stop `ora.net1.network` on member `rac1`
Stop of `ora.net1.network` on member `rac1` succeeded.
Stop of `ora.registry.acfs` on member `rac1` succeeded.
Stop of `ora.eons` on member `rac2` succeeded.
Stop of `ora.OCR_VOTE.dg` on member `rac1` succeeded.
Stop of `ora.eons` on member `rac1` succeeded.
Stop of `ora.test.db` on member `rac2` succeeded.
Stop of `ora.OCR_VOTE.dg` on member `rac2` succeeded.
Stop of `ora.test.db` on member `rac1` succeeded.
CRS-0216: Could not stop resource 'ora.LISTENER.lsnr rac2 1'.
CRS-0216: Could not stop resource 'ora.asm rac1 1'.
CRS-0216: Could not stop resource 'ora.asm rac2 1'.
CRS-0216: Could not stop resource 'ora.gsd'.
CRS-0216: Could not stop resource 'ora.gsd rac1 1'.
CRS-0216: Could not stop resource 'ora.gsd rac2 1'.
CRS-0216: Could not stop resource 'ora.oc4j'.
CRS-0216: Could not stop resource 'ora.ons rac1 1'.
CRS-0216: Could not stop resource 'ora.ons rac2 1'.
以上这些进程都没有被关闭，现在看下就跟下面这样了：
[grid@rac2 bin]$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.DATA.dg ora....up.type OFFLINE OFFLINE
ora.FRA.dg ora....up.type OFFLINE OFFLINE
ora....ER.lsnr ora....er.type OFFLINE OFFLINE
ora....N1.lsnr ora....er.type OFFLINE OFFLINE
ora....VOTE.dg ora....up.type OFFLINE OFFLINE
ora.asm ora.asm.type OFFLINE ONLINE rac2
ora.eons ora.eons.type OFFLINE OFFLINE
ora.gsd ora.gsd.type OFFLINE OFFLINE
ora....network ora....rk.type OFFLINE OFFLINE
ora.oc4j ora.oc4j.type OFFLINE OFFLINE
ora.ons ora.ons.type OFFLINE OFFLINE
ora....SM1.asm application OFFLINE ONLINE rac1
ora....C1.lsnr application OFFLINE OFFLINE
ora.rac1.gsd application OFFLINE OFFLINE
ora.rac1.ons application OFFLINE OFFLINE
ora.rac1.vip ora....t1.type OFFLINE OFFLINE
ora....SM2.asm application OFFLINE ONLINE rac2
ora....C2.lsnr application OFFLINE OFFLINE
ora.rac2.gsd application OFFLINE OFFLINE
ora.rac2.ons application OFFLINE OFFLINE
ora.rac2.vip ora....t1.type OFFLINE OFFLINE
ora....ry.acfs ora....fs.type OFFLINE OFFLINE
ora.scan1.vip ora....ip.type OFFLINE OFFLINE
ora.test.db ora....se.type OFFLINE OFFLINE
然后再尝试手工关闭asm：
[grid@rac2 bin]$ srvctl stop asm -n rac1
PRCR-1014 : Failed to stop resource ora.asm
PRCR-1065 : Failed to stop resource ora.asm
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
ORA-15097: cannot SHUTDOWN ASM instance with connected client
CRS-2675: Stop of 'ora.asm' on 'rac1' failed
CRS-2675: Stop of 'ora.asm' on 'rac1' failed
原因及解决方案：
原来在Oracle 10g 环境下，我们停RAC的顺序是：
（1）数据库
（2）ASM
（3）CRS
但是在Oracle 11gR2下，如果是用OUI来安装的话，除了数据文件之外，OCR 也是放在ASM里的。 所以这里就遇到了问题。 OCR放在ASM里，这样CRSD 就会成为ASM的客户端。 如果我们像Oracle 10g里那样直接停ASM的时候，就会报上面的错误。 所以，在11gR2下面，要停ASM实例的话，只能和CRS一起停。
用以下方法停asm：在root用户下执行：crsctl stop crs
问题就解决了。。。
注意：不要直接kill 掉ASM 进程，或者用shutdown abort 来关闭ASM 实例。 这样CRS 也会被挂掉。
所以，我们之前10g rac停止的步骤，到了srvctl stop asm -n rac1 这一行，就没法执行了。。
srvctl stop instance -d test -i test1
srvctl stop instance -d test -i test2
srvctl stop asm -n rac1
srvctl stop asm -n rac2
srvctl stop nodeapps -n rac1
srvctl stop nodeapps -n rac2
所以，一般11g rac的启动和关闭步骤，要按照下面的顺序进行：
一 关闭rac
1，确认srvctl 和ps -ef|grep smon
[grid@rac1 ~]$ srvctl status database -d RACDB
实例 RACDB1 正在节点 rac1 上运行
实例 RACDB2 正在节点 rac2 上运行
[grid@rac1 ~]$ ps -ef|grep smon
oracle 3676 1 0 06:05 ? 00:00:02 ora_smon_RACDB1
grid 12840 1 0 01:54 ? 00:00:00 asm_smon_+ASM1
grid 27890 27621 0 07:52 pts/3 00:00:00 grep smon
2,将数据库关闭并再次确认
[grid@rac1 ~]$ srvctl stop database -d RACDB
[grid@rac1 ~]$ ps -ef|grep smon
3,使用root 帐号关闭ASM
[grid@rac1 ~]$ su -
口令：
[root@rac1 ~]# cd /home/grid
[root@rac1 grid]# sh .bash_profile
4，使用crs_stat 确认集群各项资源和服务运行状态
[root@rac1 bin]# /u01/app/11.2.0/grid/bin/crs_stat -t -v
5，使用crsctl 指令关闭集群
[root@rac1 bin]# /u01/app/11.2.0/grid/bin/crsctl stop cluster -all
6，使用crs_stat 确认集群各项资源和服务运行状态
[root@rac1 bin]# /u01/app/11.2.0/grid/bin/crs_stat -t -v
[root@rac2 ~]# /u01/app/11.2.0/grid/bin/crs_stat -t -v
CRS-0184: Cannot communicate with the CRS daemon.
说明顺利关闭
二 。RAC 开启
1，root 执行grid 下面的环境变量 (可以不执行直接到/u01/app/11.2.0/grid/bin/模式下)
2，[root@rac1 ~]# /u01/app/11.2.0/grid/bin/crsctl stop cluster -all
3，开启集群
[root@rac1 ~]# /u01/app/11.2.0/grid/bin/crsctl start cluster -all
检查状态
[root@rac2 ~]# /u01/app/11.2.0/grid/bin/crs_stat -t -v
4，使用srvcel 确认数据库状态
[root@rac1 ~]# /u01/app/11.2.0/grid/bin/srvctl status database -d RACDB
实例 RACDB1 没有在 rac1 节点上运行
实例 RACDB2 没有在 rac2 节点上运行
5，打开RAC
[root@rac1 ~]# /u01/app/11.2.0/grid/bin/srvctl start database -d RACDB
确认状态
[root@rac2 ~]# /u01/app/11.2.0/grid/bin/srvctl status database -d RACDB
实例 RACDB1 正在节点 rac1 上运行
实例 RACDB2 正在节点 rac2 上运行
6,打开OEM
[root@rac1 ~]# /u01/app/11.2.0/grid/bin/emctl start RACDB