这是一套12c R2 4-nodes Oracle RAC on RHEL 7的环境，已安装0417 RU。 该库有一套Phyical DataGard， 同时也是GoldenGate的target端，存在一个replicat 进程同步数据。 一日收到该数据库归档空间(in ASM) DiskgrouP 使用率告警，后分析刚上线没多久就趟了一个雷。这里简单记录过程。
-- DB alert log
2019-04-12 17:31:43.791000 +08:00
krsd_check_stuck_arch: stuck archiver condition cleared
Unable to create archive log file '+ARCHDG'
Errors in file /oracle/app/oracle/diag/rdbms/anbob/anbob2/trace/anbob2_arc1_64728.trc:
ORA-19504: failed to create file "+ARCHDG"
ORA-17502: ksfdcre:4 Failed to create file +ARCHDG
ORA-15041: diskgroup "ARCHDG" space exhausted
ARC1: Error 19504 Creating archive log file to '+ARCHDG'
krsd_check_stuck_arch: stuck archiver: insufficient local LADs
krsd_check_stuck_arch: stuck archiver condition declared
-- 手动清理日志
RMAN> list archive log all;
...
RMAN> delete archivelog until time "sysdate-1";
released channel: ORA_DISK_1
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=7428 instance=anbob2 device type=DISK
RMAN-08137: warning: archived log not deleted, needed for standby or upstream capture process
archived log file name=+ARCHDG/anbob/ARCHIVELOG/2019_04_10/thread_1_seq_1887.326.1005219643 thread=1 sequence=1887
RMAN-08137: warning: archived log not deleted, needed for standby or upstream capture process
archived log file name=+ARCHDG/anbob/ARCHIVELOG/2019_04_10/thread_1_seq_1888.277.1005226879 thread=1 sequence=1888
RMAN-08137: warning: archived log not deleted, needed for standby or upstream capture process
archived log file name=+ARCHDG/anbob/ARCHIVELOG/2019_04_10/thread_1_seq_1889.324.1005230519 thread=1 sequence=1889
RMAN-08137: warning: archived log not deleted, needed for standby or upstream capture process
archived log file name=+ARCHDG/anbob/ARCHIVELOG/2019_04_10/thread_1_seq_1890.302.1005232111 thread=1 sequence=1890
RMAN-08137: warning: archived log not deleted, needed for standby or upstream capture process
archived log file name=+ARCHDG/anbob/ARCHIVELOG/2019_04_10/thread_1_seq_1891.269.1005233425 thread=1 sequence=1891
RMAN-08137: warning: archived log not deleted, needed for standby or upstream capture process
archived log file name=+ARCHDG/anbob/ARCHIVELOG/2019_04_10/thread_1_seq_1892.259.1005234787 thread=1 sequence=1892
RMAN-08137: warning: archived log not deleted, needed for standby or upstream capture process
archived log file name=+ARCHDG/anbob/ARCHIVELOG/2019_04_10/thread_1_seq_1893.332.1005237411 thread=1 sequence=1893
RMAN-08137: warning: archived log not deleted, needed for standby or upstream capture process
archived log file name=+ARCHDG/anbob/ARCHIVELOG/2019_04_10/thread_1_seq_1894.341.1005240287 thread=1 sequence=1894
TIP:
当然该库有DG，目录的日志还未同步应用，虽然强制(with force option)删除影响了DG环境，后期还要补救过于麻烦，当然主库恢复业务优先， 确认了一下ASM DISKGROUP可用空间和归档日志日生成量，决定先临时修改归档路径到DATA所在的ASM DISKGROUP。
SQL> @asmdg
GROUP_NUMBER NAME SECTOR_SIZE LOGICAL_SECTOR_SIZE BLOCK_SIZE ALLOCATION_UNIT_SIZE STATE TYPE TOTAL_MB FREE_MB HOT_USED_MB COLD_USED_MB REQUIRED_MIRROR_FREE_MB USABLE_FILE_MB OFFLINE_DISKS COMPATIBILITY DATABASE_COMPATIBILITY V CON_ID
------------ ------------------------------ ----------- ------------------- ---------- -------------------- ----------- ------ ---------- ---------- ----------- ------------ ----------------------- -------------- ------------- ------------------------------------------------------------ ------------------------------------------------------------ - ----------
1 ARCHDG 512 512 4096 4194304 CONNECTED EXTERN 1048576 384 0 1048192 0 384 0 12.2.0.1.0 10.1.0.0.0 N 0
2 DATADG 512 512 4096 4194304 CONNECTED EXTERN 9437184 1141244 0 8295940
