Thursday 5 June 2014

Space not recovered after house keeping.... !!!! the open file alert log from oracle database in solaris How to recover space from open files like alert log,. ...from the file system DO NOT MOVE. or Delete.. recovering the space without loosing any contents which is getting updated as well...follow the step

OS : Solaris
Oracle Database
Recover space after housekeeping
===============================

How to recover space from open files like alert log,. 

Have you ever experienced after delete or move a file in Solaris, 

the mount point space was not recovered....unless you stop the process(here we have to bounce the DB)

In case if you ever experienced such issue you may consider following step which will recover the space.

This happens because due open files. ( you may google about this)

consider a scenario where we can do only do alert log file move/delete 

( Alert log is configured with logical standby hence alertlog grows 1gb in 3 days) and have 5 database running

Below Mount point is filled with different files, but will consider housekeeping on alertlog file 

=======================================
Note :
please use the commands carefully before executing in your server its advised to test in test server.

Note : 
Implementing or trying any of the advice from this blog is at your own risk.
=======================================

Open 2 putty session for this
1st Putty session Original Putty to execute all the commands
2nd Putty Session Duplicate to tail the alert log ( Make sure you enable huge number of scroll in putty to have the entire history output just visible)
OS : Solaris

Step 1  : Check the space available in Oracle Home Mount point
========

solaris_test1:testdb1:/ora/admin/testdb1/bdump>df -h .
Filesystem             size   used  avail capacity  Mounted on
/dev/md/dsk/d100       133G   128G   3.2G    98%    /ora
solaris_test1:testdb1:/ora/admin/testdb1/bdump>

Step 2  : Size of the alert log
========

solaris_test1::/ora/admin/testdb1/bdump>du -sh alert_testdb1.log
 3.3G   alert_testdb1.log
solaris_test1::/ora/admin/testdb1/bdump>

Step 3  : other available mount point in case if space is not available in same mount point.
========

solaris_test1::/ora/admin/testdb1/bdump>df -h /ora/db052/temp
Filesystem             size   used  avail capacity  Mounted on
10.97.74.113:/vol/adeis_ps_vol10/ora_db052_temp
                       561G   151G   410G    27%    /ora/db052/temp
solaris_test1::/ora/admin/testdb1/bdump>

Step 4  : Tail the alert log file and make sure you are able to see the db events getting updated.
========

Open another putty session( duplicate of same server) , do below to trap all the alert log events in your putty window.( Make sure huge scroll in putty is enabled so that enire history is getting saved)

solaris_test1::/ora/admin/testdb1/bdump>
solaris_test1::/ora/admin/testdb1/bdump>tail -10f alert_testdb1.log


Step 5  : copy the alert log to another location where you have enough space. !!!! .....DO NOT MOVE. if you move the space will not recover.....!!!!!!

( Run in it background with the help of "&"
========

solaris_test1:testdb1:/ora/admin/testdb1/bdump>cp alert_testdb1.log /ora/db052/temp/alert_testdb1.log_june_5_2014 &
[1] 6078
solaris_test1:testdb1:/ora/admin/testdb1/bdump>

Step 6  : Monitor the background job by command "jobs"
========

solaris_test1:testdb1:/ora/admin/testdb1/bdump>jobs
[1]+  Running                 cp alert_testdb1.log /ora/db052/temp/alert_testdb1.log_june_5_2014 &
solaris_test1:testdb1:/ora/admin/testdb1/bdump>

Step 7  : To see how much copied compare source and destination file size
========

solaris_test1:testdb1:/ora/admin/testdb1/bdump>du -sh alert_testdb1.log
 3.3G   alert_testdb1.log
solaris_test1:testdb1:/ora/admin/testdb1/bdump>

solaris_test1:testdb1:/ora/admin/testdb1/bdump>du -sh /ora/db052/temp/alert_testdb1.log_june_5_2014
 717M   /ora/db052/temp/alert_testdb1.log_june_5_2014


Step 8 : Once the entire files copied then jobs output will not come. and also the size of source and copied will be same(Approximately)
========
solaris_test1:testdb1:/ora/admin/testdb1/bdump>jobs
solaris_test1:testdb1:/ora/admin/testdb1/bdump>

solaris_test1:testdb1:/ora/admin/testdb1/bdump>du -sh *
 3.3G   alert_testdb1.log
  57M   alert_testdb1.log_april_29_2014.gz
   1K   testdb1_arc0_3593.trc
  21K   testdb1_lgwr_1688.trc
   0K   nohup.out
   0K   v1
solaris_test1:testdb1:/ora/admin/testdb1/bdump>du -sh /ora/db052/temp/alert_testdb1.log_june_5_2014
 3.3G   /ora/db052/temp/alert_testdb1.log_june_5_2014

Step 9  : Purge the original file with appending with 0 byte file
========


solaris_test1:testdb1:/ora/admin/testdb1/bdump>touch test1
solaris_test1:testdb1:/ora/admin/testdb1/bdump>cat test1 > alert_testdb1.log
solaris_test1:testdb1:/ora/admin/testdb1/bdump>rm test1

solaris_test1::/ora/product/software>df -h /ora
Filesystem             size   used  avail capacity  Mounted on
/dev/md/dsk/d100       133G   126G   5.5G    96%    /ora

solaris_test1:testdb1:/ora/admin/testdb1/bdump>sync
solaris_test1:testdb1:/ora/admin/testdb1/bdump>

Step 10  : from Duplicatate or another putty window which you capture in Step 4 .. Copy and paste in the alert log file to fill the gap( some events may repeat but we are almost not losing any date
========


copy and paste the alert log output which you capturing from tail -10f output

Step 11  : zip the file
========

solaris_test1:testdb1:/ora/admin/testdb1/bdump>gzip /ora/db052/temp/alert_testdb1.log_june_5_2014 &
[1] 3341
solaris_test1:testdb1:/ora/admin/testdb1/bdump>
solaris_test1:testdb1:/ora/admin/testdb1/bdump>jobs
[1]+  Running                 gzip /ora/db052/temp/alert_testdb1.log_june_5_2014 &
solaris_test1:testdb1:/ora/admin/testdb1/bdump>


Step 12  : move it back to original location or create a soft link. 
========


solaris_test1:testdb1:/ora/admin/testdb1/bdump>du -sh /ora/db052/temp/alert_testdb1.log_june_5_2014.gz
  15M   /ora/db052/temp/alert_testdb1.log_june_5_2014.gz
solaris_test1:testdb1:/ora/admin/testdb1/bdump>mv /ora/db052/temp/alert_testdb1.log_june_5_2014.gz /ora/admin/testdb1/bdump/
solaris_test1:testdb1:/ora/admin/testdb1/bdump>