Comandos basicos del Trace File Analyzer

Hoy vamos a ver los comandos y acciones basicas del tafcl
El tafcl es el comando de ejecucion del Trace File Analyzer (TFA), este pequeño script esta incluido dentro de la suite Autonomous Health Framework (AHF) , y sera o primero que nos solicite Oracle support una vez abramos un SR

Instalacion

Nos descargaremos el paquete de la AHF de la nota de soporte Autonomous Health Framework (AHF) – Including TFA and ORAchk/EXAchk (Doc ID 2550798.1 Y lo ejecutaremos como superusuario el comando ahf_setup.
Este comando tiene varias opciones

   Usage for ./ahf_setup

   ./ahf_setup [[-ahf_loc ][-data_dir ][-nodes ][-extract[orachk|exachk|-notfasetup] [-local][-silent][-tmp_loc ][-force][-debug [-level <1-6>]]]

        -ahf_loc          -    Install into the directory supplied. (Default /opt/oracle.ahf)
        -data_dir         -    AHF Data Directory where all the collections, metadata, etc. will be stored
        -nodes            -    Comma separated Remote Node List
        -extract          -    Extract only files from Installer. (Default for non-root users)
        -nosymlink        -    Do not create symlinks on Exadata DOM0. (Default for non-root users)
        -notfasetup       -    Do not Configure TFA when used with -extract
        -local            -    Only install on the local node
        -silent           -    Do not ask any install questions
        -tmp_loc          -    Temporary location directory for AHF to extract the install archive to (must exist) (Default /tmp)
        -perlhome         -    Custom location of perl binaries
        -force            -    Force AHF Install
        -debug            -    Debug AHF Install Script
        -level            -    AHF Instal Debug Level 1-6 (Default 4 with option -debug)
                               [FATAL - 1, ERROR - 2, WARNING - 3, INFO - 4, DEBUG - 5, TRACE - 6]

Si la instalacionn se hizo de manera estandard, el mismo TFA detectara la instalacion anterior y se actualizara.
Veamos un ejemplo de actualizacion.

[oracle@rac1 tmp]$ sudo ./ahf_setup 

AHF Installer for Platform Linux Architecture x86_64

AHF Installation Log : /tmp/ahf_install_211000_21804_2021_04_01-09_22_26.log

Starting Autonomous Health Framework (AHF) Installation

AHF Version: 21.1.0 Build Date: 202104131158

AHF is already installed at /opt/oracle.ahf

Installed AHF Version: 20.2.3 Build Date: 202010132201

Do you want to upgrade AHF [Y]|N : Y

Upgrading /opt/oracle.ahf

Shutting down AHF Services
Shutting down TFA
Removed symlink /etc/systemd/system/multi-user.target.wants/oracle-tfa.service.
Removed symlink /etc/systemd/system/graphical.target.wants/oracle-tfa.service.
. . . . .
. . .
Successfully shutdown TFA..


Starting AHF Services
Starting TFA..
Created symlink from /etc/systemd/system/multi-user.target.wants/oracle-tfa.service to /etc/systemd/system/oracle-tfa.service.
Created symlink from /etc/systemd/system/graphical.target.wants/oracle-tfa.service to /etc/systemd/system/oracle-tfa.service.
Waiting up to 100 seconds for TFA to be started..
. . . . .
. . . . .
. . . . .
Successfully started TFA Process..
. . . . .
TFA Started and listening for commands
No new directories were added to TFA


INFO: Starting orachk scheduler in background. Details for the process can be found at /opt/app/oracle/oracle.ahf/data/rac1/diag/orachk/compliance_start_230421_093124.log


AHF is successfully upgraded to latest version

.------------------------------------------------------------------.
| Host       | TFA Version | TFA Build ID         | Upgrade Status |
+------------+-------------+----------------------+----------------+
| rac1 |  21.1.0.0.0 | 21100020210413115827 | UPGRADED       |
| rac2 |  20.2.3.0.0 | 20230020201013220107 | NOT UPGRADED   |
'------------+-------------+----------------------+----------------'

Moving /tmp/ahf_install_211000_21804_2021_04_01-09_22_26.log to /opt/app/oracle/oracle.ahf/data/rac1/diag/ahf/

Usos comunes

Obtener informacion de las trazas en un determninado tiempo

Si quremos recabar informacion del sistema entre unas determinadas fechas usaremos los fags -from  -to
./tfactl diagcollect -from "2021-04-07 08:00:00" -to "2021-04-07 10:00:00"

Obtener informacion ded un determinado error

Si quremos obtener informacion del sistema sobre un determinado error, usaremos el flag srdc

./tfactl diagcollect -srdc ORA-07445

Gestion del repositorio

Ver la configuration

 
tfactl> print config
.---------------------------------------------------------------------------------------------------------------------.
|                                                      wiracdev02                                                     |
+--------------------------------------------------------------------------------------------------------+------------+
| Configuration Parameter                                                                                | Value      |
+--------------------------------------------------------------------------------------------------------+------------+
| TFA Version ( tfaversion )                                                                             | 21.1.0.0.0 |
| Java Version ( javaVersion )                                                                           | 1.8        |
| Public IP Network ( publicIp )                                                                         | true       |
| Repository current size (MB) ( currentsizemegabytes )                                                  | 293        |
| Repository maximum size (MB) ( maxsizemegabytes )                                                      | 10240      |
| Cluster Event Monitor ( clustereventmonitor )                                                          | ON         |
| scandiskmon                                                                                            | OFF        |
| scanacfslog                                                                                            | OFF        |
| File Data Collection ( inventory )                                                                     | ON         |
| Automatic Purging ( autoPurge )                                                                        | ON         |
| Internal Search String ( internalSearchString )                                                        | ON         |
| ISA Data Gathering ( collection.isa )                                                                  | ON         |
| Trim Files ( trimfiles )                                                                               | ON         |
| collectTrm                                                                                             | OFF        |
| chmdataapi                                                                                             | ON         |
| chanotification ( chanotification )                                                                    | ON         |
| Managelogs Auto Purge ( manageLogsAutoPurge )                                                          | OFF        |
| scanacfseventlog                                                                                       | OFF        |
| Alert Log Scan ( rtscan )                                                                              | ON         |
| debugips                                                                                               | OFF        |
| generateZipMetadataJson                                                                                | ON         |
| collectAllDirsByFile                                                                                   | ON         |
| scanvarlog                                                                                             | OFF        |
| Auto Diagcollection ( autodiagcollect )                                                                | ON         |
| Public IP Network ( publicIp )                                                                         | ON         |
| Flood Control ( floodcontrol )                                                                         | ON         |
| Generation of Mini Collections ( minicollection )                                                      | ON         |
| odscan                                                                                                 | ON         |
| Disk Usage Monitor ( diskUsageMon )                                                                    | ON         |
| Discovery ( discovery )                                                                                | ON         |
| analyze                                                                                                | OFF        |
| indexInventory                                                                                         | ON         |
| Generation of Telemetry Data ( telemetry )                                                             | OFF        |
| chaautocollect                                                                                         | ON         |
| Granular Tracing ( granulartracing )                                                                   | OFF        |
| minPossibleSpaceForPurge                                                                               | 1024       |
| disk.threshold                                                                                         | 90         |
| mem.swapfree                                                                                           | 5120       |
| mem.util.samples                                                                                       | 4          |
| inventoryThreadPoolSize                                                                                | 1          |
| mem.swaptotal.samples                                                                                  | 2          |
| maxFileAgeToPurge                                                                                      | 1440       |
| mem.free                                                                                               | 20480      |
| actionrestartlimit                                                                                     | 30         |
| Minimum Free Space to enable Alert Log Scan (MB) ( minSpaceForRTScan )                                 | 500        |
| cpu.io.samples                                                                                         | 30         |
| mem.util                                                                                               | 80         |
| Maximum single Zip File Size (MB) ( maxZipSize )                                                       | 2048       |
| Time interval between consecutive Disk Usage Snapshot(minutes) ( diskUsageMonInterval )                | 60         |
| TFA ISA Purge Thread Delay (minutes) ( tfaDbUtlPurgeThreadDelay )                                      | 60         |
| firstDiscovery                                                                                         | 0          |
| TFA IPS Pool Size ( tfaIpsPoolSize )                                                                   | 5          |
| Maximum File Collection Size (MB) ( maxFileCollectionSize )                                            | 5120       |
| Time interval between consecutive Managelogs Auto Purge(minutes) ( manageLogsAutoPurgeInterval )       | 60         |
| arc.backupmissing.samples                                                                              | 2          |
| cpu.util.samples                                                                                       | 2          |
| cpu.usr.samples                                                                                        | 2          |
| cpu.sys                                                                                                | 50         |
| Flood Control Limit Count ( fc.limit )                                                                 | 3          |
| Flood Control Pause Time (minutes) ( fc.pauseTime )                                                    | 120        |
| Maximum Number of TFA Logs ( maxLogCount )                                                             | 10         |
| DB Backup Delay Hours ( dbbackupdelayhours )                                                           | 27         |
| cdb.backup.samples                                                                                     | 1          |
| arc.backupstatus                                                                                       | 1          |
| purgeFrequency                                                                                         | 4          |
| TFA ISA Purge Age (seconds) ( tfaDbUtlPurgeAge )                                                       | 2592000    |
| Maximum Collection Size of Core Files (MB) ( maxCoreCollectionSize )                                   | 200        |
| cpu.util                                                                                               | 80         |
| mem.swapfree.samples                                                                                   | 2          |
| cdb.backupstatus                                                                                       | 1          |
| mem.swaputl.samples                                                                                    | 2          |
| arc.backup.samples                                                                                     | 3          |
| unreachablenodeTimeOut                                                                                 | 3600       |
| Flood Control Limit Time (minutes) ( fc.limitTime )                                                    | 60         |
| mem.swaputl                                                                                            | 10         |
| mem.free.samples                                                                                       | 2          |
| Maximum Size of Core File (MB) ( maxCoreFileSize )                                                     | 20         |
| disk.samples                                                                                           | 2          |
| cpu.sys.samples                                                                                        | 30         |
| cpu.usr                                                                                                | 98         |
| arc.backupmissing                                                                                      | 1          |
| cpu.io                                                                                                 | 20         |
| Archive Backup Delay Minutes ( archbackupdelaymins )                                                   | 40         |
| inventoryPurgeThreadInterval                                                                           | 720        |
| Age of Purging Collections (Hours) ( minFileAgeToPurge )                                               | 12         |
| cpu.idle.samples                                                                                       | 2          |
| unreachablenodeSleepTime                                                                               | 300        |
| cpu.idle                                                                                               | 20         |
| mem.swaptotal                                                                                          | 24         |
| TFA ISA CRS Profile Delay (minutes) ( tfaDbUtlCrsProfileDelay )                                        | 720        |
| cdb.backupmissing                                                                                      | 1          |
| cdb.backupmissing.samples                                                                              | 2          |
| Trim Size ( trimsize )                                                                                 | 500000     |
| Maximum Size of TFA Log (MB) ( maxLogSize )                                                            | 52428800   |
| minTimeForAutoDiagCollection                                                                           | 300        |
| skipScanThreshold                                                                                      | 100        |
| fileCountInventorySwitch                                                                               | 5000       |
| TFA ISA Purge Mode ( tfaDbUtlPurgeMode )                                                               | profile    |
| country                                                                                                | US         |
| Debug Mask (Hex) ( debugmask )                                                                         | 0x000000   |
| Setting for ACR redaction (none|SANITIZE|MASK) ( redact )                                              | none       |
| language                                                                                               | en         |
| AlertLogLevel                                                                                          | ALL        |
| BaseLogPath                                                                                            | ERROR      |
| encoding                                                                                               | UTF-8      |
| UserLogLevel                                                                                           | ALL        |
| Logs older than the time period will be auto purged(days[d]|hours[h]) ( manageLogsAutoPurgePolicyAge ) | 30d        |
| isaMode                                                                                                | enabled    |
'--------------------------------------------------------------------------------------------------------+------------'


cambio de ubicacion

Supongamos que queremos mover la ubicacion a un directorio compartido.
La ultima rama del repositorio debe de llamarse repository

tfactl set repositorydir=/mnt/tfa/repository 

Desinstalacion

Para su desinstalacion simplemente tenemos que ejecutarlo con el flag uninstall

[root@rac1 ~]$ which tfactl
/opt/app/12.1/grid/bin/tfactl
[root@rac1 ~]$  /opt/app/12.1/grid/bin/tfactl uninstall

Como siempre, mas infrmacion en oracle support:

Error PRKO-2207 : Warning:–spfile option has been deprecated and will be ignored

Hoy me he encontrado con un eroor curioso.
Cuando he intentado modificar la ubicacionn del spfile con el comando

srvctl modify asm -spfile +DATA/ASM/ASMPARAMETERFILE/initasm.ora

El sistema me ha devuelto el error
PRKO-2207 : Warning:-spfile option has been deprecated and will be ignored

Cual ha sido mi sorpresa al ver que la modificacion del spfile o el listener en el asm con el comando srvctl esta deprecated.
Segun lanota de soporte «PRKO-2207 : Warning:-spfile option has been deprecated and will be ignored» for ASM instance (Doc ID 2227045.1) la manera de llevarlo a cabo ahora es mediante os comandos de oracle, estoson:

Eliminar un nodo del rac

Hoy vamos a ver la manera de eliminar de manera limpia un nodo de un RAC .
Supondremos que queremos eliminar de nuestro cluster el nododo que llamamos rac1, los pasos a llevar a cabo seran:

Eliminar del nodo las bases de datos corriendo

Supongamos que tenemos la BBDD

ORACLE_BASE=/u01/app/oracle
ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/db
ORACLE_SID=TEST1
DB_NAME=TEST
INSTANCE_NAME=TEST1
NODE_NAME=rac1

Para cada una de las bases de datos , la parariamos con :

 srvctl stop instance  -db $DB_NAME -i $INSTANCE_NAME

Y posteriormente la eliminariamos con:

rac1.pamplona.name:oracle (TEST:/u01/app/19c/grid) srvctl config database -db $DB_NAME
Database unique name: TEST
Database name:
Oracle home: /u01/app/oracle/product/12.1.0.2/db
Oracle user: oracle
Spfile: +DATA/TEST/PARAMETERFILE/spfile.357.935866381
Password file:
Domain:
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools:
Disk Groups: +DATA,+REDO,+FRA
Mount point paths:
Services:
Type: RAC
Start concurrency:
Stop concurrency:
OSDBA group: dba
OSOPER group: osoper
Database instances: TEST1,TEST2
Configured nodes: rac1,rac2
Database is administrator managed

dbca -silent -deleteInstance -nodeList rac1 -gdbName $DB_NAME -instanceName $INSTANCE_NAME -sysDBAUserName sys -sysDBAPassword syspass

Tras esto comprobariamos que no quedan bases de datos en este nodo con crsctl stat rs -t

Eliminams los recursos del cluster de ese nodo

Una vez hemos eliminado las bases de datos, eliminaremos los recursos del cluster.
El primer paso es mover la mgtdb y para el proxy

srvctl relocate mgmtdb -n rac2
srvctl stop asm -proxy -n wiractst01

Eliminamos el listener

srvctl disable listener -l LISTENER -n rac1
srvctl stop listener -l LISTENER -n rac1

Desinstalando los binarios del CRS

Con el entorno cargado el ASM procederemos a desinstalar los binarios del RAC , para este proposito usaremos el comando deinstall del $ORACLE_HOME deel nodo, este comando se encargara de hacernos una desinstalacion limpia borrandonos:

  • Configuraciones
  • Binarios
  • Interfaces de red virtuales
$ORACLE_HOME/deinstall/deinstall -local

rac1.pamplona.name:oracle (+ASM1:/u01/app/19c/grid/deinstall) $ORACLE_HOME/deinstall/deinstall -local
Checking for required files and bootstrapping ...
Please wait ...
Location of logs /u01/app/oraInventory/logs/

############ ORACLE DECONFIG TOOL START ############
######################### DECONFIG CHECK OPERATION START #########################
## [START] Install check configuration ##
Checking for existence of the Oracle home location /u01/app/19c/grid
Oracle Home type selected for deinstall is: Oracle Grid Infrastructure for a Cluster
Oracle Base selected for deinstall is: /u01/app/oracle
Checking for existence of central inventory location /u01/app/oraInventory
Checking for existence of the Oracle Grid Infrastructure home /u01/app/19c/grid
The following nodes are part of this cluster: rac1,rac2
Checking for sufficient temp space availability on node(s) : 'rac1'
## [END] Install check configuration ##

Traces log file: /u01/app/oraInventory/logs/crsdc_2021-03-23_09-16-26-PM.log
Network Configuration check config START
Network de-configuration trace file location: /u01/app/oraInventory/logs/netdc_check9_09-16-27PM.log
Network Configuration check config END
Asm Check Configuration START
ASM de-configuration trace file location: /u01/app/oraInventory/logs/asmcadc_check2021-03-23_09-16-27PM.log
Database Check Configuration START
Database de-configuration trace file location: /u01/app/oraInventory/logs/databasedc_check2021-03-23_09-16-27PM.log
Oracle Grid Management database was found in this Grid Infrastructure home
Database Check Configuration END
######################### DECONFIG CHECK OPERATION END #########################

####################### DECONFIG CHECK OPERATION SUMMARY #######################
Oracle Grid Infrastructure Home is: /u01/app/19c/grid
The following nodes are part of this cluster: rac1,rac2
The cluster node(s) on which the Oracle home deinstallation will be performed are:rac1
Oracle Home selected for deinstall is: /u01/app/19c/grid
Inventory Location where the Oracle home registered is: /u01/app/oraInventory
Option -local will not modify any ASM configuration.
Oracle Grid Management database was found in this Grid Infrastructure home
Oracle Grid Management database will be relocated to another node during deconfiguration of local node
Do you want to continue (y - yes, n - no)? [n]: y
A log of this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2021-03-23_09-16-21-PM.out'
Any error messages from this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2021-03-23_09-16-21-PM.err'

######################## DECONFIG CLEAN OPERATION START ########################
Database de-configuration trace file location: /u01/app/oraInventory/logs/databasedc_clean2021-03-23_09-16-27PM.log
ASM de-configuration trace file location: /u01/app/oraInventory/logs/asmcadc_clean2021-03-23_09-16-27PM.log
ASM Clean Configuration END

Network Configuration clean config START
Network de-configuration trace file location: /u01/app/oraInventory/logs/netdc_clean2021-03-23_09-16-27PM.log
Network Configuration clean config END
Run the following command as the root user or the administrator on node "wiractst01".
/u01/app/19c/grid/crs/install/rootcrs.sh -force  -deconfig -paramfile "/tmp/deinstall2021-03-23_09-15-25PM/response/deinstall_OraGI19Home1.rsp"
Press Enter after you finish running the above commands
<----------------------------------------

Ejecutamos el comando como root

[oracle@rac1~]$ sudo /u01/app/19c/grid/crs/install/rootcrs.sh -force  -deconfig -paramfile "/tmp/deinstall2021-03-23-15-25PM/response/deinstall_OraGI19Home1.rsp"
Using configuration parameter file: /tmp/deinstall2021-03-23_09-15-25PM/response/deinstall_OraGI19Home1.rsp
The log of current session can be found at:
  /u01/app/oraInventory/logs/crsdeconfig_wiractst01_2021-03-23_09-19-35PM.log
Redirecting to /bin/systemctl restart rsyslog.service
2021/04/09 21:23:06 CLSRSC-336: Successfully deconfigured Oracle Clusterware stack on this node

Y continuamos el proceso dandole a [ ENTER ]

######################### DECONFIG CLEAN OPERATION END #########################
####################### DECONFIG CLEAN OPERATION SUMMARY #######################
Local node configuration of Oracle Grid Management database was removed successfully
Oracle Clusterware is stopped and successfully de-configured on node "rac1"
Oracle Clusterware is stopped and de-configured successfully.
#######################################################################
############# ORACLE DECONFIG TOOL END #############
Using properties file /tmp/deinstall2021-03-23-15-25PM/response/deinstall_2021-03-23-16-21-PM.rsp
Location of logs /u01/app/oraInventory/logs/
############ ORACLE DEINSTALL TOOL START ############
####################### DEINSTALL CHECK OPERATION SUMMARY #######################
A log of this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2021-03-23-16-21-PM.out'
Any error messages from this session will be written to: '/u01/app/oraInventory/logs/deinstall_deconfig2021-03-23-16-21-PM.err'
######################## DEINSTALL CLEAN OPERATION START ########################
## [START] Preparing for Deinstall ##
Setting LOCAL_NODE to rac1
Setting CLUSTER_NODES to rac1
Setting CRS_HOME to true
Setting oracle.installer.invPtrLoc to /tmp/deinstall2021-03-23-15-25PM/oraInst.loc
Setting oracle.installer.local to true
## [END] Preparing for Deinstall ##
Setting the force flag to false
Setting the force flag to cleanup the Oracle Base
Oracle Universal Installer clean START
Detach Oracle home '/u01/app/19c/grid' from the central inventory on the local node : Done
Delete directory '/u01/app/19c/grid' on the local node : Done
The Oracle Base directory '/u01/app/oracle' will not be removed on local node. The directory is in use by Oracle Home '/u01/app/oracle/product/12.1.0.2/db'.
Oracle Universal Installer cleanup was successful.
Oracle Universal Installer clean END
## [START] Oracle install clean ##
## [END] Oracle install clean ##
######################### DEINSTALL CLEAN OPERATION END #########################

####################### DEINSTALL CLEAN OPERATION SUMMARY #######################
Successfully detached Oracle home '/u01/app/19/grid' from the central inventory on the local node.
Successfully deleted directory '/u01/app/19/grid' on the local node.
Oracle Universal Installer cleanup was successful.

Review the permissions and contents of '/u01/app/oracle' on nodes(s) 'rac1'.
If there are no Oracle home(s) associated with '/u01/app/oracle', manually delete '/u01/app/oracle' and its contents.
Oracle deinstall tool successfully cleaned up temporary directories.
#######################################################################
############# ORACLE DEINSTALL TOOL END #############

Ahora, iremos a uno de los nodos que quedan y eliminaremos el nodo 1

[root@rac2 ~]$ .oraenv
ORACLE_SID = [root] ? +ASM2
ORACLE_BASE environment variable is not being set since this
information is not available for the current user ID root.
You can set ORACLE_BASE manually if it is required.
Resetting ORACLE_BASE to its previous value or ORACLE_HOME
The Oracle base has been set to /u01/app/19c/grid

[root@rac2 ~]$  $ORACLE_HOME/bin/crsctl delete node -n  rac1
CRS-4661: Node rac1 successfully deleted.

Comprobamos de nuevo como usuario oracle el numero de nodos

rac2:oracle (+ASM2:/home/oracle) olsnodes -s -t
rac2  Active  Unpinned

Mas informacion en:

  • How to Add Node/Instance or Remove Node/Instance with Oracle Clusterware and RAC (Doc ID 1332451.1)
  • https://oracledbwr.com/step-by-step-deleting-node-in-oracle-rac-12c-release-1-environment

Mas entradas para dummies sobre RAC:
Comandos basicos en Orace RAC
Comandos basicos del RAC II
Eliminar un nodo del rac