ZFS Storage ilom fmadm faulty and fmadm repair

-> start /SP/f
faultmgmt  firmware
-> start /SP/faultmgmt/shell/
Are you sure you want to start /SP/faultmgmt/shell (y/n)? y
faultmgmtsp> fmadm faulty
------------------- ------------------------------------ -------------- --------
Time                UUID                                 msgid          Severity
------------------- ------------------------------------ -------------- --------
2016-08-09/22:31:33 3e724a2d-55a5-4a51-c5b4-e2abb767dff9 DISK-8000-0X Major
Problem Status    : open
Diag Engine       : eft 1.16
System
   Manufacturer   : Oracle Corporation
   Name           : SuperCluster T5-8
   Part_Number    : SuperCluster T5-8
   Serial_Number  : AK00358593
System Component
   Manufacturer   : Oracle Corporation
   Name           : SUN FIRE X4170 M3
   Part_Number    : 7095777
   Serial_Number  : xxxxxx
----------------------------------------
Suspect 1 of 1
   Fault class  : fault.io.disk.predictive-failure
   Certainty    : 100%
   Affects      : dev:///:devid=id1,sd@n5000cca23b1bc088//scsi_vhci/disk@g5000cca23b1bc088
   Status       : faulted
   FRU
      Status            : faulty
      Location          : DISK  7
      Chassis
         Manufacturer   : Oracle Corporation
         Name           : SUN FIRE X4170 M3
         Part_Number    : 7095777
         Serial_Number  : xxxxxxxx
Description : A fault was diagnosed by the Host Operating System.
Action      : Please refer to the associated reference document at
              http://support.oracle.com/msg/DISK-8000-0X for a complete,
              detailed description and the latest service procedures and
              policies regarding this diagnosis.
faultmgmtsp>
faultmgmtsp> fmadm repair 3e724a2d-55a5-4a51-c5b4-e2abb767dff9
faultmgmtsp> fmadm faulty
No faults found
faultmgmtsp>



-> start SP/
-> start SP/f
faultmgmt  firmware
-> start SP/faultmgmt/shell/
Are you sure you want to start /SP/faultmgmt/shell (y/n)? y
faultmgmtsp> help
Built-in commands:
  echo   - Display information to user.
           Typical use: echo $?
  help   - Produces this help.
           Use 'help ' for more information about an external command.
  exit   - Exit this shell.
External commands:
  fmadm  - Administers the fault management service
  fmdump - Displays contents of the fault and ereport/error logs
  fmstat - Displays statistics on fault management operations
faultmgmtsp> fmstat
fdd statistics    2017-05-18/17:41:30
engine     status      evts_in evts_out errors   rtave, s rtmax, s
platform   empty             1        0      0      0.001    0.001
repair     empty             4        0      0     0.0763    0.237
hysteresis empty            22        1      0     0.0005    0.001
SERD       empty             0        0      0          0        0
simple     empty             0        0      0          0        0
Rules Engine specific stats
faultmgmtsp> fmadm faulty
------------------- ------------------------------------ -------------- --------
Time                UUID                                 msgid          Severity
------------------- ------------------------------------ -------------- --------
2016-08-09/22:31:33 3e724a2d-55a5-4a51-c5b4-e2abb767dff9 DISK-8000-0X Major
Problem Status    : open
Diag Engine       : eft 1.16
System
   Manufacturer   : Oracle Corporation
   Name           : SuperCluster T5-8
   Part_Number    : SuperCluster T5-8
   Serial_Number  : AK00358593
System Component
   Manufacturer   : Oracle Corporation
   Name           : SUN FIRE X4170 M3
   Part_Number    : 7095777
   Serial_Number  : xxxxxxx
----------------------------------------
Suspect 1 of 1
   Fault class  : fault.io.disk.predictive-failure
   Certainty    : 100%
   Affects      : dev:///:devid=id1,sd@n5000cca23b1bc088//scsi_vhci/disk@g5000cca23b1bc088
   Status       : faulted
   FRU
      Status            : faulty
      Location          : DISK  7
      Chassis
         Manufacturer   : Oracle Corporation
         Name           : SUN FIRE X4170 M3
         Part_Number    : 7095777
         Serial_Number  : xxxxxxx
Description : A fault was diagnosed by the Host Operating System.
Action      : Please refer to the associated reference document at
              http://support.oracle.com/msg/DISK-8000-0X for a complete,
              detailed description and the latest service procedures and
              policies regarding this diagnosis.
faultmgmtsp>

faultmgmtsp> fmstat
fdd statistics    2017-05-18/17:41:30
engine     status      evts_in evts_out errors   rtave, s rtmax, s
platform   empty             1        0      0      0.001    0.001
repair     empty             4        0      0     0.0763    0.237
hysteresis empty            22        1      0     0.0005    0.001
SERD       empty             0        0      0          0        0
simple     empty             0        0      0          0        0
Rules Engine specific stats
faultmgmtsp> fmadm faulty
------------------- ------------------------------------ -------------- --------
Time                UUID                                 msgid          Severity
------------------- ------------------------------------ -------------- --------
2016-08-09/22:31:33 3e724a2d-55a5-4a51-c5b4-e2abb767dff9 DISK-8000-0X Major
Problem Status    : open
Diag Engine       : eft 1.16
System
   Manufacturer   : Oracle Corporation
   Name           : SuperCluster T5-8
   Part_Number    : SuperCluster T5-8
   Serial_Number  : xxxxxxx
System Component
   Manufacturer   : Oracle Corporation
   Name           : SUN FIRE X4170 M3
   Part_Number    : 7095777
   Serial_Number  : 1535NML00M
----------------------------------------
Suspect 1 of 1
   Fault class  : fault.io.disk.predictive-failure
   Certainty    : 100%
   Affects      : dev:///:devid=id1,sd@n5000cca23b1bc088//scsi_vhci/disk@g5000cca23b1bc088
   Status       : faulted
   FRU
      Status            : faulty
      Location          : DISK  7
      Chassis
         Manufacturer   : Oracle Corporation
         Name           : SUN FIRE X4170 M3
         Part_Number    : 7095777
         Serial_Number  : xxxxxxx
Description : A fault was diagnosed by the Host Operating System.
Action      : Please refer to the associated reference document at
              http://support.oracle.com/msg/DISK-8000-0X for a complete,
              detailed description and the latest service procedures and
              policies regarding this diagnosis.
faultmgmtsp> fmadm repair 3e724a2d-55a5-4a51-c5b4-e2abb767dff9
faultmgmtsp> fmadm faulty
No faults found
faultmgmtsp>

6/9/2017 - all looks good
efsp-h1-storadm:status storage> show
Storage:
   ssc:
      Used     2.13T bytes
      Available  62.0T bytes
      Free     61.0T bytes
      State          online
      Compression:   1x
      Dedup:         1x
sp-h1-storadm:status storage>

Fwd: TechQue



1. server 1 is pingable from my desktop , but its not pingable from other server , what is the problem.
Check the IPtables on that destination server

2.how to downgrade patch if its installed with RPM and if its installed with yum
# yum downgrade package
# yum downgrade sqlite

3.Server is not pingale and found hung state in console , what can be done.
Login to console and try to see run level # who -r and try to start in MultiUser mode runlevel 5
 
4.Server is not pingable and in console its stopped in grub , what can be done.

--> you'll boot Linux from an installation CD or other 'rescue media'--->mount the system's root partition onto /mnt, then the file you would normally see as /etc/fstab will be seen as /mnt/etc/fstab. fix if there is any issue with fstab and reboot
----> if it is corrupted bcas you have 2 different Linux release on the same boot disk then do as follows.
But try remember why it was stuck at Grub, what happened before that...it all depends on that
$ sudo /mnt/usr/sbin/grub

grub> root (hd0,0)
grub> setup (hd0) (hd0,0)
 Checking if "/boot/grub/stage1" exists... yes
 Checking if "/boot/grub/stage2" exists... yes
 Checking if "/boot/grub/e2fs_stage1_5" exists... yes
 Running "embed /boot/grub/e2fs_stage1_5 (hd0)"... 15 sectors are embedded. succeeded
 Running "install /boot/grub/stage1 (hd0) (hd0)1+15 p (hd0,0)/boot/grub/stage2 /boot/grub/menu.lst"... succeeded
Done.
grub> quit
The key command here is setup (hd0) (hd0,0) which tells Grub to re-install the original stage 1 into the MBR of (hd0), and copies the original (Linux img) stage 1.5 into the sectors immediately following. (For a more detailed description of this, look up the install and embed commands in the GNU manual at www.gnu.org/software/grub/manual/grub.html) Once this is done, you can simply reboot
 
5. How to save LVM in case of drive failure
...
You may be able to find the UUID for the physical volume that was overwritten by looking in the /etc/lvm/archive directory. Look in the file VolumeGroupName_xxxx.vg for the last known valid archived LVM metadata for that volume group.
/etc/lvm/backup
# vgcfgbackup vg00(vgname)
# lvs -a -o +devices
  Couldn't find device with uuid 'FmGRh3-zhok-iVI8-7qTD-S5BI-MAEN-NYM5Sk'.
  Couldn't find all physical volumes for volume group VG.
  Couldn't find device with uuid 'FmGRh3-zhok-iVI8-7qTD-S5BI-MAEN-NYM5Sk'.
  Couldn't find all physical volumes for volume group VG.
# vgchange -an --partial
  Partial mode. Incomplete volume groups will be activated read-only.
  Couldn't find device with uuid 'FmGRh3-zhok-iVI8-7qTD-S5BI-MAEN-NYM5Sk'.
  Couldn't find device with uuid 'FmGRh3-zhok-iVI8-7qTD-S5BI-MAEN-NYM5Sk'.
  ...
# pvcreate --uuid "FmGRh3-zhok-iVI8-7qTD-S5BI-MAEN-NYM5Sk" --restorefile /etc/lvm/archive/VG_00050.vg /dev/sdh1
  Physical volume "/dev/sdh1" successfully created

# vgcfgrestore VG
  Restored volume group VG

# lvs -a -o +devices
  LV     VG   Attr   LSize   Origin Snap%  Move Log Copy%  Devices
  stripe VG   -wi--- 300.00G                               /dev/sdh1 (0),/dev/sda1(0)
  stripe VG   -wi--- 300.00G                               /dev/sdh1 (34728),/dev/sdb1(0)

# lvchange -ay /dev/VG/stripe
[root@link-07 backup]# lvs -a -o +devices
  LV     VG   Attr   LSize   Origin Snap%  Move Log Copy%  Devices
  stripe VG   -wi-a- 300.00G                               /dev/sdh1 (0),/dev/sda1(0)
  stripe VG   -wi-a- 300.00G                               /dev/sdh1 (34728),/dev/sdb1(0)
 
6. How to retrieve data from vmx. file in case VM get corrupted
From the VMware vsphere or VMware player follow the same steps as creating a new VM and choose the VMX file when it will ask you to.








Installing Ingress Controller - Kubernetes

Installing the Ingress Controller Prerequisites Make sure you have access to the Ingress controller image: For NGINX Ingress controll...