UNIX DevOps Advice

1. server 1 is pingable from my desktop , but its not pingable from other server , what is the problem.
Check the IPtables on that destination server

2.how to downgrade patch if its installed with RPM and if its installed with yum
# yum downgrade package
# yum downgrade sqlite

3.Server is not pingale and found hung state in console , what can be done.

4.Server is not pingable and in console its stopped in grub , what can be done.

--> you'll boot Linux from an installation CD or other 'rescue media'--->mount the system's root partition onto /mnt, then the file you would normally see as /etc/fstab will be seen as /mnt/etc/fstab. fix if there is any issue with fstab and reboot
----> if it is corrupted bcas you have 2 different Linux release on the same boot disk then do as follows.
But try remember why it was stuck at Grub, what happened before that...it all depends on that

$ sudo /mnt/usr/sbin/grub

grub> root (hd0,0)
grub> setup (hd0) (hd0,0)
 Checking if "/boot/grub/stage1" exists... yes
 Checking if "/boot/grub/stage2" exists... yes
 Checking if "/boot/grub/e2fs_stage1_5" exists... yes
 Running "embed /boot/grub/e2fs_stage1_5 (hd0)"... 15 sectors are embedded. succeeded
 Running "install /boot/grub/stage1 (hd0) (hd0)1+15 p (hd0,0)/boot/grub/stage2 /boot/grub/menu.lst"... succeeded
Done.
grub> quit

The key command here is setup (hd0) (hd0,0) which tells Grub to re-install the original stage 1 into the MBR of (hd0), and copies the original (Linux img) stage 1.5 into the sectors immediately following. (For a more detailed description of this, look up the install and embed commands in the GNU manual at www.gnu.org/software/grub/manual/grub.html) Once this is done, you can simply reboot

5. How to save LVM in case of drive failure
...

You may be able to find the UUID for the physical volume that was overwritten by looking in the /etc/lvm/archive directory. Look in the file VolumeGroupName_xxxx.vg for the last known valid archived LVM metadata for that volume group.

/etc/lvm/backup
# vgcfgbackup vg00(vgname)

# lvs -a -o +devices
Couldn't find device with uuid 'FmGRh3-zhok-iVI8-7qTD-S5BI-MAEN-NYM5Sk'.
Couldn't find all physical volumes for volume group VG.
Couldn't find device with uuid 'FmGRh3-zhok-iVI8-7qTD-S5BI-MAEN-NYM5Sk'.
Couldn't find all physical volumes for volume group VG.
# vgchange -an --partial
Partial mode. Incomplete volume groups will be activated read-only.
Couldn't find device with uuid 'FmGRh3-zhok-iVI8-7qTD-S5BI-MAEN-NYM5Sk'.
Couldn't find device with uuid 'FmGRh3-zhok-iVI8-7qTD-S5BI-MAEN-NYM5Sk'.
...
# pvcreate --uuid "FmGRh3-zhok-iVI8-7qTD-S5BI-MAEN-NYM5Sk" --restorefile /etc/lvm/archive/VG_00050.vg /dev/sdh1
Physical volume "/dev/sdh1" successfully created

# vgcfgrestore VG
Restored volume group VG

# lvs -a -o +devices
LV     VG   Attr   LSize   Origin Snap% Move Log Copy% Devices
stripe VG   -wi--- 300.00G                               /dev/sdh1 (0),/dev/sda1(0)
stripe VG   -wi--- 300.00G                               /dev/sdh1 (34728),/dev/sdb1(0)

# lvchange -ay /dev/VG/stripe
[root@link-07 backup]# lvs -a -o +devices
LV     VG   Attr   LSize   Origin Snap% Move Log Copy% Devices
stripe VG   -wi-a- 300.00G                               /dev/sdh1 (0),/dev/sda1(0)
stripe VG   -wi-a- 300.00G                               /dev/sdh1 (34728),/dev/sdb1(0)

6. How to retrieve data from vmx. file in case VM get corrupted
From the VMware vsphere or VMware player follow the same steps as creating a new VM and choose the VMX file when it will ask you to.

There are 2 different ways of doing iSCSI target and initiator configuration.
In below example I have done it using "tgtd" on the Server and "open-iscsi" on the clients side.
I have used Centos7 as the "Target" and in the Centos7 I have used KVM and created a "Kali Linux" as a VM. So kali linux VM I have used as iScsi Initiator.

Centos7 Server side output:

167 yum install scsi-target-utils
168 systemctl tgtd start
169 systemctl start tgtd
170 systemctl status tgtd
171 systemctl enable tgtd
172 df -h
173 fdisk
174 fdisk -l
175 vgs
502 yum provides gnome-disk-utility
503 yum install gnome-disk-utility-3.14.0-2.el7.x86_64
504 gnome-disks
505   cat /etc/tgt/targets.conf
506 vi /etc/tgt/targets.conf
507   cat /etc/tgt/targets.conf
508 systemctl restart tgtd
509 journalctl -xe
510 hostname
511 cd /var/log/
512 ls
513 more secure
514
515 date
516 ps -ef |grep 16522
517 systemctl restart tgtd
518 systemctl status tgtd.service
519   cat /etc/tgt/targets.conf
520 vi /etc/tgt/tgtd.conf
521 vi /etc/tgt/tgtd.conf
522   cat /etc/tgt/targets.conf
523 vi /etc/tgt/tgtd.conf
524 systemctl restart tgtd
525 systemctl status tgtd
526 df -k
527 tgt-admin --show
528 lvdisplay
529 tgt-admin --show
530 tgtadm --lld isci --op bind --mode target --tid 1 -I ALL
531 history

[root@myPC log]# ps -ef |grep 16522
root     16834 13494 0 22:15 pts/1    00:00:00 grep --color=auto 16522
[root@myPC log]# systemctl restart tgtd
Job for tgtd.service failed because the control process exited with error code. See "systemctl statu
s tgtd.service" and "journalctl -xe" for details.
[root@myPC log]# systemctl status tgtd.service
● tgtd.service - tgtd iSCSI target daemon
   Loaded: loaded (/usr/lib/systemd/system/tgtd.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Wed 2017-01-11 22:17:50 EST; 40s ago
Process: 16944 ExecStop=/usr/sbin/tgtadm --op delete --mode system (code=exited, status=0/SUCCESS)
Process: 16940 ExecStop=/usr/sbin/tgt-admin --update ALL -c /dev/null (code=exited, status=0/SUCCE
SS)
Process: 16938 ExecStop=/usr/sbin/tgtadm --op update --mode sys --name State -v offline (code=exit
ed, status=0/SUCCESS)
Process: 3189 ExecStartPost=/usr/sbin/tgtadm --op update --mode sys --name State -v ready (code=ex
ited, status=0/SUCCESS)
Process: 16934 ExecStartPost=/usr/sbin/tgt-admin -e -c $TGTD_CONFIG (code=exited, status=1/FAILURE
)
Process: 16932 ExecStartPost=/usr/sbin/tgtadm --op update --mode sys --name State -v offline (code
=exited, status=0/SUCCESS)
Process: 16928 ExecStartPost=/bin/sleep 5 (code=exited, status=0/SUCCESS)
Process: 16926 ExecStart=/usr/sbin/tgtd -f $TGTD_OPTS (code=exited, status=0/SUCCESS)
Main PID: 16926 (code=exited, status=0/SUCCESS)

Jan 11 22:17:45 myPC tgtd[16926]: tgtd: iser_ib_init(3436) Failed to initialize RDMA; load kernel mo
dules?
Jan 11 22:17:45 myPC tgtd[16926]: tgtd: work_timer_start(146) use timer_fd based scheduler
Jan 11 22:17:45 myPC tgtd[16926]: tgtd: bs_init_signalfd(267) could not open backing-store module di
rectory /usr/l...-store
Jan 11 22:17:45 myPC tgtd[16926]: tgtd: bs_init(386) use signalfd notification
Jan 11 22:17:50 myPC tgt-admin[16934]: Multiple default-driver definitions are not allowed!
Jan 11 22:17:50 myPC tgt-admin[16934]: Check your config file for errors.
Jan 11 22:17:50 myPC systemd[1]: tgtd.service: control process exited, code=exited status=1
Jan 11 22:17:50 myPC systemd[1]: Failed to start tgtd iSCSI target daemon.
Jan 11 22:17:50 myPC systemd[1]: Unit tgtd.service entered failed state.
Jan 11 22:17:50 myPC systemd[1]: tgtd.service failed.
Hint: Some lines were ellipsized, use -l to show in full.
[root@myPC log]# cat /etc/tgt/targets.conf
# This is a sample config file for tgt-admin.
#
# The "#" symbol disables the processing of a line.

# Set the driver. If not specified, defaults to "iscsi".
default-driver iscsi

# Set iSNS parameters, if needed
#iSNSServerIP 192.168.111.222
#iSNSServerPort 3205
#iSNSAccessControl On
#iSNS On

# Continue if tgtadm exits with non-zero code (equivalent of
# --ignore-errors command line option)
#ignore-errors yes
default-driver iscsi

        backing-store /dev/sda3
        initiator-address 192.168.122.1

[root@myPC log]# vi /etc/tgt/tgtd.conf
[root@myPC log]# vi /etc/tgt/tgtd.conf
[root@myPC log]# cat /etc/tgt/targets.conf
# This is a sample config file for tgt-admin.
#
# The "#" symbol disables the processing of a line.

# Set the driver. If not specified, defaults to "iscsi".
default-driver iscsi

# Set iSNS parameters, if needed
#iSNSServerIP 192.168.111.222
Jan 11 22:17:50 myPC systemd[1]: Unit tgtd.service entered failed state.
Jan 11 22:17:50 myPC systemd[1]: tgtd.service failed.
Hint: Some lines were ellipsized, use -l to show in full.
[root@myPC log]# cat /etc/tgt/targets.conf
# This is a sample config file for tgt-admin.
#
# The "#" symbol disables the processing of a line.

# Set the driver. If not specified, defaults to "iscsi".
default-driver iscsi

# Set iSNS parameters, if needed
#iSNSServerIP 192.168.111.222
#iSNSServerPort 3205
#iSNSAccessControl On
#iSNS On

# Continue if tgtadm exits with non-zero code (equivalent of
# --ignore-errors command line option)
#ignore-errors yes
default-driver iscsi

        backing-store /dev/sda3
        initiator-address 192.168.122.1

[root@myPC log]# vi /etc/tgt/tgtd.conf
[root@myPC log]# vi /etc/tgt/tgtd.conf
[root@myPC log]# cat /etc/tgt/targets.conf
# This is a sample config file for tgt-admin.
#
# The "#" symbol disables the processing of a line.

# Set the driver. If not specified, defaults to "iscsi".
default-driver iscsi

# Set iSNS parameters, if needed
#iSNSServerIP 192.168.111.222
Set iSNS parameters, if needed
#iSNSServerIP 192.168.111.222
#iSNSServerPort 3205
#iSNSAccessControl On
#iSNS On

# Continue if tgtadm exits with non-zero code (equivalent of
# --ignore-errors command line option)
#ignore-errors yes
default-driver iscsi

        backing-store /dev/sda3
        initiator-address 192.168.122.1

[root@myPC log]# vi /etc/tgt/tgtd.conf
[root@myPC log]#
[root@myPC log]#
[root@myPC log]# systemctl restart tgtd
[root@myPC log]# systemctl status tgtd
● tgtd.service - tgtd iSCSI target daemon
   Loaded: loaded (/usr/lib/systemd/system/tgtd.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2017-01-11 22:31:39 EST; 14s ago
Process: 16944 ExecStop=/usr/sbin/tgtadm --op delete --mode system (code=exited, status=0/SUCCESS)
Process: 16940 ExecStop=/usr/sbin/tgt-admin --update ALL -c /dev/null (code=exited, status=0/SUCCE
SS)
Process: 16938 ExecStop=/usr/sbin/tgtadm --op update --mode sys --name State -v offline (code=exit
ed, status=0/SUCCESS)
Process: 17769 ExecStartPost=/usr/sbin/tgtadm --op update --mode sys --name State -v ready (code=e
xited, status=0/SUCCESS)
Process: 17741 ExecStartPost=/usr/sbin/tgt-admin -e -c $TGTD_CONFIG (code=exited, status=0/SUCCESS
)
Process: 17739 ExecStartPost=/usr/sbin/tgtadm --op update --mode sys --name State -v offline (code
=exited, status=0/SUCCESS)
Process: 17737 ExecStartPost=/bin/sleep 5 (code=exited, status=0/SUCCESS)
Main PID: 17735 (tgtd)
   CGroup: /system.slice/tgtd.service
           └─17735 /usr/sbin/tgtd -f
Process: 17737 ExecStartPost=/bin/sleep 5 (code=exited, status=0/SUCCESS)
Main PID: 17735 (tgtd)
   CGroup: /system.slice/tgtd.service
           └─17735 /usr/sbin/tgtd -f

Jan 11 22:31:34 myPC tgtd[17735]: librdmacm: Warning: couldn't read ABI version.
Jan 11 22:31:34 myPC tgtd[17735]: librdmacm: Warning: assuming: 4
Jan 11 22:31:34 myPC tgtd[17735]: librdmacm: Fatal: unable to get RDMA device list
Jan 11 22:31:34 myPC tgtd[17735]: tgtd: iser_ib_init(3436) Failed to initialize RDMA; load kernel mo
dules?
Jan 11 22:31:34 myPC tgtd[17735]: tgtd: work_timer_start(146) use timer_fd based scheduler
Jan 11 22:31:34 myPC tgtd[17735]: tgtd: bs_init_signalfd(267) could not open backing-store module di
rectory /usr/l...-store
Jan 11 22:31:34 myPC tgtd[17735]: tgtd: bs_init(386) use signalfd notification
Jan 11 22:31:39 myPC tgtd[17735]: tgtd: device_mgmt(246) sz:15 params:path=/dev/sda3
Jan 11 22:31:39 myPC tgtd[17735]: tgtd: bs_thread_open(408) 16
Jan 11 22:31:39 myPC systemd[1]: Started tgtd iSCSI target daemon.
Hint: Some lines were ellipsized, use -l to show in full.
[root@myPC log]#
[root@myPC log]#
[root@myPC log]# df -k
Filesystem              1K-blocks     Used Available Use% Mounted on
/dev/mapper/centos-root 52403200 32285408 20117792 62% /
devtmpfs                  1869656        0   1869656   0% /dev
tmpfs                     1879784   624300   1255484 34% /dev/shm
tmpfs                     1879784     9256   1870528   1% /run
tmpfs                     1879784        0   1879784   0% /sys/fs/cgroup
/dev/mapper/centos-home 61410000 26627544 34782456 44% /home
/dev/sda1                  508588   127596    380992 26% /boot
tmpfs                      375960        0    375960   0% /run/user/1001
tmpfs                      375960       16    375944   1% /run/user/1000

[root@myPC log]# tgt-admin --show
Target 1: iqn.2017-01.home.myiPC:target1
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: null
            Backing store path: None
            Backing store flags:
        LUN: 1
            Type: disk
            SCSI ID: IET     00010001
            SCSI SN: beaf11
            Size: 0 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rdwr
            Backing store path: /dev/sda3
            Backing store flags:
    Account information:
    ACL information:
        192.168.122.1
[root@myPC log]#
vi /etc/tgt/targets.conf
default-driver iscsi

        backing-store /dev/sda3
        initiator-address 192.168.122.166
Note:
The initiator address in the file above needs to be the "Client's" IP address not the iscsi target IP..
The firewall to be enabled for port 3260.
Command - # firewall-cmd --add port=3260/tcp

UNIX DevOps Advice

Fwd: TechQue

iscsi configuration on Centos 7 / RHEL7/ Kali

Installing Ingress Controller - Kubernetes