AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  CARME  ELOG logo
 List  |  New  |  Edit  |  Delete  |  Reply  |  Duplicate  |  Find  |  Config  |  Help 
Message ID: 182     Entry time: Mon Jan 31 08:58:38 2022
Author: OH, TD, JM 
Subject: centos server configuration  
Steps taken:
- MIDAS_Releases, Embedded and Exports files brought across from the previously working SL6 aida-gsi install. Brought as tar balls to preserve ownership
- dhcpd.conf set up with same IP addresses as used on the SL6 install from aida-gsi
- nfs-utils package installed on centos7
- Paths to directories listed in /etc/exports checked that they point to the correct locations on the centos system
- Firewall disabled to confirm that it is not interfering
- setenforce 0 to confirm selinux is not interfering.


- Attachment 1 Screenshot of FEE console for aida04 as it stops booting.
- Attachment 2 text dump of /var/log/messages during the FEE boot sequence
- Attachment 3 text dump from tcpdump filtered to aida04
- Attachment 4 /var/lib/dhcp/dhcp.leases file as mentioned in the /var/log/messages dump
- Attachment 5 A copy of dhcpd.conf
- Attachment 6 A copy of /etc/hosts
- Attachment 7 A copy of /etc/exports    
- Attachment 8 Text dump of ifconfig -a (Note the FEEs are conencted to p4p1)

From the messages dump it seems there is an issue being encountered with dhcp leases for the static IP addresses of the FEEs. They are then getting assigned new IP addresses which are getting put in the leases file.
Attachment 1: 220131_0950_FEE_CONSOLE.png  23 kB  | Hide | Hide all | Show all
220131_0950_FEE_CONSOLE.png
Attachment 2: 220131_0951_logmessages.txt  6 kB  | Hide | Hide all | Show all 
an 31 09:48:40 carme-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:d0:05 via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:d0:05 via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPOFFER on 192.168.11.3 to d8:80:39:41:d0:05 via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPREQUEST for 192.168.11.3 (192.168.11.99) from d8:80:39:41:d0:05 via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPREQUEST for 192.168.11.3 (192.168.11.99) from d8:80:39:41:d0:05 via p4p1: unknown lease 192.168.11.3.
Jan 31 09:48:40 carme-gsi dhcpd: DHCPACK on 192.168.11.3 to d8:80:39:41:d0:05 via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:b4:0b via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPOFFER on 192.168.11.5 to d8:80:39:41:b4:0b via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:b4:0b via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:d7:cd via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:d7:cd via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPOFFER on 192.168.11.8 to d8:80:39:41:d7:cd via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPREQUEST for 192.168.11.5 (192.168.11.99) from d8:80:39:41:b4:0b via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPACK on 192.168.11.5 to d8:80:39:41:b4:0b via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPREQUEST for 192.168.11.5 (192.168.11.99) from d8:80:39:41:b4:0b via p4p1: unknown lease 192.168.11.5.
Jan 31 09:48:40 carme-gsi dhcpd: DHCPREQUEST for 192.168.11.8 (192.168.11.99) from d8:80:39:41:d7:cd via p4p1: unknown lease 192.168.11.8.
Jan 31 09:48:40 carme-gsi dhcpd: DHCPREQUEST for 192.168.11.8 (192.168.11.99) from d8:80:39:41:d7:cd via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPACK on 192.168.11.8 to d8:80:39:41:d7:cd via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:d8:20 via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPOFFER on 192.168.11.6 to d8:80:39:41:d8:20 via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:d8:20 via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPREQUEST for 192.168.11.6 (192.168.11.99) from d8:80:39:41:d8:20 via p4p1
Jan 31 09:48:40 carme-gsi dhcpd: DHCPREQUEST for 192.168.11.6 (192.168.11.99) from d8:80:39:41:d8:20 via p4p1: unknown lease 192.168.11.6.
Jan 31 09:48:40 carme-gsi dhcpd: DHCPACK on 192.168.11.6 to d8:80:39:41:d8:20 via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:d7:c2 via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPOFFER on 192.168.11.4 to d8:80:39:41:d7:c2 via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:d7:c2 via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPREQUEST for 192.168.11.4 (192.168.11.99) from d8:80:39:41:d7:c2 via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPREQUEST for 192.168.11.4 (192.168.11.99) from d8:80:39:41:d7:c2 via p4p1: unknown lease 192.168.11.4.
Jan 31 09:48:41 carme-gsi dhcpd: DHCPACK on 192.168.11.4 to d8:80:39:41:d7:c2 via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPDISCOVER from d8:80:39:42:0d:0c via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPDISCOVER from d8:80:39:42:0d:0c via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:b4:16 via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:b4:16 via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPOFFER on 192.168.11.7 to d8:80:39:41:b4:16 via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPREQUEST for 192.168.11.7 (192.168.11.99) from d8:80:39:41:b4:16 via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPREQUEST for 192.168.11.7 (192.168.11.99) from d8:80:39:41:b4:16 via p4p1: unknown lease 192.168.11.7.
Jan 31 09:48:41 carme-gsi dhcpd: DHCPACK on 192.168.11.7 to d8:80:39:41:b4:16 via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:d8:2b via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:d8:2b via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPOFFER on 192.168.11.2 to d8:80:39:41:d8:2b via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPREQUEST for 192.168.11.2 (192.168.11.99) from d8:80:39:41:d8:2b via p4p1: unknown lease 192.168.11.2.
Jan 31 09:48:41 carme-gsi dhcpd: DHCPREQUEST for 192.168.11.2 (192.168.11.99) from d8:80:39:41:d8:2b via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPACK on 192.168.11.2 to d8:80:39:41:d8:2b via p4p1
Jan 31 09:48:41 carme-gsi rpc.mountd[79344]: authenticated mount request from 192.168.11.3:956 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida03 (/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Jan 31 09:48:41 carme-gsi rpc.mountd[79344]: authenticated mount request from 192.168.11.8:675 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida08 (/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Jan 31 09:48:41 carme-gsi rpc.mountd[79344]: authenticated mount request from 192.168.11.5:907 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida05 (/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Jan 31 09:48:41 carme-gsi rpc.mountd[79344]: authenticated mount request from 192.168.11.6:883 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida06 (/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Jan 31 09:48:41 carme-gsi rpc.mountd[79344]: authenticated mount request from 192.168.11.4:985 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida04 (/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Jan 31 09:48:41 carme-gsi rpc.mountd[79344]: authenticated mount request from 192.168.11.7:682 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida07 (/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Jan 31 09:48:41 carme-gsi rpc.mountd[79344]: authenticated mount request from 192.168.11.2:707 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida02 (/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Jan 31 09:48:41 carme-gsi dhcpd: DHCPOFFER on 192.168.11.121 to d8:80:39:41:d0:05 via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPOFFER on 192.168.11.122 to d8:80:39:41:b4:0b via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPOFFER on 192.168.11.120 to d8:80:39:41:d7:cd via p4p1
Jan 31 09:48:41 carme-gsi dhcpd: DHCPOFFER on 192.168.11.118 to d8:80:39:41:d8:20 via p4p1
Jan 31 09:48:42 carme-gsi dhcpd: DHCPOFFER on 192.168.11.119 to d8:80:39:41:d7:c2 via p4p1
Jan 31 09:48:42 carme-gsi dhcpd: DHCPOFFER on 192.168.11.123 to d8:80:39:42:0d:0c via p4p1
Jan 31 09:48:42 carme-gsi dhcpd: DHCPOFFER on 192.168.11.123 to d8:80:39:42:0d:0c via p4p1
Jan 31 09:48:42 carme-gsi dhcpd: Can't create new lease file: Permission denied
Jan 31 09:48:42 carme-gsi dhcpd: DHCPREQUEST for 192.168.11.123 (192.168.11.99) from d8:80:39:42:0d:0c via p4p1
Jan 31 09:48:42 carme-gsi dhcpd: DHCPACK on 192.168.11.123 to d8:80:39:42:0d:0c via p4p1
Jan 31 09:48:42 carme-gsi dhcpd: Wrote 0 deleted host decls to leases file.
Jan 31 09:48:42 carme-gsi dhcpd: Wrote 0 new dynamic host decls to leases file.
Jan 31 09:48:42 carme-gsi dhcpd: Wrote 8 leases to leases file.
Jan 31 09:48:42 carme-gsi dhcpd: DHCPREQUEST for 192.168.11.123 (192.168.11.99) from d8:80:39:42:0d:0c via p4p1
Jan 31 09:48:42 carme-gsi dhcpd: DHCPACK on 192.168.11.123 to d8:80:39:42:0d:0c via p4p1
Jan 31 09:48:42 carme-gsi dhcpd: DHCPOFFER on 192.168.11.124 to d8:80:39:41:b4:16 via p4p1
Jan 31 09:48:42 carme-gsi dhcpd: DHCPOFFER on 192.168.11.125 to d8:80:39:41:d8:2b via p4p1
Attachment 3: 220131_0952_tcpdump.txt  2 kB  | Show | Hide all | Show all
Attachment 4: 220131_0953_dhcpdleases.txt  1 kB  | Hide | Hide all | Show all 
# The format of this file is documented in the dhcpd.leases(5) manual page.
# This lease file was written by isc-dhcp-4.2.5

lease 192.168.11.118 {
  starts 1 2022/01/31 08:48:40;
  ends 1 2022/01/31 08:50:40;
  cltt 1 2022/01/31 08:48:40;
  binding state free;
  hardware ethernet d8:80:39:41:d8:20;
}
lease 192.168.11.120 {
  starts 1 2022/01/31 08:48:40;
  ends 1 2022/01/31 08:50:40;
  cltt 1 2022/01/31 08:48:40;
  binding state free;
  hardware ethernet d8:80:39:41:d7:cd;
}
lease 192.168.11.122 {
  starts 1 2022/01/31 08:48:40;
  ends 1 2022/01/31 08:50:40;
  cltt 1 2022/01/31 08:48:40;
  binding state free;
  hardware ethernet d8:80:39:41:b4:0b;
}
lease 192.168.11.121 {
  starts 1 2022/01/31 08:48:40;
  ends 1 2022/01/31 08:50:40;
  cltt 1 2022/01/31 08:48:40;
  binding state free;
  hardware ethernet d8:80:39:41:d0:05;
}
lease 192.168.11.125 {
  starts 1 2022/01/31 08:48:41;
  ends 1 2022/01/31 08:50:41;
  cltt 1 2022/01/31 08:48:41;
  binding state free;
  hardware ethernet d8:80:39:41:d8:2b;
}
lease 192.168.11.124 {
  starts 1 2022/01/31 08:48:41;
  ends 1 2022/01/31 08:50:41;
  cltt 1 2022/01/31 08:48:41;
  binding state free;
  hardware ethernet d8:80:39:41:b4:16;
}
lease 192.168.11.119 {
  starts 1 2022/01/31 08:48:41;
  ends 1 2022/01/31 08:50:41;
  cltt 1 2022/01/31 08:48:41;
  binding state free;
  hardware ethernet d8:80:39:41:d7:c2;
}
lease 192.168.11.123 {
  starts 1 2022/01/31 08:48:42;
  ends 1 2022/01/31 20:48:42;
  cltt 1 2022/01/31 08:48:42;
  binding state active;
  next binding state free;
  rewind binding state free;
  hardware ethernet d8:80:39:42:0d:0c;
}
server-duid "\000\001\000\001)\211c\037\000\340L\241\026\257";
Attachment 5: dhcpd.conf  4 kB  | Show | Hide all | Show all
Attachment 6: hosts.txt  342 Bytes  | Hide | Hide all | Show all 
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6


192.168.11.251 nnrpi1
192.168.11.1 aida01
192.168.11.2 aida02
192.168.11.3 aida03
192.168.11.4 aida04
192.168.11.5 aida05
192.168.11.6 aida06
192.168.11.7 aida07
192.168.11.8 aida08
Attachment 7: exports.txt  235 Bytes  | Hide | Hide all | Show all 
/home/Embedded/XilinxLinux/ppc_4xx/rfs  192.168.0.0/255.255.0.0(rw,sync,nohide,no_root_squash)
/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 192.168.0.0/255.255.0.0(rw,sync,nohide)
/MIDAS_130718 192.168.0.0/255.255.0.0(rw,sync,nohide)
Attachment 8: ifconfig.txt  3 kB  | Hide | Hide all | Show all 
enp0s31f6: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether a4:bb:6d:d1:81:9a  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 16  memory 0x90300000-90320000  

enp2s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.207.142  netmask 255.255.255.0  broadcast 192.168.207.255
        inet6 fd43:525a:5249:4e47:d923:1a54:9ea8:e9b1  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::4194:b06d:5173:1c42  prefixlen 64  scopeid 0x20<link>
        ether a4:bb:6d:d1:7f:ca  txqueuelen 1000  (Ethernet)
        RX packets 15104702  bytes 22030764707 (20.5 GiB)
        RX errors 0  dropped 29  overruns 0  frame 0
        TX packets 2199234  bytes 424845717 (405.1 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0x90100000-9017ffff  

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 1157441  bytes 870407343 (830.0 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1157441  bytes 870407343 (830.0 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

p4p1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.11.99  netmask 255.255.255.0  broadcast 192.168.11.255
        inet6 fe80::74a5:671:a523:2bc5  prefixlen 64  scopeid 0x20<link>
        ether 00:e0:4c:a1:16:af  txqueuelen 1000  (Ethernet)
        RX packets 2283907  bytes 3266470097 (3.0 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 256271  bytes 19134505 (18.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

p4p2: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 00:e0:4c:a1:16:b0  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

p4p3: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 00:e0:4c:a1:16:b1  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

p4p4: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 00:e0:4c:a1:16:b2  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 52:54:00:9c:18:b1  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0-nic: flags=4098<BROADCAST,MULTICAST>  mtu 1500
        ether 52:54:00:9c:18:b1  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
ELOG V3.1.4-unknown