OVH Community, your new community space.

Server Lagging


Myatu
20-03-2011, 12:27
Hmm. Can you show the full output of smartctl -a ?

Also, there seems to be a regression bug in the latest kernels that causes a similar problem as you've described. One work around appears to be using the kernel option libata.force=noncq. Just do a lookup on "kernel ncq" for related info, and see if it applies.
zemon
20-03-2011, 08:56
This is top results

Code:
top - 09:49:09 up 25 min,  1 user,  load average: 7.44, 8.07, 6.44
Tasks: 141 total,   1 running, 133 sleeping,   7 stopped,   0 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni, 14.2%id, 85.8%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   4008008k total,  1209028k used,  2798980k free,     8808k buffers
Swap:  2095096k total,        0k used,  2095096k free,   909404k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    7 root      20   0     0    0    0 S    0  0.0   0:00.02 events/0
    1 root      20   0 23740 1952 1296 S    0  0.0   0:00.89 init
    2 root      20   0     0    0    0 S    0  0.0   0:00.00 kthreadd
    3 root      RT   0     0    0    0 S    0  0.0   0:00.00 migration/0
    4 root      20   0     0    0    0 S    0  0.0   0:00.00 ksoftirqd/0
    5 root      RT   0     0    0    0 S    0  0.0   0:00.00 migration/1
    6 root      20   0     0    0    0 S    0  0.0   0:00.12 ksoftirqd/1
    8 root      20   0     0    0    0 S    0  0.0   0:00.04 events/1
    9 root      20   0     0    0    0 S    0  0.0   0:00.00 cpuset
   10 root      20   0     0    0    0 S    0  0.0   0:00.01 khelper
   13 root      20   0     0    0    0 S    0  0.0   0:00.00 netns
   16 root      20   0     0    0    0 S    0  0.0   0:00.00 async/mgr
  259 root      20   0     0    0    0 S    0  0.0   0:00.00 sync_supers
  261 root      20   0     0    0    0 S    0  0.0   0:00.00 bdi-default
  262 root      20   0     0    0    0 S    0  0.0   0:00.00 kintegrityd/0
  263 root      20   0     0    0    0 S    0  0.0   0:00.00 kintegrityd/1
  265 root      20   0     0    0    0 S    0  0.0   0:00.00 kblockd/0
  266 root      20   0     0    0    0 S    0  0.0   0:00.00 kblockd/1
  267 root      20   0     0    0    0 S    0  0.0   0:00.00 kacpid
  268 root      20   0     0    0    0 S    0  0.0   0:00.00 kacpi_notify
Iotop

Code:
Total DISK READ: 0.00 B/s | Total DISK WRITE: 0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
    1 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % init
    2 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kthreadd]
    3 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/0]
    4 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [ksoftirqd/0]
    5 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/1]
    6 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [ksoftirqd/1]
    7 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [events/0]
    8 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [events/1]
    9 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [cpuset]
   10 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [khelper]
   13 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [netns]
   16 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [async/mgr]
 3119 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % -bash
 3167 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % vnstatd -d
 2672 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [flush-9:1]
 3243 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % top
  370 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [ata_aux]
 2739 be/4 ntp         0.00 B/s    0.00 B/s  0.00 %  0.00 % ntpd -p /var/run/ntpd.pid -g -u 104:111
 3258 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % getty -8 38400 tty1
 1211 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [iscsi_eh]
 1220 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [fc_exch_workque]
 1221 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [fc_rport_eq]
 1222 be/0 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [fcoethread/0]
 1223 be/0 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [fcoethread/1]
 3274 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % nano /etc/fstab
 2769 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [flush-9:3]
 1235 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [scsi_eh_0]
 2773 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kjournald]
 1238 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [scsi_eh_1]
 3300 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % vnstat -l
 3301 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % python /usr/bin/iotop
 2278 be/3 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % udevd --daemon
 2279 be/3 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % udevd --daemon
 1256 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [mtdblockd]
 2808 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % getty -8 38400 tty4
 3322 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % python /usr/bin/iotop
 2811 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % getty -8 38400 tty5
 3327 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % python /usr/bin/iotop
 2818 be/4 syslog      0.00 B/s    0.00 B/s  0.00 %  0.00 % rsyslogd -c4
  259 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [sync_supers]
 3332 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % python /usr/bin/iotop
  261 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [bdi-default]
  262 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kintegrityd/0]
  263 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kintegrityd/1]
  265 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kblockd/0]
  266 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kblockd/1]
  267 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kacpid]
  268 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kacpi_notify]
  269 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kacpi_hotplug]
 3345 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % top
 3346 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % python /usr/bin/iotop
 2835 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % getty -8 38400 tty2
 2836 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % getty -8 38400 tty3
 2838 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % getty -8 38400 tty6
 2839 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % sshd -D
 1316 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kpsmoused]
 3377 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % top
 2860 be/4 bind        0.00 B/s    0.00 B/s  0.00 %  0.00 % named -u bind
 2861 be/4 bind        0.00 B/s    0.00 B/s  0.00 %  0.00 % named -u bind
zemon
20-03-2011, 08:46
I run smart tools but no errors were reported.
Winit
19-03-2011, 20:46
Install smartmontools and run smartctl against the drive.
zemon
19-03-2011, 16:19
Can anyone point me in the right direction to correct this problem.

The server is really lagging because of the hard drive.

This is taken from my logs and this same message is repeated every minute.

Code:
Mar 19 01:47:59 ns380846 kernel: ata1: link is slow to respond, please be patient (ready=0)
Mar 19 01:48:04 ns380846 kernel: ata1: device not ready (errno=-16), forcing hardreset
Mar 19 01:48:04 ns380846 kernel: ata1: soft resetting link
Mar 19 01:48:09 ns380846 kernel: ata1: link is slow to respond, please be patient (ready=0)
Mar 19 01:48:14 ns380846 kernel: ata1: soft resetting link
Mar 19 01:48:19 ns380846 kernel: ata1: link is slow to respond, please be patient (ready=0)
Mar 19 01:48:22 ns380846 kernel: ata1.00: configured for UDMA/100
Mar 19 01:48:22 ns380846 kernel: ata1.00: device reported invalid CHS sector 0
Mar 19 01:48:22 ns380846 kernel: ata1: EH complete