CPU/Memory Problem after two days, process shutdown

[minersday]

The CPU hogging dashd thread is, please fix it, if possible:
from top -H:
21921 xxxxx 22 2 908200 179908 36092 R 58.1 8.8 49:39.70 dash-msghand

getting messages like this, continious flood, like screen is scrolling as fast as it can..
strace -p 21921:
endto(39, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(31, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(35, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(36, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(37, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(33, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(20, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(26, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(24, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(28, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(30, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(32, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(21, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(23, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(29, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(27, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(39, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(31, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(35, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(36, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(37, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(33, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(20, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(26, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(24, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(28, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(30, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(32, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(21, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(23, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(29, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(27, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(39, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140
sendto(31, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140

---

minimal hogging:
futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140475, {1439662667, 560970182}, fffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140477, {1439662667, 562308164}, fffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140479, {1439662667, 563645382}, fffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140481, {1439662667, 565038070}, fffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140483, {1439662667, 566340425}, fffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140485, {1439662667, 567818400}, fffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140487, {1439662667, 569213288}, fffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140489, {1439662667, 570554175}, fffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140491, {1439662667, 571893022}, fffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140493, {1439662667, 573248748}, fffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140495, {1439662667, 574590053}, fffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140497, {1439662667, 575932126}, fffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0

---

Don't know if it's from BOOST_FOREACH doing the dseep looping, maybe wait/sleep has to be in there?

[minersday]

Issue fixed with version ..45

[darkcoinproject]

Perfect thanks

On Sun, Aug 16, 2015 at 1:33 PM, minersday notifications@github.com wrote:

Issue fixed with version ..45

—
Reply to this email directly or view it on GitHub
#535 (comment).

[minersday]

There's something happening after two day's in 0.45 release, it seems CPU / Memory goes up and down like a chainsaw..

spamming the same thing connection timed out.

[minersday]

[minersday]

Spamming the (Connection timed out), in the dash-msghand process thread.
And memory usage seems to be going up.

[eduffield222]

Do you have a "connect=" in your configuration?

[minersday]

Nope, in any of the dash.conf:s.

[minersday]

People having daemon shutdown, could be because of the memory/CPU erratic use after two days. Could trigger daemon shutdown in hosting services.
I don't have that problem, hosting my own instances..

[minersday]

Could this be the problem?
http://stackoverflow.com/questions/28320969/futex-return-etimedout-quicker-than-the-parameter-timeout

all of masternodes are configured to keep the time with ntpd, and in VM-hosting enviroments, there are clock cycle skips, because of the shared enviroment. And ntpd needs to adjust time.

[minersday]

Changed the clocksource in VM to acpi_pm:
echo acpi_pm > /sys/devices/system/clocksource/clocksource0/current_clocksource

now getting mostly spammed in strace with:
gettimeofday({1439921834, 663640}, NULL) = 0
gettimeofday({1439921834, 663765}, NULL) = 0
gettimeofday({1439921834, 663841}, NULL) = 0

[minersday]

With .046
CPU/MEM/Disk/Network usage, seems to gone back down to normal levels and not going up and down rapidly.