Weird BOINC Issue – 385 days old

[ZemaTalon]

I've been seeing this on one machine, I haven't had a chance to really investigate it much, & I didn't find much relevant info on Google. This is on a Linux Mint 12 machine running KDE. It's in another room and I only check on it a few times a week, but every now and then (maybe once every 5-14 days) I'll find that BOINC has stopped crunching and it reports that it's not connected to the client - and that it's not connected to localhost. When I try to get it going again it indicates that it's trying to connect o localhost, which always fails. Completely shutting down BOINC manager and clients makes no difference. Usually by the time I'm checking up on it there are some Mint updates so I install those and reboot, after which BOINC works perfectly fine...until the next time. I've never run into this before, anyone have any clues what might be going on here?

[Dark Angel]

It's not hibernating or something silly is it?

Check the stderr file when it happens and see if there's anything there. Alternatively start it from the command line and leave the terminal open, then you'll see any messages that come up and might catch the cause.

[ZemaTalon]

Thanks DA, it doesn't seem to be hibernating. I'll give those a try next time

[ZemaTalon]

Happened again, I didn't find an stderr file, but there is an stderrdae.txt file which I guess is probably the one? But it doesn't look like it's been written to since March. Anyway here's what's inside it:

SIGSEGV: segmentation violation
Stack trace (14 frames):
/home/zema/BOINC/boinc(boinc_catch_signal+0x65)[0x499545]
/lib/x86_64-linux-gnu/libc.so.6(+0x36420)[0x7feab4a05420]
/lib/x86_64-linux-gnu/libssl.so.1.0.0(+0x20867)[0x7feab3cc9867]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(+0x269a5)[0x7feab5d889a5]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(Curl_ssl_connect_nonblocking+0x28)[0x7feab5d9e478]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(+0x1258e)[0x7feab5d7458e]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(Curl_protocol_connect+0x8a)[0x7feab5d852ea]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(+0x36b88)[0x7feab5d98b88]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(curl_multi_perform+0xb5)[0x7feab5d99035]
/home/zema/BOINC/boinc[0x47e0ff]
/home/zema/BOINC/boinc[0x43e6f7]
/home/zema/BOINC/boinc[0x4826be]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7feab49f030d]
/home/zema/BOINC/boinc[0x42e1e9]

Exiting...
SIGSEGV: segmentation violation
Stack trace (14 frames):
/home/zema/BOINC/boinc(boinc_catch_signal+0x65)[0x499545]
/lib/x86_64-linux-gnu/libc.so.6(+0x36420)[0x7f2442047420]
/lib/x86_64-linux-gnu/libssl.so.1.0.0(+0x20867)[0x7f244130b867]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(+0x269a5)[0x7f24433ca9a5]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(Curl_ssl_connect_nonblocking+0x28)[0x7f24433e0478]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(+0x1258e)[0x7f24433b658e]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(Curl_protocol_connect+0x8a)[0x7f24433c72ea]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(+0x36b88)[0x7f24433dab88]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(curl_multi_perform+0xb5)[0x7f24433db035]
/home/zema/BOINC/boinc[0x47e0ff]
/home/zema/BOINC/boinc[0x43e6f7]
/home/zema/BOINC/boinc[0x4826be]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7f244203230d]
/home/zema/BOINC/boinc[0x42e1e9]

Exiting...
SIGSEGV: segmentation violation
Stack trace (14 frames):
/home/zema/BOINC/boinc(boinc_catch_signal+0x65)[0x499545]
/lib/x86_64-linux-gnu/libc.so.6(+0x36420)[0x7f6e2164c420]
/lib/x86_64-linux-gnu/libssl.so.1.0.0(+0x20867)[0x7f6e20910867]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(+0x269a5)[0x7f6e229cf9a5]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(Curl_ssl_connect_nonblocking+0x28)[0x7f6e229e5478]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(+0x1258e)[0x7f6e229bb58e]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(Curl_protocol_connect+0x8a)[0x7f6e229cc2ea]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(+0x36b88)[0x7f6e229dfb88]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(curl_multi_perform+0xb5)[0x7f6e229e0035]
/home/zema/BOINC/boinc[0x47e0ff]
/home/zema/BOINC/boinc[0x43e6f7]
/home/zema/BOINC/boinc[0x4826be]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7f6e2163730d]
/home/zema/BOINC/boinc[0x42e1e9]

Exiting...

[ZemaTalon]

Relaunching it from a terminal, and it says:

Initialization completed
20-May-2012 14:39:07 [---] Running CPU benchmarks
20-May-2012 14:39:07 [---] Suspending computation - CPU benchmarks in progress
20-May-2012 14:39:07 [World Community Grid] Sending scheduler request: Project initialization.
20-May-2012 14:39:07 [World Community Grid] Requesting new tasks for CPU and NVIDIA GPU
20-May-2012 14:39:38 [---] Benchmark results:
20-May-2012 14:39:38 [---] Number of CPUs: 8
20-May-2012 14:39:38 [---] 2400 floating point MIPS (Whetstone) per CPU
20-May-2012 14:39:38 [---] 9213 integer MIPS (Dhrystone) per CPU
20-May-2012 14:41:09 [---] Project communication failed: attempting access to reference site
20-May-2012 14:41:09 [World Community Grid] Scheduler request failed: Timeout was reached
20-May-2012 14:41:10 [---] Internet access OK - project servers may be temporarily down.
20-May-2012 14:42:34 [World Community Grid] Sending scheduler request: Project initialization.
20-May-2012 14:42:34 [World Community Grid] Requesting new tasks for CPU and NVIDIA GPU
SIGSEGV: segmentation violation
Stack trace (14 frames):
./boinc(boinc_catch_signal+0x65)[0x499545]
/lib/x86_64-linux-gnu/libc.so.6(+0x36420)[0x7f00751eb420]
/lib/x86_64-linux-gnu/libssl.so.1.0.0(+0x20867)[0x7f00744af867]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(+0x269a5)[0x7f00765709a5]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(Curl_ssl_connect_nonblocking+0x28)[0x7f0076586478]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(+0x1258e)[0x7f007655c58e]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(Curl_protocol_connect+0x8a)[0x7f007656d2ea]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(+0x36b88)[0x7f0076580b88]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(curl_multi_perform+0xb5)[0x7f0076581035]
./boinc[0x47e0ff]
./boinc[0x43e6f7]
./boinc[0x4826be]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7f00751d630d]
./boinc[0x42e1e9]

Exiting...
zema@zema3 ~/BOINC $

So, somethin screwie

...and as usual, rebooting the machine always fixes it.

[Dark Angel]

A bit of hunting from that SIGSEGV: segmentation violation part suggests it's got something to do with either an error from the graphics card or the driver. Does the same error happen if you disable the GPU in the BOINC settings?

[ZemaTalon]

Just now disabled it, I'll wait an see if that takes care of it

[Grogan]

No, that error is the application crashing because of something to do with libcurl or the way it's compiled. It has nothing to do with graphics drivers.

You probably have an unexpected version of libcurl. That's the problem with binary software... if that would have been compiled, it would have failed to compile with a version of libcurl that it doesn't like, and the package builder/distributor would know it.

Curl is a library that provides an interface for URL based internet transfer protocols (mostly http and https, but also ftp and others).

The segmentation fault is just a typical error... the one the kernel catches and then it kills the program. That in itself is not at all diagnostic, it is the stack trace that is more meaningful. It basically means that an application or process tried to access a memory address that is either invalid, or not allocated to the application or process. Usually a symptom of a malfunction rather than a cause.

[Dark Angel]

Ok then, we'll try again. Zema, what version of BOINC are you running? You appear to have libssl-1.0.0 so either you've tinkered with things or you're running a latest crop distro. If your BOINC client is too old you might have issues (we're up to v7.0.25 from Berkley and I believe they compile it on Ubuntu so the repo package should work without a hitch).

[ZemaTalon]

Looks like 6.12.34 - I got it from Berkley earlier in the year. This isn't my main system like I said above, just a secondary machine I don't check on too often, but it's running Mint 12 KDE final with all updates. I'll try a newer client

[ZemaTalon]

Got 7.0.25 up and running, I checked the Ubuntu archeives and they're still at 6.12.34 so I got it from Berkley again

[Dark Angel]

Ubuntu 12.04 is serving BOINC 7.0.24, you must be using an earlier release/repos.

[ZemaTalon]

I wonder if it has to do with this install starting out as a RC?

[Dark Angel]

There could be some inconsistencies causing problems, it's hard to tell.

[ZemaTalon]

Well despite uninstalling the old BOINC and installing the latest release, this problem continues. Even after I kill every trace of BOINC I can find in the tree, if I try to just launch the client it says BOINC is already running, and that there's no config file. And as before, if I try to launch the manager, it can't connect to localhost. So after scouring the process tree and finding nothing remotely BOINC-related to kill, the only thing that fixes it is still to reboot. This is the same Mint version and same version of BOINC I have running without any problems on my main machine, so based on that and Gro's analysis maybe some time in memtest might be in order.

[Grogan]

No, I said it was a software issue, not problems with your physical memory (though I suppose it could be and it certainly wouldn't hurt to run memtest)

A segmentation fault is a typical program crash in unix. The stack trace indicates that it's having a problem with your libcurl, or at least that's where the fault is occurring.

[Bill J]

Having the same exact problems with 2 machines. Both are home built I7 2600 CPUs running MINT 12 which I check for updates appx once per week. both are using Boinc 6.12.33 which was loaded with the software manager. I used to update to the lates BOINC verson immediatly but had a bit of trouble starting with 7.25, so now I wait.

Problem is that somewhere between 2 and 4 weeks of 24/7 trouble free crunching, I find the the client has disconnected. I've tried everthing I know to re-attach but the only thing that works it a reboot. The Reboot always works. I have four Win 7 machines and 1 Mint 10 that never have this problem. This is more than a minor annoyance because one of the machines is at a remote location that I access with TEAM VIEWER. If I reboot, the machine is lost to me for weeks because i lose my connection with team viewer in the reboot.

I'll admit up front that I one of the guys that use LINUX because it is free, not because I dislike windows (well, except for VISTA).

Advice appriciated.

Thanks

B2I

[ZemaTalon]

Hi Bill, and welcome to the forum! Well sadly I never did find a solution to that frustrating problem. I ended up reworking that machine recently, turning it into a work machine, but before I made any changes I attempted to run memtest on it, but memtest refused to run. I forget the error it threw now, that was back in June. Whether there was really any problem with the memory I don't know - I read about some long-standing issue with memtest producing the kind of error I was seeing so it may have been a coincidence, but it was suspicious. To get it ready to use for work I upgraded the memory and replaced Mint12 KDE with Mint 13 XFCE. And with preparation for work taking precedence the last few weeks I haven't yet gotten around to reinstalling BOINC on it. I'll be doing that this week and then see if the problem is finally gone.

[Grogan]

My money is on being related to the libcurl in that distro. It may not happen in Mint 10, or 13 etc.

No, it may not happen consistently on all hardware either and wouldn't necessarily be an obvious problem that the right people would notice.

[Dark Angel]

The only consistent issue I've had with any of the BOINC series since 6.12.x and continuing with the 7.0.x series is that Berkley insist on linking to wxwidgets 2.8.11 while most newer distros are using 2.8.12 The result being that in the Advanced view the top menu (File View etc) is flakey and often doesn't display. This can be worked around temporarily by pressing Alt+v and switching to simple view and then switching back to advanced view.

[Troy]

Welcome to the forum, Bill J.