FahMon 2.2.2

uncle_fungus · 7 Jul 2007 at 02:44

FahMon 2.2.2 is now out!

This is mainly a bugfix release to (hopefully) remove some of the problems people encountered with previous versions.

Here's the complete list of changes:

Added preliminary detection for hung clients
Fixed bug which prevented reloading selected client data after being restored from the system tray
Fixed regression which caused inaccessible clients to report an incorrect state on FahMon launch
Fixed regression that often caused an incorrect xyz file to be saved when experimental reload system was activated
Fixed (longstanding) major issue with state detection and timezone conversion which caused clients to have an unknown state when the local timezone has passed 00:00 and the client hasn't.
Fixed bug which prevented you from running FahMon in "start minimised" mode without "enable tray icon" being set
Fixed another bug relating to window size not restoring correctly. If the frame is ever detected to be 0x0, default values are used instead
Client list colour scheme now follows system on both Windows and Linux
Made more code compatible with newer versions of wxWidgets (Win32)

Hung client notification kicks in when the current frame is > 5x the avg frame time, inactive notification kicks in at 3x the avg.

The major showstopper bug that has been fixed now means your clients won't get stuck in the yellow/orange state for extended periods of time. The states should now work correctly and stay green for active clients.

The alternating background colours in the client list should follow your system colour scheme, although if you change it whilst FahMon is running, the colours won't update until the client list itself is updated.

Snapshot · 7 Jul 2007 at 06:42

Downloaded and installed. Okay so far. Will try it on the school server as that has a huge list of inaccessible clients at the moment.

A quick install and test on the server shows that the minimise/restore bug is well and truly squashed.

SiriusB · 7 Jul 2007 at 08:05

Will install this later and let you know how it goes.

Keep up the good work.

jaric · 7 Jul 2007 at 10:33

Both my Linux SMP client are reporting as hung even thought they are both working OK.

[07/07/07 - 10:21:02] Reloading core1
[07/07/07 - 10:21:02] Project number found for core2 from FAHlog.txt: 3404
[07/07/07 - 10:21:02] Reloading Ubuntu Server 1
[07/07/07 - 10:21:02] Reloading server
[07/07/07 - 10:21:02] Project number found for core1 from FAHlog.txt: 1170
[07/07/07 - 10:21:02] core2 is stopped (The line "Folding@Home Client Shutdown." was found)
[07/07/07 - 10:21:02] Reloading Ubuntu Server 2
[07/07/07 - 10:21:02] core1 is stopped (The line "Folding@Home Client Shutdown." was found)
[07/07/07 - 10:21:03] Project number found for server from FAHlog.txt: 3402
[07/07/07 - 10:21:03] server is on frame 19
[07/07/07 - 10:21:03] server [ETA] | avg=00:42:12 | ref=00:42:15 | adj=00:42:15 | left=56:20:00
[07/07/07 - 10:21:03] Project number found for Ubuntu Server 2 from FAHlog.txt: 2604
[07/07/07 - 10:21:03] Ubuntu Server 2 is on frame 11
[07/07/07 - 10:21:03] Ubuntu Server 2 [ETA] | avg=00:21:13 | ref=00:26:57 | adj=00:26:57 | left=39:31:36
[07/07/07 - 10:21:03] ! Ubuntu Server 2 seems to have hung : Elapsed time is 1367m and limit is 127m
[07/07/07 - 10:21:03] Project number found for Ubuntu Server 1 from FAHlog.txt: 2609
[07/07/07 - 10:21:03] Ubuntu Server 1 is on frame 84
[07/07/07 - 10:21:03] Ubuntu Server 1 [ETA] | avg=00:27:12 | ref=00:26:21 | adj=00:26:21 | left=06:35:15
[07/07/07 - 10:21:03] ! Ubuntu Server 1 seems to have hung : Elapsed time is 1371m and limit is 163m

I think the problem may be that the clock on the Linux VM is out of sync with the Windows host.

Edit : Just set the timezone in FahMon to UTC + 22 & it's now reporting the clients correctly.

uncle_fungus · 7 Jul 2007 at 11:45

jaric said:
Both my Linux SMP client are reporting as hung even thought they are both working OK.

I think the problem may be that the clock on the Linux VM is out of sync with the Windows host.

Edit : Just set the timezone in FahMon to UTC + 22 & it's now reporting the clients correctly.

You shouldn't have to do that, since now FahMon thinks it's in the wrong timezone.

Try resetting the timezone, and enabling the "ignore asynchronous clocks" option, the clients should now go blue instead of red.
If that doesn't work, I probably have to look at the interaction between the two features as they don't seem to be working quite right. The funny thing is, my SMP machine has an asynch clock and doesn't report itself as hung...

jaric · 7 Jul 2007 at 18:09

uncle_fungus said:
You shouldn't have to do that, since now FahMon thinks it's in the wrong timezone.

Try resetting the timezone, and enabling the "ignore asynchronous clocks" option, the clients should now go blue instead of red.
If that doesn't work, I probably have to look at the interaction between the two features as they don't seem to be working quite right. The funny thing is, my SMP machine has an asynch clock and doesn't report itself as hung...

Set timezone back to normal & "ignore asynchronous clocks" was already enabled. Clients are now showing as hung again.

uncle_fungus · 7 Jul 2007 at 20:21

jaric said:
Set timezone back to normal & "ignore asynchronous clocks" was already enabled. Clients are now showing as hung again.

Ok, given the above, and what people on FCF have also been reporting, it looks like the priorities for hung clients and asynch clients need adjusting such that asynch overrides hung.

A quick question though, can you confirm that all your clients are reporting the correct time in UTC when they write to FAHlog.txt?

shadowscotland · 7 Jul 2007 at 21:38

nice to see you working hard UF

Keep up the good work and we'll continue to be your most loyal ginnipigs

jaric · 7 Jul 2007 at 22:38

uncle_fungus said:
A quick question though, can you confirm that all your clients are reporting the correct time in UTC when they write to FAHlog.txt?

The clients are reporting in UTC but the VM clock is about 90 mins ahead of the host machine.

uncle_fungus · 7 Jul 2007 at 22:48

jaric said:
The clients are reporting in UTC but the VM clock is about 90 mins ahead of the host machine.

Ah, well that's why then. If the clock is that far out of sync it's bound to cause problems

.
Adjusting the state priorities may help, but with Vmware's somewhat dodgy timekeeping you can never guarantee accurate results. I think the best you can hope for is a blue async state for them in the next version.

Mattus · 14 Jul 2007 at 22:23

I'm having similar problems, especially when the VM clock is out of sync with the host machine.

Could FahMon possibly have an option to monitor frame times irrespective of the times in FAH's log file?

ie. Say a frame is completed at 22:20. The logs report that it's 15:00, but that's irrelevant since FahMon has internally logged the frame at 22:20. Then, when the next frame completes at 22:35, FahMon can see that all is OK even though the logs are out.

uncle_fungus · 15 Jul 2007 at 01:03

Mattus said:
I'm having similar problems, especially when the VM clock is out of sync with the host machine.

Could FahMon possibly have an option to monitor frame times irrespective of the times in FAH's log file?

ie. Say a frame is completed at 22:20. The logs report that it's 15:00, but that's irrelevant since FahMon has internally logged the frame at 22:20. Then, when the next frame completes at 22:35, FahMon can see that all is OK even though the logs are out.

The simplest answer to that question would be no.
FahMon doesn't even do any internal logging as it stands now, it relies on reading the timestamps in the log. Altering that behaviour would require a major rewrite of the log parsing and processing system.
FahMon just wasn't designed to produce accurate results when the clocks of monitored machines are so badly out of sync with "real time", which IMO is not unreasonable.

2.2.3 should have the hung client problem fixed, but you're still going to have to enable the "ignore asynchronous clocks" option, albeit your PPD and ETA values will still be wrong because of the slowdown/speedup that's causing the asynchrony.

jaric · 15 Jul 2007 at 10:20

Just as an update. I installed ntpd on the VM to try to keep the clocks in sync and FahMon has been fine since.

Thanks for your help uncle_fungus.