Folding Forum

by **Judas** » Thu Jul 26, 2012 5:16 pm

Many people may have noticed that there seems to be plenty of threads regarding the issue of the ATI GPUs refusing to "fold" for no apparent reason, nothing seems to solve the problem.

After an excessive amount of testing and toying and failing to complete WU's unfortunately due to starting them, testing and finding them either unable to actually start at all on them ever regardless of how long you wait for it to progress, i've determined the only reasonable explanation.

Running an Nvidia GPU along side an ATI GPU folding results in the nvidia taking priority and killing any folding being doen on the ATI gpus.

Worst yet, If an Nvidia GPU is ever present in the machine, removing it and then attempting to fold on the ati GPU has a HIGH probability of failing to work. That's right, if for example you are running lets say an ATI GPU right now, and folding along just fine, installing an nvidia gpu will kill it, removing it and all evidence of it's drivers will result in the ATI GPU failing to be able to fold still.

Having spent plenty of time trying to figure it out, i can only conclude thus far that reminents of the nvidia driver or even physx/cuda/opencl of sorts conflicts with ATI's and all the various forms of cleaning even a System Restore to a date prior to installing an nvidia gpu STILL results in the ATI gpu refusing to fold.

I've setup multiple test machines with different configurations and OSes, all produce the same results. It doesn't matter if the nvidia gpu was primary or secondary or even the 3rd or 4th GPU in the system, Once the nvidia gpu is there, it kills all the other ATI GPUs.

You can run 4 ATI gpu's just fine together.

you can run 4 nvidia GPU just fine together

Mixing nvidia and ati, only the nvidia gpus will fold.

Waiting a period of 1 week (which is well past the ati folding deadline) for a 5450, 5770, a 6850, 6950, 6990, 7750, and a 7970, and numerous models between stays @ 0% running state indefinitely, never any progress made.

This is with a range of CPUs and motherboard configurations as well, Even disabling SMP completely in order to ensure the ATI Folding (which seems to always require a steady use of a single CPU thread/Core otherwise folding is slowed dramatically on ATI cards)

The only way i was able to restore GPU folding on the ATI cards was to do a full format/Clean install of windows. My initial attempt at a Repair install resulted in the GPUS still unable to fold.

the Video cards however do work fine otherwise, there are no crashes, no 3D issues, video playback, etc while a monitor is attached to these ATI cards. The only issue is SPECIFIC to Folding @ Home.

Once a full reinstall of windows was completed, GPU folding confirmed working fine on the ATI cards, an nvidia card was reinserted into the machine, and upon completeing the driver installation of the card, GPU folding on the ATI cards was emediately halted and GPU activity dropped to 0%. A full format/reinstall was again required to get the cards funtioning.

I've discussed this with a few others that have complained about this same problem and pretty much confirmed the exact same experience.

For referrence, only vista/win7/win8 machines were tested.

minimal specs of the various machines include

Intel 3930K CPU with Asus P9X79 PRO
Intel 3770K CPU with Asus P8Z77-V PRO
Intel i3 2120 CPU with Asus P8H67
intel i3 2120 cpu with Asus P8Z77-LK
intel P4 Northwood CPU with MSI intel 945 chipset
AMD Phenom II x4 940 with Gigabyte Board (AMD chipset)
AMD 4400+ x2 with Asus A8R32-MVP (ATI/AMD Chipset)
AMD 3700+ with Asus A8N-E (Nforce 4 Ultra chipset)
and more

Additionally the nvidia video cards i've tested include everything from the nvidia 9xxx series (9600gt/9800/9400) up through to the 285,540,560,680 series.

All boards support 2 or more PCI-ex 16x Video cards, with more than sufficient Power Supplies as well as either win7/win8 and all are up to date with the latest drivers as well tested with older known to work drivers to ensure new drivers aren't creating the problem.

Sorry for the numerous failed WU due to attempting to find the problem.

I was going to report this issue nearly a month ago, but it took over a month for me to finally receive my activation email. I suggest an ADMIN look into this system as i know about 5 others that have registered but have no received the email activation even after resubmitting for it several times like me. Mine only arrived due to getting an email notification that my account still wasn't active.

by **7im** » Thu Jul 26, 2012 6:00 pm

Next time try a forum search before wasting all of those work units. How to do the setup with mixed cards is already detailed here in the forum: http://foldingforum.org/viewtopic.php?f=67&t=19989&p=199379#p199379 and works quite well.

by **Judas** » Thu Jul 26, 2012 6:55 pm

not at all wasting anything.

Using that method in that thread results in no change being applied... the ATI gpus continue to "run" yet will not fold, 0% activity, no load, no progress ever made.

As i pretty much stated in the initial thread post... i've exhausted all solutions presented on the forums and other 3rd party faqs/resources.

by **jimerickson** » Thu Jul 26, 2012 6:57 pm

should have used a "captured" WU.

by **bruce** » Thu Jul 26, 2012 7:20 pm

Judas wrote:not at all wasting anything.

Every WU which is downloaded and returned with an error delays the science so there's a cost associated with it. If you want to test something, do it in a way that minimizes the consumption of good WUs from the servers.

by **7im** » Thu Jul 26, 2012 8:13 pm

Judas wrote:not at all wasting anything.

Using that method in that thread results in no change being applied... the ATI gpus continue to "run" yet will not fold, 0% activity, no load, no progress ever made.

As i pretty much stated in the initial thread post... i've exhausted all solutions presented on the forums and other 3rd party faqs/resources.

It didn't work for you... yet. It has worked well with others.

There are certain driver minimums to meet, etc., and other potential issues that can prevent this from working, so apparently you missed one.

Now that you have access to the forum, let's work through the problem, and find out what you are doing that is different from other people, then fix that difference so it does work.

by **Judas** » Thu Jul 26, 2012 8:21 pm

bruce wrote:
Judas wrote:not at all wasting anything.

Every WU which is downloaded and returned with an error delays the science so there's a cost associated with it. If you want to test something, do it in a way that minimizes the consumption of good WUs from the servers.

I'm aware of that and i had no intention of wasting WUs.... actually trying to avoid it however deleteing my post asking about "captured" WU doesn't help the situation as had it been explained would have presented a potential solution to unintentional WU wasting. The only real good thing about unfortunate WU failures on the ATI GPUs is there relatively SHORT lifespan unlike cpus that can have almost 2 month deadlines. At which point the server would recognize it didn't complete and resupply it as a valid wu to be worked on.

The little bit i did look up on captured work units however would present a potential problem as in order to eliminate all potential causes of the failed gpu folding, a new WU may have been required to make certain in the end anyways.

by **Judas** » Thu Jul 26, 2012 8:26 pm

7im wrote:It didn't work for you... yet. It has worked well with others.

There are certain driver minimums to meet, etc., and other potential issues that can prevent this from working, so apparently you missed one.

Now that you have access to the forum, let's work through the problem, and find out what you are doing that is different from other people, then fix that difference so it does work.

A good place to start would be an indication as to why when the nvidia cards are removed why F@H refuses to fold on the single video card still left. Leaves very little left to determine a potential fix...

Actually i'm quite interested in why a work GPU installed and identified as slot 00, works, then a nvidia gpu installed showing as slot 01 kills slot 00 progress... when nvidia gpu is removed, the slot 00 no longer proceeds.

manually forcing openCL on slot 00 changes nothing, a forced reset of all folding information/deletion and reinstall continues to change nothing even though the nvidia gpu is not present.

This raises a few questions.

by **bruce** » Thu Jul 26, 2012 8:57 pm

A WU which is assigned to CUDA device cannot be folded by an OpenCL device and vice-versa, When you change slot numbers or when you rearrange the types of GPUs that you have, you stand a good chance of causing a core to hang. V7 does not account for unexpected changes, and V7.1.52 can assign them incorrectly due to differing methods of enumeration. These are discussed in open bug tickets and are being worked on by Development.

The URL mentioned above points to a systematic method of working around the current limitations. I don't see any useful conclusions from your semi-random testing that are not already documented. "Do not run..." isn't a useful conclusion because I'm quite sure with some help, we could get them to work.

by **Judas** » Thu Jul 26, 2012 10:16 pm

I've already attempted to force a openCL WU to run on the openCL device.... confirming the project numbers and information pertaining to them that they are indeed openCL and matching.

further more a test in which a WU was already started and working, would hang once an nvidia gpu was installed, and confirming that the WU was running on the correct device.

Having attempted to manual setup each individual device, the same results occur, a failure to continue. You may be surprised how large of a population that have issued with F@H never report or post here.

by **uncle_fungus** » Fri Jul 27, 2012 7:13 am

Judas wrote:I was going to report this issue nearly a month ago, but it took over a month for me to finally receive my activation email. I suggest an ADMIN look into this system as i know about 5 others that have registered but have no received the email activation even after resubmitting for it several times like me. Mine only arrived due to getting an email notification that my account still wasn't active.

I apologise for this inconvenience. There was a server email problem which was out of our control resulting in many emails being rejected at the receiving end. This has now been fixed and I sent out reminder emails to everyone who had registered but not activated.

In response to your post topic, do you have a log showing the problem occurring? If so please post it here as it might help us deduce the problem and therefore help you work towards a solution.

by **7im** » Fri Jul 27, 2012 7:24 am

One of the latest CAT drivers doesn't work with a lot of cards to do OpenCL. Using that version to test would have been pointless.
And 30x.xx or newer is strongly recommended for late model NV cards to fold without problems. And only certain past versions worked for folding. 285xx, 296xx, etc.

Please be more specific with what GPU driver versions were used for your testing.

by **Judas** » Fri Jul 27, 2012 4:12 pm

I'm trying to compile a set of logs in order to post here, However between recent work and personal things, i've been kinda swamped..

I could however post the one machine i haven't fully formated yet with just a 6850 installed with the 12.6 catalyst drivers that refuses to fold AFTER having an nvidia card installed and then removed.

However i don't have a captured WU.... if i add it back it'll automatically download a new wu... This is however my current test rig as i really don't want to fowl up the other machines at this time as they are used for work.

All the latest drivers are in use, my initial experience/start of folding was started on June 12th 2012, As of that specific date, all drivers would be up to date prior to installing folding v7 client.

I will however provide a quick shot of the log when it attempts to run and never changes.. only the CPU continues to proceed.... gpu can run a week solid without ever completing a single percentage..

Code: Select all: 16:15:24:FS02:Unpaused 16:15:24:FS00:Unpaused 16:15:24:WU00:FS02:Starting 16:15:24:WARNING:WU00:FS02:Changed SMP threads from 8 to 6 this can cause some work units to fail 16:15:24:WU00:FS02:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 00 -suffix 01 -version 701 -lifeline 3352 -checkpoint 15 -np 6 16:15:24:WU00:FS02:Started FahCore on PID 7748 16:15:24:WU00:FS02:Core PID:6280 16:15:24:WU00:FS02:FahCore 0xa4 started 16:15:24:WU01:FS00:Starting 16:15:24:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/ATI/R600/Core_16.fah/FahCore_16.exe -dir 01 -suffix 01 -version 701 -lifeline 3352 -checkpoint 15 -gpu 0 16:15:24:WU01:FS00:Started FahCore on PID 7912 16:15:24:WU01:FS00:Core PID:7060 16:15:24:WU01:FS00:FahCore 0x16 started 16:15:24:WU00:FS02:0xa4: 16:15:24:WU00:FS02:0xa4:*------------------------------* 16:15:24:WU00:FS02:0xa4:Folding@Home Gromacs GB Core 16:15:24:WU00:FS02:0xa4:Version 2.27 (Dec. 15, 2010) 16:15:24:WU00:FS02:0xa4: 16:15:24:WU00:FS02:0xa4:Preparing to commence simulation 16:15:24:WU00:FS02:0xa4:- Looking at optimizations... 16:15:24:WU00:FS02:0xa4:- Files status OK 16:15:24:WU00:FS02:0xa4:- Expanded 886729 -> 2027628 (decompressed 228.6 percent) 16:15:24:WU00:FS02:0xa4:Called DecompressByteArray: compressed_data_size=886729 data_size=2027628, decompressed_data_size=2027628 diff=0 16:15:24:WU00:FS02:0xa4:- Digital signature verified 16:15:24:WU00:FS02:0xa4: 16:15:24:WU00:FS02:0xa4:Project: 8013 (Run 61, Clone 22, Gen 98) 16:15:24:WU00:FS02:0xa4: 16:15:24:WU00:FS02:0xa4:Assembly optimizations on if available. 16:15:24:WU00:FS02:0xa4:Entering M.D. 16:15:24:WU01:FS00:0x16: 16:15:24:WU01:FS00:0x16:*------------------------------* 16:15:24:WU01:FS00:0x16:Folding@Home GPU Core 16:15:24:WU01:FS00:0x16:Version 2.11 (Thu Dec 9 15:00:14 PST 2010) 16:15:24:WU01:FS00:0x16: 16:15:24:WU01:FS00:0x16:Compiler : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01 for 80x86 16:15:24:WU01:FS00:0x16:Build host: user-f6d030f24f 16:15:24:WU01:FS00:0x16:Board Type: AMD/OpenCL 16:15:24:WU01:FS00:0x16:Core : x=16 16:15:24:WU01:FS00:0x16: Window's signal control handler registered. 16:15:24:WU01:FS00:0x16:Preparing to commence simulation 16:15:24:WU01:FS00:0x16:- Looking at optimizations... 16:15:24:WU01:FS00:0x16:- Files status OK 16:15:24:WU01:FS00:0x16:sizeof(CORE_PACKET_HDR) = 512 file=<> 16:15:24:WU01:FS00:0x16:- Expanded 45123 -> 171163 (decompressed 379.3 percent) 16:15:24:WU01:FS00:0x16:Called DecompressByteArray: compressed_data_size=45123 data_size=171163, decompressed_data_size=171163 diff=0 16:15:24:WU01:FS00:0x16:- Digital signature verified 16:15:24:WU01:FS00:0x16: 16:15:24:WU01:FS00:0x16:Project: 11293 (Run 15, Clone 444, Gen 20) 16:15:24:WU01:FS00:0x16: 16:15:24:WU01:FS00:0x16:Assembly optimizations on if available. 16:15:24:WU01:FS00:0x16:Entering M.D. 16:15:26:WU01:FS00:0x16:Tpr hash 01/wudata_01.tpr: 1320017940 401753266 4054896448 3537629650 1520673740 16:15:26:WU01:FS00:0x16:Working on ALZHEIMER DISEASE AMYLOID 16:15:26:WU01:FS00:0x16:Client config unavailable. 16:15:26:WU01:FS00:0x16:Starting GUI Server 16:15:30:WU00:FS02:0xa4:Using Gromacs checkpoints 16:15:30:WU00:FS02:0xa4:Mapping NT from 6 to 6 16:15:30:WU00:FS02:0xa4:Resuming from checkpoint 16:15:30:WU00:FS02:0xa4:Verified 00/wudata_01.log 16:15:30:WU00:FS02:0xa4:Verified 00/wudata_01.trr 16:15:30:WU00:FS02:0xa4:Verified 00/wudata_01.xtc 16:15:30:WU00:FS02:0xa4:Verified 00/wudata_01.edr 16:15:30:WU00:FS02:0xa4:Completed 146040 out of 250000 steps (58%) 16:16:15:WU00:FS02:0xa4:Completed 147500 out of 250000 steps (59%)

by **uncle_fungus** » Fri Jul 27, 2012 4:35 pm

OK. Could you stop your client and run

Code: Select all: FAHClient --lspci

and post the output?

by **Judas** » Fri Jul 27, 2012 4:43 pm

Code: Select all: C:\Users\Drakon>fahclient --lspci VendorID:DeviceID:Vendor Name:Description 0x1002:0x6739:Advanced Micro Devices [AMD] nee ATI:Barts PRO [ATI Radeon HD 6800 Series] 0x1002:0xaa88:Advanced Micro Devices [AMD] nee ATI:High Definition Audio Control ler 0x168c:0x0032:Atheros Communications Inc.:Atheros AR9485 Wireless Network Adapte r 0x1b21:0x1042:ASMedia Technology Inc.:ASMedia XHCI Controller 0x1b21:0x1080:ASMedia Technology Inc.:PCI standard PCI-to-PCI bridge 0x8086:0x0150:Intel Corporation:Xeon(R) processor E3-1200 v2/3rd Gen Core proces sor DRAM Controller - 0150 0x8086:0x0151:Intel Corporation:Xeon(R) processor E3-1200 v2/3rd Gen Core proces sor PCI Express Root Port - 0151 0x8086:0x1503:Intel Corporation:Intel(R) 82579V Gigabit Network Connection 0x8086:0x1e10:Intel Corporation:Intel(R) 7 Series/C216 Chipset Family PCI Expres s Root Port 1 - 1E10 0x8086:0x1e1c:Intel Corporation:Intel(R) 7 Series/C216 Chipset Family PCI Expres s Root Port 7 - 1E1C 0x8086:0x1e1e:Intel Corporation:Intel(R) 7 Series/C216 Chipset Family PCI Expres s Root Port 8 - 1E1E 0x8086:0x1e20:Intel Corporation:High Definition Audio Controller 0x8086:0x1e22:Intel Corporation:Intel(R) 7 Series/C216 Chipset Family SMBus Host Controller - 1E22 0x8086:0x1e26:Intel Corporation:Intel(R) 7 Series/C216 Chipset Family USB Enhanc ed Host Controller - 1E26 0x8086:0x1e2d:Intel Corporation:Intel(R) 7 Series/C216 Chipset Family USB Enhanc ed Host Controller - 1E2D 0x8086:0x1e31:Intel Corporation:Intel(R) USB 3.0 eXtensible Host Controller 0x8086:0x1e3a:Intel Corporation:Intel(R) Management Engine Interface 0x8086:0x1e44:Intel Corporation:Intel(R) Z77 Express Chipset LPC Controller - 1E 44 0x8086:0x244e:Intel Corporation:Intel(R) 82801 PCI Bridge - 244E 0x8086:0x2822:Intel Corporation:Intel(R) Desktop/Workstation/Server Express Chip set SATA RAID Controller

Folding Forum

Do not run ATI/AMD GPU alongside Nvidia GPU...

Do not run ATI/AMD GPU alongside Nvidia GPU...

Re: Do not run ATI/AMD GPU alongside Nvidia GPU...

Re: Do not run ATI/AMD GPU alongside Nvidia GPU...

Re: Do not run ATI/AMD GPU alongside Nvidia GPU...

Re: Do not run ATI/AMD GPU alongside Nvidia GPU...

Re: Do not run ATI/AMD GPU alongside Nvidia GPU...

Re: Do not run ATI/AMD GPU alongside Nvidia GPU...

Re: Do not run ATI/AMD GPU alongside Nvidia GPU...

Re: Do not run ATI/AMD GPU alongside Nvidia GPU...

Re: Do not run ATI/AMD GPU alongside Nvidia GPU...

Re: Do not run ATI/AMD GPU alongside Nvidia GPU...

Re: Do not run ATI/AMD GPU alongside Nvidia GPU...

Re: Do not run ATI/AMD GPU alongside Nvidia GPU...

Re: Do not run ATI/AMD GPU alongside Nvidia GPU...

Re: Do not run ATI/AMD GPU alongside Nvidia GPU...

Who is online