I've looked at this post a few times and wondered because I have the same problem on one machine on my much smaller network. My machines are all 32 bit and recently (Year Ago) restored to factory default. I was tearing my hair out like you.
I blamed the NIC card but a new card made no difference.
I blamed a failing hard drive but overnight tests gave it 100% OK
Rather than restore it again, I cleaned it up. 7000 windows junk files, 364MB Internet junk (forgot to re-set the maximum whoever set it up) about 4000 cookies, error reports, set up logs, boot logs and the rest. Sorry, but I do it manually. Empty the recycle bin and defrag. 2 files unable to defrag. msconfig and disable the junk.
I'm happy with 5 seconds.