Fluke Demo Part II ... beware of bluetooth

10 of us (GCC IT staff and volunteers and Dave Mast all the way from Ohio) crammed into my office Tuesday evening to get a demo of Fluke Networks EtherScope and OmniView products.  Kevin Fitzgerald, a Fluke (Sales?) Engineer, did a great job showing off the features of each unit live on our church network as well as fielding our questions.  Kevin not only knows the products very well, he knows networking very VERY well.

We've been experimenting with our loaner EtherScope (~$10K) for a few days ... sweet product.  Of course once we saw the OmniView (~$20K) in use it made the EtherScope not as appealing anymore ... it's packet capture and analysis is a thing of beauty :-)
About the only thing either of these units does not do is continous trending ... like What'sUpGold, OpManager, Orion, etc.  So these Fluke devices are killer for network troubleshooting, they are not designed for long term network monitoring ... crud.  UPDATE: You CAN do long term network monitoring, just requires purchasing additional Fluke software like OptiView Console.

I was pleased that Kevin found nothing crazy happening on our network and he did actively look ... he did find several items of interest that we're investigating further ... like why are some of the macs on the network using multicast?

The BIG take away of the night for all of us was the incredible impact bluetooth devices have on access points!  A single bluetooth device in use causes an amazing amount of interference in the .11b/g range.  Kevin said he's seen as few as 12 bluetooth devices take down an access point!  What?!  We all started turning on our bluetooth phones and doing partner searches and the real-time wifi graphs on the Fluke's were going nuts!  It makes perfect sense now in thinking about it ... but somehow I and the rest in the room had no clue bluetooth devices could take down access points.  Why haven't I seen this talked about in the various IT publications I follow?  How did is miss this piece of info?  This is huge news ... at least to me.

The timing of this discovery is perfect as we've been experiencing some random off the wall wifi issues lately on the weekends.  Guess what wifi range we're using for our checkin kiosks?  .11g ... take a guess as to how many people with bluetooth enabled phones (most prob don't even know it's on by default) walk up to and past our kiosks each service?  Hmm, could this be the cause of our bizzaro wifi drops over the past months?  "All signs point to yes" is my gut feeling.  So afterwards Tom went down and switched all the kiosks and their access point to .11a only.  I'm very curious to see if that eliminates the strange wifi issues we've been experiencing.  I watched the wireless kiosks the entire time for our Thursday night service ... not a single hiccup.  Now to see how they do Sat and Sun.

Last September at the IT Roundtable at GCC we had some strange wifi issues down where we held our meeting ... it acted like the access point was periodically rebooting ... the access point was located directly across the hall.  Hmm, there were 24 of us in the room ... most everyone using a laptop.  I recall seeing many wireless mice (bluetooth?) and a couple bluetooth phone earpieces ... hmm.  There was also another nearby room having a workshop and I'd wager a few people with laptops and a few bluetooth devices.  Hmmmmmmmmmmmmmmm

Now, without more investigation I can't blame bluetooth for any wifi issues we've had.  BUT, now having seen with my own eyes and heard from a very knowledgable networking engineer about bluetooth's impact on .11b/g devices ... wow, it's hard not to immediately point fingers.  I already know wireless mice can sometimes operate at ridiculously long ranges and cause, in our case, ridiculous amounts of stress and panic.  So the notion that some phones and mice could wig out wifi has a high probability in my book.

Of note is that the hardware/software to do this uber detailed wifi analysis doesn't require either Fluke device demoed.  It's a separate software/hardware bundle called AnalyzeAir.  A quick check at CDW shows it's right at $4K.  Given the amount of wifi stuff we do at GCC, we need this caliber of tool.  Our Wi-Spy is helpful for basic wifi needs, but it can't come close to the features or resolution of AnalyzeAir.

So in the end, I think at some point down the road we'll need something like the OmniView device.  Yes, $20K is a lot, but break that down over say 4 years and you're talking $5K/year.  As our network becomes increasingly larger and more complex, $5K/yr is a no-brainer.  Hmm, I think I just convinced myself to get one ;-)

Here's a couple picks for your viewing pleasure ...

Comments

[shellie]

We recently discovered this on our network as well.

It scares us greatly, as we are a 24/7 mission critical shop. The network MUST work. 22% of our 700+ clients are wireless.

Not to mention being in a public service industry. So 'can you please turn that off?' isn't gonna cut it.

[Bryan Johnson]

Several years ago, i saw some Macs doing multicast on our network too. The multicast traffic was killing some Linksys gear and causing general havoc on the network. We've since removed just about all "Best Buy" class gear and haven't seen any additional problems.

I'm interested to see what you discover.

[Brian Marquis]

Regarding Macs and Multicast, all Macs from 10.2 forward include MulticastDNS (UDP 224.0.0.251:5353) for zero configuration service discovery, AKA Bonjour or Rendezvous. To see what machines are available use a tool like Bonjour Browser (http://www.tildesoft.com/Programs.html)

For more info on see:
http://en.wikipedia.org/wiki/Bonjour_(software)

The inevitable question - How to disable it:
sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.mDNSResponder.plist

The -w makes the change permanent.

Try this to reenable it.
sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.mDNSResponder.plist

Note that Bonjour is pervasive on the Mac and is used by many different programs to discover what's available on the network. I'd recommend against disabling Bonjour and instead limit what is being broadcast by turning off services that are not needed in System Preferences, Sharing.

You might also have success by disabling multicast in your WAPs, provided that they have that option.

[Brian Slezak]

Dude!

Your bluetooth discovery actually makes sense to me now. I had just recently noticed that when I have my bluetooth usb adapter plugged in, that I experienced connection drops on my G wireless link. Never researching this, I would not have thought bluetooth and 802.11g would collide as much as they do. Duh. :)

Thanks man. This makes me want to turn off my Treo's bluetooth when I'm not using it. ;)