PROBLEMS with DC switches at or near FM tower locations

[sirhc]

New development in this problem.

We have learned that the noise is definitely coming in the DC cables, battery cables, and chargers.

We will work on making an external filter and then later integrate that filter into future DC2DC power supplies.

[sirhc]

An inline filter will not fix this, we tried.

We are adding filters to the power supply and switch board.

I am sorry for this unforeseen issue but we are working to resolve it as fast as we can but this is not something that will be solved in a week or 2 by slapping a few components on a PCB as it will take weeks to order in experimental PCB's then to have them assembled then test them.

Part of the solution will be re-routing traces on the board on different PCB layers to provide better shielding, possibly changing the location on the board we pull I2C from, and finally adding filters to some circuits.

To get a little technical normal I2C communication has filtering mechanisms built in but the way we are achieving I2C is a process referred to as bit banging with GPIO pins which is perfectly acceptable in this application and widely used but sadly this method does not have these filter mechinisums built in.

This is also an extremely expensive process to fix this for just a few people having an issue when installing these DC switches on FM broadcast tower sites but none the less we are going to resolve it. This more than likely will cost us "many" thousands of dollars and increase our cost to produce these units in the future so that they will work in these difficult FM environments which we will have to absorb as we already have MSRP established and we will not raise the price.

The best suggestion I can give you at this time is use our AC powered models until we work this out as they do not appear to have this issue.

Our goal is to find a solution and modify the design and have it tested before we order the next production batch which has to be placed within 2-3 weeks as it is a 6-8+ weeks lead time from the time we place an order until we receive products or we will run out of stock (which we do not want to have happen) so you can see we are very much trying to speed this process along as fast as we can but this is not so simple or inexpensive.

You have to remember the idea behind making our DC switches (WS-8-250-DC, WS-12-DC, and WS-12-250-DC) was for OFF-GRID installations where WISPs have a small tower on a hill with a couple solar panels and a battery bank and no AC grid.

We were NOT thinking people would put these in boxes 300 feet in the air on FM tower towers or even near FM tower sites because normally if you are at an FM tower site they have generators for backup and you would be using our AC models.

So please have a little understanding that just because some people decided this would work as a TOWER Switch (which we are working on a TOWER Switch which does not have this issue with FM) have a little understanding that we did not foresee this nor did we test for this type of environment.

But with that said we are going to resolve this but it will not be fixed with firmware nor will it be fixed in a couple weeks, very sorry but physics is physics.

[keefe007]

#1 Never, ever, ever, ever,ever locate on an AM tower

#2 Try not to locate on an FM tower

Problems will haunt you forever.

[kbloch]

The site where we installed the DC switch has occasional AC power issues from So-Cal Edison. It happens a few times per year.

We are a utility company and the site is primarily used for our voice radio system. Last June we had to install a three hop microwave path to our office in the area as Verizon was shutting down the point to point T1 that the office was using. The microwave network (UBNT AC) now offers more than 20 times the previous bandwidth.

The reason to use the DC model is because we have DC batteries as backup power for our radio system. As a way to keep things as simple as possible we didn't want to add a separate UPS system as it would have to be very large compared to an all DC battery system for all of the equipment.


Glad to hear that you have now identified the problem and are working on a solution.

[wispwest]

I am not willing to try a firmware upgrade at the moment, it is Christmas Eve and we need 24-48 hours to book transportation to the site (top of a Volcano).

Could we build our own type of 'faraday' cage around it next week when we can get up there?

Thx

Dave

That's sick!!! I always wanted a tower on top of a volcano, although we have sites all around Yellowstone. What volcano is it, if you don't mind?

[sirhc]

The site where we installed the DC switch has occasional AC power issues from So-Cal Edison. It happens a few times per year.

We are a utility company and the site is primarily used for our voice radio system. Last June we had to install a three hop microwave path to our office in the area as Verizon was shutting down the point to point T1 that the office was using. The microwave network (UBNT AC) now offers more than 20 times the previous bandwidth.

The reason to use the DC model is because we have DC batteries as backup power for our radio system. As a way to keep things as simple as possible we didn't want to add a separate UPS system as it would have to be very large compared to an all DC battery system for all of the equipment.


Glad to hear that you have now identified the problem and are working on a solution.

You can use an inverter for now tapping those existing batteries and use our AC model until we resolve this issue, I am sorry for the troubles.

[yahel]

My quick take on all this... If you locate on a broadcast tower you have to take extra precautions (and it's generally a very bad idea - avoid if possible).
The Netonix switches would not be the only equipment that may suffer...
I think it's ridiculous that a small company like Netonix will attempt to make their gear RF-proof for such locations. You guys should be using your time for better things.... and you have a solution already --- people at broadcast towers should use the AC models.

Just my two cents!

Yahel.

[kshogan]

We have fought a similar issue on FM towers with the AC switches as well. Solution is to put the switch in a Faraday cage, and isolate all ethernet cables going up tower in steel EMT as far as possible. Then baluns on the ethernet cables that are exposed at 18" intervals, and even baluns at the switch ingress.

Our issue was not the i2c readings, but ethernet training issues. The switch would show 100Mb FD, but the radio would be at 100Mb HD. the mismatch causes lots of errors on the ethernet. When we see this, we typically reboot the radio from the switch, and sometimes it will autonegotiate correctly. If not, we hardcode to 100Mb FD on each end and it will usually run fine until the errors re-occur for unknown reasons. Reboots solve the issue again at times. Running 1G on FM colo is dicey at best.

We have noticed that this is most prominent when heavy ice is built up on the tower. This covers the FM antennas, and creates a lot of reverse power RF back into the shack that creates all the noise.

[Dave]

To All

Yes, if you have a very strong FM source, and it is close enough to the switch/cables, the switch simply can't filter out the RF, it is just to much to ask for. And for sure, all those Ethernet cables act just like radiators bringing in the picked up RF into the switch.

However, I am trying to come up with a few simple fixes to get rid of a more typical FM environment, one that is more close to the edge of not being interference, as opposed to way to much RF power that I can't filter. I am focused on fixing any RF noise issues that are coming into switch via DC power cables.

I know what to do to improve the switch, and am working on verification, but as Chris said, it will be awhile before fixes get into production.

As a note, I do not think most companies would spend the time/resources to do anything with this, they would just through up there arms & say not our problem, but that is not our viewpoint at Netonix...always learn, and improve is a much better viewpoint I think.

Dave

[sirhc]

With all this being said we have been working on a NEW product that will come in very handy on FM towers with the use of fiber and our SMART DC2DC power ability to overcome voltage drops and keep Ethernet cable length from this unit to the radios. This model was designed with consideration of FM noise following the DC cables.

This product will be announced in the weeks to come and hopefully start shipping in Q1 this year. This device is sort of a hybrid/grey area between the WISP Switch and our future TOWER Switch line.