This works well for us

[sirhc]

So we have been trying different configurations and so far this works best.

So if you have an hour and a half click HERE to view the movie.

Right now we have the airFIBER back hauls configured as mid-spans because when we had them traversing the Static LAG with VLANs they were traversing the same port(s) to the router as the AP's and since we found it better to enable Flow Control on the ports feeding the AP's and of course then we had to enable it on the ports to the routers for Flow Control to be effective it would cause pause frames to interfere with the back haul radio communications even packets bound for the next tower out in the ring.

We hope to eventually figure out how to have 2 LAGs with both un-tagged to the default VLAN which would allow us to have a separate LAG for back hauls without being affected by Flow Control Pause Frames generated by the AP's on their Static LAG ports. When we figure this out I will post that config as it will save a couple ports on the switch WHICH WE NEED.

But anyway since most of our towers are pretty much identical here is the standard configuration we are using and here is a speed test I am getting at home right now which is 7 hops from our Level 3 fiber including my last wireless link to my house traversing 6 towers/routers/switches to my house.

I connect to an AP like all my customers. I share this AP with about 10 other people and this AP is a standard M5 using a 30 MHz wide channel.

Flow control is needed on ports going to AP's and your WISP Switch up-link port(s) going to your tower router.

CLICK IMAGES BELOW TO SEE FULL SIZE

Status Tab

qr-s.png (121.45 KiB) Viewed 47369 times

Ports Tab

qr-p.png (119.79 KiB) Viewed 47369 times

VLAN Tab

qr-v.png (84.35 KiB) Viewed 47369 times

LAG Tab

qr-l.png (149.92 KiB) Viewed 47369 times

Speed Test

home st.png (734.32 KiB) Viewed 47369 times

new.jpg (264.77 KiB) Viewed 47049 times

[Mola9850]

TY nice INFO

[rebelwireless]

Does the 'QR' in your VLAN labels mean something?

[rebelwireless]

also, can you clarify the lag group issue, are you saying that when you had two AF plugged into the switch, you were forced to use flow control which caused issues?

[sirhc]

Does the 'QR' in your VLAN labels mean something?

QR Stands for Quarry Road because that is the road the water tower is on.

also, can you clarify the lag group issue, are you saying that when you had two AF plugged into the switch, you were forced to use flow control which caused issues?

We had originally had a single LAG between the switch and the router but since Flow Control wanted to issue PAUSE FRAMED to interfaces that meant the LAG interface that also carried the AF Back Hauls also got PAUSE FRAMES for 50 micro-seconds which we did not like so we broke the AF bach hauls our and configured the switch as mid-span POE for the back hauls. We eventually want to have 2 LAGs, one for back hauls and one for tower radios but we were having a hard time figuring out how to do that in Cisco without cause VLAN packet bleed.

It is on our list of things to try and figure out how to have 2 STATIC LAGS ti the router from the switch, one for AP's and such and one for back hauls to keep the PAUSE FRAMES generated from the AP's away from the back hauls as we do a ring topology so traffic flows in through and out of the tower on the back hauls without having to contend with interface pause frames generated from slow HD wireless links plugged into 100M FD or 1G FD ports but the wireless can not push the link speed because it is a dynamic wireless capacity based on many environmental variables.

[rebelwireless]

I see. I was just talking to a buddy that runs a local datacenter about tech stuff. He was asking why we didn't use x or y because he'd never thought about all these techs being designed for known values, not dynamically variable links etc. All of these things, flow control, lacp, etc, were all designed with wired or optical interfaces with known link speeds in mind. Further, they were designed thinking the jumps were 10>100>1000 etc, so throwing in a 75Mbps link wasn't on the list of test scenarios. He is also adamantly against mikrotik routers, which is a fair position to have, but he doesn't have an alternative with the feature set anywhere near the price range.

[ircactus]

So we have been trying different configurations and so far this works best.

Right now we have the airFIBER back hauls configured as mid-spans because when we had them traversing the Static LAG with VLANs they were traversing the same port(s) to the router as the AP's and since we found it better to enable Flow Control on the ports feeding the AP's and of course then we had to enable it on the ports to the routers for Flow Control to be effective it would cause pause frames to interfere with the back haul radio communications even packets bound for the next tower out in the ring.

We hope to eventually figure out how to have 2 LAGs with both un-tagged to the default VLAN which would allow us to have a separate LAG for back hauls without being affected by Flow Control Pause Frames generated by the AP's on their Static LAG ports. When we figure this out I will post that config as it will save a couple ports on the switch WHICH WE NEED.

But anyway since most of our towers are pretty much identical here is the standard configuration we are using and here is a speed test I am getting at home right now which is 7 hops from our Level 3 fiber including my last wireless link to my house traversing 6 towers/routers/switches to my house.

I connect to an AP like all my customers. I share this AP with about 10 other people and this AP is a standard M5 using a 30 MHz wide channel.

Flow control is needed on ports going to AP's and your WISP Switch up-link port(s) going to your tower router.

You know what I'd like to see. The router side of the config (Sanitized of the ip's) but still how you are tying back to the router / routers as the case may be.

[sirhc]

Here is a sterilized config of HC Tower, an older config before BFD was implemented, hope this is what you wanted?

HC01#sh run
Building configuration...
Current configuration : 15133 bytes
!
version 15.3
service nagle
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname HC01
!
boot-start-marker
boot system flash0:/c2951-universalk9-mz.SPA.153-1.T1.bin
boot config flash0:/startup-config
warm-reboot count 50
boot-end-marker
!
logging buffered 10000
no logging console
no logging monitor
enable secret 4 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
!
no aaa new-model
clock timezone EST -5 0
clock summer-time EDT recurring
!
ip cef
!
ip domain name xxxxxxxxxxxx.xxx
ip name-server 0.0.0.0
ip name-server 0.0.0.0
!
multilink bundle-name authenticated
!
!
license udi pid CISCO2951/K9 sn XXXXXXXXXXXXXXX
!
!
archive
path tftp://0.0.0.0/HC1-config
write-memory
!
spanning-tree portfast bpduguard
no spanning-tree vlan 1
no spanning-tree vlan 16
no spanning-tree vlan 20
no spanning-tree vlan 21
no spanning-tree vlan 22
no spanning-tree vlan 51
no spanning-tree vlan 52
no spanning-tree vlan 53
no spanning-tree vlan 54
no spanning-tree vlan 55
no spanning-tree vlan 56
no spanning-tree vlan 60
no spanning-tree vlan 70
no spanning-tree vlan 71
no spanning-tree vlan 72
no spanning-tree vlan 97
no spanning-tree vlan 100
!
redundancy
!
ip tcp ecn
ip tcp path-mtu-discovery
!
interface Port-channel10
description Tank Vlan
no ip address
no ip redirects
load-interval 30
carrier-delay msec 0
no mop enabled
hold-queue 150 in
!
interface Port-channel10.1
encapsulation dot1Q 1 native
ip address 0.0.0.0 255.255.255.0
no ip redirects
!
interface Port-channel10.20
description HC20 AP
encapsulation dot1Q 20
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252
no ip redirects
!
interface Port-channel10.21
description HC21 AP
encapsulation dot1Q 21
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252
no ip redirects
!
interface Port-channel10.22
description HC22 AP
encapsulation dot1Q 22
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252
no ip redirects
!
interface Port-channel10.51
description HC51 AP
encapsulation dot1Q 51
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252
no ip redirects
!
interface Port-channel10.52
description HC52 AP
encapsulation dot1Q 52
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252
no ip redirects
!
interface Port-channel10.53
description HC53 AP
encapsulation dot1Q 53
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252
no ip redirects
!
interface Port-channel10.54
description HC54 AP
encapsulation dot1Q 54
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252
no ip redirects
!
interface Port-channel10.55
description HC55 AP
encapsulation dot1Q 55
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252
no ip redirects
!
interface Port-channel10.56
description HC56 AP
encapsulation dot1Q 56
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252
no ip redirects
!
interface Port-channel10.60
description HC2CDC
encapsulation dot1Q 60
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.248
no ip redirects
ip ospf message-digest-key 1 md5 7 XXXXXXXXXXXXXXXXXXXXXXXXXXXX
!
interface Port-channel10.70
description HC to RTC.MA 5Ghz
encapsulation dot1Q 70
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.248
no ip redirects
ip ospf message-digest-key 1 md5 7 XXXXXXXXXXXXXXXXXXXXXXXXXXXX
ip ospf dead-interval minimal hello-multiplier 5
!
interface Port-channel10.71
description HC to RTC.IC 5Ghz
encapsulation dot1Q 71
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.248
no ip redirects
ip ospf message-digest-key 1 md5 7 XXXXXXXXXXXXXXXXXXXXXXXXXXXX
!
interface Port-channel10.72
description HC to RTC.MS 5Ghz P2P
encapsulation dot1Q 72
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252
no ip redirects
!
interface Port-channel10.96
description TMC Access Points
encapsulation dot1Q 96
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.0 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252
no ip redirects
!
interface Embedded-Service-Engine0/0
no ip address
shutdown
!
interface GigabitEthernet0/0
description Lagg to WS1 P22
no ip address
no ip redirects
load-interval 30
carrier-delay msec 0
duplex auto
speed auto
channel-group 10
no cdp enable
no mop enabled
!
interface GigabitEthernet0/1
description Lagg to WS1 P23
no ip address
no ip redirects
load-interval 30
carrier-delay msec 0
duplex auto
speed auto
media-type rj45
no keepalive
channel-group 10
no cdp enable
no mop enabled
!
interface GigabitEthernet0/2
description HC to QR AF24Ghz
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.248
no ip redirects
ip ospf message-digest-key 1 md5 7 XXXXXXXXXXXXXXXXXXXXXXXXXXXX
ip ospf dead-interval minimal hello-multiplier 5
load-interval 30
carrier-delay msec 0
duplex auto
speed auto
no cdp enable
no mop enabled
!
interface GigabitEthernet0/1/0
description HC to NL Fiber 1
ip address 0.0.0.0 255.255.255.248
no ip redirects
ip ospf message-digest-key 1 md5 7 XXXXXXXXXXXXXXXXXXXXXXXXXXXX
ip ospf dead-interval minimal hello-multiplier 5
load-interval 30
carrier-delay msec 0
media-type sfp
no cdp enable
no mop enabled
!
interface GigabitEthernet0/2/0
description HC to NL Fiber 2
ip address 0.0.0.0 255.255.255.248
ip ospf message-digest-key 1 md5 7 XXXXXXXXXXXXXXXXXXXXXXXXXXXX
ip ospf dead-interval minimal hello-multiplier 5
ip ospf cost 65000
load-interval 30
shutdown
media-type sfp
no cdp enable
no mop enabled
!
interface GigabitEthernet0/3/0
description HC to BV AF24Ghz
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.248
no ip redirects
ip ospf message-digest-key 1 md5 7 XXXXXXXXXXXXXXXXXXXXXXXXXXXX
duplex auto
speed auto
media-type rj45
!
interface GigabitEthernet0/0/0
no ip address
!
interface GigabitEthernet0/0/1
no ip address
!
interface GigabitEthernet0/0/2
description HC to GT RocketM5 (Vlan 97)
switchport access vlan 97
no ip address
!
interface GigabitEthernet0/0/3
description HC to SR Mimosa B5Ghz (Vlan 100)
switchport access vlan 100
no ip address
!
interface Vlan1
no ip address
!
interface Vlan97
description HC to GT RocketM5 on G0/0/2
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.248
no ip redirects
ip ospf message-digest-key 1 md5 7 XXXXXXXXXXXXXXXXXXXXXXXXXXXX
ip ospf hello-interval 20
!
interface Vlan100
description HC to SR Mimosa B5Ghz on G0/0/3
ip address 0.0.0.0 255.255.255.252 secondary
ip address 0.0.0.0 255.255.255.252
ip ospf message-digest-key 1 md5 7 XXXXXXXXXXXXXXXXXXXXXXXXXXXX
ip ospf dead-interval minimal hello-multiplier 5
ip ospf cost 60000
load-interval 30
carrier-delay msec 0
no mop enabled
!
router ospf 1
router-id 0.0.0.0
auto-cost reference-bandwidth 1000
area 0 authentication message-digest
area 50 authentication message-digest
area 51 authentication message-digest
area 60 authentication message-digest
timers throttle spf 1 10000 20000
timers throttle lsa 100 10000 45000
redistribute connected subnets
redistribute static subnets
network 0.0.0.0 0.0.0.7 area 0
network 0.0.0.0 0.0.0.7 area 0
network 0.0.0.0 0.0.0.7 area 0
network 0.0.0.0 0 0.0.0.7 area 0
network 0.0.0.0 0.0.0.7 area 0
network 0.0.0.0 0.0.0.7 area 50
network 0.0.0 0 0.0.7 area 51
network 0.0.0.0 0.0.0.7 area 60
network 0.0.0.0 0.0.0.3 area 0
!
no ip forward-protocol nd
!
no ip http server
no ip http secure-server
!
ip route 0.0.0.0 255.255.255.252 0.0.0.0
ip route 0.0.0.0 255.255.255.252 0.0.0.0
ip route 0.0.0.0 255.255.255.252 0.0.0.0
ip ospf name-lookup
!
no cdp run
!
snmp-server community XXXXXXXXXX XX XX
snmp-server enable traps ospf state-change
snmp-server enable traps ospf errors
snmp-server enable traps envmon
snmp-server enable traps entity-sensor threshold
access-list 20 permit 0.0.0.0 0.0.0.255
access-list 20 permit 0.0.0.0 0.0.7.255
access-list 20 permit 0.0.0.0 0.0.255.255
access-list 20 deny any
access-list 30 permit 0.0.0.0 0.0.0.127
!
control-plane
!
banner motd ^C
****************************************************************************
* *
* *
* NO UNAUTHORIZED ACCESS *
* *
* *
****************************************************************************
^C
!
line con 0
line aux 0
line 2
no activation-character
no exec
transport preferred none
transport input all
transport output none
stopbits 1
line vty 0 4
password 7 00000000
login
transport input telnet ssh
line vty 5 15
password 7 00000000
login
transport input telnet ssh
!
scheduler process-watchdog terminate
scheduler isr-watchdog
no scheduler allocate
scheduler interval 100
ntp server ip XXXXXX.XXXXXX.XXX prefer
!
end
HC01#

[LRL]

All those /30s make me cringe. Do you guys receive your IP space from L3 or are you direct to ARIN? Any criticism about assigning every customer a /30?

[tossman101]

We get our IP's from Level3. the IP addressing scheme was started before I worked here, but as far as I know Level3 has never complained about it. We have two /24's and one /21 from them. Which is what we advertise back to them, we're not doing world routes.