Jabra Goodness

Let’s face it.  The reason there are so many headsets on the market is because everyone’s head is different.  Personally, I like a nice stereo (Binaural to be technically correct) headset that is DECT.  Wireless to the base, USB to the workstation.  Great range.  For those long conference calls this technology can only be beat by not calling in.  I can get up and walk around, get coffee, and get tasks done while monitoring the call.  Using wide-band, my DECT has at least 200 feet of test range.  So, based on this foreknowledge, I played with a few Jabra headsets lately.  Remember that my existing DECT is the gold standard.

Jabra BIZ 2300 USB Duo

Such a title for a simple headset that has great audio quality, excellent comfort, and zero effort plug n pray.  I think the VOLUME was a bit lacking, but cranking up the sound card fixed that.  Adjustable boom mike.  What appears to be pretty good sound canceling.  Folds flat for storage.  Optimized for Lync.  Nice set of workable controls on the inline cord module. Here is the market speak straight from Jabra.

I chose this item for test because it hits right where I work.  As a 50% travel, I live with Lync as my communications center.  On the road, you need some tough stuff; it is going to take a beating.  Take a look at that market-speak, I have to agree with all of it.

Lync

  Oh yes.  Well, I got nothing.  I plugged it in, and it worked.  Perfect. You could not ask for more.  You can get one right here.

Adelante.

Jabra Pro 9465 Duo

Oh Wow.  Here is the market-speak.  I will repeat.  Oh wow.  What a great piece of gear.  Just running through the setup makes you appreciate what went into this great unit.  Right at the moment, the 9465 is playing headset to one of my lab common area phones, doing BT to my iPhone, and is also connected to my lab laptop as the preferred Lync headset.  All at the same time, all seamlessly, all at the same time.  WITH NO READING ON MY PART.  The dedicated reader will know my aversion to reading device documentation (the precept being that the average user will not read the documentation either). From hooking this unit to power, it presented, in color, a walk-through for setup that was PERFECT, and I hooked it to three different devices.  The 9465 even has a “phone home” feature that configures the sound on the base unit based on the deployed environment.  So nice.

The Jabra marketing professionals came up with this:

The PRO 9465 Duo offers the ultimate in connectivity. Specifically designed for executives, managers and other professionals, this headset enables employees to connect with their mobile, desk and softphone, simultaneously. The microphone features advanced noise canceling technology, enabling clear, understandable conversations for your employees and customers.

While I usually decry marketing types as overblown hyperbole (is that redundant?  can you have overblown exaggeration), in this case, I quoted it so that you can see what I don’t want to take the time typing.  And I agree with the content and intent.  ‘Nuff said. 

Lync

Seamless.  Plugged it in and it worked.  What a great piece of gear. I like it, I like it, I like it. You can get one right here.

Jabra Pro 930

Here is the market-speak.  According to Jabra, the Pro 930 is “…Designed for PC based telephony and Unified Communications systems…A professional, wireless headset optimized for Microsoft OC and Lync.”

DECT, with great range.  Excellent audio quality.  I tried this one because it comes with the ear-hook-thingy that gets rid of the headache-making single ear head band.  Ouch, I hate those.  I hate them even more than I hate one-ear setups.  But, being the even-handed type that I am, I figured I would at least try it before I dissed it.

Following the assembly instructions proved to be my only out for assembly – I failed at figuring it out.  Admittedly, I did not try to hard; maybe a few minutes before I decided that it had to be a press-fit, and then I read the documentation before I broke something.

As a personal note, I did not like the one ear angle of dangle when I had it up and running and actually using it.  It just did not feel “right.”  I tried doing the wiggle and reposition techniques, but in the end, I never got comfortable with being only on one ear.

Lync

Hooked right up and started working.  What more could you ask for?

Summary

All three of these Jabra units are solid, top-of-the-pack choices.  You could do a lot worse.  Solid design, high quality build, great feel, excellent performance.  Jabra even has some spiffy software you can download to keep things easily configured and the base unit firmware updated.

YMMV.

Lync 2013 Edge Hairpin

Many thanks go to Chard Johnston (AudioCodes), and Jeremy Silber (CDW)

Scenario

The project was Lync 2013 Enterprise, two sites, full HA, DR, and call recording using AudioCodes SmartTap. The edges in both sites were DNSLB.

The Symptoms

Once we started making more than a few calls to external numbers, we noticed that the SmartTap was not recording as expected.  This caused a few calls to the helpful AudioCodes support engineers.  It turns out that SmartTap does a little call-redirection magic, and captures all the necessary traffic to record both sides of a phone call from the edge servers.  And when one user lands on Edge1 and the other user lands on Edge2, we start seeing calls failing.

We were also failing regular calls between Lync users that used the Edge servers.  Same symptoms.  Calls would start, then fail when the time came to establish media.  Needless to say, this was not good.

Interestingly, this problem has been around for bit.  Jeremy Silber has an excellent article that outlines the problem, the cause, and the fix in explicit detail.  Even better, if you talk to Jeremy (I happen to have direct access) he can translate the contents of that blog into English!  Highly recommended reading. Having it translated to English so that either of my brain cells could comprehend was priceless.  Firewall rules had been through the change order process at least a month ago, so we thought we were good there.  All previous testing had been good.  But we had not tested voice/video yet.

What is going on here?

I had a heck of time getting those firewall rules in, not at the technical level, but at the explaining the “why” in English.  So to get my skills up to speed, I discussed the issue (and the fix) with Chard Johnston of AudioCodes – seeing as how he was buried in trying to get SmartTap working correctly.  After I showed Chard the hairpin requirement – see previous reference to Jeremy Silber – Chard went to Microsoft using his channels.  Apparently this discussion went on for a bit. Chard came back and created the following diagrams.

 

 

Why is this needed?  Well, if you look back at Jeremy’s blog, you will see that the candidate pairs that are exchanged between the users don’t have FQDN, they have IP.  And, done correctly, the IP will be that of the EXTERNAL PUBLIC IP of the AV service per edge.  So, the firewall must allow traffic from one public IP to simply hairpin back to the other edge public IP.

Remember that while SmartTap highlighted the issue, it was a firewall configuration that was the real culprit.  If we look at the reference article in the Lync 2013 documentation, we don’t find the requirement for the Edge server AV service to talk to the other edge pool server AV service via the public IP address.  Once you know the requirement exists, the information is there if you read between the lines a bit.

In the end, the requested firewall rules were not implemented correctly, so we had some one-way conversations going, and some quick adjustments by the firewall team had everything ironed out.

YMMV.

Lync 2013 SIP trunk with a twist

Scenario

Maybe like me, you have a split environment with an Lync 2013 EE pool, with a Lync 2013 SE, and you want to get a SIP trunk installed so that you can play with pilot Dial in Conferencing and maybe some light Enterprise Voice?  The guidance on direct SIP trunks is to stand up a separate Lync 2013 Mediation Server.  You can read up on that right here.  The mediation server “strong recommendation” is here.   Be that as it may, you might decide that it would be a more efficient usage of resources, especially for a pilot, to use the Lync 2013 SE as your mediation server.  And using a internet-based SIP trunk provider will get you the most bang for the buck albeit at the expense (maybe) of reliability.  I personally have had great results using internet-based SIP trunks, YMMV.

After reading up a bit, you realize that you are going to need a non-routable IP added to the Lync 2013 SE to make things work.  Why would you need that?  In my case, the internal subnetting and security was such that the SE needed another subnet to work with – security would not allow an unsecure connection ( a SIP Trunk straight to a production network server with no SBC on-premises).

How to

As luck would have it, Intelepeer – for a wide variety of reasons, my first choice for net new SIP trunks in an environment – was willing to work on a semi-custom plan to get our pilot up and running.  SIP trunks in Lync 2013 did not change much since Lync 2010, so we can use this guide from MVP Brian Ricks to get the basics accomplished.  Another MVP, Curtis Johnstone, has another SIP trunk article that is well worth reading.

Before you start though, what about that need for a mediation server?  In our scenario, we need to arrange for another NIC/IP on that SE so the mediation server can have a separate subnet.

This blog entry from Norway will walk you through what to do for the second NIC/IP needed. The sharp-eyed reader will note that the NIC setup looks like an external interface for a Lync Edge server.  Moving forward, you will need make up some firewall rules to get the requisite SIP call setup (TCP) and media flow (UDP) between your new mediation server NIC and the service up in SIP trunk land.  Depending on the firewall, you may want to double check to make sure that the NAT you setup is taking your mediation server traffic and sending it out the correct address.

While you may not have an SBC on-premises, you can be assured that the SIP trunk provider is going to have one, and that SBC will not communicate with an IP that is doesn’t have defined in the trunk setup.  I strongly recommend creating a group in your firewall, and restricting your SIP trunk traffic (that leaves your firewall) to only communicate with the provider.  1:1 NAT may not be possible on your firewall (why I cannot imagine, but there you go) so that is something you may want to consider before getting started.

Here is what we need:

The Results

Right out of the box, setup using the given guidance, outbound calls work, but would never disconnect if/when the called party hung up.  Inbound calls just failed. Turns out that there were two things, one Lync and one Intelepeer.

Lync

Based on this bit of Lync 2013 documentation, the “Centralized media processing” needs clearing.  In our case, Intelepeer is using TCP on one IP and UDP on two others.  Hence, clear the box.  In this case, we were also doing no encryption and Intelepeer basically told me that support for “Refer” ain’t there yet.

Intelepeer

On the Intelepeer side, their SBC was looking into the packets, finding the IP of the outside of SE/Mediation server (10.10.10.62) and trying to send SIP signaling traffic to that IP.  Obviously, that would not work.  At any rate, the Intelepeer engineer (a most helpful fellow) twiddled some bit on his end, and wala! Instant telephony in and out.  Fabulous.

Conclusions

If you need a path through a network maze, you can come up with one.  In this case, we needed to allow for the Security Mavens to have their (understandable in this case) way, and still be able to provide Lync with a SIP trunk to pilot Dial-in Conferencing and EV.  Total time that involved actual network and Lync hands-on touching?  Maybe two hours total over several weeks.

YMMV

Duplicate IP error

Just ran into a little something I have not seen in a long time. Building edge servers with Windows Server 2012R2 and the external NIC kept coming up as duplicate address. Doing all the cross checking showed that the server had only two interfaces and that they were configured properly. Turns out that this little gem from 2008 R2 days is still valid:

http://social.technet.microsoft.com/forums/windowsserver/en-US/d7bda315-6366-4e0a-bdcf-dc875ff6963e/win-2008-r2-servers-will-not-connect-to-network-says-duplicate-ip

The firewall doing the NAT to the public IP was ASA. The registry change for disabling gratuitous ARP fixed all of the my edges.

This might be of interest also:  http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1028373

YMMV

Frontier FIOS and static IP

Background

About four months ago, I made the decision to extend my lab, Tsoorad.net, out to the larger world of the internet.  I have several reasons for wanting to do this; most of them selfish, but a surprising number of them are for client testing purposes.  I need/want to implement a full SBC for a SIP trunk, which leads to needing a SIP trunk, et cetera.  So for this exercise I need more than one (1) static IP.  I could probably figure out a way to futz with ports per service and get something worked up, but I want to be able to duplicate customer scenarios.  Internally I have plenty of network.  Externally, I need a static IP configuration.  And with needing at least three IP externally, the logical choice (see how I justified all that?) a /29 seemed to be a good fit.

Enter Frontier FIOS

I have been a fan of FIOS since the halcyon days of Verizon replacing my DSL with fiber to my house.  Awesome.  None of the crap that you put up with if you use cable.  I don’t share my bandwidth with the rest of the neighborhood.  Nice and consistent on the speeds up and down.  And the 15/5 package is more than sufficient for my 2 person internal domain and my lab domain. 

But, it turns out that FIOS has this ONT thing (Optical Network Thingy)(actually, Optical Network Terminal) mounted to the side of the house.  Which leads us to the The Problem.

The Problem

On the Frontier side, the sales folks sold me.  I know, I should have been smarter.  I should have known.  Sales types around the world have a mantra:  “yes yes yes, please sign here” and they are compensated on their ability to get the signature.  I get it.  But I hate misinformation. 

Knowing that I was going to need at least three of the /29 workable, asked the obvious (and brilliant) question: “Will the Frontier supplied router be able to handle a /29 or will I need my own firewall/router?”  I should have heard the alarm bells go off when I had to explain what a /29 was and why I was asking the firewall/router question.  But, I was assured, by someone who assured me that they worked on the business provisioning side of things, that the router supplied on the install would handle the requisite services.

NOT!

When the Frontier install tech showed up onsite, he expressed his doubts.  But, armed with the knowledge (however faulty) of the sales professional, we moved forward.  The actual connection part was easy.  When I first moved in, I had the technician run Cat5 from the ONT to the internal drop.  There are pluses and minuses to this approach.  But seeing how I could care less about the TV service (I use a popular satellite vendor), using an Ethernet drop versus coax was a no brainer.  As it turns out, a necessary one as we shall see.

Firstly, if you have coax to your local demarc, then you will need to use the Frontier-supplied router/modem.  The Frontier-supplied unit, an Actiontec MI424WR, certainly LOOKED like it should handle things.  In the network definition section in the setup, there is a place to put in IP’s, and it would take the full definition complete with mask and gateway.  But, we could not get more than one address working.  At one point we had a second address working, but it abruptly quit after a few hours.  For reasons we shall see, this was caused by the ONT and in the end, had the Frontier Tier 1 non-help desk been more informed and less script-driven, I might still be using the ActionTec.  You see, the problem, while it appeared with the original Frontier-supplied router, might not definitely indicate that the ActionTec won’t work.  However (on the flip side) I did have a longish chat with an Actiontec “engineer” who blithely told me that their router would only do 1:1 static NATs, and not dynamic 1:M NAT.  I am also not sure if the Actiontec would be able to do 1:1 AND 1:M NAT at the same time.  I suppose that particular test will be done at some point in the future when I am snowed in or can’t sleep at 0230.

Adelante!

Next, having proved over and over that we could get all 5 IPs working off a switch on five different pieces of gear, and all at the same time, we started looking at how the network was delivered from the Frontier side of things.

<Cue creepy techno-vibe music from CSI with lots of tracert, pathping, and telnetting with multiple interfaces on different machines for configuring the various firewalls and other appliances>

Back to the Frontier Tier 1 non-help folks who (reading from their script) had me retest everything again, and then proclaimed that I had a DNS issue.  <sigh>

Over and over this went.  Then it turns out that the network technician who was advising the Tier 1 person was reading from the standard, world-wide network engineer script that goes something like this:  “Everything is good here.  You have a problem.”  And the idiot user (me in the case) is not allowed to speak directly to the actual network technician – so you have this wonderfully frustrating back ‘n forth between the idiot (me), the non-support Tier 1 person, and the network technician who blindly insists that all of their gear is provisioned correctly.  Because I was asking for support for something that was not common to their network, this entire effort failed because I was off script.

This went on for several months.  In that time I tried, in order of implementation, the following firewalls:  Cisco PIX 515e, Cisco ASA 5520, Cisco ASA 5505.  Nuttin’!  All this time, each time I called for some more non-help to be told how everything was peachy on their end, but gees, maybe you could clear your ARP tables please, I was told how the issue had to be on my gear being misconfigured.  Now, I am not in any fashion a Cisco ASA expert.  But I have some knowledge.  So when it came to about 1/2 way through the 5520 efforts and for ALL of the 5505 efforts, I trotted out the hired guns.  Namely a completely certified (he is really out there) Cisco dood – who lives and breathes Cisco security as his primary job.

As a side note, this guy was awesome.  But clearly, in the end, he was clearly puzzled as to why what works for the rest of the civilized world refused to work for the TsoorRad environment.  And he did this several times.  We had nothing wrong.  My network is solid.

The Fix Begins

At about this time, I am starting to doubt my sanity.  Stupid thing should be working but it is not.  I think that I have eliminated any possible error on my side – Yet Frontier network techs still insisted that there was nothing wrong on their end.  However, if you read some of the material to be found online which I magnanimously have provided here:

http://www.dslreports.com/forum/r23683580-Does-the-ONT-have-its-own-IP-address

http://www.dslreports.com/forum/r27214458-Number-of-internet-IPs-on-FIOS-

http://www.dslreports.com/forum/r27022337-Cisco-ASA-5505-with-FiOS-Multiple-IPs-WORKING

http://www.dslreports.com/forum/r21956488-northeast-Proxy-ARP-with-FIOS-business-static-IP

http://www.dslreports.com/faq/verizonfios/3.0_Networking#16077

…you can see that I am NOT the only one.  However, in defense of fairness, in some of the research the firewalls were called into question.  So, I will try one last time.  I purchased a NetGear FVS318G.  Same results as before.  I then discovered a piece of the Frontier website that allowed me to find email addresses for executives in my area.  So I fired up my email and sent a politely worded notice of my dissatisfaction.  Several.  But the first one got serious attention.  So the buck does stop somewhere. 

So now I have Frontier calling me, managers, technicians, and technician supervisors all have been told to fix this.  About time.  But guess what?  The network team on their side is still insisting that their configuration is perfect.  Here is a little blurb of the back-n-forth:

“…the ONT cannot (empirically) deliver the entire /29 all at one time to one device such as a firewall.  This is not acceptable. 

It would appear that something in the ONT is not able to send the entire /29 block to one device.  I have now used 4 different firewalls, ranging from bottom end business class (netgear fvs318g) to industry standard business class firewalls (PIX 515E, ASA 5520, ASA 5505) and they all do the same thing – specifically, the devices work just fine on one IP address, but the ONT fails to deliver the entire block to the single device.

I think that the ball is back in Frontier’s court.  Something must be able to be done to the ONT to bridge my entire /29 to the Ethernet so that what I paid for, namely business class service, is actually delivered to me.  Bridge the ONT, tell the ONT to handle multiple IP’s, upgrade the firmware on the ONT to do this, SOMETHING…”

The Fix is IN

As it turns out, the ONT not being able to deliver all the addresses was the issue all along.  While I cannot get direct information as what EXACTLY was done, my service now works with a complete /29.  Somebody up in Frontier-land did something to either the ONT (several folks did several things, all very hush-hush) and then an upper-level regional engineer did something to the upstream router (from my service perspective) and magically the NetGear FVS318 is cranking on all five cylinders.  I cannot verify the regional engineer thing, it was only mentioned once, and then when I enquired about it again the subject was changed very quickly.

Here is what Frontier told me was the issue:

“Our Gateway Router sends out an ARP request every 6 hours or so to see if the same MAC address is still using an IP address.  It sends it out on 0.0.0.0 and if it does not get a reply, it shuts down that IP addresses' ability to get online.  The reason this happens is because many firewalls identify an incoming request on 0.0.0.0 as an attack.  So the Firewall sees it and then discards the packet.  The discarded packet never gets back to the Gateway Router, so the Gateway Router in essence says: don't let this IP address get online, I cannot verify the Mac Address.

At the time you were experiencing this problem, we were introducing a new optical network terminal (ONT) into our network; Calix.  It operates much different that the way the Alcatel Lucent system operated.  We plan on exclusively using the Calix ONTs for small business customers where it has been added, which is about 95% complete in the Oregon FiOS markets.”

Conclusions

1. Remember the Actiontec that did two addresses but then quit?  IF the ONT is configured correctly, maybe, just maybe the Actiontec CAN handle a /29 like the original sales person stated?
2. The ASA is a multi-talented device.  Dang did I learn a lot.  But in the end, not able to work for me.  My buck-twenty NetGear POS is working just fine.  Complete with SIP trunk.
3. Frontier has assured me that the process has been changed as any business-class (fixed IP) service will require the new Calix ONT (mentioned above).
4. Sales types are a necessary evil.  Business doesn’t work without them.  Caveat Emptor.  Be educated for yourself.  I don’t know if I could have done anything different, but I will be thinking about it for future actions.
5. When I got ahold of the correct layer of responsibility inside of Frontier, things went very smoothly, and I was treated much differently.  Like maybe I knew just a little of what I said.
6. Network technicians need to learn to look a tad past the CLI on their screen and stop assuming that “everything” is correct on their side of things.

In the end, I am happy, but I sure wish this had not taken so long to resolve.

YMMV

Lync Mirror RPC Server is unavailable

Don’t ask me why, I just know what I did to work around it.

Situation

I was engaged in establishing a database mirror for a new monitoring database for an established Lync 2013 CU4 EE pool.  When it came time to publish the topology and install the new databases, the mirror refused to instantiate with an error of:

An error occurred: "System.Runtime.InteropServices.COMException" "The RPC server is unavailable. (Exception from HRESULT: 0x800706BA

Lovely, eh wot?

The Fix

The SQL server version for all three servers in question is SQL 2012 SP1. I went through the usual routine of checking the services, the firewall rules, the rights and permissions on the SQL primary, mirror, and FSW.  All of that looked as it should.  Here is a good outline for you to follow should you need it.

What fixed it for me?  We turned off the firewall for all interfaces on all three SQL servers.  I know, but don’t ask me.  After the successful mirror activity, we turned the firewalls back on for all interfaces on all three SQL servers and everything seems to be happy.

Go figure.

YMMV