About Me

My Photo
TsooRad is a blog for John Weber. John is a Lync Server MVP (2010-2013). My day job is titled "Principal Consulting Engineer" - I work with an awesome group of people at CDW, LLC. I’ve been at this gig in one fashion or another since 1988 - starting with desktops (remember Z-248’s?) and now I am in Portland, Oregon. I focus on collaboration and infrastructure. This means Exchange of all flavors, LCS/OCS/Lync, Windows, business process, and learning new stuff. I have a variety of interests - some of which may rear their ugly head in this forum. I have a variety of certifications dating back to Novell CNE and working up through the Microsoft MCP stack to MCITP multiple times. FWIW, I am on my third career - ex-USMC, retired US Army. I have a fancy MBA. One of these days, I intend to start teaching. The opinions expressed on this blog are mine and mine alone.

2013/04/26

Is my HLB device a Reverse Proxy?

Ever since Microsoft announced that TMG was no longer to be sold, the Exchange and Lync implementers of the world have been somewhat at a loss.  Why?  Because the officially supported stance in the product documentation for both Exchange and Lync still says that the best practice is to abstract the environment from the nastiness in the outside world with a Reverse Proxy.  In other words, bring your outside web traffic in through your firewall, but don’t let it touch your servers.  Land that traffic on a Reverse Proxy (which one is the big question) and keep those baddies away from your internal environment. 

This raises the question of which Reverse Proxy to use.  Most of my projects have included some form of load balancer, and the load balancer product vendors will blithely state that “by definition, a load balancer is a Reverse Proxy.”  I know the fine folks over a Kemp Technologies certainly said that (guess where that quote comes from…) (to be fair, I get the same answer from Barracuda and F5 also)– and assuming that an HLB/VLB from one vendor operates pretty much the same as the next vendor, the purpose of this article is to put the concept to the test. In the immortal words of Ronald Reagan, “…trust but verify…” Let’s dive in.

Scenario

For our test, we will be using the following setup: 

  • An external user on IP 10.10.10.104. 
  • A firewall (TMG in this case) with an external IP of 10.10.10.58, and an internal IP of 1.1.1.2. 
  • A Kemp VLB on the internal net of 1.1.1.0/24; a WAS VIP of 1.1.1.45, and two VIP member servers of 1.1.1.46 and 1.1.1.47. 
  • I took the liberty of disabling the 1.1.1.47 node.  This enabled me to know exactly where the traffic would be going – making the network tracing much easier. 
  • The firewall is doing a NAT from 10.10.10.58 to the VIP on 1.1.1.45. DNS is setup all the way around so that the workstation is attempting to contact https://lswaspoolext.tsoorad.net which DNS tells it is 10.10.10.58.

Here is a graphical representation:

image

Here is what the Kemp looks like, just in case you don’t trust me:

image

Workstation Traffic Analysis

From the workstation the browser saw this:

image

From the workstation, NetMon saw this:

image

From the top, the workstation on 10.10.10.104 is talking to the network printer, the domain controller, the network printer (again), (and again), www.google.com, the DNS resolution of 10.10.10.58 (our target URL of lswaspoolext.tsoorad.net), the firewall IP (again), Google (again), my network WAP (10.10.10.16), my DirectTV (10.10.10.103) and this workstation.  The IPV6 traffic is 100% between my work laptop and my Twist (which was my external workstation for this test).

Workstation Conclusion:

The workstation for this test is talking ONLY to the 10.10.10.58 address to get the data from lswaspoolext.tsoorad.net/hosting/discovery.

Firewall Traffic Analysis

Here is the Netmon trace on the Firewall:

image

Without proving the point again, I can identify all the traffic sources that the firewall is talking to on both interfaces.  At no time during our test did the firewall talk to 1.1.1.46 or 1.1.1.47.  The firewall did trade traffic with 1.1.1.45, the WAS VIP on the Kemp.

Office Web Apps Server Analysis

And finally, to whom was the WAS talking?  Let’s see:

image

It would appear (to me at least) that the server ls2013was1.tsoorad.net – which is responding to and delivering the content requested by our workstation  - has no knowledge of the workstation, or for that matter, the firewall through which the workstation is connected.  Slick!  Apparently, the techie mavens I talk to regarding using an HLB device as a Reverse Proxy might have known what they were talking about (I will never tell them – their heads are already big enough).

Conclusion

We traced the traffic all the way from the origination workstation through each step in the path.  For each hop, with our interest on who talked to who, we saw that the workstation never communicated directly to our target.  The traffic was definitely PROXIED by the Kemp VLB – to whit, traffic stopped at the HLB, the HLB retrieved the data requested, then returned that to the requestor.  To be fair, the TMG (our firewall) was doing the same thing, but I think you get my point.  Now you can tell your client with all equanimity, yes, the Kemp HLB is also a Reverse Proxy. Can we make the same assumption for other vendors?  I defer to Ronald Reagan.

YMMV

2013/04/25

Plantronics Calisto 620-M Device Review

Disclaimer:  Plantronics provided the device.

What it is

The Plantronics Calisto is a “Blutetooth Wireless Speakerphone + Bluetooth USB Adapter” device.  Long words short, a speaker phone device.  It does make me wonder about the marketing/media-speak though.  The Calisto comes with a charging cable that is for charging only.  OK.  Got it.  But if the device REQUIRES the Bluetooth connection to operate, isn’t it reasonable to assume that a device-specific adapter would come with?  Does it operate better with the custom adapter than with my laptop’s built-in Bluetooth?  On a side note, does anyone know why this technology is labeled “Bluetooth?” 

Ok, I went and looked it up.  Provided we can trust the ubiquitous wikipedia.com -image

No matter.  Let’s take a look at this Calisto and how it operates with Lync – our primary reason for living, yes?

What is in the box

Here is what comes out of the box. 

image

I did not include the nifty carrying bag in the photo. Also, I did not include the warranty and safety paperwork.  Which brings me to a quick question:  Safety notice?  Do we expect that folks are going to take this unit into the bathtub or use it in a flooded basement? I get it, I really do.  But this litigious society we live in slays me.  What ever happened to personal responsibility?  </Rant Off>  Adelante! 

Build Quality

From me playing with the Calisto, it feels very solid.  The USB connections are nice and tight.  The USB adapter holder thingy in the Calisto bottom panel has some rubberized feet and I don’t think the dongle will fall out of the holder without some secret agent work.  You could, of course, put the dongle in backwards – at which point  it will fall out at the worst possible moment.  The Calisto bottom panel itself has rubber booties; it did not slide around on my desk.  Nice.  The USB adapter connection to my USB port was tight.

image

 

Features

The list is not too long.  The Calisto is pretty simple.  Volume up/down.  Call button; Mute button, USB and wall charger, and a USB dongle.  It will also pair with your cell phone.  The instructions are in the four-language instruction sheet that comes in the box. Odd.  Why does the Calisto come with only four languages on the instruction sheet but a headset gets 15?  Is the Calisto somehow less worthy?  Out of the box, the instructions lied to me, or, my USB hub produces more power than the average laptop USB port.  The claim was 2.5 hours to fully charge from the USB port.  And that 30 minutes is needed to initially charge.  I did neither.  From plug-in to setup to fully charged took less than 20 minutes.  In fact, when the battery charge indicator went dark I was concerned that the unit had somehow failed.  Maybe the unit was partially charged coming to me in the tidy retail box?

Lync Integration

As an “Optimzed for Microsoft Lync” device, the Calisto supposedly works with Lync, understands Lync, and controls Lync to the extent that the function buttons on the Calisto will affect the corresponding function in the Lync client.

Well, yes it does.  The Lync integration was completely seamless. The instructions would have you download some software. I have no idea what the software does, I never needed it, so I never downloaded it.  Why mess with something that is already working perfectly?

The instructions cautioned against using a USB hub, but I like to live on the edge, so I plugged the dongle into my hub.  I plugged my charger cable into the same hub, and I was done.  Done as in I did nothing else and Lync 2013 was operational with the Calisto 620.  Mute muted.  Call button terminated calls.  Volume volumed.  All with pleasing dulcet tones to announce function.  The Calisto took over as my audio device.  The only thing it cannot do is dial the number for you.  But if you make a call with the Calisto plugged in and selected as your audio device, you will be pleased with the outcome.  A very minor complaint is that the Calisto became my primary and it relegated my headset (my preferred device) to secondary.  Even when turned off, the Calisto was primary and I had to manually swap the preferred and secondary devices (This could be a Lync thing and not a Plantronics thing).

Here you see the Calisto with a call in progress, with the microphone muted, and the Lync 2013 client showing the same thing.

image image

Complaint Department

It could be just me but I found it difficult to tell if the button got pushed in terms of feel.  The unit responds directly with sounds and lights, but John is a button-feel person.  I think a little tactile clicky or break or something would be in order.  I also think the USB pairing lights could be a touch a lot brighter.  If you aren’t paying attention, you might think the USB connection died. Also, why not have a dial pad on this thing?  Then you could press the call button, dial (DTMF actually) and have a complete speaker phone.  Of course, I may have missed something, but I don’t want to push the call button on one device and dial from another.  I guess if you want that you need to pony up the extra $50 and look the Calisto 800 series.

General Observations

Even plugged into my USB hub, overall performance and audio quality is excellent.  Both send and receive is very clear.  The microphone picks up the human voice spectrum nicely.  The speaker has enough volume and richness for the task; it sounds very nice. Excellent, really.  I can see placing this unit into a small conference room where 4-6 people are around a small table and one person brings their laptop or perhaps there is a conference room workstation.  The Calisto 620 would shine in that role.

YMMV

2013/04/24

Plantronics Blackwire C320-M Lync Device Review

Disclaimer:  Plantronics provided the headset.

What it is

The Plantronics Blackwire C320-M is a “Optimized for Microsoft Lync” stereo (bi-aural) USB headset.

image

Build Quality

This headset demonstrates the first class build quality that we have come to expect from Plantronics.  In particular, on my laptop, the USB connection is nice and tight – not all loose and nasty like some devices.The rotating boom mike stays put rather than flopping around, the foam earpieces look and feel good, and the wire leads in and out of the inline control button thingy have reinforcements that make me think that the leads won’t be breaking or pulling out anytime soon.  Overall, pretty doggone nice.

Features

The headset lead has the typical built-in buttons for call control, volume and muting – except that as a Lync-optimized device, these controls actually control Lync also.  The microphone boom has very nice feel to it.  And you can bend it to fit whatever position you want it to be in, and it stays there.  I like it.

image

Lync Integration

The documentation for this headset consisted of a single multiple-folded (made up that word just now!) 15 language, single piece of paper.  Of which 4 pages actually covered everything you need to know and then some.  What is more important was that on multiple systems that I checked this device on, I never needed to read anything, configure anything, or mess with anything.  I plugged it it, and it worked.  The call button takes Lync off hook as expected and terminates the call if you press it again.  The mute button makes a rising beep when it mutes, and a descending beep tone when you unmute.  And yes, the buttons controls the Lync client as well.

The buttons light up to show function.  In a call and muted:

image image

The volume up/down makes louder and softer tones also – with no discernable change in the system settings (volume mixer) – but the volume goes up and down.  Nice trick, eh wot?

The Plantronics official tear sheet is here. I have used this headset for a week on four different workstations; used it for meetings, regular calls, Lync meetings, calls to cell phones and calls to other Lync users.  In each case the audio was crisp and clean.  The microphone is sensitive, but the device also seems to be very good at blocking the extraneous background (i.e.: something that is not right in front of the microphone boom).  According to the official brag material up on plantrontics.com, this headset has “…Dynamic EQ…” – whatever, it seems to work well.  When it came to Lync, both 2010 and 2013, this thing just worked, and worked well.

Complaint Department

There were two things I did not like about this unit.  One of them might be because Plantronics shipped me a bulk-order unit and not a boxed unit.  The first is that there is no clip or something to hold the headset lead to your person/clothing/otherlocation.  I tend to find the inline control module under my forearm on my desk with me accidently pushing buttons in an un-wanted fashion.  Annoying.  And such a little detail.  The second is that the wire/cable from the USB connection to the headset itself is sort of … stiff.  It might be just me, and I am wire bundle challenged – but the headset lead kept getting in my way.  Over the course of a week, it never got more flexible.  OTOH, this might indicate that it won’t break!

General Observations

Overall, I like this device.  When you consider that a quick Google-fu session prices this unit in the $35-47.50 – this headset starts to look better and better.  And comfy for my brain housing group also.  Over the week, I not once thought that I had something uncomfortable latched onto my gourd.  Combined with the overall workings and build quality, that might be what makes the decision.

YMMV.

2013/04/22

Lync 2013 EE pool won’t start

Sub-title: We get to explore the Reset-CSPoolRegistrarState cmdlet

Life is tough.  Apparently the drunk that took out the power station the other day and caused a major power outage in my zip code is going to find that out.

What I found out was that when my lab restarted, the Lync 2013 Pool FE servers would not start any service except for the Replication Service (Lync Server Replica Replicator Agent – who names this stuff anyway?).  Wowzer.  This is not good.  I know that having only two nodes puts me living on the edge, but it is only a techie lab with no real users!  Never had this bad of a failure before.  I have been good and followed the restart instructions for a two-node EE pool.  You have read that, right? If not, here you go.  Look down at the bottom of the page – you will catch on quick. 

Issues like this is why I always recommend having Lync 2013 EE pools with at least three members.  Can’t have that quorum deficit biting you!  At the very least, I rant and rave at some length about setting up for failure and make sure the administrators of the impending doom system know what is what and the risks and the mitigation process.

In our little story, the command

Reset-CSPoolRegistrarState –ResetType QuorumLossRecovery –PoolFQDN ls2013pool.tsoorad.net

(runs on each pool FE member) did not achieve the desired pool state.  At about this time, I am really starting to exercise my military jargon skills.

When the power did come back on, there was no way to follow the recommended practice of starting the servers in reverse order they went down – they both went down at the same time!  The event viewers had enough red errors in them to allow me to open a paint store.

What to do, what to do?  Well, our intrepid hero simply dusted off the command syntax for Reset-CSPoolRegistrarState. Take a gander at this explanation of the reset types:

* ServiceReset – The RtcSrv and fabricHostSvc services are stopped and restarted. A service reset will be performed if the ResetType is not specified.

* QuorumLossRecovery – Reloads user data from the backup store for any routing groups currently in quorum loss. (A quorum loss occurs when neither a database nor its replicas are available.) Data not yet written to the database could be lost when you do this type of reset.

* FullReset – performs the same type of reset as QuorumLossRecovery but, in addition, rebuilds the local Lync Server databases. This type of reset can be potentially long and resource-intensive.

* MachineStateRemoved -- Removes the specified server from the pool. This type of reset should be used only when the server in question (or its databases) have been permanently lost.

A short period of pondering removed the “MachineStateRemoved” from being a valid fix, and the “QuorumLossRecovery” had already failed. so I ran the following (you might want to run the Lync Management Shell as Administrator, or you will find out why I always run PowerShell as Administrator): 

Reset-CSPoolRegistrarState –ResetType FullReset –PoolFQDN ls2013pool.tsoorad.net

I then did a “Shutdown /r /t 0” to get a system restart.  One at a time, mind you, but I booted both my EE nodes after running the reset-cspooregistrar cmdlet.  Success!  The sun is in my sky once again.

Now, this is a little drastic, but what the heck?  To quote Alfred E. Newman, “What, me worry?”  In my case, the lab pool was not starting anyway.  In a real life situation the data loss would be that data that had not committed to the local database prior to the failure.If you run into this, and the pool won’t start anyway, there is probably a bigger issue than losing a few contacts or a few chats.  .

YMMV.

2013/04/18

OCS R2 client and Lync 2013 Server

Scenario

I have a coexistence issue: I need to have a boatload of R2 clients connect to a new Lync 2013 environment.  Backwards compatibility has been installed, the R2 environment has been imported with Topology Builder, and users can be moved between the R2 pool and the new 2013 pools.  We can chat back and forth, send files, use A/V and even participate in the same conferences.  External, remote, and Federation scenarios work just fine.

So what is the issue?  Today, the _sipinternaltls._tcp.domain.com SRV record points to the FQDN of the R2 pool.  Which works just fine.  When the 2013 pilot group cranked up, they found the lyncdiscoverinternal.domain.com record, hit the 2013 pool as expected, and logged in.  Perfect.  The issue in front of us today is moving the target of the _sipinternaltls._tcp.domain.com from ocspool1.domain.com to sip.domain.com  - we want to decommission the R2 pool at the end of this project.

Our plan is to simply set the SRV record TTL and the associated A record TTL to something very short, and then after waiting for that propogate, change the SRV target to sip.domain.com.  To test this, we went to an R2 client and set the manual configuration as shown. 

clip_image002

However, this failed with “server not available.”  We also noted that on the initial setting of “manual” the TLS option was greyed out and the initial connection attempt failed.  Checking the manual settings again revealed the TCP setting selected, but now we could change to TLS as the settings were no longer greyed out.  Odd.  Strange.  And totally consistent from machine to machine.  Can’t explain it.  After the initial failure, the TLS setting would hold, but still we had “server not available.”

The Fix

After running around the settings several times, and checking the Client Version policy at least twice, and making sure that the workstations in question could resolve sip.domain.com properly, we turned to the Client Version Policy again. It turns out that this is the default setting for the Lync 2013 Client Version Policy.  Note that it says OC 3.5.6907.233 and OLDER should be blocked. 

image

A cursory check of the clients tested showed versions of 3.5.6907.261 and 253.  Newer right?  Apparently not!  Setting this policy element to ALLOW fixed the issue and now the mix of R2 clients can login just fine using sip.domain.com as the “director.” which points to the new 2013 pool.  Because the SIP domain and the AD domain names match, there are no certificate issues.

YMMV.

2013/04/12

Lync 2013 Internal and External Web Sites

Lync 2013 (and 2010 for that matter) documentation recommends using a Reverse Proxy to abstract the Lync Server Front End Web Services from the baddies on the outside world.  A cursory look at a Lync Front End Server will reveal that the Lync Server installation process does indeed create two websites.  But what is the difference?  What might be the possible reasons for having two websites?

Here is a typical Lync EE pool server IIS manager showing the Lync Web Services.

image_thumb[1]

Two websites. Because the websites are both using the same IP address, different ports are needed. If you look at the binding for each you will see the External answering on ports 8080 and 4443 while the Internal is setup on 80 and 443. 

image_thumb[5] image_thumb[6]

Because we are exposing our internal server we want the Reverse Proxy to stop traffic and play fetch for us.  Understandable, yes?  Because we have two websites that are different but on the same IP, we need different ports.  Again, understandable.  But if you look at the following eye-chart – you can also see that there are differences in the virtual directories. The Internal virtuals down below that are different from the external website (in addition to, and similar but named differently) simply have no business being exposed to non-internal users; I think the virtual directory names provide all the clues you need to agree with that assessment: CSCP (the control panel), OCSPowerShell, and RGSconfig (Response Group Configuration page) jump out at me. 

image

If your Lync Web Services get their authentication scheme messed up (now how would that happen?) you can also use this handy chart to return the website and virtual directories to their initial state.  How convenient is that?

So, we can see that there is a few reasons to have two websites:  two websites on the same IP but using different ports, the difference between virtual directories on the internal v external websites, and the content that is not very desirable to expose to external users who just might not be your best friend.

YMMV

2013/04/03

Lync 2013 CU1 UCWA Errors

Just had a small issue trying to publish Lync 2013 Topology where it would appear that the Lync Server 2013 CU1 update duplicated all the 2013 FE Server MSRTCSIP-TrustedService entries for the UCWAInternal and UCWAExternal services.  Everything looked fairly good until it came time to enable-cstopology.  Here is the error we got.

“Multiple Active Directory entries were found for type “MS-RTC-TrustedService” with ID of fqdn.domain.com/UcwaExternal.

image

Fixing ONE of these resulted in the next item on the list being found and then erroring out.

Ouch.

I went through and renamed the service names on all 4 FE servers to update so the enable-cstopology only saw one valid entry.  This was needed on three (3) Enterprise FE nodes and on an SE FE server in the environment.

image

Will update this accordingly if this proves to have borked up something else.  Why this occurred is not known to me, other than running the CU1 updates into the environment.

YMMV.