Thursday, April 4, 2013

BigIP F5 as Reverse Proxy for Lync Server, Windows 8 and Lync Metro App

I worked on interesting case today. A large enterprise customer began testing the new Windows 8 OS (desktop and Surface) and received numerous reports for sign-in issues with Lync Metro App. The client will spin for a while and go back to login screen. This seemed strange, since Lync 2010 and 2013 desktop client would work just fine, as well as Lync 2010 Mobile. In this post I will not only explain the problem and provide solution, but also share my method of troubleshooting and some techniques as well.

My approach to every problem is half academic, half technical – in this precise order. I don’t know how you guys work, but my brain retains information relevant to specific case only for relatively short period of time, or to be precise, until I move to the next case and so, for this task, I went back to my notes and refreshed the knowledge as of how Lync Autodiscover works. I decided to concentrate on Autodiscover process because Lync Metro App does not honor SRV records. Instead, it relies on Autodiscover to receive sign-in FQDN and/or URL.
  1. Client will try and resolve either lyncdiscoverinternal.contoso.com or lyncdiscover.contoso.com
  2. Client will request https://lyncdiscoverinternal.contoso.com
  3. Autodiscover will send back the Autodiscover service URL (typically Director Web Services)
  4. Client will authenticate and web ticket.
  5. Client will make new request while submitting the web ticket so that the Director can retrieve the user’s home pool information.
  6. Director will redirect the client to the home pool Web Services.
  7. Client will authenticate again and submit the web ticket
  8. Home pool web services (via Autodiscover) will respond with internal and external Lync services for the user’s home pool.
  9. Client (Lync Metro App in this case) will sign to the Registrar or the Edge server just like desktop client would after discover the registrar via SR record.
All right, how that we know how the process works, next step is to identify at which step Lync Metro App fails. I could only use Wireshark for this troubleshooting session. It is possible to use WS to decrypt SSL traffic indeed if… you have the server private key, which I did not in this case. So, the logical step was to just capture sign-in attempt and see if there is something I can hang on.

The first logical step was to examine the certificate returned from the server. I know, I know – all other service are working at the moment but… just in case. There are many ways to examine server certificate:


By using Internet Explorer. Simply visit the web site by using https protocol and review the certificate.




***While this is certificate from my lab, the process used was the same. Earlier today I verified that the required names (lyncdiscover.contoso.com, director_web.contosof.com and home_pool.contoso.com) were present. So, it is not certificate issue!

Alternatively, you can use tool called RUCT (Remote UC Troubleshooting Tool) http://www.insideocs.com/Tools/RUCT/RUCT.htm. Simply start it, go to Certificate Information tab, input the target URL and port, retrieve the certificate and examine it.


Because the issue was Metro App which runs on Windows 8 OS only, my next test will be from W8 machine. I opened IE10 and went to https://lyncdiscoverinternal.customer.com



I received the expected output and when opend it with notepad, everything looked just fine.


So far I’ve established that DNS resolves to correct IP, the certificate presented from the server have the correct names in the SAN list and the web service is functional.

This is the place to mention the fact that this customer uses F5 Hardware Load Balancer not only as HLB but Reverse Proxy for the External Lync web services. I checked with the network team earlier today and found that the current fimware on F5 is version 10.2

With this in mind, my next stop was… you guessed – F5 compatibility matrix (http://support.f5.com/kb/en-us/products/big-ip_apm/manuals/product/apm-clientcompmatrix-11-1-0.html)


Ouch! As one of my favorite standup comedians Russel Peters often say: "Somebody gonna get a hurt real bad".

I was really not sure what to make of this statement “Windows Phone versions 7 and 8, and Windows RT are not supported”. Any way, as stubborn as I am, next logical step was to "wireshak it" and see what’s up.


I see the normal “Client Hello” followed by “Server Hello”, “Certificate , “Server Hello Done” and “Change Cipher, Encrypted Handshake Message”. Absolutely normal handshake, nothing out of ordinary… handshake done via TLSv1. No RST (reset) i.e both client and server are “talking”. Since I don’t have the private key, I cannot decrypt the traffic and so, for now I will assume the client visited lyncdiscover URL.


I see second handshake in the sequence described above. This must be the authentication attempt. Remember, we must authenticate first in order to receive web ticket.

Oops. This is something new. Note that the TLS version the client requested is v1.2


The server responded with Server Hello TLS v1.2 ...


…but then… nothing. The client dropped the connection.

I already knew from my research that F5 at firmware 10.2 does not support Windows 8 (not that it is clear what exactly is not supported…). I also know that TLS v1.0 works fine (when I visit https://lyncdiscover.contoso.com Internet Explorer 10 does not bombs and so, my conclusion was – this has something to do with the interpretation of TLS v1.2 on both sides.

At this point, just for fun, I decided to disable TLS v1.2 on Windows OS level, which I hoped would force Lync Metro App to use TLS v1.0 and examine the SSL traffic again.

Protocols can be enabled or disabled via registry. To make your life easier, here is link to .reg file which will disable TLS v1.2 on your Windows 8 powered device.

After modifying the registry and reboot, Lync Metro App signed immediately. Wireshark trace confirmed the app used TLS v1.0 to communicate with the Web Services via F5 Reverse Proxy and signed successfully. Download, unzip, move the file to your Windows 8, Merge and reboot.

Simply said, we introduce two new registry keys:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Server


To revert to the default security settings when such "fix" will be no longer necessary, delete "TLS 1.2" key and reboot.

By no means I encourage you to reduce the level of security on your operation system. This “fix” is proposed with the sole purpose that you as IT professional can continue testing all features of Windows 8 and/or Lync Server until such “fix” will be no longer necessary.



29 comments:

Anonymous said...

Very nice case!

Anonymous said...

The problem with this one is that Windows update will then be disabled.. for some reason updates cannot be retrieved anymore..

EXARAY said...

Finally KB2973337 (http://support.microsoft.com/kb/2973337/en-us) solves this Problem.
So update and everything will be fine now :)

sheela rajesh said...

Really nice blog and impressive information you gave us.Thank you and i will expect more in future.
JAVA Training in Chennai
JAVA Training in Velachery
Software testing training in chennai
Android Training in Chennai
Selenium Training in Chennai
Hadoop Training in Chennai
JAVA Training in Chennai
Java Training in Tnagar

Chris Hemsworth said...

The article is so informative. This is more helpful. Thanks for sharing

Learn best software testing online certification course class in chennai with placement
Best selenium testing online course training in chennai
Best online software testing training course institute in chennai with placement
magento developer training

divi said...

thanks for your information really good and very nice web design company in velachery

Quickbooks Pro Support Phone Number said...

this is really great to know about.

Servo said...

Thanks for sharing this information.
Want to Buy Step Down Transformer or Are you Curious about What is Step Down Transformer? Read the blog to get your queries resolved before making a purchase.

Webroot Geek Squad said...

it is really great to know about. Thanks for sharing this with us.

Quickbooks Enterprise Support Phone Number said...

it was really good to know about. thanks for sharing this.

Shahil said...

Wonderful post and I hope you more updates from your blog. Thanks to you...
JMeter Training in Chennai
JMeter Training
Spark Training in Chennai
Pega Training in Chennai
Power BI Training in Chennai
Job Openings in Chennai
Linux Training in Chennai
Oracle Training in Chennai
Tableau Training in Chennai
Oracle DBA Training in Chennai
JMeter Training in Velachery

The India said...

Rice Bags Manufacturers
Pouch Manufacturers
wall putty bag manufacturers
fertilizer bag manufacturers
seed bag manufacturers
gusseted bag manufacturers
bopp laminated bags manufacturer
Lyrics with music

The India said...

we have provide the best ppc service.
ppc company in gurgaon
website designing company in Gurgaon
PPC company in Noida
seo company in gurgaon
PPC company in Mumbai
PPC company in Chandigarh
Digital Marketing Company

The India said...

we have provide the best fridge repair service.
Washing Machine Repair In Faridabad
LG Washing Machine Repair In Faridabad
Videocon Washing Machine Repair In Faridabad
IFB Washing Machine Repair In Faridabad
Samsung Washing Machine Repair In Faridabad
Washing Machine Repair in Noida
godrej washing machine repair in noida
whirlpool Washing Machine Repair in Noida
IFB washing Machine Repair in Noida
LG Washing Machine Repair in Noida

The India said...

iso certification in noida
iso certification in delhi
ce certification in delhi
iso 14001 certification in delhi
iso 22000 certification cost
iso consultants in noida

The India said...

iso 27001 certification services
iso 27001 certification cost
ISO 9001 Certification in Noida

Norton Setup with Key said...

it was really good to know about it. Thanks for sharing this with us.

Norton Product key said...

it was really great to know about.

mwww-office.com said...

Download Or Install Office Setup Step 1. Go to www.office.com/setup - Sign in, 2nd. Enter product key, and 3rd. we can also help you with office.com/setup online.

Mark William said...

www.trendmicro.com/downloadme
For Trend Micro Download , you must create a Trend Micro account from trend micro that can help you in smooth Trend Micro installation. Protect your systems and online data from multiple threats, including viruses, spyware, identity theft, malware, rootkits, Trojans, phishing attacks, ransomware, Adware, Worms, Bots, Horses, and other threats.You can activate the Trend Micro software by installing the Trend Micro download

html tutorial said...

Thanks for the Article.
html tutorial

css tutorial said...

Thanks for the Article.
css tutorial

javascript tutorial said...

Thanks
javascript tutorial

wordpress tutorial said...

Thanks for the Information
wordpress tutorial

smith machinist said...

Norton.com/setup helps you to deploy Norton setup on your computing devices. Here are the steps for deployment of Norton setup without any interruptions.

norton.com/setup

lily grace said...

the user must purchase a license of Avg antivirus from the retail store or online. Though download AVG software is the easiest process, your system must match the essential prerequisites of Avg antivirus. Once you will get AVG activation code and will create an AVG account , then you can install AVG with license number. install-avg-with-license-number | www.avg.com/activation | www.avg.com/retail

www.trendmicro.com/downloadme said...

Thanks for Sharing this Article.
www.trendmicro.com/downloadme
www.trendmicro.com/bestbuypc |
trendmicro.com/bestbuypc |
www.trendmicro.com besbuypc

norton.com/setup said...

Norton is a reputed and cost-effective antivirus suite
company which offers protection so that no virus can damage
your computer. It also provides many other products and
services apart from antivirus.

phautecouture said...

Panache Haute Couture, a leading Indian Designer House for Indian Dresses. You can Buy Designer indian wedding dresses lehenga dresses, anarkali suits, indo western bridal gowns,
lehenga choli online, Designer Sarees Online at Online.

lehenga choli


lehenga online