Wednesday, December 11, 2013

User might experince delay when join Lync Online Meeting.

Recently I’ve worked with large enterprise customer on troubleshooting strange issue with Lync Meeting join. Users from the pilot (50 people) group would report prolonged (up to 30 seconds) delay between the splash screen and the actual Lync Meeting join. A pattern was not observed (although there is one, just not obvious at first) – it would happen to random users, at random time, joining different meeting ID’s on different pools.

In this article I will explain the background of the “issue” and describe possible solution.

When user click Lync meeting link in format https://meet.contoso.com/user/M33T1NG1D , the request is processed by Director Pool member (if Simple URL points to Director), where after Meeting Discovery, Director Web Service will respond with “301 Moved permanently” to the original request. Client then sends new GET request with destination Web Service of the pool where the meeting will be hosted and meeting join completes.

As Lync server is installed on Windows Server, HTTP requests are served by Internet Information Services (IIS). The web sites (Internal or External) and Virtual Directories within the sites are driven by IIS Worker Process and are logically defined within IIS Application Pools.



As we see, not only whole Web sites (Internal or External), but also Virtual Directory within a web site might be configured to run under dedicated Application Pool.

One aspect of IIS Application Pool is “Recycling”. This process is defined in TechNet as “Internet Information Services (IIS) application pools can be periodically recycled to avoid unstable states that can lead to application crashes, hangs, or memory leaks.” When we install Lync server components, a default value is configured - 1740 minutes (29 hours).




When the Application Pool recycles, all items in the memory are released, a new w3wp process is created, configuration read and applied, .NET reloaded and we site is back serving requests. Under normal circumstances, had we had a request for simple web page as https://dialin.contoso.com , user would not experience visible delay because there is no complexity in assembling and serving this page.

However, Meeting Join request requires extensive server side processing, not discussed in this article. In reality, it takes about 22 seconds for the very first meeting join request to be served after Application Pool Recycle. This fact might cause inconvenience under certain circumstances – in my case a very large front end pool (nine pool members) and only 50 pilot users on this pool. Because the pool web services are Hardware Load Balanced, there is great chance that unique user would hit pool member where the Application Pool was just recycled and expertise 22+ seconds delay. Reports of type “It took forever to join the meeting this morning” were pouring in and frustration was growing fast.

The solution to eliminate the delay is rather simple – monitor the System Event Log for EventID 5074, examine the event description for which Application Pool was just recycled, and fire up Scheduled Task to execute PS script which will simulate real meeting join request. The request will perform “Application Pool warm-up” and because we do so immediately after recycle even, chances a real life user to attempt to join after Pool Recycling, but prior to Scheduled Task execution is practically non-existent.



Steps to configure Application Pool meeting join warm-up:
  

Step 1: Create folder “Scripts” on C:\
  • Inside this folder, create text file MeetExtWarUp.txt
  • The file should contain single line Invoke-WebRequest https://FE_Server_FQDN:4443/Meet?key=A1B2C3D4 where FE_Server_FQDN is the FQDN of your SE or EE Lync front end server.
  • Rename the file to MeetExtWarUp.ps1
  • Follow the steps above to create second .ps1 file where the text is 
    Invoke-WebRequest https://FE_Server_FQDN/Meet?key=A1B2C3D4
***Note that we use “:4443” for the External Web Site. It will be omitted in the file used for the Internal Web Site.


Step 2: Create Scheduled Task to execute the .ps1 script when AppPool recycle event is detected.

  • Start Task Manager as Administrator
  • Expand Microsoft/Windows node, right click and select "Create Task"



  • Name the task (WarmInternalAppPool in this example)
  • Check "Run with highest privileges"
  • Click "Change User or Group" button and select SYSTEM, then click OK.




  • Select "Triggers" tab.
  • Click "New"
  • In the "New Trigger" window, select "On an event" from "Begin the task:"
  • Toggle "Custom" under Settings
  • Click "New Event Filter" button



  • In New Event Filter window, click XML tab.
  • Check "Edit Query manually" check box
  • paste the following Query, where pay attention on the string describing which AppPool we want to use as trigger. For the Internal Web Site, we use LyncIntFeature and for External - LyncExtFeature.

<QueryList> <Query Id="0" Path="System"> <Select Path="System"> *[System[Provider[@Name='Microsoft-Windows-WAS'] and (EventID=5074)]] and *[EventData[Data[@Name='AppPoolID'] and (Data='LyncIntFeature')]] </Select> </Query> </QueryList>

  • Click OK on this window.
  • Click "Action" tab.
  • Click "New"
  • Type powershell.exe in "Program/script" box.
  • Add the following in "Add adgument (Optional):" box -ExecutionPolicy Unrestricted c:\Scripts\MeetExtWarmUp.ps1 where MeetXXXWarmUp.ps1 is the the name of the script we prepared earlier for the appropriate AppPool.



Complete the setup. your final configuration will look similar to mine.


What we just configured:

  1. Monitor the System Event Log for EventID 5074 (indicating AppPool recycle event"
  2. Examine the event to determine which AppPool was just now recycled
  3. Fire up the appropriate PowerShell script to sent fake meeting join request, thus forcing the server perform Meeting Join procedure (the server is not aware this is not a real meeting ID)

Step 3: Verify if our scheduled task works as expected.

Upon scheduled AppPool recycle, we find the corresponding entries as shown


Also, when examine the IIS log, we expect to see appropriate entries, indicating the meeting join request was invoked and precessed.


In conclusion, it is possible to "warm-up" the app pool of Lync Server Web Site to avoid meeting join delay.

This article can also be used as reference as of how to capture and evaluate specific event and execute Scheduled Task based on the input.

20 comments:

Mark Poulton said...

Thanks for the post. I took your idea and made a more generic script. Should work. Basically I wanted a batch file I could run that would automagically generate the FQDN since I tend to deploy large pools.

@echo off

REM
REM Determing the domain the compmuter is in
REM

FOR /F "tokens=1* delims=REG_SZ " %%A IN ('REG QUERY HKLM\System\CurrentControlSet\Services\Tcpip\Parameters /v Domain') DO (SET CURR_DOMAIN=%%B)

REM
REM Converting domain to upper case curr_domain
REM

IF [%curr_domain%]==[] goto huh
SET curr_domain=%curr_domain:a=A%
SET curr_domain=%curr_domain:b=B%
SET curr_domain=%curr_domain:c=C%
SET curr_domain=%curr_domain:d=D%
SET curr_domain=%curr_domain:e=E%
SET curr_domain=%curr_domain:f=F%
SET curr_domain=%curr_domain:g=G%
SET curr_domain=%curr_domain:h=H%
SET curr_domain=%curr_domain:i=I%
SET curr_domain=%curr_domain:j=J%
SET curr_domain=%curr_domain:k=K%
SET curr_domain=%curr_domain:l=L%
SET curr_domain=%curr_domain:m=M%
SET curr_domain=%curr_domain:n=N%
SET curr_domain=%curr_domain:o=O%
SET curr_domain=%curr_domain:p=P%
SET curr_domain=%curr_domain:q=Q%
SET curr_domain=%curr_domain:r=R%
SET curr_domain=%curr_domain:s=S%
SET curr_domain=%curr_domain:t=T%
SET curr_domain=%curr_domain:u=U%
SET curr_domain=%curr_domain:v=V%
SET curr_domain=%curr_domain:w=W%
SET curr_domain=%curr_domain:x=X%
SET curr_domain=%curr_domain:y=Y%
SET curr_domain=%curr_domain:z=Z%


echo.
echo Computer Name: %computername%
echo.
echo Current Domain: %curr_domain%
echo.
echo.

choice /t 1 /n /d y


PowerShell -NoProfile -ExecutionPolicy Bypass -Command "& {Invoke-WebRequest https://%computername%.%curr_domain%:4443/Meet?key=A1B2C3D4}"

rem PowerShell -NoProfile -ExecutionPolicy Bypass -Command "& {Invoke-WebRequest https://%computername%.%curr_domain%:443/Meet?key=A1B2C3D4}"


rem choice /t 60 /n /d y

Oliver Maurice said...

You should also visit this blog for some information about homework and how it was originated. Its pretty interesting and funny article and I think you will be impressed on how thing turned out.

norhan said...

مكافحة حشرات بالخبر افضل شركة مكافحة حشرات بالخبر
مكافحة حشرات بمكة افضل شركة رش حشرات بمكة
مكافحة حشرات بالمدينة المنورة افضل شركة رش حشرات بالمدينة المنورة
مكافحة حشرات بالدمام افضل شركة مكافحة حشرات بالدمام

Harper Chelsea said...

Wow, such an awesome blog you have written there and you and I get exactly what information I am looking for, in the third paragraph you put amazing effort to explain the theme of the content. As a content writer, I can understand efforts because when students ask me for programming assignment help,I do the same.

Kevin Wick said...

Use Finance assignment help to finish your finance or accounting papers without messing yourself. You will get assured solutions to your numerical questions within your defined date or time.

Lilly said...

We are here for a significant time frame and give the best finance assignment help to the students.

Casestudysamples said...

Aside from conveying outstanding work, we have ensures that buy case study stand apart from the rest. We can show you how the conditions and your own mentality impact your presentation and show you what to do to improve as student.

Chris ray said...

Transfer money online in seconds with PayPal money transfer. All you need is an email address.
Paypal Login
Paypal Login
Paypal Login

lucifer1413 said...

The MyTV Code is a free application that can be downloaded for free and will let you watch TV on your PC.
www.amazon.com/mytv |
Xfinity Login |
DirecTV login |
ij.start.cannon |
Cox login |
Comcast login|

Jackson Roy said...

Are you facing login problems with your PayPal login account? If yes! Then refer to the solutions that are mentioned below. paypal-prepaid.com login
So, stay with us. We are a self-dependent technical help service provider for Computer and other devices related issues. Our experienced & certified experts are available 24*7 for you. Our technicians will be always there to help you.
If you are having trouble while using canon printer on your device and you have tried every solution to fix the issue, then the last thing you are left with is to uninstall and re-install the Canon Printer driver. So, in this article, we have clearly discussed how to uninstall the printer on a Mac. This will be followed by a discussion on the re-installation process that can be easily done via canon.com/ijsetup canon,com/ijsetup
The streaming of video content online is trendy today as people move to modern entertainment. They subscribe to various streaming services like Netflix, Hulu, etc. Among other things, Amazon Prime is one such streaming service. This service not only allows you to stream videos but also allows you to utilise many other features. And this is all possible by getting amazon.com/code and redeeming it. how to use amazon discount code and redeeming it.
TD Bank Login is known to be a Canadian international subsidiary of the Toronto-Dominion Bank and its original town is Toronto, as suggested by the name. Their locations are worldwide and several thousand staff endeavour to fulfil customer requirements.
wells fargo login has become a renowned and trustworthy financial services firm, whose main focus is on aiding and benefiting society as a whole. They always highlight their assets, which are reported to be over a trillion dollars.
Yeah, at the moment, I don't remember my user ID and need to retrieve it, unfortunately. Whenever I hit Reset Password or Retrieve User ID and try to verify my information, the page just sits there. I've left it sitting for about 20 minutes before and it never did anything.Read More about: best buy login credit card

devin wood said...

Greetings for the day,

I am from Ozpaperhelp.com, One of the top networks of assignments and thesis would like to say that these posts ​are very unique and authentic. Basically, Our Institution is worked for making well-organized project reports, which are available at an affordable cost.

Dissertation Experts
Doctoral Dissertation Help
Dissertation Proofreading Services
Write My Assignment for Me
All Assignment Help
Physicist Assignment Help
PYTHON Assignment Help
HTML Assignment Help
Common Lisp Assignment Help
Computer Network Assignment Help
Civil Engineering Assignment Help
Environmental Engineering Assignment Help
Data Warehouse Assignment Help
HND Warehousing Management
HND Diploma in Web Design
HND Diploma in Digital Marketing
HND Diploma in Project Management
HND Diploma in Entrepreneurship
Strategic Value of Human Resource Management
Innovation Management at Apple Company
Strategic Management Analysis
Risk Management Committees
Operations Management Assignment

Regards,


Gloria Benedict said...

A set of software designed for a specific range of tasks and a specific user. In order to choose the best software or develop it, you need to use the following principles: performance, economy, control, and data security, compatibility, and flexibility, as well as hire excel expert. You will have fun, get positive feedback and achieve your goals.

Monnika Jacob said...

No matter how big and successful the brand is, consumers always have problems. Working as an expert author for Dissertation Proposal Writing Services, I consider only successful brands that assist their customers in a timely manner.

Unknown said...

Oh, I had such a problem when I tried to join Lync Online Meeting because it was necessary for my writing a case brief https://fabulouswritings.com/services/case-brief-writing-help Thanks for this explanation and now I know more about it.

devanderson said...

A consumer must ensure that all of the guidelines for Canon IJ Printer Configuration provided in an instruction manual that comes with the Canon printer are followed. The printer receives the necessary electricity, as well as ink that is properly installed in the printer. Search for an Ethernet connection to connect the printer to the network, and the router will also require an Ethernet wire. The following move is to search the driver and applications for your Canon printer mounted on your desktop. Using it for Canon IJ configuration when the user receives some CD with canon printer, then the user can set it up for all the CD, then the user can set up the program with it
Get More info: canon.com/ijsetup
ij.start.canon/setup http /ij.start.canon/setup Follow the instructions below to start the process of establishing your Canon Inkjet Printer. A gadget, a laptop or a PC is the initial stage in this procedure. Now, open your device's online browser and choose for any Google Chrome, Internet Edge, or Mozilla Firefox web browser. Place the address bar on the top panel of the browser. Click on the "ij.start.canon" bar and enter. Introduce. Now you are going to visit the website's homepage. You'll notice tabs on different operations on the homepage. The "Set Up" icon has to be clicked. You're moved to a new window when you click on it.
Get More info: ij.start.cannon

Carl Wood said...

If the Samsung printer is offline or has an offline status, it displays offline or offline pop up windows that do not convey or display errors. It's not much to correct how a Samsung Offline Printer should be brought online. Technical persons are most often willing to give Samsung's printer the excellent features to record the results of the printing process. The printing necessity of your paper is first and foremost that your device and others understand the presence and other existence of Samsung's printer module. Finally, it is recommended that you take the Samsung printer solution offline with the assistance of a professional team. We never use a short-cut to reduce the presence of a problematic issue. If you only want to access a positive effect, you must consult with our expert team. Otherwise, you'll be surrounded by a slew of technical issues. Please do not hesitate to bring your questions to our expert, who will respond as soon as possible.
visit our site for further steps: Fix Samsung Printer Offline

Rian said...

event planner Viewers enjoy visual novelty. Visually changing things up can be as important as the content itself when it comes to keeping eyes on the stream happy birthday mom, happy birthday sister and happy birthday brother

Irony said...

There is no way to get this useful info from your way. Its very fabulous and easy for me to get it from FMWhatsApp. I am very thankful and I appreciate your intelligent work. Awesome!!

Bad Credit Finance Help said...

A six figure salary is a standard in the corporate world. This is a salary of at least $100,000. A five-figure salary is for those who earn less than $100,000 but more than $10,000. A million dollars is a good salary.

Kaylee Brown said...

Thanks for sharing such a fabulous article; you are helping both students and paper helpers with your insightful articles.