Visual Studio: Web Load Testing

Source code on my Git Repository.

Contents

• Running the Test Web Server
• Hello (Web Test) Word
• Validating the Response
• Using variables (a.k.a. “Context Parameters”)
• Using a Data Source
• Using a Web Test as a Load Test
• Errors

Visual Studio Enterprise Edition has a project type called the Web Performance and Load Test Project.

If you can’t see it then you probably just have to run the Visual Studio Installer and install it.

Easy.

Running the Test Web Server

This is very easy!

If you clone the Git repository then you’ll see that as well as the Web Performance and Load Test Project there’s a console project too. The console project is a self hosting web server that we can use for running the web tests. If you run it without admin privileges (which is how you’re going to run it if you’re sensible) then it’ll throw a AutomaticUrlReservationCreationFailureException. In order to avoid this you’ll need to open the port it’s listening on. Assuming you’re using the default port of 9000 then you’ll need to run the following in a command prompt that is running as admin.

netsh http add urlacl url="http://+:9000}/" user="Everyone"

The web server lets you add messages, they’re saved in memory and can be retrieved either by ID or as a list of all of them.

Method	Path	Description
GET	~/	Return all messages
GET	~/{id}	Get specific message
POST	~/{message}	Store new message

Hello (Web Test) Word

Add a new item to the project and select ‘Web Performance Test…’ Annoyingly, it will assume that you want to record a test and start recording one for you and open IE for it too:

Anyhow, close it and click ‘Stop Recording’ and you’ll have an empty web test page where we can put our first web call.

To rename the test from boring ‘WebTest1’ to something useful like ‘HelloWorldTest’ we just change the name of the file. The test will update itself.

Now, we’ll get our first web request. Right-click on the node and select ‘Add Request’. It’ll create a request node pointing to http://localhost/. Right click and select ‘Properties’ (or press F4) and you’ll see the editable properties.

Here we can change the URL, the method and more. We’ll post a new message to our test server (get it running without debugging if it’s not).

Set the following properties:

Property	Value
Url	http://localhost:9000/hello-world
Method	POST

Click the run button on the top left to run the test.

All going well you should see a successful run like this:

NB: If you get a 404, check the test server is running

If you click on the test then the tabs underneath will contain details about the call:

• Web Browser will attempt to render the response in a browser. In this case, it just shows the JSON we return {"Id": 1 }.
• Request will give you some details about the request including headers, cookies, query string parameters and form parameters.
• Response will give you the headers and body for the response.
• Context isn’t important yet. It contains the variables for this test case.
• Details gives you, well, other stuff.

Validating the Response

Go back to the web test again and right-click on the request node (the one with http://localhost:9000/hello-world). Select ‘Add Validation Rule’ from the context menu. You’ll be presented an ugly dialogue box with some validation options. We’re going for the very simplest here so select ‘Find Text’ and set the text to some random thing so that we can see the test fail before we see it pass.

You’ll see the validation rule appear. If we run the test then we’ll see it fail.

Now, to see what exactly went wrong you can click on the tabs. When you get to the ‘Details’ tab you’ll see what happened:

Rule	Type	Result	Parameters
Find Text	Validation	The required text ‘Incorrect’ did not appear in the HTML response.	FindText=Incorrect, IgnoreCase=False, UseRegularExpression=False, PassIfTextFound=True

Seems pretty conclusive. Let’s make it pass now, we’ll just hard code it to start with. My ID here is 2, so I know the next one will be 3 so I’ll set it to expect {"Id": 3 }.

When we run it again we get a pass. If we look in the details tab we can see the validation rule again, but passing this time.

That was fun, but we need to confirm that we can get the message as well. We could just get all the messages and search for what we want or we could get the inserted message by its ID and check it’s the same. Whichever we choose we’re going to need another web request. Right click the root node and select ‘Add Request’ again. You could select ‘Add Dependent Request’ here but, well, don’t.

You should now see another request like before. Change the properties to be:

Property	Value
Url	http://localhost:9000/4
Method	GET

This should retrieve the message with ID 4 that we create when we run it this time.

You’ll also have to update the validation rule to expect {"Id": 4 } new or that will fail.

If you’ve done everything correctly then you’ll see the passing test.

Of course, this is useless as a load test as it isn’t re-runnable. Every time you run it you need to figure out what the next ID is and manually type it in. If that’s all you were planning on doing then use Fiddler instead.

Using variables (a.k.a. “Context Parameters”)

Context parameter are variable data that is scoped to a specific test. We can use them to store and retrieve data so that our tests are dynamic and re-runnable. We’re going to use one to store the ID of the created message and the message itself. This means that in the future we can run the test a lot of times, in parallel and do actual load testing.

First of all, lets add one. The only node you can add a context parameter to is the root node. This makes sense as context parameters are scoped to the whole test. Right click again and select ‘Add Context Parameter’. It’ll add a folder to keep all your context parameters same and give it the default name Parameter1 and no value. Create another one and use the properties to rename them and give them values.

Name	Value
ID	5
Message	hello-world

Now we’ll plug these parameters in. We reference a parameters with the parameter name in double parentheses. e.g.:

• {{ID}}
• {{Message}}

So, change the URLs for web requests to:

• http://localhost:9000/{{Message}}
• http://localhost:9000/{{ID}}

You’ll also have to change the test in our Validation Rule to include the token for the ID parameter:

{"Id": {{ID}} }

(Substitute the ID for whatever one you’re on now, I’m sure you’ve grasped this bit by now)

You should have something like this:

Run the test and you’ll see that the values in the context parameters have replaced the tokens.

We still have to manually update the ID, but at least it’s only in one place now. What we really want to be able to do is capture the ID from the initial POST and use that. The default extraction options are pretty rubbish, you can write custom ones but we’ll make to.

Right click on the first request and select ‘Add Extraction Rule’. Choose the ‘Extract Text’ option and enter the following values.

Property Name	Value
Context Parameter Name	ID
Starts With	{"Id":
Ends With	}
Required	True
HTML Decode	False

Your screen should look like this now.

If we run it then it fails.

If you click on the failed test and click on the details we can see what went wrong. It seems that the extraction worked but the validation is failing now. The reason it that the validation runs before the extraction runs. This isn’t great, there is a solution but it’s for a later date. Right now we’ll have to make do with using a regular expression to check that some integer is returned.

We can use this regex:

^\{\s*"Id"\s*:\s\d*\s*\}$

Now, run it again and we get a nice green test run.

Lastly, we have the message being sent being hard coded to hello-world. It would be nice if it could be tokenized too.

Using a Data Source

First we’ll create a CSV file as a data source. For the test driver to read the file the first line must contain the headers.

I’ve created a file called Messages.csv with the following content:

Messages
Lorem-ipsum-dolor
Sit-amet-a
Tortor-nec-nulla
A-lacinia-enim-felis-ut-sit
Sed-laoreet-etiam
Mi-sagittis-ut
Nulla-nec-fringilla-aptent
Justo-semper-volutpat
Justo-integer-architecto
Blandit-eum-pretium
Pellentesque-lobortis-nulla
Non-sit-qui
Maecenas-vivamus-at
Aliquam-consequat-pellentesque-sapien-ante-ultricies-proin

Right click on the root node again and select ‘Add Data Source’. You’ll be presented with a dialogue box. Name it ‘Messages’ and select ‘CSV File’ as the Data Source Type. Use the … to browse to the file or just type in the path. You’ll see a sample of the content in the window. Click Finish.

Unless you’ve added the file to the project you’ll see a dialogue asking if you want to add it. Select ‘Yes’

Now you should have a screen like this:

Just to be different, change the ‘Access method’ property from ‘Sequential’ to ‘Random’.

The token for the value in the data source needs to be qualified and again in double parentheses.

{{DataSourceName.Filename#Extension.ColumnName}}

For us this is:

{{Messages.Messages#csv.Messages}}

Which means our original URL should now be:

http://localhost:9000/{{Messages.Messages#csv.Messages}}

Run this and you’ll see it pass with a random message taken from our CSV file.

Lastly, lets validate that the correct message is being returned. Add another validation rule, this time to the GET request. It’ll be a ‘Find Text’ validation rule again. Set the ‘Use Regular Expression’ property to True again and set the ‘Find Text’ value to

^\s*\{\s*"message"\s*:\s"{{Messages.Messages#csv.Messages}}"*\s*\}\s*$`

We don’t need the Message parameter at all any more, so you can delete it.

Our finished test should now look like:

Run it and you should see it all green again. Have a look in the details tabs to see the validations and extractions etc.

Using a Web Test as a Load Test

Finally, we’ll make this into a basic load test. The crude way it to just set the test to run a certain number of times. You can do this in by opening the local.testsettings file. It’s probably in the Solution Items folder. Double click it to open the edit dialogue.

Click ‘Web Test’ and change the ‘Fixed Run Count’ to something bigger than 1 or select the ‘One run per data source row’.

Save it.

Run the test again and you’ll see it runs many times.

I don’t know why the highlighting is so horrible in Visual Studio.

We don’t get many useful statistics here, we can copy everything to clipboard but all we get is:

Passed  Run 1               
Passed      http://localhost:9000/Non-sit-qui   200 OK  0.018 sec   0.018 sec   11
Passed      http://localhost:9000/31    200 OK  0.001 sec   0.001 sec   27
Passed  Run 2               
Passed      http://localhost:9000/Blandit-eum-pretium   200 OK  0.000 sec   0.000 sec   11
Passed      http://localhost:9000/32    200 OK  0.000 sec   0.000 sec   35
Passed  Run 3               
Passed      http://localhost:9000/Pellentesque-lobortis-nulla   200 OK  0.000 sec   0.000 sec   11
Passed      http://localhost:9000/33    200 OK  0.000 sec   0.000 sec   43
Passed  Run 4               
Passed      http://localhost:9000/Blandit-eum-pretium   200 OK  0.000 sec   0.000 sec   11
Passed      http://localhost:9000/34    200 OK  0.000 sec   0.000 sec   35
Passed  Run 5               
Passed      http://localhost:9000/A-lacinia-enim-felis-ut-sit   200 OK  0.000 sec   0.000 sec   11
Passed      http://localhost:9000/35    200 OK  0.000 sec   0.000 sec   43
Passed  Run 6               
Passed      http://localhost:9000/Lorem-ipsum-dolor 200 OK  0.000 sec   0.000 sec   11
Passed      http://localhost:9000/36    200 OK  0.000 sec   0.000 sec   33
Passed  Run 7               
Passed      http://localhost:9000/Lorem-ipsum-dolor 200 OK  0.000 sec   0.000 sec   11
Passed      http://localhost:9000/37    200 OK  0.000 sec   0.000 sec   33
Passed  Run 8               
Passed      http://localhost:9000/Non-sit-qui   200 OK  0.000 sec   0.000 sec   11
Passed      http://localhost:9000/38    200 OK  0.000 sec   0.000 sec   27
Passed  Run 9               
Passed      http://localhost:9000/Blandit-eum-pretium   200 OK  0.000 sec   0.000 sec   11
Passed      http://localhost:9000/39    200 OK  0.000 sec   0.000 sec   35
Passed  Run 10              
Passed      http://localhost:9000/Pellentesque-lobortis-nulla   200 OK  0.000 sec   0.000 sec   11
Passed      http://localhost:9000/40    200 OK  0.000 sec   0.000 sec   43

Let’s do something a bit more useful. Put the number of web test interactions back to 1.

We need to create a load test. The load will run the web test we created multiple times and generate statistics for you.

Right click on the project in the Solution Explorer and select ‘Add Load Test’. This will present you with another dialogue for you to define your test.

Here we can define run settings, like duration or iterations (I’m just going to choose 100 iterations).

We can also define our scenario, the scenario contains the individual tests to include, load patterns, number of users to simulate and details of how they’ll act along with other things you can see by clicking about.

I’m going to call the Scenario ‘Hello World’, give it five minutes duration, and a constant load of twenty five users.

In the ‘Test Mix’ we add the tests that will be included and can say what percentage of users will do what. We only have one test, so add that.

Click ‘Finish’ and you’ll see the load test.

Now, after all that, you can click the run button. You’ll see a counter in the top right corner saying how much time is left. You’ll also see some graphs with performance metrics etc.. If you can’t see the graphs then you just need to click the ‘Graphs’ button.

Errors

If you’re running this against the test server provided you’ll probably start seeing errors occurring. These are represented by the purple line in the top left graph for me. It might be different for you. You can see the legend at the bottom.

To see details of the errors click on the ‘Tables’ button.

These are because by carefully crafted web server can’t handle this many users at once. The reason why? I’ll leave you to have a look at the code and see but I’ll give you a hint. Dictionary is not a thread-safe type for writing. Have a look at the MSDN reference.

Hope this was useful.

Thanks for reading.

Feel free to contact me @BanksySan!

Corrections always welcome (especially spelling mistakes).