IDataReader, .NET Core & Multiple Tables

With .NET Core maturing it’s worth noting a missing class previously in System.Data. It’s probably one you’ve forgotten about since the early days. System.Data.DataSet seems to have seen its day (at least in .NET Core). Whether you liked it or not it was useful for receiving multiple data sets.

What do you do now? Now we have to work directly with the data reader. Turns out it’s really easy, I just hadn’t ever looked at it before. If we have the following two tables:

ID Description
1 1-1
2 1-2
3 1-3
4 1-4
5 1-5
ID Description
1 2-1
2 2-2
3 2-3
4 2-4
5 2-5

Then we can get them both from a single call to the database sending two SELECT statements.

SELECT [Id],[Description] 
FROM [dbo].[Table1]; 

SELECT [Id],[Description] 
FROM [dbo].[Table2];

In order to read the two tables, we need to use two methods off the IDataReader interface.

  • .Read() will advance the reader to the next row in the table, returning false when there are no more rows left.
  • .NextResult() will advance the reader to the next table, again returning false when there are no more data sets.

The full code to get both the above tables in a single database call might look like this:

using (var connection = new SqlConnection(CONNECTION_STRING))
{
  const int WIDTH = 15;
  const string COMMAND_TEXT =
    @"SELECT [Id],[Description] 
     FROM [dbo].[Table1]; 
     SELECT [Id],[Description] 
     FROM [dbo].[Table2];";

  using (var command = connection.CreateCommand())
  {
    command.CommandText = COMMAND_TEXT;
    command.CommandType = CommandType.Text;
    connection.Open();

    using (var dataReader = command.ExecuteReader())
    {
      do
      {
        var columnCount = dataReader.FieldCount;
        WriteLine();
        WriteLine($"|{new string('-', WIDTH)}|{new string('-', WIDTH)}|");
        WriteLine($"|{dataReader.GetName(0),WIDTH}|{dataReader.GetName(1),WIDTH}|");
        WriteLine($"|{new string('-', WIDTH)}|{new string('-', WIDTH)}|");

        while (dataReader.Read())
        {
          for (var i = 0; i < columnCount; i++)
          {
            Write($"|{dataReader[i],WIDTH}");
          }
          WriteLine("|");
        }
        WriteLine($"|{new string('-', WIDTH)}|{new string('-', WIDTH)}|");
    } while (dataReader.NextResult());
  }
}

This will give you a nice output too:

|---------------|---------------|
|             Id|    Description|
|---------------|---------------|
|              1|            1-1|
|              2|            1-2|
|              3|            1-3|
|              4|            1-4|
|              5|            1-5|
|---------------|---------------|

|---------------|---------------|
|             Id|    Description|
|---------------|---------------|
|              1|            2-1|
|              2|            2-2|
|              3|            2-3|
|              4|            2-4|
|              5|            2-5|
|---------------|---------------|

No need to miss the DataSet any more.

For almost all use-cases this will be much more memory efficient as well.

Thanks for reading.

Please feel free to get in contact @BanksySan.

Redirecting Application Insights

Application Insights is a great tool, really powerful and very easy to set up. That’s its weakness in some respects. If, like me, you work in a company that handles sensitive data and is subject to various laws around how you process and store that data then the idea of just sending everything into “The Cloud” will get some or your stakeholders quite concerned (and rightfully so).

You may well have security policies, compliance policies, governing bodies as well as contractual and other legal obligations in relation to the data you store and how you process it.

In order to proceed with sending your metrics you really need to be able to prove what you’re sending and prove that you can control and monitor it. To do that, you need to be able to report on it.

How on Earth does one go about reporting on telemetry?

The problem is a bit of a Catch-22 in that without actually implementing the telemetry we can’t do any better than hypothesising and if we implement it then we’ll be sending data out that’s not been vetted.

This is the problem that I’m going to try to solve here.

We’re going to do this in four parts.

  1. Use a console application to see what’s being sent to where.
  2. Create a web application and install Application Insights into it and look at what’s going on there.
  3. Using the same web application, look at the client side (JavaScript) logging that’s going on.
  4. Look at how we can control this data.

Locking up

I’m jumping ahead here and telling you what you need to block to experiment with Application Insights without leaking information. I did this by firing up Fiddler on a computer that I was OK with details being sent to the cloud. If you don’t have one then I’d suggest firing up a virtual machine in Azure and using that.

It seems that the telemetry for Application Insights is currently (on the 11 May 2017) going to two endpoints:

  • dc.services.visualstudio.com/v2/track
  • rt.services.visualstudio.com/v2/track

dc.services.visualstudio.com/v2/track is the endpoint for most of the metrics. rt.services.visualstudio.com/v2/track seems to be exclusively for hitting something called the “Quick Pulse Service”, I’ve yet to figure out what it’s for,

NB You may also see metrics sent off to other endpoints from background metric gathering, e.g. have a look at what’s being sent to vortex-win.data.microsoft.com; it is very generous and probably a lot more than you think you are leaking.

You could just add these hosts to your hosts file and direct them to 127.0.0.1. Doing that won’t let you see what’s being sent though. I think a more pleasing solution is to use Fiddler and use its Auto Responder function to catch the calls and instead of sending them out to the World to just return a dummy response.

Console application

Track Event

Create a console application and install the Application Insights Nuget package.

Install-Package Microsoft.ApplicationInsights

This will install the Microsoft.ApplicationInsights.dll assembly. The .NET telemetry client which contains the methods to fire off metrics. The simplest it the TrackEvent(string) method.

Add the following and run it. The argument of the TrackEvent method is the name you want to give this event.

private static void DefaultTelemetryClient()
{
    var telemetryConfiguration = new TelemetryConfiguration(INSTRUMENTATION_KEY);
    var client = new TelemetryClient(telemetryConfiguration);
    client.TrackEvent("Application Started");
    client.TrackEvent("Application Stopping");
    client.Flush();
}

NB The client will buffer metrics until you call Flush().
NB Even though we have a Flush() method, the TelemetryClient class isn’t disposable. That’s odd, more odd is that the TelemetryConfiguration class is disposable.

If you run this you should see a single request transmitted with two events. The headers will look similar to:

POST https://dc.services.visualstudio.com/v2/track HTTP/1.1
Content-Type: application/x-json-stream
Host: dc.services.visualstudio.com
Content-Length: 665
Expect: 100-continue
Connection: Keep-Alive

The body contains all the metrics:

{
    "name": "Microsoft.ApplicationInsights.INSTRUMENTATION_KEY.Event",
    "time": "2017-05-11T22:13:10.1409683Z",
    "iKey": "INSTRUMENTATION_KEY",
    "tags": {
        "ai.cloud.roleInstance": "MyPersonalComputer",
        "ai.internal.sdkVersion": "dotnet:2.3.0-41907"
    },
    "data": {
        "baseType": "EventData",
        "baseData": {
            "ver": 2,
            "name": "Application Started"
        }
    }
}

You’ll have two of these. You can already see that it’s transmitting more than just the time and the event name. We also have the name of my computer in there and the version of the SDK. It’s unlikely that the SDK could constitute PII by is something that could maybe be used by an attacker. The computer’s name will probably worry your security team too.

Application Insights sets these automatically. We can override these but Microsoft haven’t made it nearly as simple as it could be.

The telemetry has several contexts that are used to hold meta information about the wider environment the the metrics are being gathered in.

  • public ComponentContext
  • public DeviceContext
  • public CloudContext
  • public SessionContext
  • public UserContext
  • public OperationContext
  • public LocationContext
  • internal InternalContext

To set these we need to use the TrackEvent(EventTelemetry) overload and create an EventTelemetry object and set the properties.

  • ai.cloud.roleInstance is a property of the CloudContext class.
  • ai.internal.sdkVersion is a property of the InternalContext class.

We can access all of these directly with the exception on the InternalClass which, for some reason, is internal. There’s an extension method that provides you with access to it. (It seems a little overly convoluted though).

The new code looks like this:

private static void DefaultTelemetryClient(string mask)
{
    var telemetryConfiguration = new TelemetryConfiguration(INSTRUMENTATION_KEY);
    var client = new TelemetryClient(telemetryConfiguration);
    var applicationStartedEvent = new EventTelemetry("Application Started");
    applicationStartedEvent.Context.Cloud.RoleInstance = mask;      applicationStartedEvent.Context.GetInternalContext().SdkVersion = mask;

    var applicationStopEvent = new EventTelemetry("Application Stopping");
    applicationStopEvent.Context.Cloud.RoleInstance = mask;
    applicationStopEvent.Context.GetInternalContext().SdkVersion = mask;
client.TrackEvent(applicationStartedEvent);
client.TrackEvent(applicationStopEvent);
    client.Flush();
}

What do we set the mask parameter to? null or string.Empty might seem likely candidates. Unfortunately if they are set to either of these, then the properties are still set to be the defaults and so make no difference at all. We need to set them to a non-zero length string. If I set it to "." then we get this:

{
    "name": "Microsoft.ApplicationInsights.INSTRUMENTATION_KEY.Event",
    "time": "2017-05-12T20:16:01.0834153Z",
    "iKey": "INSTRUMENTATION_KEY",
    "tags": {
        "ai.cloud.roleInstance": ".",
        "ai.internal.nodeName": "MyPersonalComputer",
        "ai.internal.sdkVersion": "."
    },
    "data": {
        "baseType": "EventData",
        "baseData": {
            "ver": 2,
            "name": "Application Started"
        }
    }
}

Now Application Insights adds a new context property called nodeName which contains the computer name I just tried to get rid of. Luckily, if we set this too we can get everything covered.

var telemetryConfiguration = new TelemetryConfiguration(INSTRUMENTATION_KEY);
var client = new TelemetryClient(telemetryConfiguration);

var applicationStartedEvent = new EventTelemetry("Application Started");
applicationStartedEvent.Context.Cloud.RoleInstance = mask;
var internalContext = applicationStartedEvent.Context.GetInternalContext();
internalContext.SdkVersion = mask;
internalContext.NodeName = mask;

client.TrackEvent(applicationStartedEvent);
client.Flush();

Finally we get a clean POST:

{
    "name": "Microsoft.ApplicationInsights.instrumentation_key.Event",
    "time": "2017-05-12T20:28:01.8214775Z",
    "iKey": "INSTRUMENTATION_KEY",
    "tags": {
        "ai.cloud.roleInstance": ".",
        "ai.internal.nodeName": ".",
        "ai.internal.sdkVersion": "."
    },
    "data": {
        "baseType": "EventData",
        "baseData": {
            "ver": 2,
            "name": "Application Started"
        }
    }
}

The Other Seven Tracking Methods

There are eight methods in total (ignoring the one marked as “DO NOT USE” by Microsoft). The methods are:

  • TrackEvent
  • TrackException
  • TrackAvailability
  • TrackDependency
  • TrackMetric
  • TrackPageView
  • TrackRequest
  • TrackTrace

They all seem to act in the same was as far as auto metrics gathering goes (at least in a console application). We can write a generic MaskContext(ITelemetry, string) method to deal with them all.

Try running the following code:

internal class Program
{
    private const string INSTRUMENTATION_KEY = "INSTRUMENTATION_KEY";

    class CustomException : Exception
    {
        public CustomException(string message = null)
            : base(message)
        { }
    }

    private static void Main()
    {
        DefaultTelemetryClient(".");
    }

    private static void DefaultTelemetryClient(string mask)
    {
        var telemetryConfiguration = new TelemetryConfiguration(INSTRUMENTATION_KEY);
        var client = new TelemetryClient(telemetryConfiguration);
        var eventTelemetry = new EventTelemetry("An event happened");
        var customException = new CustomException("It's dead Jim.");
        var expectionTelemetry = new ExceptionTelemetry(customException);
        var availabliltyTelemetry =
            new AvailabilityTelemetry("AI Application", DateTimeOffset.Now, TimeSpan.FromSeconds(2), "My house", true, "It's alive");
        var dependencyTelemetry = new DependencyTelemetry("SomeDependentType", "AI Application", "My dependency", "Here's some data");
        var metricDependency = new MetricTelemetry("Bank balance", 1000000);
        var pageViewTelemetry = new PageViewTelemetry("Home page");
        var requestTelemetry = new RequestTelemetry("List bank accounts", DateTimeOffset.Now, TimeSpan.FromSeconds(3), "OK", true);
        var traceTelemetry = new TraceTelemetry("Don't look here, everything's fine", SeverityLevel.Critical);

        MaskContext(eventTelemetry, mask);
        MaskContext(expectionTelemetry, mask);
        MaskContext(availabliltyTelemetry, mask);
        MaskContext(dependencyTelemetry, mask);
        MaskContext(metricDependency, mask);
        MaskContext(pageViewTelemetry, mask);
        MaskContext(requestTelemetry, mask);
        MaskContext(traceTelemetry, mask);

        client.TrackEvent(eventTelemetry);
        client.TrackException(expectionTelemetry);
        client.TrackAvailability(availabliltyTelemetry);
        client.TrackDependency(dependencyTelemetry);
        client.TrackMetric(metricDependency);
        client.TrackPageView(pageViewTelemetry);
        client.TrackRequest(requestTelemetry);
        client.TrackTrace(traceTelemetry);

        client.Flush();
    }

    private static void MaskContext(ITelemetry telemetry, string mask)
    {
        var context = telemetry.Context;
        MaskContext(context.GetInternalContext(), mask);
        MaskContext(context.Cloud, mask);
    }

    private static void MaskContext(CloudContext cloudContext, string mask)
    {
        cloudContext.RoleInstance = mask;
    }

    private static void MaskContext(InternalContext internalContext, string mask)
    {
        internalContext.NodeName = mask;
        internalContext.SdkVersion = mask;
    }
}

It’ll fire off one of each track type in a single call (because we just call Flush() once at the end.

{
    "name": "Microsoft.ApplicationInsights.instrumentation_key.Event",
    "time": "2017-05-13T11:29:51.3312151Z",
    "iKey": "INSTRUMENTATION_KEY",
    "tags": {
        "ai.cloud.roleInstance": ".",
        "ai.internal.nodeName": ".",
        "ai.internal.sdkVersion": "."
    },
    "data": {
        "baseType": "EventData",
        "baseData": {
            "ver": 2,
            "name": "An event happened"
        }
    }
}{
    "name": "Microsoft.ApplicationInsights.instrumentation_key.Exception",
    "time": "2017-05-13T11:29:51.3522228Z",
    "iKey": "INSTRUMENTATION_KEY",
    "tags": {
        "ai.cloud.roleInstance": ".",
        "ai.internal.nodeName": ".",
        "ai.internal.sdkVersion": "."
    },
    "data": {
        "baseType": "ExceptionData",
        "baseData": {
            "ver": 2,
            "exceptions": [
                {
                    "id": 4094363,
                    "typeName": "BanksySan.Learning.Scratch.Program+CustomException",
                    "message": "It's dead Jim.",
                    "hasFullStack": true
                }
            ]
        }
    }
}{
    "name": "Microsoft.ApplicationInsights.instrumentation_key.Availability",
    "time": "2017-05-13T11:29:51.3072079Z",
    "iKey": "INSTRUMENTATION_KEY",
    "tags": {
        "ai.cloud.roleInstance": ".",
        "ai.internal.nodeName": ".",
        "ai.internal.sdkVersion": "."
    },
    "data": {
        "baseType": "AvailabilityData",
        "baseData": {
            "ver": 2,
            "id": "w+xlXps84pI=",
            "name": "AI Application",
            "duration": "00:00:02",
            "success": true,
            "runLocation": "My house",
            "message": "It's alive"
        }
    }
}{
    "name": "Microsoft.ApplicationInsights.instrumentation_key.RemoteDependency",
    "time": "2017-05-13T11:29:51.3592249Z",
    "iKey": "INSTRUMENTATION_KEY",
    "tags": {
        "ai.cloud.roleInstance": ".",
        "ai.internal.nodeName": ".",
        "ai.internal.sdkVersion": "."
    },
    "data": {
        "baseType": "RemoteDependencyData",
        "baseData": {
            "ver": 2,
            "name": "My dependency",
            "id": "nK4TqamvipU=",
            "data": "Here's some data",
            "success": true,
            "type": "SomeDependentType",
            "target": "AI Application"
        }
    }
}{
    "name": "Microsoft.ApplicationInsights.instrumentation_key.Metric",
    "time": "2017-05-13T11:29:51.3622256Z",
    "iKey": "INSTRUMENTATION_KEY",
    "tags": {
        "ai.cloud.roleInstance": ".",
        "ai.internal.nodeName": ".",
        "ai.internal.sdkVersion": "."
    },
    "data": {
        "baseType": "MetricData",
        "baseData": {
            "ver": 2,
            "metrics": [
                {
                    "name": "Bank balance",
                    "kind": "Aggregation",
                    "value": 1000000,
                    "count": 1
                }
            ]
        }
    }
}{
    "name": "Microsoft.ApplicationInsights.instrumentation_key.PageView",
    "time": "2017-05-13T11:29:51.3662274Z",
    "iKey": "INSTRUMENTATION_KEY",
    "tags": {
        "ai.cloud.roleInstance": ".",
        "ai.internal.nodeName": ".",
        "ai.internal.sdkVersion": "."
    },
    "data": {
        "baseType": "PageViewData",
        "baseData": {
            "ver": 2,
            "name": "Home page"
        }
    }
}{
    "name": "Microsoft.ApplicationInsights.instrumentation_key.Request",
    "time": "2017-05-13T11:29:51.3162108Z",
    "iKey": "INSTRUMENTATION_KEY",
    "tags": {
        "ai.cloud.roleInstance": ".",
        "ai.internal.nodeName": ".",
        "ai.internal.sdkVersion": "."
    },
    "data": {
        "baseType": "RequestData",
        "baseData": {
            "ver": 2,
            "id": "XIN69SQNXZQ=",
            "name": "List bank accounts",
            "duration": "00:00:03",
            "success": true,
            "responseCode": "OK"
        }
    }
}{
    "name": "Microsoft.ApplicationInsights.instrumentation_key.Message",
    "time": "2017-05-13T11:29:51.3722295Z",
    "iKey": "INSTRUMENTATION_KEY",
    "tags": {
        "ai.cloud.roleInstance": ".",
        "ai.internal.nodeName": ".",
        "ai.internal.sdkVersion": "."
    },
    "data": {
        "baseType": "MessageData",
        "baseData": {
            "ver": 2,
            "message": "Don't look here, everything's fine",
            "severityLevel": "Critical"
        }
    }
}

This looks better. Some of the calls have an id field added though.

Auto Generated ID fields

I guess we need to know what’s encoded into these id values as they could also be leaking some information.
They affect:

  • Exception telemetries.
  • Availability telemetries.
  • Dependency telemetries.
  • Request telemetries.

Well look at those next.

Exception Telemetry

This one has a numeric ID field. This is actually just the result of calling GetHashCode() on the exception object. This means that whenever you restart the application you will get exactly the same numbers generated again (regardless of Exception type, message or anything else).

Does this leak anything important? The only thing I can think of it that it reveals how many application starts there are. If you have a server farm then this could be the number of servers you have running.

On a practical note, it does mean that this auto generated id is not unique, so you can’t rely on it for a primary key.

Availability, Dependency & Request

The id fields on a three of these are random and 64 bit. They don’t reveal anything about you and are probably OK for you to use as a primary key.

Web Application

Creating a default web application. There are a few different options, so you may well find that yours of different.

I’m making a .NET 4.7 MVC template with libraries for MVC and WebAPI added. I’m also installing Application Insights by right clicking on the project and selecting “Configure Application Insights”.

Configure Application Insights

I’m not adding the trace collection either.

Changes after Installation

  • App_Start/FilterConfig.cs
  • ApplicationInsights.config
  • BanksySan.Learning.ApplicationInsights.Web.csproj
  • Connected Services/Application Insights/ConnectedService.json
  • ErrorHandler/AiHandleErrorAttribute.cs
  • Views/Shared/_Layout.cshtml
  • Web.Config

Filters (Exception Filter)

Application Insights replaces the HandleError filter with an AiHandleError filter and registers it in place of the (now deleted) HandleError filter. This filter will only cause an tracking to be sent if you have custom errors enabled and you are actually running with a non-null HttpContext.

Web & Application Insights Config

The web.config registers the ApplicationInsightsHttpModule HTTP handler. This is used to collect information about requests and responses.

ApplicationInsights.config contains the majority of the config for Application Insights. In here we have definitions for lots of different metric gathering activities as well as definitions for how those metrics should be processed

Layout file

A script is added to the _Layout.cshtml that will download the Application Insights JavaScript file on page load. (This file a much bigger than this small script).

Where Are Metrics Collected?

Looking at the files changes, there are several points that metric gathering is set up:

  • Registering filters in the Global.asax or FilterConfig.
  • Adding a bespoke script to the client via the script added in _Layout.cshtml.
  • Registering HTTP handlers in the Web.Config.
  • Modules, processors and channels in the applicationinsights.config file.

Going through all of these would be beyond my understanding, however we can still control what we send with some bespoke code.

Controlling the Metrics

We are going to redirect all the output so that we can, effectively, turn on all the metrics gathering functionality but have it all directed as some endpoint we own. This way we can record it, check what we’re sending, share the collected metrics with any stakeholders and even manipulate them before sending them on.

This way can also set up a regression suite to look at metrics in the development environment to confirm that development changes we make don’t start leaking metrics we don’t want leaked.

Client-side JavaScript

Application Insights adds a script to the _Layout.cshtml which configures and downloads the main Application Insights script. The URL for this script is, by default, az416426.vo.msecnd.net/scripts/a/ai.0.js but we can change this. Look at the script you’ll see where we pass in the instrumentation key.

// ... removed for brevity...
{
    instrumentationKey: "INSTRUMENTATION_KEY"
}

We can add a different URL here for the application to get the script from.

// ... removed for brevity...
{
    instrumentationKey: "31c04897-c554-430c-840b-84407f0fe4d8",
    url: "//some.other.url"
}

This will cause Application Insights to try to get the script from some.other.url, which can be a script you’ve crafted.

The URL for the final end point, that being the one that your metrics will be sent to, is defined in that script that is downloaded. We need to correct that too (assuming you haven’t crafted a bespoke script). To change this that we can pass in a different config property:

{
    instrumentationKey: "INSTRUMENTATION_KEY",
    endpointUrl: "//some.other.url"
} 

NB You might need to enable CORS to get this to work.

Now the client-side metrics are all being sent to an endpoint that you own for you to analyse and process as you please.

Server-side

Server side, the endpoint is controlled by sending metrics down an ITelemetryChannel implementation. The default is ServerTelemetryChannel.

There are two ways we can change the endpoint for this

  1. Change the endpoint.
  2. Use a custom ITelemetryChannel.

Changing the Endpoint

There are two\three options here too:

  1. Set the endpoint in the applicationinstights.config.
  2. Set the endpoint in the Application_Start() in Global.asax.
  3. Set it on individual instances of the TelemetryClient.

The first is probably the easiest. We just need to add the following:

  <TelemetryChannel Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.ServerTelemetryChannel, Microsoft.AI.ServerTelemetryChannel">
    <EndpointAddress>//hostname:port/My-Endpoint</EndpointAddress>
  </TelemetryChannel>

NB localhost seems to just stop it sending any telemetry. Using the machine name, or localhost.fiddler (if you have fiddler running) works fine.

Your other option is to set it explicitly. This is done by adding the following to the Application_Start().

TelemetryConfiguration.Active.TelemetryChannel.EndpointAddress =  "http://hostname:port/My-Endpoint";

NB If you choose either of these then you will need to add support for application/x-json-stream as a content-type.

The last option is to just write your own channel and replace the one provided completely.

Implementing a Custom Telemetry Channel

Telemetry channels implement the ITelemetryChannel interface:

public interface ITelemetryChannel : IDisposable
{
  void Send(ITelemetry item);
  void Flush();
  bool? DeveloperMode { get; set; }
  string EndpointAddress { get; set; }
}

To change the telemetry channel you can either do so in the Application_Start() method like above, or replace the one in the applicationinsights.config.

<TelemetryChannel Type="Fully.Qualified.Name.FileSystemChannel, The.Assembly.Name" />

If you’re planning on storing your data on the local disk, or in a database then the last option would be the most performant as it doesn’t involve any unnecessary HTTP calls.

Example Custom File System Telemetry Channel

The following example will save metrics to the file system. I also does some sanitising of metrics which could be easily used for sending metrics to the cloud.

public class FileSystemChannel : ITelemetryChannel
{
    private static readonly DirectoryInfo BASE_DIRECTORY = new DirectoryInfo(@"...\TelemetryDump");

    private bool? _developerMode;

    /// <inheritdoc />
    static FileSystemChannel()
    {
        if (!BASE_DIRECTORY.Exists) { BASE_DIRECTORY.Create(); }
    }

    public void Dispose() { }

    public void Send(ITelemetry item)
    {
        var saneItem = new SaneTelemetry
        {
            TimeStamp = item.Timestamp,
            Type = item.GetType().Name
        };

        saneItem.Details = GetDetails(item);

        var fileName = BASE_DIRECTORY.FullName + "\\" + DateTime.Now.Ticks;
        using (var writer = File.CreateText(fileName))
        {
            writer.Write(JsonConvert.SerializeObject(saneItem));
        }
    }

    public void Flush() { }

    public bool? DeveloperMode { get; set; }

    public string EndpointAddress { get; set; }

    private dynamic GetDetails(ITelemetry item)
    {
        if (item is ExceptionTelemetry)
        {
            return GetDetails((ExceptionTelemetry)item);
        }

        if (item is RequestTelemetry)
        {
            return GetDetails((RequestTelemetry)item);
        }

        // Implement the rest as you see fit.

        return null;
    }


    private dynamic GetDetails(RequestTelemetry requestTelemetry)
    {
        return new
        {
            requestTelemetry.Duration,
            requestTelemetry.Name,
            requestTelemetry.Url
        };
    }

    private dynamic GetDetails(ExceptionTelemetry exceptionTelemetry)
    {
        return new
        {
            exceptionTelemetry.Exception.Message,
            exceptionTelemetry.Exception.GetType().Name
        };
    }
}

public class SaneTelemetry
{
    public string Type { get; set; }
    public DateTimeOffset TimeStamp { get; set; }
    public dynamic Details { get; set; }
}

I hope this has been helpful. You might find better ways to achieve these things, but this might kick you off. If you do find anything better then please message me!

Thanks for reading.

Please feel free to get in contact

@BanksySan

General Purpose Fiddler Filter

Fiddler is great.

I usually have a few windows open, maybe Netflix too and it get's very noisy. I have a filter that I routinely use.  It's available at my Github Gist.

Just paste it into the hosts to hide bit in Fiddler.

Hope it's useful.  Please feel free to leave any comments.

Regards

@BanksySan