I am having issues working out how to get around a problem I have to do with the use of custom logging fields in IIS 8.5 and Checkpointing together.
Overview:
So basically I am using the log parser to automatically gather all the IIS logs for all our live web sites from all our webservers and compile them into database so we can sue this for centralised support/troubleshooting and giving developers visibility. I use the custom login fields to record the IP address for our customer as due to the way our load balancers need to interact with our webservers the client IP will only show the load balancers IP. To make this available in the logs I have setup the load balancer to insert a custom host header and then in IIS I have picked up this host header and written it to the log.
So that works brilliantly.
Obviously I also want to use check points so that it does have to parse all the IIS logs and also it doesn’t inset duplicate data.
Problem:
In order to use checkpoints I have to set the input type to IISW3C. This then gives me an error as it does not recognise the additional field (so I assume it has a static list it expects).
Parse Errors: Unknown field x-forwarded-for found in #fields directive
This will read this information fine is I use the input format of W3C but from what I can see that means you cannot use check points.
Error: invalid parameter "icheckpoint"
So basically I am left with either:
Choosing IISW3C and not recording the Customer IP information (which is one of the key reasons we want these logs) but being able to checkpoint.
Or
Choosing W3C so we can have out custom field and then not using the existing check pointing functionality. I would then have to look at developing my own version of check pointing (Massive shame as it works so well and frustrating to re-invent the wheel)
Example log parser code that shows issue below:
If use IISW3C:
C:\Program Files (x86)\Log Parser 2.2>logparser "select * from c:\inetpub\logs\LogFiles\W3SVC2\*x.log" -i: IISW3C -iCheckpoint: c:\inetpub\logs\LogFiles\W3SVC2\Checkpoint.lpc
If use W3C
C:\Program Files (x86)\Log Parser 2.2>logparser "select * from c:\inetpub\logs\LogFiles\W3SVC2\*x.log" -i:W3C -iCheckpoint: c:\inetpub\logs\LogFiles\W3SVC2\Checkpoint.lpc
I really appreciate that you have taken the time to read this and if you have any suggestions/advice or ideas on what I can do this would be greatly appreciated.