Split a string select everything after delimiter

[habs3]

Hi,

I am trying to query the esx logs for warnings and remove duplicates of the same error to limit what s returned. i am stating with removeing everything after the date so i can do a -unique hoping that will limit the returns. i was wondering ho i could split the following example and keep everything after the first space no mate what the data is?

So this

2018-01-24T20:01:37.652Z cpu5:33394)NMP: nmp_ResetDeviceLogThrottling:3349: last error status from device naa.00c00d00f00t00h00y00w00q repeated 2 times

Would look like this

cpu5:33394)NMP: nmp_ResetDeviceLogThrottling:3349: last error status from device naa.00c00d00f00t00h00y00w00q repeated 2 times

Or if there are any other ideas to query esx logs, return errors and remove the duplicates returned that would be very helpful.

Thanks for any assistance

[LucD]

Would something like this work?

I created some test data, and did two Group-Object loops on specific parts of the log entry.

$text = '2018-01-24T20:01:37.652Z cpu5:33394)NMP: nmp_ResetDeviceLogThrottling:3349: last error status from device naa.00c00d00f00t00h00y00w00q',

    '2018-01-24T20:01:38.652Z cpu5:33394)NMP: nmp_ResetDeviceLogThrottling:3349: last error status from device naa.00c00d00f00t00h00y00w00q',

    '2018-01-24T20:01:39.652Z cpu5:33394)NMP: nmp_ResetDeviceLogThrottling:3349: last error status from device naa.00c00d00f00t00h00y00w00q',

    '2018-01-24T20:01:37.652Z cpu6:33394)NMP: nmp_ResetDeviceLogThrottling:3349: last error status from device naa.00c00d00f00t00h00y00w00q',

    '2018-01-24T20:01:38.652Z cpu6:33394)NMP: nmp_ResetDeviceLogThrottling:3349: last error status from device naa.00c00d00f00t00h00y00w00q',

    '2018-01-24T20:01:39.652Z cpu7:33394)NMP: nmp_ResetDeviceLogThrottling:3349: last error status from device naa.00c00d00f00t00h00y00w00q'

$text | Group-Object -Property {$_.Split(' ')[1]} | %{

    $name = $_.Name

    $_.Group | Group-Object -Property {($_.Split(' ')[2] -join ' ')} | %{

       Write-Output "On $($Name) we saw error $($_.Name) $($_.Group.Count) times"

    }

}

[habs3]

Thank you,

What i am trying to accomplish is to audit a log file for anything that is an errors and find the instance of it once. I was trying the select-object -unique but since the time changes in the entries they are all different even though the error may be the same

.

so here is a different one and it is the complete line

2018-01-26T11:03:46.351Z cpu10:38380)lpfc: lpfc_scsi_cmd_iocb_cmpl:2185: 1:(0):3271: FCP cmd x1a failed <0/8> sid x011600, did x010800, oxid xffff iotag x9e2 SCSI Reservation Conflict -

2018-01-26T11:03:46.371Z cpu10:38914)lpfc: lpfc_scsi_cmd_iocb_cmpl:2185: 1:(0):3271: FCP cmd x1a failed <0/8> sid x011600, did x010800, oxid xffff iotag x114f SCSI Reservation Conflict -

2018-01-26T11:03:46.391Z cpu10:38914)lpfc: lpfc_scsi_cmd_iocb_cmpl:2185: 1:(0):3271: FCP cmd x1a failed <0/8> sid x011600, did x010800, oxid xffff iotag x997 SCSI Reservation Conflict -

2018-01-26T11:03:46.412Z cpu10:38367)lpfc: lpfc_scsi_cmd_iocb_cmpl:2185: 1:(0):3271: FCP cmd x1a failed <0/8> sid x011600, did x010800, oxid xffff iotag xc29 SCSI Reservation Conflict -

2018-01-26T11:03:46.432Z cpu10:37888)lpfc: lpfc_scsi_cmd_iocb_cmpl:2185: 1:(0):3271: FCP cmd x1a failed <0/8> sid x011600, did x010800, oxid xffff iotag xa2e SCSI Reservation Conflict -

So see how it is the same error but the time is different so i have not figured out a way to filter out all except the ones that are single instances of the error. the errors and warnings are returning different messages so ultimately i am trying to get individual errors messages for each log file so i can look at them for issues and  follow up on the ones that are real. There is a lot of data so being able to get one instance of each error or warning would be great.

Thanks again

[LucD]

Could be that I'm misunderstanding your request, but that is exactly what my earlier code is doing.

It leaves out the timestring, takes the identification of the system, and then tells how many times that messages was logged, independent of the time string.

[habs3]

Hmmm,

OK I think i may have been looking at it incorrectly. Let me try again in a full query on a log that will be a real test.

Thanks for the help and i will let you know how this comes out.

[habs3]

Good Morning LucD

I tried what you sent me and the following are the results. This appears on different errors. See how it shows the same error i was looking to get this to show and the amount of times is not a concern just looking to do a query of the logs and get events. I want to cut down the amount of events so i can view what comes back in a timely manner so i am not going through a 100 events and 75 of them are repeats.

On cpu8:38366)lpfc: we saw error lpfc_scsi_cmd_iocb_cmpl:2185: 1 times

On cpu8:38366)lpfc: we saw error lpfc_scsi_cmd_iocb_cmpl:2185: 1 times

On cpu8:38366)lpfc: we saw error lpfc_scsi_cmd_iocb_cmpl:2185: 1 times

On cpu8:38366)lpfc: we saw error lpfc_scsi_cmd_iocb_cmpl:2185: 1 times

On cpu8:38366)lpfc: we saw error lpfc_scsi_cmd_iocb_cmpl:2185: 1 times

On cpu8:38366)lpfc: we saw error lpfc_scsi_cmd_iocb_cmpl:2185: 1 times

On cpu8:38366)lpfc: we saw error lpfc_scsi_cmd_iocb_cmpl:2185: 1 times

On cpu8:38366)lpfc: we saw error lpfc_scsi_cmd_iocb_cmpl:2185: 1 times

On cpu8:38366)lpfc: we saw error lpfc_scsi_cmd_iocb_cmpl:2185: 1 times

On cpu8:38366)lpfc: we saw error lpfc_scsi_cmd_iocb_cmpl:2185: 1 times

On cpu8:38366)lpfc: we saw error lpfc_scsi_cmd_iocb_cmpl:2185: 1 times

Here is what i ran:

$Events = (Get-Log -VMHost (Get-VMHost "host name") vmkernel).Entries

ForEach($Item in $Events)

{

$item | Group-Object -Property {$_.Split(' ')[1]} | %{$name = $_.Name

   $_.Group | Group-Object -Property {($_.Split(' ')[2] -join ' ')} | %{

   Write-Output "On $($Name) we saw error $($_.Name) $($_.Group.Count) times"

  }

}

}

[LucD · Accepted solution]

If you look line by line, you will never find duplicates.

Try like this

$Events = (Get-Log -VMHost (Get-VMHost "hostname") vmkernel).Entries

$Events | Group-Object -Property {$_.Split(' ')[1]} | %{

    $name = $_.Name

    $_.Group | Group-Object -Property {($_.Split(' ')[2] -join ' ')} | %{

        Write-Output "On $($Name) we saw error $($_.Name) $($_.Group.Count) times"

    }

}

[habs3]

Thank you

This worked for me and what i was looking to compete.  I completely missed the foreach as a single instance and now see where you fixed that with the group.

Thanks again