To resolve this week’s issue, we went for the most obvious resolution path and performed a CDR to SS7 reconciliation against a days worth of traffic. After performing the reconciliation, we were still left with a large amount of SS7 records with billable duration that didn’t match to CDR’s. So we then started to analyse these SS7 records and the underlying customers ie
Had the customers produced any CDR’s – Yes,
Were calls to the specific destinations producing CDR’s – Yes,
Was it a specific duration issue – No,
Was there a unique flag on the SS7 record – No,
Did the SS7 records just originate from one switch – No – but not the entire network.
It was this last point that started to hone us in on the actual issue, all of the additional SS7 records came from a specific region of the country, which was served by 8 switches. According to the company’s engineers, there was nothing unique with this part of the network, so it was back to the analysis. At this point we started to look at individual customers calling profiles and it was when we did this that we found the problem. We effectively had duplicate SS7 records – although they weren’t true duplicates in that the call times and durations were ever so slightly different. We had duplicate checked the data right at the beginning of the analysis (as any good analyst will do), however because the records were different – they weren’t flagged up to us.
Each pair of SS7 records were being recorded by different probes, the overall SS7 solution worked in such a way that only the originating probe generated the dummy SS7 CDR, so the fact there were two records, both stating they were the originating probe, suggested something was wrong with the network ie double call routing. At this point the issue was handed back to the engineers to work out what the cause was.
The network had originally operated an “A-Link” signalling network (signalling that goes down it’s own pipe via an SCP), however the operator had moved to an “F-Link signalling network (signalling is carried by the same pipe as the call) a few years prior to the investigation described above. The old A-Link signalling was supposed to have been fully decommissioned once the new F-Link was put in place, however it transpired that in this region of the country, the decommissioning had somehow been missed. So what was happening, was that for each call being made, the signalling was being sent across two pipes and hence two originating SS7 probe CDR’s were being made -as there was a probe on every part of the signalling network(I know – you would have thought they would have spotted the issue when installing the probes!!)
An SS7 to CDR reconciliation is one of the best pieces of analysis an RA department can perform – it allows us to verify the one piece of information we rely on for the majority of our other reconciliations (the switch CDR). Ultimately though, I hope the above issue has demonstrated that an SS7 record cannot be treated as definitive and does require the same levels of authentication as any other record produced by a network.
Entries (RSS)