The first MOOC I completed ended on February 10th. Foundations of Data Analysis, a course by the University of Texas at Austin was hosted on edx and essentially, was an overview of statistics from regression to hypothesis testing. The course itself was excellent at introducing concepts and putting forth the theory while at the same time it introduced R, a statistical analysis tool. Personally, the material was a review for me as I had taken a statistics course, as well as an econometrics course during my undergraduate studies. That was a few (11 since graduation) years ago and apart from making you think differently there has not been any practical application of this knowledge.
Each lesson was released on a Tuesday consisting of a reading (pdf), lecture videos, R tutorial videos, pre-lab, lab and problem set. The readings were chapters from a statistics textbook and provided the foundation of the theory for each topic. Lecture videos was additional theory and insight on the topic by Dr Mahometra, while the R tutorial videos introduced the related R commands. The pre-lab and lab were a series of exercises were you could apply the theory and R commands. The lab and pre-lab were very similar in nature; basically the pre-lab was guided by the videos while the lab was work you had to complete unguided. Finally, there was a problem set at the end for additional practice.
Statistics adds a lot to the way you view data and information, either at work or even in the news. However, unless you are directly working at a company that analyses data in this way you will not come across most of these concepts. You might use some basic descriptive stats but inferential statistics as well as modelling are rare. In RA specifically, statistics will be used above and beyond day to day activities. They could be used initially to “sniff out” some characteristics of the data, but the inferential and modelling parts will come only at a stage when everything is running so smoothly that RA is providing additional information from the checks demanded.
For anyone wanting to get a good overview of statistics this course is ideal. The fact that you also learn a modern statistical tool in the process is an added benefit. My only reservation is not specifically with this course but with MOOCs and training outside of working in general. Depending on your stage in life your time can be scarce and this means that you need to be selective with how you choose to spend it. There will be times when you might feel like the knowledge you gain is not immediately applicable and that it is not worth it. However, another way to look at it is that you are adding to and building your resume as well. You are building for the future or you can simply be reviewing. If you deal with data on a daily basis – as most RA professionals do – then some knowledge of statistics would help you even if it’s not something you can use right away.
The next courses I have registered to take are:
1. Communicating Strategically – PurdueX – started this week – duration 5 weeks – “Improving communication skills and presentation effectiveness for scientists, engineers, and professionals.”
2. Networks, Crowds and Markets – CornellX – 16 Feb 2015 – duration 10 weeks – “an exploration of fundamental questions about how our social, economic, and technological worlds are connected. Students will explore game theory, the structure of the Internet, social contagion, the spread of social power and popularity, and information cascades.”
3. The analytics edge – MITx – 3 Mar 2015 – duration 12 weeks – “apply analytics to real-world applications” – looks like the most challenging but also the most relevant and rewarding one for an analyst.
4. Introduction to Big Data with Apache Spark – UC BerkeleyX – April 2015 – duration 4 weeks (python or other programming language prerequisite) – “Learn how to apply data science techniques using parallel programming in Apache Spark to explore big (and small) data.”
For each course, I will usually decide if I will follow it to the end during the first week. This will also depend on my time availability of course… For the Data Analysis MOOC I added pressure to myself because I wanted to see it through so I paid for an ID verified certificate. I crammed at the end to achieve the required score to get the certificate, but it was worth it. If nothing more than for the sense of accomplishment.
Posted by: Eric in News, Opinion
The Intercept has broken a major story about US and UK intelligence agencies hacking into a SIM card manufacturer and stealing encryption keys used to protect the privacy of mobile phone users worldwide; see the full story here. Whistleblower Edward Snowden provided them with a secret 2010 document from GCHQ, the UK’s electronic surveillance agency. Dutch SIM card manufacturer Gemalto is named in the document, which states:
…successfully implanted several machines and believe we have their entire network – TDSD are working the data
Gemalto is the world’s biggest manufacturer of SIM cards, making 2 billion of them every year. The firm provides a variety of encryption-based security solutions to governments, telcos and banks. Gemalto describe themselves as “a world leader in digital security” and their motto is “security to be free”.
The article also reports that:
Additionally, the spy agency targeted unnamed cellular companies’ core networks, giving it access to “sales staff machines for customer information and network engineers machines for network maps.” GCHQ also claimed the ability to manipulate the billing servers of cell companies to “suppress” charges in an effort to conceal the spy agency’s secret actions against an individual’s phone. Most significantly, GCHQ also penetrated “authentication servers,” allowing it to decrypt data and voice communications between a targeted individual’s phone and his or her telecom provider’s network. A note accompanying the slide asserted that the spy agency was “very happy with the data so far and [was] working through the vast quantity of product.”
The Intercept also provides some insight into how the hacking took place:
Top-secret GCHQ documents reveal that the intelligence agencies accessed the email and Facebook accounts of engineers and other employees of major telecom corporations and SIM card manufacturers in an effort to secretly obtain information that could give them access to millions of encryption keys. They did this by utilizing the NSA’s X-KEYSCORE program, which allowed them access to private emails hosted by the SIM card and mobile companies’ servers, as well as those of major tech corporations, including Yahoo and Google.
I recommend you read the full story for yourself. It makes for fascinating reading for anyone who works in the telecoms sector, describing the infiltration of mobile networks in various countries and naming Telenor, Mobilink, MTN and Belgacom amongst the victims.
This news makes me angry. Whilst the UK and US governments have repeatedly talked about the need for the legal power to intercept communications in order to protect the public from criminals and extremists, there is no legal or moral justification for how these spy agencies have behaved in this instance. The possession of these keys would allow spy agencies to spy on mobile communications, both at home and abroad, without anyone knowing. Moreover, this particular hacking was not aimed at criminals or terrorists. The targets were private companies and their employees, all of whom were going about their lawful business. Western governments have often chided the Chinese government for stealing commercial secrets, but that is exactly what the NSA and GCHQ did when they eavesdropped on the communications of employees of SIM makers, equipment manufacturers and telcos. And where is the court order that permitted them to do that? The US and UK governments have lost any moral high ground that they previously pretended to. And US President Barack Obama has been outed as a baldfaced liar, or an ignorant buffoon, when he said in January 2014:
…people around the world, regardless of their nationality, should know that the United States is not spying on ordinary people who don’t threaten our national security and that we take their privacy concerns into account in our policies and procedures…
You work in telcos. These governments have shown they can, and will, spy on you too, just because you work in the communications sector. You need to get angry, and stay angry. Telcos in the private sector have become increasingly robust when standing up to governments and their insatiable desire for surveillance. Yet, at the same time, politicians like Obama call for ever more government control of the communications sector. The public needs a strong, healthy, vibrant and privately-owned communications industry that will serve its customers by opposing government overreach. The public needs people like you, to stand up for them.
Posted by: Eric in News, Opinion
Have you ever left a smiley :) or other emoticon at the end of a text message? Maybe you need to check your handset and your tariff before you do, warns British consumer website MoneySavingExpert.com. After being warned by readers, they wrote a brilliant article which elegantly explains how unanticipated interdependencies between systems can lead to very unhappy customers. Some handsets will automatically convert the text of a smiley into an image, prompting the message to be sent as an MMS instead of being an SMS. Many tariffs include free text messages in the customer’s allowance, but charge for every picture message. As a consequence, prolific texters might rack up big bills before realizing that the use of emoticons will cost them money.
MoneySavingExpert.com does a great job of listing the handsets that will convert emoticons into pictures. But the real story is that the telecoms industry insists on giving itself a black eye, when there is no need. As pointed out by MoneySavingExpert.com, apps like Facebook Chat and Whatsapp allow customers to send any message, including images. Telcos do not want customers to switch to these apps. However, we know that confusing tariffs and bill shock drives customers towards experimenting with these apps. When people turn to a website like MoneySavingExpert.com to understand bill shock, it inevitably means more people will try alternative messaging apps, further undermining telco revenue streams.
Perhaps some telcos calculate it is worth upsetting a few customers in order to benefit from inflated bills and increased revenues. After all, they can always try to blame the handset manufacturers for converting the text into a picture. However, the response on MoneySavingExpert.com shows it is foolish for telcos to think this way. Customers invariably blame big bills on the comms provider that sent them. And why should handset manufacturers care about telco tariffs? The manufacturers make the same money, irrespective of whether the customer uses Whatsapp or sends an SMS.
Telcos need to protect their own interests by proactively informing customers of potential charges for silly little elements like emoticons. Better still, they should present themselves as champions who will defend customers by fighting against unhelpful handset features that drive up bills. A good way to start would be to give customers guidance about how handsets vary in how efficiently they consume data. Modern electrical appliances are rated, so customers can compare the electricity used by two different washing machines, or two different televisions. Why not rate handsets based on how efficiently they use data bandwidth? The differences may not be large, but the increased focus on handset manufacturers will discourage the tendency to solely blame telco ‘greed’ for network congestion and all the other issues that anger net neutrality advocates.
When people visit sites like MoneySavingExpert.com to get important tariff information they have not been told elsewhere, it means the comms provider is failing to effectively communicate with their customers. Unexpectedly large bills are proof of a failure to communicate. Every failure encourages those who push for more price controls to be imposed on telcos. The only sensible strategy is to keep our customers looking like :) and never like :(
Posted by: Eric in News, Opinion
Those who work in telecoms assurance know it is not the centre of the universe, though some believe and wish it was closer to the centre. However, the attention currently being given to assurance in Ghana offers some insights into how and why assurance can rise up the public agenda. It also shows that such exposure has downsides as well as benefits. Recently talkRA has featured stories about the implementation of a new national interconnect clearing house in Ghana, and how this has been linked to the prevalence of simbox fraud and the consequential impact on government tax revenues; see here and here. In the last week, there have been yet more developments in this story. Opposition MP Kwaku Kwarteng has blasted the government’s clearing house plans, saying they are “dishonest”. In an article for Ghana’s JoyOnline news station, the MP wrote:
Government is in the process of appointing an Interconnect Clearing House (ICH) operator to connect calls from one network to the other on behalf of the telecommunications service providers.
This is nothing but connivance between government and its cronies posing as some Interconnect Clearing House company to make undeserved money from subscribers and users of telecommunications services…
…perhaps the strangest of all government conduct is the attempt to justify the ICH by presenting it as an answer to sim box fraud. Not only is this deceitful, it is also laughable. Sim box fraud is the practice of routing international calls through the internet in order to bypass the legitimate route for international calls. This is done criminally to avoid taxes. There is nothing about an ICH that can check this bypassing, and government knows this.
I must agree with Kwaku Kwarteng, who also sits on the Communications Select Committee in the Ghanian Parliament. It makes no sense to combat the illegal bypass of legitimate telecoms operators by imposing a new mechanism for transferring interconnect traffic between them.
The real issue is whether Ghana’s government is getting all the tax they are entitled to, and want, from Ghana’s telcos. You can decide for yourself if what they want is the same as what they are entitled to. However, we can all agree there is a fine line between assurance work designed to validate the accuracy of revenues reported, and audit work designed to extract more tax from businesses. The distinction lies in the objectives and motives of the professionals who review accuracy, and of the people who employ them. Clearly political tensions are running high following a special government-mandated audit of Ghanian telcos. In response to various insinuations, Vodafone Ghana has felt it necessary to make a public statement about the results of the audit, and how it should be interpreted. When discussing one fine subsequently imposed by the regulator, Vodafone asserted the following:
The fact that Vodafone was providing handling fees to International Carriers for securing its traffic from going via SIMbox fraudsters was perceived by the NCA [Ghana's regulator], based on an ambiguous regulatory clause, as a regulatory breach (not a revenue assurance breach) and that accounted for the penalty.
In other words, Vodafone took care to say their data was accurate but they disagreed with how the regulatory rules were interpreted and applied.
Vodafone clearly feel they have received unfair criticism from some quarters, as emphasized by the closing paragraph of their statement:
In the name of fairness and transparency, Vodafone reserves the right to exercise all available options, including legal, against any individual or group, who impugns on the hard won image and reputation of the company by peddling falsehoods about this issue.
Meanwhile, rivals MTN Ghana released an extraordinary press release stating they ‘remain committed to the fight against simboxing‘. What makes the press release so unusual is that it efficiently explains what simboxing is, before detailing the various ways that MTN detects and responds to the crime. They then go on to give a table comparing local call rates to international termination costs in several African countries. It shows that simbox fraud occurs where the difference between these prices creates the incentive for crime. This leads MTN to conclude:
The so-called monitoring solution that is practiced by many countries in Africa is one that attempts to treat the problem after the fact, in other words, attempt to put out the fire after it has started. By its very definition monitoring is aimed at detecting the fraud while it is already in progress. Due to the lucrative nature of the business, SIMboxers are able to employ highly skilled technicians adept at detection avoidance. They are also able to side-step controls instituted by telcos to streamline SIM registration.
MTN believes steps must be taken to stop the illegal practice altogether. If the US$0.19 mandatory pricing is removed SIMBox fraudsters will have no commercial reason to even practice SIMboxing in Ghana.
Ghana is not the first country to have debated the purpose of telecoms assurance, only to discover that political and selfish reasons can lead to contradictory conclusions. However, I doubt that any country in the world has ever engaged in such a high-profile and tense public debate about what is, and what is not, telecoms revenue assurance. In my view, this is a foretaste of an increasingly public debate that will occur in more countries. Industry consolidation will lead to less competition between telcos. This will encourage increased political speculation about whether telcos charge ‘fair’ prices, and if their revenues, profits and taxes have been transparently and accurately calculated. Distinctions will be drawn between assurance work done within telcos, and audits conducted on behalf of the government. In turn, this will draw attention to grey areas in the calculation and reporting of some numbers. If our profession is weak, there will be plenty of people employed on either side, each pulling in different directions. It would be better to have a strong profession, that avoids such conflict by being rigorous about its techniques and its terminology. But either way, this tension will make additional work for audit and assurance practitioners.
Posted by: Eric in News, Opinion
If you work in revenue assurance, you surely need the instinct to spot numbers which look suspicious. I am often bemused by market ‘research’ reports which sell for several thousand dollars but present funny-looking numbers and questionable facts amongst the few snippets they give away for free. The latest example is from marketsandmarkets.com, who want your company to pay USD7150 to discover their predictions for the RA market from 2014 to 2019. The report covers all industries, which immediately begs a question of how easy it is to research the extent of ‘revenue assurance’ in banking, healthcare, and logistics… or even what counts as revenue assurance in these contexts. Nevertheless, the report tell us that:
The global revenue assurance market is expected to grow from $ 1.7 Billion in 2014 to $ 2.9 Billion in 2019
So the market is already worth USD1.7bn? Really? I have arguably the most popular website dedicated to revenue assurance, and I can assure you it does not feel like I am working in a USD1.7bn market. But I kept an open mind, and continued to read their press release. After all, it is very possible that when marketsandmarkets.com sized the market, they included revenues from lots of businesses that provide software and services which I would not include in the scope of revenue assurance. And then I noticed this…
The Global Revenue Assurance market consists of large players like Cvidya, Subex, Nec, Accenture, and others – which offer services and solutions in this market. The market has seen these players grab high amount of market share.
Let me pick one of these ‘large players’ and do some quick common sense maths to verify the reports’ claims. For my example, I will work with numbers from cVidya for no other reason than they were first in the list. ***ahem***
We can reasonably estimate that cVidya’s annual revenues are about USD40mn. They are smaller than Subex, a publicly traded company whose numbers are audited, and WeDo, whose reported numbers can be sense-checked by reviewing the performance of their division within their parent group’s accounts. Corroboration that cVidya is smaller than Subex and WeDo is available in the form of other reports from other research firms; for example, Stratecast recently reported that WeDo led competitors with a 14% share of the telco assurance market. But comparing cVidya’s revenues to the total pan-sector assurance market would give USD40mn/USD1.7bn = 2.3% market share. If cVidya has a 2.3% share of a USD1.7bn market, how can this be reconciled with the assertion that they are a “large player” with “a high amount of market share?”
If we assume Stratecast’s numbers are accurate, and use the thumbnail rule that WeDo told Stratecast that their annual revenues are around USD70mn, then that means Stratecast sized the telco assurance market at approximately USD500mn. For marketandmarket.com to have accurately sized the pan-sector assurance market, that would mean the non-telco market must be currently worth USD1.2bn. It then seems odd to me that marketsandmarkets.com start their list of “large players” with cVidya and Subex. Neither of these companies is the current market share leader in telco assurance, and neither firm generates significant revenues outside of telecoms. In fact, WeDo has made much more effort to diversify outside of telcos. And whilst WeDo are pleased with their non-telecoms growth, they still admit that the vast majority of their revenues comes from telco customers.
Not long ago, Subex CEO Surjeet Singh spoke to me about his hopes for his company, saying he wanted to grow Subex revenues to USD100mn within the next few years. But even USD100mn of revenues would only translate into a 5.9% share of what marketsandmarkets.com say is the total value of the pan-sector market.
In short, there is something very wrong with the research of marketsandmarkets.com. Either they have massively exaggerated the size of the pan-sector assurance market, or they have badly miscalculated the revenues of cVidya and Subex, or they have deliberately misrepresented cVidya and Subex as “large players” whilst choosing not to mention much larger players, or there are no large players because there is only a long long list of small players – which begs a question about the coherence of this market. Whatever the truth, it is not worth paying USD7150 to discover where they went wrong.