Big data – the analysis of large volumes of changing data from structured and unstructured sources – has promise in business and government. Government “open data” is seen by many observers as an economic opportunity for business. Gartner analysts Doug Laney and Frank Buytendijk described trends and opportunities in a recent webinar (storified below). Some takeaways from the webinar include:
Vendors (as usual) are confusing customers by focusing on some elements of big data rather than what’s important – the variety of data
Disruption of business models where data can be monetized by organizations
Tipping point has arrived for senior management interest
Disruption in traditional “business intelligence” disciplines like data warehousing
I’ve been somewhat vocal about how these two landmark surveys are socialized. It seems to me that they help persist inaccurate narratives about developing countries. That’s not to say that these surveys are not helpful in the anticorruption fight. It’s that observers can extrapolate some fairly fantastic narratives from the tool. Cobham points out that “in general the media emphasis on corruption in lower-income countries is, if anything disproportionate.”
I took exception to an article yesterday that suggested that because of higher trust of NGOs compared to governments, donors should reconsider the use of country systems. I collected a set of odd conclusions from the GCB last week. Alternative explanations to perception changes over time do not seem to be considered. For example, increased transparency and accountability can expose more corruption leading to the perception that there is an increase. Or, the increase in corruption reporting may have the same perception as crime statistics in many countries where the incidence of violent crime is reducing but the perception is the opposite. And, the GCB focus on bribery, although useful, measures but one aspect of the corruption problem.
Cobham minces no words in his assessment that “CPI embeds a powerful and misleading elite bias in popular perceptions of corruption, potentially contributing to a vicious cycle and at the same time incentivizing inappropriate policy responses”
My colleague Carlos Lipari has pointed out some of the problems in measuring corruption. Cobham describes the “sample size” problem of the CPI that looks at perceptions of a small sample size of elites and GCB that has a broader sample. Cobham points out that in CPI “the same kinds of people are being asked for their perceptions.”
Where does that leave us then? How can we find more effective measurement for corruption?
Policy responses and legal reform such as the introduction of an anti-corruption commission is often window dressing, because, in the words of Matt Andrews, often “what you see is not what you get.”
Open data standards IATI (aid), EITI (resource transparency), CoST (construction) and the emerging OpenContracting (procurement) provides “big data” of structured and unstructured data. Many countries have transparency portals that provide additional open information. Much of this information can be processed on inexpensive cloud computing platforms using open source tools.
Social media and the use of mobile devices provide more tools in the hands of citizens. We’ve seen the impact in the Arab Spring and recent anti-corruption protests in Brazil, Bulgaria and Turkey. Crowdsourcing open-source tools like Ushahidi provides platforms to increase the sample size for corruption perception while collecting real data on corruption incidents. Big data tools used for marketing like “sentiment analysis” could also be leveraged.
Civil society reports on corruption incidents. This information can be captured via RSS feeds and other mechanisms to build up a statistical picture that could be compared to other mechanisms. This means that “top down” and “bottom up” measurement methods could be used. This will expose potential gaps that will help to improve the measurement process.
My sense is that breakthroughs in anti-corruption enablement will occur when a critical mass of open information can be analyzed through the use of semantic web techniques will expose patterns. This will show us the weak corruption signals in countries that we do not have significant amounts of data. And, it will put pressure by the international community on what kinds of information is useful for combating corruption.
We’ve been tweeting from 27th. Annual International Consortium on Governmental Financial Management (ICGFM) conference in Miami. You’ll find the chronological “storified” version of these tweets from each presentation posted on our storify site We’ve also summarized some of the themes from the conference.
Government open data advocates are shooting themselves in their collective feet. Skeptics suggest that there is little value in some, many or all open data initiatives. These pragmatists and curmudgeons demand to see the Return on Investment (ROI) calculation.
What do they get from open data advocates?
Anecdotes. Often the same anecdotes.
I had intended to review a recent study on open data. But, it’s just a bunch of attractive pictures with anecdotes. Some anecdotes stretch to an entire page and are presented as “case studies”.
A page does not a case study make.
I also got the impression that the information was stretched out to fill the page. Of course, with just enough information to determine that the conclusions are dubious.
Should it be surprising that governments in developing countries are adopting transparency measures? Many are surprised that developing country governments adopt transparency portals and open data mechanisms. Yet, we see great interest in leapfrog transparency among our customers. It has been a major theme for discussion in the last 5 FreeBalance International Steering Committee (FISC) conferences. There are important incentives. We hope to document some of these over the next few weeks, first by reaching out to experts on Twitter. Then through discussions with Public Financial Management (PFM) professionals attending FISC in Ottawa.
We’ll be collecting comments here. And, we’ve “storified” the tweet stream on the subject
GRP or Integrated Financial Management Information Systems (IFMIS) can be cornerstone to anti-corruption
Doug Hadden, VP Products
We were asked last week about how FreeBalance software contributes to reduced corruption in developing countries. There can be a significant effect on reducing corruption.
What is the linkage between “anti-corruption” and GRP?
Governments implement Government Resource Planning (GRP) or Integrated Financial Management Information Systems (IFMIS) to improve fiscal management and governance as part of Public Financial Management (PFM) reform
GRP has a significant effect on financial institution and investor corruption and is enabler for reducing corruption by executives. Based on the World Bank study in 2000 , GRP has a possible positive effect on more than half of attributable corruption in developing countries.
What are the corruption effects on public finances?
Decentralization: of functions to impede centralized corruption
Payments: eliminate cash payments through secure cheque printing and electronic funds transfer making financial transactions fully traceable and reducing cash fungibility including use of the Treasury Single Account to prevent manipulation
Reporting: knowledge that fraud patterns can be uncovered in reporting or audit changes the behaviour of politicians and public servants
Segregation: of duties for processes across many people reduces the ability of any one person to commit fraud
Countries with low perceived corruption enjoy higher human development
What forms of corruption are restricted through the use of GRP?
Accrual Accounting: prevents hiding arrears and assets that are possible in cash accounting
Briefly, exo means outside; exopolitics therefore, for our purpose, politics outside politics. Because politics outside politics is emerging as the core phenomenon of American culture. And on the scale at which we are experiencing it it is novel.
Phenomena like the tea party and occupy movement are exopolitics symptoms. Social media, open data and Government 2.0 enables exopolitcs, according to Cameron.
Observers like Evgeny Morozov see technology more as tools of repression than any kind of democratic enablement. And, Morozov questions any concept of technology determinism. Cameron is among the current thinkers like Jeff Jarvis and Clay Shirky who have a broader understanding of the effects of technology on society. (Of course, Harold Innis, Marshall McLuhan and Friedrich Kittler provide a much better perspective on technology and change covering centuries.)
Technology provides opportunities for organization outside traditional organizations. It’s no surprise that the traditional opposition in Egypt was a surprised as the regime with the Arab Spring. This technology doesn’t immediately have effects. It takes some time for the medium to become the message. So, traditional methods of political interaction will remain dominant while pressure for transparency mounts to a tipping point.
Representative Democracy and Demigods
Here’s where I differ with Cameron. Cameron suggests that representative democracy should remain the primary vehicle for change. He points to how direct democracy has failed such as the ballot system in California that limits fiscal options or the way in which Napoleon manipulated plebiscites. This is a pretty thin argument in my opinion. Why? There is always the tendency to think that traditional methods will be used in new eras.
It’s true that various methods of “Government 2.0″ outreach have resulted in support for marijuana legalization. We can’t expect traditional thinking to immediately change thanks to new technology. Look how long it has taken for the obsolete medium known as newspapers to finally see subscription reductions.
My view is that Government 2.0 will open up new “exopolitics” avenues – not so-called “direct democracy” but richer citizen to government interaction because:
Open data and social media will provide more evidence-based debates (eventually – maybe not in the current US Presidential election)
Social tools will enable interaction with citizens on important matters, particularly with expert networks at first rather than traditional crowdsourcing
Instant organization around important subjects will disintermediate traditional power structures such as lobbyists (but not immediately)
Effects will be seen more in local government where there is a more direct relationship with citizens
This will take some time. Governments will continue to struggle open data and culture change. Traditional media will continue to polarize citizens. And, representative democracy will be the main avenue for political change. But, these time are achangin’.
The American television network, NBC, has exclusive broadcast rights to the 2012 London games. Broadcast coverage by NBC has generated the twitter hash tag of #NBCfail. The use of social media, including twitter has generated an undercurrent of satire and ridicule for:
The problem for NBC as for other media is that it is trying to preserve old business models in a new reality. To experiment with alternatives when billions are at stake is risky. But so is not experimenting and not learning when millions of your viewers can complain about you on Twitter.
From Broadcast to Always On
What’s changed from, say 1996? There was tape delays during the 1996 Atlanta games, in the United States, for events like the Men’s 100 Meter Dash [see below].
Broadcast electronic media enables control. Control of the message. Control of the advertising revenue. But, there is limited control in cyber space which is always on. Where people in stadiums and those watching in other countries can post results.
NBC executives seem to think that American citizens need the context presented. As if Americans are not bright enough to understand what is going on (or find out through a search on the Internet.) This is an patronizing attitude by elites is described in full by John Ralston Saul.
Lessons for Governments
Public servants need to consider that they do not have all of the answers or the best answers. Governments can leverage citizen cognitive surplus.
Citizens view delays of information as indicative of incompetence or hiding something so transparency can increase citizen trust.
Governments cannot stop information flow to citizens, so it’s better to be proactive with open data rather than wait for access to information requests.
Governments will be satirized and ridiculed – as they have been for centuries, just on cyberspace. Get used to it. Adjust.
Transparency will reveal incompetence, corruption and inefficiencies. Embrace these to improve government effectiveness.
1996 100 Meter Dash
I watched the event on CBC. There was a lot of tension given the strong field. After watching the stress of false starts and the eventual win by Donovan Bailey, I flipped to the American channel (we can do that in Canada, although there doesn’t seem to be much reciprocity) to find that the network was setting up the event. Providing context. Here’s British commentary from the event. Wasn’t this enough tension? Isn’t it better than reality television?