BIG Data The New Oil

The ability to harness and store large volumes of data to enable future services and add extensions to current services to improve can only be achieved with big data. Every project or deployment can be woven into a fabric using the threads of big data such that city managers can see where their deployments are adding value and how the synergies are greater than the individual components.
<p></p>
<p> Back in 2006, when <span style="font-weight: bold;">Clive Humby, Mathematician and Architect, Tesco Clubcard</span>, first coined the phrase, ‘Data is the new oil’, no one took him seriously, until now! A decade later, data is truly realised to be the oil of the digital era. In fact, with the synergy between big data and artificial intelligence (AI), they seem to be echoing, ‘I see everything’. They not only see, but also sort and identify information accurately. Now, one may wonder what big data has to do with one of the most niche sectors – infrastructure. In fact, it has a lot to do, principally because the sector requires lot of coordination during the development, construction, maintenance and day-to-day operations of projects and the systems they operate in. Yet, with that entire sophisticated organisation, the often overlooked issue is communication and the real-time flow of management information.</p>
<p>As rightly observed by <span style="font-weight: bold;">Gautam Balakrishnan, Vice President and Head û Smart Cities Business, Tata Projects, </span>’The adage ‘garbage in, garbage out’ applies to big data in a huge way’. If the input data for decision-making is inconsequential, then the likely outcome of any decision made on the basis of that data will be negative. In case of public-private-partnership (PPP) projects, where financial viability and project paybacks are critical to financing it, the need for reliable data models becomes paramount. </p>
<p>To this, <span style="font-weight: bold;">Sudhir Aggarwal, Vice President &amp; Head, Government Relations, Thomson Reuters South Asia,</span> agrees with Balakrishnan. ‘Absolutely! Infrastructure projects are often hit by various financial irregularities and risks.'</p>
<p><span style="font-weight: bold;">Jagdish Salgaonkar, Senior Vice President, Civil &amp; Infrastructure, Aecom, </span>explains the role of big data with an example of the Dholera project, which is a part of the ambitious Delhi-Mumbai Industrial Corridor (DMIC), by saying, ‘The role of big data during project implementation is different from that during project operations.’ </p>
<p>And why not? The implementation phase typically consists of planning, design and construction. In a city building project like Dholera there are at least seven different disciplines being simultaneously planned, designed and constructed. These are water, wastewater, recycled water, storm water, power, ICT and transport (roads) systems. Elaborate software systems which use variations of big data analytics are available to optimise the design for each system. During the design phase, each element (asset) is given a unique identification number which later becomes part of the asset management program. Performing designs in 3D with extensive use of BIM and other collaboration platforms is the key to making designs efficient <span style="font-weight: bold;">Lux Rao, Director &amp; Leader, Digital Transformation Office, Cisco India &amp; SAARC,</span> explains, ‘The role of data-driven insights is increasingly becoming the cornerstone for decision-making across industries and domains, and increasingly so for large-scale projects’.</p>
<p>He further adds, ‘Large-scale infrastructure projects require constant communication and real-time flow of information. The primary role of big data is to facilitate these information flows and mechanisms for learning and coordination among varied individuals’.</p>
<p>Most large-scale infrastructure projects today are a result of PPPs, where data-driven outcomes are critical because these projects need to generate value for money, be innovative and attain operational efficiency in a defined time frame. </p>
<p> <span style="font-weight: bold;">Big data: Value for money</span><br />
These are exciting, if not daunting times for governments, planners, asset managers and infrastructure developers. The confluence of information technology and a complex web of infrastructure unlocks a wealth of opportunity. The data that flows from this convergence should improve performance as well as transparency and save money for everyone. In terms of PPPs, data-driven outcomes are essential when value for money, disclosure, innovation and operational efficiency are at the core of a project’s objectives. </p>
<p>There are many examples of deployment of big data in infrastructure projects; however, a classic example is how geographic information system (GIS) data has been used extensively for planning of roads in Tamil Nadu. (To know more, refer, ‘How GIS can help in the planning of roads’ in page 33) </p>
<p>Another case in point is the logistic data bank (LDB) connecting all the major ports in India, providing visibility and transparency to EXIM container movement along the western corridor of India. Starting at the ports, the containers moving through rail or road are tracked until they reach the inland container depots (ICDs) or container freight stations (CFSs). LDB has already covered around 2.5 million containers since its launch in July 2016 at India’s biggest port, the Jawaharlal Nehru Port (JNPT) in Mumbai, where it is currently operational in four port terminals. And now, with the introduction of LDB in Mundra and Hazira ports, nearly 90 per cent of the traffic along the western corridor and 70 per cent of the total container volume of the country can be tracked under one system. (To know more, refer, ‘How LDB works’ in page 37)</p>
<p> <span style="font-weight: bold;">Socially proven</span><br />
A much-needed experiment on the use of big data was carried out in social infrastructure. The district was Chandrapur, which despite having vast natural resources of coal, lime, wood and more, still remains underdeveloped. Around 33 per cent of the district is under forest cover. There are immense coal reserves in the Wardha Valley Coal?eld and there are many cement factories. Yet, around 50 per cent of the houses in Chandrapur are kachcha buildings and a mere 10 per cent of households use LPG for cooking. </p>
<p>Chandrapur desperately needs development! Yet a standard development plan cannot account for the diversity of Chandrapur’s blocks. This is because most of the citizens were off the radar as far as government schemes were concerned. They were unable to reap the benefits of various schemes like employment, food security, water and electricity.</p>
<p>To help district officials drive better budget and policy decisions in Mul, Pombhurna, and Jiwati blocks of Chandrapur, Tata Trusts has partnered with SocialCops and local NGOs. A data intelligence platform was deployed to create a centralised planning tool that would be used to effectively micro-target development initiatives. Though one might find it hard to believe, whopping 6.9 million data points from 290 villages and 900 surveys were created and collected for this exercise. (To know more, refer, ‘Collect, transform and visualise’ in page 29) Realising the value of data, some of the cities in India have actively taken up the agenda of working towards data-driven governance (DDG) to inform, impact and improve policymaking, with a view to facilitate holistic development of the nation. At present, three cities û Pune, Surat and Jamshedpur -have implemented DDG in association with the World Council on City Data (WCCD).</p>
<p>What have these cities done? Says <span style="font-weight: bold;">Patricia McCarney, President and CEO, World Council, </span>’With the DDG programme, these cities have implemented a pilot project, by micro-targeting interventions using the data from mobile-based real-time data survey and creating a model of convergence of government schemes. Further, they have enabled last-mile linkage of individuals to schemes, thus empowering communities’. (To know more, refer, ‘City data, a new currency’ in page 40)</p>
<p> <span style="font-weight: bold;">Financial outcome </span><br />
Meanwhile, Tata Projects, one of the largest infrastructure companies, uses big data and predictive modelling techniques to ensure that they factor in all possible variables that can affect the financial outcome of any project. This information, says Balakrishnan, ‘àis shared with potential lenders to give them comfort that the project is viable and will yield sufficient funds from the monetisation of transactions to enable financial closure of the investment’. </p>
<p>In the absence of reliable and widespread data being made available, most lenders would shy away from projects and would prefer to be conservative in their approach. This is substantiated by the value of stalled projects in India that stand at a whopping Rs 11.70 lakh crore. While the reasons can be many, the common note was decimating financial closure. </p>
<p>This insecurity strips a perfectly good project of its funds. To prevent this situation, Tata Projects is working with big data tools to ensure that all relevant data and statistical analyses are completed well ahead of the project funding stage.</p>
<p>So how can the government avoid strained situations, where the private players are in deep financial trouble and the government is under continuous scrutiny from the opposition? To answer this question, <span style="font-weight: bold;">Vishal Dhupar, Managing Director, South Asia, NVIDIA,</span> suggests the use of data science that focuses on the processes and systems to enable extraction of knowledge or insights from data in various forms, either structured or unstructured. In practice, data science has evolved as an interdisciplinary field that integrates approaches from data analysis such as statistics, data mining and predictive analytics, drawing on diverse observational domains. </p>
<p> <span style="font-weight: bold;">PPP monitoring</span><br />
The rigid and pre-determined structure for post-implementation evaluation is a major hurdle to innovation in PPP projects. Lack of flexibility renders even small changes very difficult to incorporate. </p>
<p>Many infrastructure developers have struggled in the past to convince the customer, especially when there is a positive indicator in the data that is being generated, showing that the project is actually getting better than the expected results. If big data platforms are used to measure the efficacy of PPP projects, the true picture û whether it is for better or worse – will emerge. The accuracy of measurements would improve and payments to the partners and the city would become more transparent. An example to cite is DMIC. To monitor progress, DMIC has appointed a project management consultancy. They have also developed an in-house project monitoring and implementation system (PMIS), says <span style="font-weight: bold;">Alkesh Kumar Sharma, CEO and Managing Director, Delhi-Mumbai Industrial Corridor Development Corporation</span> adding, ‘through which we monitor the progress of each phase of the project.’ </p>
<p>Through PMIS, DMIC is able to monitor real-time progress on a dashboard, by which they perceive the amount of work that is completed or is in progress. Interestingly, this dashboard can be accessed from anywhere in the world. ‘You only require an Internet connection. Here, officials responsible for their tasks are required to update the IT-based monitoring system on a real-time basis, as there are multiple contractors involved,’ he says. </p>
<p> <span style="font-weight: bold;">Procurement and data </span><br />
To strengthen the infrastructure project procurement process, big data is inevitable. As Lux Rao believes, ‘one of the greatest opportunities available from the competent utilisation of big data is that of accurate forecasting – not just in terms of spend.’ A comprehensive analysis of the market from widely available data sets can help in accurately predicting the direction of markets and the demand for specific resources, thus allowing procurement to adapt strategies accordingly. </p>
<p>A plethora of internet databases offering financial data, statistics, market analysis and opinions on potential suppliers can be harnessed to provide a real-time big data set that, with proper analysis, could warn against those suppliers falling in financial difficulty. This would give them an opportunity to react immediately and minimise risks to supply before any impact is felt. </p>
<p>The biggest advantage of big data-led approach is its ability to predict, thus enabling preparedness. The platform digests millions of data points and creates micro patterns which it identifies and relates to specific outcomes, quicker and more efficiently than a human mind. This capability of generating and adapting complex patterns is crucial not only to predict but also to allocate resources within a city’s complex infrastructure to deal with unanticipated events or calamities. The ability to predict a tsunami or an earthquake, even if partially right, would save several lives and limit the extent of damage. This kind of payback from big data adoption cannot be quantified in financial terms alone. Traffic pattern analysis and ability to change traffic flows can have life-saving results, if ambulance routes can be prioritised and green corridors enabled them through. </p>
<p>To conclude, in the emerging era of big data, infrastructure and asset management, no organisation is an island. Projects are assets that can be managed with multiple independent, yet interconnected systems designed to deliver essential services. PPP will attract many companies to join experts from different fields of the project spectrum to provide essential information and optimal performance, closing the gaps between systems. The client is ultimately the citizen, who will get the best possible service as well as value for money. </p>
<p> <span style="font-weight: bold;">Collect, transform and visualise</span><br />
The entire exercise of data intelligence in Chandrapur was based on three principals -collect, transform and visualise. Through a mobile data collection application, volunteers collected and mapped data for each household, as well as infrastructure, healthcare facilities, schools and other data for each village. During the data collection process, around 3,000 to 4,000 survey responses – with a total of 0.6 million data points – came in from the field every day. This data was cleaned, verified and structured to build aggregate village profiles, development plans and priority scores. The transformed data was visualised in an interactive dashboard with geo-clustering, village-level comparisons, household-wise lists, village profiles and printable village development plans. </p>
<p>For this data-driven exercise, around 900 volunteers were trained and a total of 1.60 lakh people were surveyed. The outcome is a collection of 6.9 million data points, which enabled the project to kick-start last year. </p>
<p> <span style="font-weight: bold;">The result </span><br />
This project helped the district administration understand the socio-economic dynamics and development challenges of each village by creating a robust requirement sheet for the villages in Mul, Pombhurna and Jiwati taluks. Village development plans were sent to Gram Panchayat heads for all 290 villages to help in their planning. In addition, the Guardian Minister adopted 18 villages to convert them into model villages using the plans. Now, this led the block development officer of Mul to add 60 per cent of the plans’ suggestions to Mul’s 2016û17 development plan.</p>
<p>The district collector started using the dashboard to improve his field visits. He started crosschecking village development priorities with those identified on the dashboard. This helps him eliminate hearsay and less important complaints and focus on serious matters that need attention in each village.</p>
<p> <span style="font-weight: bold;">How GIS can help in the planning of roads </span><br />
The task of setting a realistic criterion to decide which roads to repair on priority has become more difficult with limited funds for road maintenance, an ever-increasing road network and the related voluminous data. To maintain its large transport infrastructure data, the Tamil Nadu Highways Department (TNHD) has established a web-based road maintenance management system (RMMS), consisting of database of road conditions collected through special data-collection vehicles. TNHD is primarily responsible for the construction of new roads and maintenance of its vast existing road network in Tamil Nadu.</p>
<p>RMMS consists of a web-enabled road information system and a planned maintenance system (PMS), which is a planning system for prioritisation of roads to suit the budget. Although RMMS generates a variety of reports related to road and bridge data, it lacks visualisation capabilities. To allocate resources for the road sector in a better manner, the need of TNHD was an application that would provide visualisation capabilities. Geographical Information System (GIS) was a befitting solution that helped visualise and enhance the analytical, problem-solving and decision-making capabilities of TNHD.</p>
<p> <span style="font-weight: bold;">How it works</span><br />
A GIS map with data on roads and bridges can help decision-makers in the planning, monitoring, and maintaining of roads and related assets effectively. TNHD deployed a solution, centred around Esri’s ArcGIS server technology. It was a web-based GIS road and bridge information system (based on the clientûserver architecture) under the e-Pathai (electronic project, administration, traffic, highway assets and information management system) programme, which integrated the GIS solution with RMMS and project and finance management system (P&amp;FMS).</p>
<p>e-Pathai GIS helps to view, understand, question, interpret and visualise data in many ways that reveal relationships, patterns and trends. The system is a mix of digital base maps for Tamil Nadu consisting of several layers (spatial data) compiled from different sources, such as Survey of India (SOI). It attributes data (non-spatial data) on roads and bridges from RMMS database besides other attribute data of interest such as demographic details from the Census of India, average annual rainfall data from the Meteorological Department of India.</p>
<p>Several spatial and non-spatial data from various sources have been integrated into e-Pathai GIS. The key sources for spatial and non-spatial data are SOI, Open Series Maps (OSM) and RMMS. The SOI layers in e-Pathai GIS are available only to authenticated users due to the sensitive nature of data. Spatial data for 20,000 km of state highways and major district roads based on GPS data stored currently in RMMS has been used to create a graphical representation of the roads in e-Pathai GIS. The spatial layer thus created has been linked with the related non-spatial or attribute data in RMMS after cleanup of the various graphical data inconsistencies.</p>
<p> <span style="font-weight: bold;">How LDB works</span><br />
Using radio-frequency identification (RFID) technology, LDB helps shippers and end users track the whereabouts of their cargo in near-real time through a dedicated single window from any device. It also collects data that can help identify bottlenecks and set up industry benchmarks for the logistics entities to streamline their operations. A collaborated effort backed by strong analytics and insights will help in reducing the logistics cost and transportation time and also in improving the ease of doing business in India.</p>
<p>’LDB can contribute immensely to revitalise the entire logistics sector, which is essential for the country to realise its ambition of becoming a global manufacturing powerhouse. India rose up 19 places to the 35th in the recent World Bank’s Logistics Performance Index 2016, but we have the potential to do a lot more by streamlining processes and improving efficiency across the entire supply chain. Scaling the LDB service is a major step towards that goal,’ said <span style="font-weight: bold;">Abhishek Chaudhary, VP – Corporate Affairs, DMICDC, and Director, DLDS.</span></p>
<p>LDB is designed to be a large-scale project and the DLDS aims to expand it to other container handling ports in the western and southern corridors in the near future for pan-India reach, which would maximise the effectiveness of the project.</p>
<p> <span style="font-weight: bold;">City data, a new currency</span><br />
A pilot project on DDG implemented by the three cities – Pune, Surat and Jamshedpur – had some surprise elements. In case of Pune, the WCCD data indicated that the city has one of the highest numbers of new patents per 1 lakh population per year at 10.77 as compared with other cities such as Buenos Aires (8.9) and London (3.14). </p>
<p>Registering for patents is an important element of innovation in a city. Also, data indicates that Pune has built its urban green spaces with 1,376 trees planted per 1 lakh population annually, which is nearly or more than twice that of cities such as London (527) and Los Angeles (772). In addition, the city has an exceptional solid waste recycling system with 56.16 per cent of the city’s solid waste being recycled, while London is at 30.56 per cent and Amsterdam at 27 per cent.</p>
<p><span style="font-weight: bold;">Kunal Kumar, Commissioner, Pune Municipal Corporation,</span> told IT that the city is and will always be the front runner when it comes to DDG. ‘We have always been a data-driven city, which has helped us to manage and plan better than our peers,’ he said. Another surprise element is the decrease in the number of crimes. The WCCD ISO 37120 data indicates that Pune has a relatively low number of crimes against property with 148.11 crimes being committed per 1 lakh population as compared to other cities such as Dubai with 367.54 crimes.</p>
<p>In case of Surat, WCCD ISO37120 indicates that 50 per cent of Surat’s total elected officials to city-level offices are women. In fact, this is even higher than London with 30.77 per cent and Johannesburg with 38.5 per cent.</p>
<p>Jamshedpur scores high in education. Data indicates that Jamshedpur achieves a high level of female student enrolment of 99.3 per cent, which is greater than Bogota (98.7 per cent), Amsterdam (98 per cent) and Buenos Aires (96.8 per cent).</p>
<p>Access to undisrupted electricity services is essential in ensuring reliability of a city’s electric utility services. According to WCCD data, Surat has an average of 0.03 electricity interruptions per consumer per year, which demonstrates the city’s capability to provide reliable electricity services to its citizens. In fact, this is much lower than London with 0.19 electricity interruptions and Dubai with 0.14.</p>
<p>Jamshedpur has a reputation for providing high-quality, reliable infrastructure services. The city provides close to complete provision of authorised electricity services at 99.2 per cent. It is also reliable, with an average of 1.8 service interruptions per year. The city’s water infra is also reliable, with only 0.04 average annual hours of water service interruptions per household.</p>