Deficiencies in China’s innovation systems for coal-bed methane development: comparison with the USA

4.1 The US national system of innovation

The US NSI dates back to the early years of the twentieth century when private industrial enterprises, notably chemicals companies, started to realise the value of scientific research to innovation and established their own laboratories. In the years leading up to World War Two, universities built up their research capabilities and developed links with industry. The structures developed for military R&D during World War Two formed the basis for the subsequent structures and systems (Mowery and Rosenberg, 1993).

The period since 1953 has seen a sustained increase in total funding for R&D in the USA, with an acceleration after the mid-1970s. Federal expenditure on R&D climbed steeply during the late 1970s and 1980s, the period under consideration in this paper. Since the 1980s, industrial funding for R&D has risen more rapidly than that from the federal government. Further, the share of R&D funding deployed by industry has consistently outweighed that of the federal government and universities. Certain states, such as Michigan, Massachusetts and California, also have a strong track record of funding research (Simons and Walls, 2010).

A vital attribute of the US NSI is the dominant role of private actors and their diversity, as is the case in the oil and gas industry (see Section 4.2). Although federal laboratories play an essential role, private enterprise carries out and funds most R&D. These firms have rapid reaction times and can absorb new technologies rapidly (Popper and Wagner, 2002). Further, venture capital has played an increasingly important role since the 1990s (Simons and Walls, 2010). Despite their dominance of R&D, industrial firms began to increase their collaboration with universities and federal laboratories in the 1980s due to competitive pressures, as well as forming other types of research alliances (Mowery, 1998). These practices were then carried over into innovation for CBM development, as will be explained in Section 4.3.

The high level of industrial innovation achieved by the USA in recent decades has been attributed to several underlying conditions in addition to the levels of funding and the role of private enterprise. These include the protection of intellectual property rights, tax incentives, a diverse range of enterprises, the low barriers to entry and exit for both business and technology, a mix of basic and applied research, a strong tertiary education system and active capital markets (Popper and Wagner, 2002; Simons and Walls, 2010). In addition, the federal government has played an important role, where needed, as a convener and builder of networks (Popper and Wagner, 2002), as will be illustrated in the case of gas research in Section 4.3.

4.2 Key features of the US oil and gas industry

The US oil and gas industry possesses two features that distinguish it from those of other countries. Both relate to private ownership. First, and uniquely today, the rights to subsurface resources in the USA lie with the individual landowner, except in the case of State and Federal lands. As a result, the landowner has the incentive to encourage the exploitation of any resources below their land to benefit from the required royalty payments. This system also helps to prevent any one company being able to monopolize production rights to a resource.

The second feature is the large number and diversity of companies involved in the US oil and gas industry. From its origins in 1859, the US oil and gas industry has been populated by private actors, large and small, with no state-owned enterprises. After the break-up of the Standard Oil Company of New Jersey in 1911, no single company has dominated the domestic industry. By the 1950s, the number of companies involved along the US oil and gas supply chain was estimated to be about 30,000 (British Petroleum, 1958). Between 1955 and 1970, the largest 20 US integrated oil and gas companies increased their share of domestic oil production from 46 to 68%.

In the 1970s, the extent of joint ventures in upstream projects had raised concerns over competition and price-fixing (Flaim, 1979). However, the dominance of the top eight companies over natural gas reserves and production in 1976 (39 and 36%, respectively) was less than for crude oil reserves and production (57 and 40%, respectively; Jones et al., 1978). Today, the American Petroleum Institute claims more than 600 members along the entire length of the supply chain [1]. According to the Independent Petroleum Association of America, there are about 9,000 independent oil and gas producers in the USA. In their definition, an independent company has no more than US$5m per year of sales or a refining output of no more than 75,000 barrels per day [2].

A study by IHS Markit (Fryklund et al., 2019) identified 128 oil and gas independent companies, distinct from eight global companies, as well as 2,079 small private companies. This report also showed that the independent companies, large and small, had played a pivotal role in developing the country’s unconventional oil and gas reserves in the second decade of the twenty-first century. We have not been able to find information on how many companies were involved in developing the US’s CBM resource in the 1980s and 1990s as production grew. However, data at the time of peak CBM production in 2008 reveals that the most productive basins each hosted tens of companies, for example: 68 in the Powder River basin, 55 in the San Juan Basin, 41 in the Arkoma Basin, 36 in the Cherokee/Forest City Basin and 32 in the Anadarko Basin (US Environmental Protection Agency, 2010). Although, many companies will have operated in multiple basins, these numbers reflect low barriers to entry and a relatively high degree of competition.

4.3 The US sectoral system for innovation as it relates to CBM

Such a high degree of competition and diversity in the oil and gas industry might have undermined collaborative research. However, the US Government took active steps to build on the practice of cooperation that was growing in the NSI by creating an organisation with the primary role of ensuring that relevant actors in the SSI cooperated in their research efforts. This was part of a wider effort to support and coordinate R&D for gas production, including CBM, in the late 1970s and 1980s.

The oil crises of the 1970s triggered a rapid response by the US Government to boost the domestic production of different forms of energy, including natural gas. The main measures taken were:

• The 1978 Natural Gas Policy Act;

• Section 29 of the 1980 tax credit on energy;

• Funding from the US Department of Energy (DoE);

• Field and laboratory research by the US Geological Survey (USGS); and

• The establishment of the Gas Research Institute, later reorganised to become the Gas Technology Institute.

The Natural Gas Policy Act aimed to create a national market for natural gas. It gave the newly created Federal Energy Regulatory Commission (FERC) authority over intra-state and inter-state gas markets (US Energy Information Administration, undated b). The 1980 tax credit gave a significant additional incentive for unconventional natural gas producers, much more than any tax deduction could have given (Soot, 1991). Between 1978 and 1982, the DoE provided US$29 million to the Methane Recovery from Coalbed Program (MCRP), a project to define the size and recoverability of domestic coalbed methane (Anonymous, 2021). It has been estimated that the US$29 million of DoE funding yielded economic benefits amounting to US$200 million (1991 dollars) (National Academic of Sciences of the US, 2001). The USGS followed up the work of the MCRP assessing the resource. This work has continued to the present day, and the results are publicly available (US Geological Survey, 2021).

At the heart of this wide-ranging initiative lay the Gas Research Institute (GRI), an organisation created explicitly to build networks between companies, research institutes and universities. This organisation was established in 1976 as an independent agency that justified its research budget to FERC, its financer. In 2000, it merged with the Institute of Gas Technology, a technical training institution, to form the Gas Technology Institute (GTI) that continues to operate today.

The mission of the GRI was to select the most cost-effective technologies to support the production, transportation and use of natural gas to the benefit of all stakeholders along the supply chain. Its primary function was to decide which projects to fund, who to fund and how much to invest. All of its research was contracted out and it did not possess any of its own laboratories. In this way, the GRI was a project management agency. Its total staff numbers reached 271 by 1987, of whom 179 were directly involved in managing research projects (Gas Research Institute, 1987).

The funding for the institute came from surcharges on the shipment of gas via interstate pipelines. R&D expenditure rose from US$20 million in 1979 to US$118 million in 1986 (Gas Research Institute, 1987). By the 1990s, the annual budget was around US$200 million. The GRI would submit to FERC a research and development strategy every five years and would request a certain percentage of funds from the inter-state pipeline tariff revenue that the FERC received (Burnett et al., 1993). Whilst some projects were funded solely by the institute, those that were moving beyond the proof-of-concept stage were required to involve the participation of and funding from one or more industry partners. Between 1980 and 1986, the annual income from industry for such projects rose from US$15 million to US$57 million. The number of funders reached 272 by 1986, of which 56 were gas-producing companies and nine were oilfield service companies (Gas Research Institute, 1987). This approach ensured the commitment of these firms to the collaborative effort.

In its strategic approach, the GRI was careful to seek an appropriate balance between long-term R&D, including basic research, and near-term R&D. Although, its top priority was R&D related to the end use of gas, it devoted about 38% of its funds to R&D at the production end of the supply chain during the period 1978–1986. It applied a rigorous, quantitative project appraisal methodology to choose R&D projects that aligned with its goals. A key requirement was to terminate projects that were unlikely to yield significant economic benefits. This minimised wasteful funding.

In the 1980s, GRI had four board-level advisory bodies to guide programme design and budgeting and also received technical support for specific projects from 15 project advisory groups (Burnett et al., 1993). By 2001, the Natural Gas Supply Project Advisor Group of the renamed Gas Technology Institute consisted of 32 exploration and production companies. Contract research relating to coalbed methane had involved about 70 organisations, including exploration and production companies, oilfield service companies, universities, consultancies, research organisations, industry associations and state geological surveys (Perry, 2001). This was an extensive collaborative effort.

Technological advances of relevance to CBM were quick to come. By 1986, these included fracture placement, fracture azimuth measurement, integrated log analysis systems, portable real-time monitoring systems, and two-dimensional seismic borehole tomography (Gas Research Institute, 1987). Between 1978 and 1993, Gas Research Institute completed and commercialised 30 gas technologies with calculated benefits of US$2 billion in net present value. Over the same period, it achieved a project success rate of 30% and a benefit to cost ratio of seven to one. Moreover, the institute’s research lead to the discovery of between 700 bcm and 2,500 bcm of additional gas resources by 1989 (Burnett et al., 1993). This resulted in annual CBM production rising from 1.0 bcm in 1988 to 35.1 bcm in 1999, a period of just 12 years (Figure 1).

Most of the research yielded outcomes that were not amenable to patent protection or licensing. In those cases where patentable technology was developed, the shares of the intellectual property rights were allocated in proportion to the amount of funding and background technology provided by each participant (Perry, 2001).

This short account of US strategies to promote the development of unconventional natural gas, including CBM, reveals three key success factors. First, the federal government acted with a major coordinated push from the very beginning, led by the legislature and four key agencies: the Department of Treasury, the FERC, the Department of Energy and the USGS. Second, the tax credit provided a very significant economic stimulus for unconventional gas production. Third, and most important in the context of this analysis, was the creation of the Gas Research Institute to manage and coordinate R&D projects to ensure economic benefit. In this way, many diverse actors across the highly competitive US oil and gas industry were brought together to address the challenge of unconventional gas development. As will described in the next section, the prevailing conditions and strategies deployed in China were quite different.

5.1 China’s national system of innovation

China’s NSI has evolved significantly since the 1950s when the Soviet Union played a significant role in the overall design of the economy, including the systems and structures for R&D. As with the wider economy, R&D was centrally planned with a focus on the military and on heavy industry, but implementation was fragmented. The gradual marketisation of the economy in the 1980s was accompanied by a sustained growth in expenditure on R&D by both government and enterprises, and by a degree of decentralisation of decision-making on R&D (Liu and White, 2001; Gu and Lundvall, 2006).

During the 1980s and 1990s the government launched several strategic research programmes aimed variously at selected industries, universities, state laboratories and science parks. The growing R&D activity resulted in a rise in the number of patents filed by manufacturers, though primarily for design rather than utility or invention (Liu and White, 2001) and in the number of publications in international academic journals by university staff (Zhang et al., 2013). Among enterprises, state-owned enterprises (SOEs) were generally less innovative. They invested proportionately less in R&D than non-SOEs, and small and medium-sized enterprises were more innovative than larger companies (Huang, 2010).

At this stage, the level of diffusion of ideas and technologies in China’s NSI remained low for a number of institutional reasons. The SOEs were slow to develop links with universities and government research institutes, and local clusters and networks were weak (Liu and White, 2001; Gu and Lundvall, 2006). Organisations were reluctant to cooperate in innovation as the legal framework for IPR was still weak (Liu and White, 2001). The gap between the research of government research institutes and industry requirements, inherited from the earlier Soviet approach, persisted (Liu, 2009). Moreover, the downstream integration of universities and government research institutes further constrained the flow of technologies to manufacturers through sale or licensing (Liu and White, 2001).

In response, the government progressively established specialised units in government research institutes to make links with industry in selected sectors, including energy, and set up high technology incubation centres and local technology markets. Together these measures succeeded in raising the level of diffusion (Liu and White, 2001; Hu and Matthews, 2008). However, the influence of the central government over science and technology policy remained strong, though was steadily declining (Chen and Naughton, 2016).

Since the early 2000s, China’s Government has placed progressively greater emphasis on domestic R&D and indigenous innovation. As a result, sales of domestic technology has outstripped that of foreign technology (Gu and Lundvall, 2006; Chen and Naughton, 2016). This success has involved a central role for the government in supporting R&D as, variously, a strategic investor, a business partner, a customer and a supervisor (Bazavan, 2019).

Since 2014, the government has reinforced its support for R&D by making innovation a key component of the national development strategy. Key documents included the 13th Five-Year Plan for Science and Technology 2016–2020 and Made in China 2025. To support these aspirations, the government has increased its role in managing science and technology (Cao et al., 2018). It has provided clear focus areas for R&D and taken steps to improve coordination across government and remove institutional and structural barriers to innovation (Cyranoski, 2014). The government has also rationalised the numerous funding programmes into five categories and outsourced the management of these programmes to professional agencies. The Ministry of Science and Technology remains central to R&D policy-making, leading the inter-ministerial deliberations that set the strategic R&D agenda. The National Natural Science Foundation was removed from the purview of the State Council, and now lies under the Ministry of Science and Technology. More rigorous procedures have been introduced to evaluate projects through domestic and international peer review and in terms of their impact.

Despite the efforts put into these reforms, many assessments of their impacts have been unfavourable. The approach is seen as excessively top-down and supply-side, rather than demand-side (Wang and Li, 2019). The efficiency of innovation remains low; in other words, the ratio of resource deployed versus innovative results achieved is low, though is improving (Qiu, 2020). The role of scientists in setting the government’s research agenda and managing research programmes appears to have declined (Zhi and Pearson, 2017). Government strategy appears to emphasise low risk, applied research with short-term commercial deliverables (Zhi et al., 2013). R&D collaboration has remained weak, as have technology adoption and diffusion. The education system is seen as still being inappropriate to spur creativity and multi-disciplinary R&D, despite the rhetoric in support of innovation and the IPR framework requires further enhancement (Wang and Li, 2019).

The industrial focus of these R&D efforts resulted in a large share of funding flowing to enterprises, with a greater proportion allocated to SOEs as they were still perceived as vital actors in the economy. Studies to evaluate the innovation performance of SOEs over the period 2000–2015 revealed that SOEs continued to less innovative than non-SOEs in terms of patents and innovative products for a given input of R&D finance (Zhou et al., 2017; Kroll and Kou, 2019). Nevertheless, specific categories of SOEs showed a higher level of innovation than others, notably: those in which the state held a minority rather than a majority share; those operated in more competitive markets; those that were founded as start-ups; and those in high-tech industries and those directly facing the market (Cao et al., 2016; Zhou et al., 2017).

Some of these weaknesses in the NSI can be seen in the SSI for CBM, notably preferential funding for SOEs, inadequate financing of basic research and weak coordination (Section 5.3).

5.2 Key features of China’s oil and gas industry

Unlike the diverse and competitive situation in the USA, China’s oil and gas industry is dominated by three wholly state-owned national oil companies (NOCs) and their listed subsidiaries: CNPC and PetroChina Ltd, Sinopec Group and Sinopec Ltd and CNOOC and CNOOC Ltd. The listed subsidiaries have long held the resource rights to most of the country’s onshore oil and gas resources. However, it was the Ministry of Coal Industries that first took an interest in CBM. To address resource rights and to promote the exploitation of CBM, the government created the China United Coalbed Methane Corporation (CUCBM) in 1996, giving it a monopoly over all CBM resources. The two shareholders were CNPC (later PetroChina) and the China National Coal Corporation (“China Coal”). Little progress was made in developing the CBM resource. PetroChina had little interest in CBM as it was directing its funds towards other sources of gas, and China Coal lacked expertise in exploration drilling as well as the funds, and was focusing its efforts on coal mining. Meanwhile the Ministry of Land and Resources was carrying out some exploratory drilling in coal mining areas, and the Jincheng Anthracite Mining Group ran some pilot CBM development projects (Regan and Zhu, 2014)

In 2007, the government removed CUCBM’s monopoly on foreign cooperation. A year later PetroChina withdrew from CUCBM and became a CBM player in its own right. Sinopec and a small number of other local state-owned Chinese enterprises also began to explore for CBM. In 2010, CNOOC Ltd bought a 50% stake in CUCBM and upped its stake to 70% in 2013 (Regan and Zhu, 2014).

The first foreign companies to explore for CBM in China during the 1990s were USA majors such as Chevron, ConocoPhillips, Arco (later BP) and Texaco. A combination of poor flow rates, low demand for gas and low gas prices resulted in these large companies moving out and passing the opportunity to smaller foreign players in the early 2000s. The most active of these have been Green Dragon, Sino Gas and Energy and Far East Energy. One major oil company, Shell, took out a PSC for CBM in 2007. Other foreign companies included Dart Energy (previously Arrow), Asia American Gas and Pacific Asia Petroleum (Beveridge and Chan, 2010). Until recently, foreign companies were obliged to operate under production sharing contracts (PSCs) in partnership with Chinese state-owned resource right holders, notably CUCBM and PetroChina. Whilst a few foreign companies succeeded in producing small quantities of CBM, the structure of the standard Chinese PSC was quite unsuited to the nature of CBM development (Regan and Zhu, 2014). As a result, CUCBM and PetroChina continued to dominate the CBM industry and held most of the CBM production and exploration licenses. Two other SOEs, CNOOC and Sinopec played a much smaller role. In 2016, these four SOEs held 64 out of the 104 CBM exploration licenses. The other 40 licenses were held by 24 local enterprises across 14 provinces, many of which were not highly prospective [3].

As mentioned above (Section 2), the government has recently taken two steps to increase the number and diversity of actors involved in the development of CBM (Jin et al., 2019). In 2017, it delegated authority for granting CBM resource rights to six provincial governments on a trial basis: Shanxi, Fujian, Jiangxi, Hubei, Guizhou and Xinjiang. This resulted in an increase in the number of CBM exploration licenses in operation from 104 in 2016 to 121 in 2020. More importantly, the number of non-SOE license holders increased from 24 to 36. Twenty of these were new actors, 9 enterprises having withdrawn from CBM exploration. The total number exploration licences held by non-SOEs rose from 40 to 68. Over the same period the share of these licenses held by the SOEs declined from 62% in 2016 to 44% in 2019. Of the 6 regions which had been granted licensing rights, only Shanxi and Guizhou saw an increase in the number of active exploration licenses, from 3 to 21 in Shanxi and from 7 to 19 in Guizhou. These are both coal-rich provinces, and Shanxi is the source of most of China’s CBM production today. The other four provinces showed no increase in license numbers [4].

While the increased number of companies active in exploration is to be welcomed, it is too early to judge whether this will help accelerate technological innovation and the production of CBM. Many of the new actors will likely face steep learning curves, as CBM development is quite unlike that for other forms of unconventional gas such as shale gas. Also, the resource quality in many of these new license areas may not be high and the area of each block is seen as being too small (Wood Mackenzie, 2019; Xu, 2021).

In a second move, in 2019, the government removed the requirement for foreign enterprises to operate under a PSC or in joint venture with a Chinese company. While a few small independent companies continue to operate, the majors such as Chevron, Shell and BP have withdrawn from CBM, and no new foreign investors had taken up the opportunity offered by the relaxation of rules on foreign investment as of May 2021 [5].

5.3 China’s sectoral system for innovation as it relates to coal-bed methane

As mentioned in Section 2, the government is estimated to have directed billions of dollars towards R&D in support of coalbed methane development since the beginning of the century. However, insufficient amounts have been directed at basic research. Rather the majority of the funds were spent on technological development and pilot projects (Wu, 2012; Tao et al., 2019a). Moreover, our own incomplete collation from available primary online Chinese sources indicates that a substantial share of government R&D funds for CBM has been granted to a relatively small number of enterprises, research institutes and university laboratories. The major recipients were the NOCs and their research institutes, and to CUCBM, as these enterprises were expected to play a leading role in the CBM industry. Thus earlier observations of the central role played by SOEs in industrial research in China’s energy sector (Karplus, 2007; Anadon, 2012) appear to be valid today, at least for CBM. The China University of Geosciences and Chongqing University received a disproportionate share of the funding to universities [6].

A further deficiency has been the apparent lack of effective mechanisms for coordinating CBM R&D. In 2006, the NDRC approved the establishment of a National Engineering Research Center for the Development and Utilization of Coalbed Methane. The founding shareholders were CUCBM, PetroChina, Sinopec, Huaibei Mining, Beijing Orion Energy and the China University of Geosciences in Beijing [7]. However, this organisation does not have the capacity or authority to play a national coordinating role [8].

In 2009, 14 enterprises, universities and research institutes created the CBM Industry Technological Innovation Strategic Alliance [9]. Membership grew to 41 by 2019 [10]. While the formal aims of the Alliance included promoting collaboration, disseminating findings and promoting the adoption of technologies [11], interviews with the secretariat and with academic researchers in Beijing indicated that the organisation functioned very informally, unlike the US Gas Research Institute. It did not fund research, neither did it actively build collaborative partnerships [12]. These observations support Anadon’s (2012) contention that links between basic and applied research and between SOEs and private enterprises remain weak in China’s energy sector. Further, there is no evidence that the government has created new organisations to promote and coordinate innovation in the oil and gas industry, let alone for CBM.

In summary, the relative lack of success of technological innovation in China’s CBM industry can be attributed, in part at least, to the preferential directing of R&D funds to SOEs and a limited number of universities for applied research, at the expense of basic research. This has been exacerbated by the historically small number of actors involved in CBM development and the high barriers to entry. In addition, the systems for research collaboration and the diffusion of results continue to be weak in both the NSI and the SSI. These features are in sharp contrast to the context of innovation in support of CBM development in the USA and go some way to explaining the different outcomes. During the eleven years after annual output reached 1.0 bcm, China’s production reached 6.0 bcm while that of the USA reached 35.1 bcm (Figure 1).