A literature search was conducted on scientific and medical databases from the earliest time permitted electronically using PubMed, CIRRIE, EBSCO Host and Scopus. Keywords used for searching titles (and title or abstract for PubMed) in alphabetical order were: wheelchair + ANSI/RESNA, assessment, comparison, environment, evaluation, ISO, performance, review, standards and testing. There was no limitation placed on the year of publication. Duplicates were removed and titles of the selected articles were screened by the author and assisting researcher and saved for further screening. Articles were then retrieved using the University of Pittsburgh library. Further review of articles based on abstracts was carried out by the author and the researcher. If any article was deemed relevant to the topics of interest by only one reviewer as per the abstract, then both reviewers read through the article to determine its relevance. Studies on motorised wheelchairs, scooters and manual suspension wheelchairs were not taken into account as the available wheelchairs used in LREs are mostly manual (Hof et al. 1993). The articles that were deemed relevant were read entirely and reviewed by the author and other researcher for inclusion in this literature review. References found from screened articles were searched using PubMed and Google Scholar or physically retrieved. Included articles were categorised into the three categories: (1) ISO standards development, (2) wheelchair testing with ISO standards and (3) field evaluation studies reporting wheelchair failures in the community. Studies conducted in REs were excluded from the third category.

Data collection and analysis were performed by the primary author. The articles related to ISO standards were evaluated for understanding whether LRE conditions were considered during the test method development process. Extracted elements from studies on ISO wheelchair testing included wheelchair sample size, ISO durability testing results and part failures. For articles related to wheelchair evaluation in LRE communities, information was retrieved on study design, wheelchair ISO qualification, maintenance status and field failures.

Advice on additional test development was sought from nine members of the ISWP Standards Working Group (ISWP-SWG). This expert group is composed of wheelchair manufacturers, designers and providers from charitable organisations and field experts with work experiences in LREs. All experts were familiar with ISO 7176 test methods. Information on failures in LREs and test development was collected through biweekly group discussions through Web conferencing via Adobe Connect (Adobe Systems Incorporated 2016). ISWP-SWG members provided pictures of broken and inoperable parts that they had collected through their work to demonstrate the types of failures common in LREs. Group discussions were centred around these failures that are not predicted by ISO 7176 tests. The failure photos were instrumental in gaining consensus about the common failures and making suggestions for the additional tests needed. Votes were taken within the group to nominate parts for testing consideration.

A systematic process was used to generate a prioritised list of the new tests recommended from this work. First, a product testing matrix was generated that includes a column listing the failures common in LREs that were identified through the literature review and by the members of the ISWP-SWG. Test conditions responsible for failures were noted. Second, experts determined whether the test conditions are already included in ISO 7176. Third, if a need for additional testing was identified, an effort was made to leverage existing test methods from relevant ISO standards, American Society for Testing and Materials (ASTM) standards and United States Military Standards (MIL-SPEC). If it was determined that a suitable test method did not already exist, members from the ISWP-SWG made suggestions for new test methods. Voting was carried out in the group to select test methods to be developed by ISWP.

International Organization for Standardization work commenced in the early 1980s with participation from UK, Sweden, Germany, France, Denmark, United States, Canada, Austria and Japan (Cooper et al. 1996a; McLaurin 1986; Staros 1981), but no LREs were involved. In the 1990s, the ISO committee developed and published wheelchair standards (Cooper et al. 1996a; Hobson 1999) and expanded participation (International Organization for Standardization 2014). Currently, there are 24 countries participating in the ISO standards committee (plus 11 observing countries) including Brazil, China and India which are considered less-resourced countries. Working groups under this subcommittee typically meet twice a year for developing new standards and revising existing standards (Cooper et al. 1996a). There are now 34 standards published by the committee with expanded categories that include power wheelchairs, scooters and stair-climbing devices. Standards specify disclosure requirements for testing and methods of measurement for: static stability (§1), dynamic stability (§2), brake effectiveness (§3), energy consumption (§4), wheelchair and seat dimensions (§5), maximum speed, acceleration and deceleration (§6), determination of seating and wheel dimensions (§7), static, impact and fatigue strength testing (§8), climatic testing (§9), obstacle climbing ability (§10), test dummy specifications (§11), power and control system (§14), flammability requirements (§16), electromagnetic compatibility (§21), setup procedures (§22) and vocabulary (§26) (International Organization for Standardization 2014). In all, wheelchair standards tests consist of durability, safety and performance tests along with measurement and reporting of wheelchair dimensions and characteristics. Some test procedures allow for comparison between wheelchair safety and performance, while certain tests need the wheelchair to pass minimum requirements (Cooper et al. 1996a; Hobson 1999; International Organization for Standardization 2014).

The ISO 7176-8 suite of durability tests includes tests for strength, impact and fatigue which primarily assess a wheelchair’s quality. Strength tests require static loading of armrests, footrests, handgrips, push handles and tipping levers. Impact tests are conducted with a test pendulum on backrests, hand rims, footrests and castors. Fatigue tests consist of a multidrum test (MDT) of 200 000 cycles and a curb-drop test (CDT) of 6666 cycles (see Figure 3). Failures of the MDT and CDT are classified into three classes: Class I and Class II failures are because of maintenance issues and can be fixed by a user or dealer, while Class III failures are caused by structural damage and require a major repair or part replacement (Cooper 1998). A Class III failure indicates failure of the test.

The literature review on wheelchair testing with ISO standards focused on 12 articles (Cooper et al. 1996b, 1997, 1999; Cooper, Stewart & VanSickle 1994; Fitzgerald et al. 2001; Gebrosky et al. 2013; Liu et al. 2008, 2010; Rentschler et al. 2001; Toro et al. 2013; Wang et al. 2010; Zipfel et al. 2007) that deal with laboratory testing of different wheelchair designs. Included were hospital style (HWC), lightweight (LWC) and ultra-lightweight (UWC) wheelchairs (see Figure 4). Wheelchair models were in new condition and were already available on the market. Information regarding their prior ISO testing was not available. Some testing studies referred to ANSI/RESNA standards. Table 1 presents study results from ISO section 8 tests and lists the observed failures. Among different designs, UWCs were found to be more durable and cost-effective compared to LWCs and HWCs except in the most recent study by Liu et al. (2010). UWCs experienced higher Class I failures that could be repaired by users, whereas HWCs had greater Class III failures (Fitzgerald et al. 2001).

Failures found in field studies with different wheelchair models are listed in Table 2. Five of the reviewed studies (Armstrong et al. 2007; Reese & Rispin 2015; Rispin et al. 2012, 2013; Toro, Eke & Pearlman 2016) evaluated usability or durability aspects of wheelchairs designed for LREs. These models (see Figure 5) have passed ISO durability tests (Hof et al. 1993; USAID 2014) and were developed by non-profit organisations (Pearlman et al. 2008; Rispin et al. 2013). They are adjustable and more appropriate for rigorous use in rugged LRE conditions (Hotchkiss 1985; Sheldon & Jacobs 2006; Toro et al. 2016; Visagie et al. 2015a, 2015b, 2016). Despite wheelchairs passing ISO testing, breakdowns and failures occurred frequently and within months of the wheelchair being delivered, which reinforces the recommendation from the WHO guidelines that additional tests should be developed.

The ISWP-SWG members are co-authors of this article and their expertise is listed in Table 3.

International Society of Wheelchair Professionals Standards Working Group members reported minimal LRE participation in ISO 7176 standards development. They noted several product quality issues and failures seen typically in LREs as seen in Figure 6. Service delivery method and the status of maintenance, repairs and user skills training for wheelchairs in Figure 6 are unknown. These failures and breakdowns are irrespective of location of manufacture (locally produced or imported) and the context for use (REs or LREs). ISWP-SWG members identified certain unique quality-affecting elements such as corrosion, ageing and high impact forces (e.g. if a wheelchair is dropped from a bus) as causes for these failures. These elements are not present in ISO durability tests. Rapid breakdowns of components such as upholstery, anti-tippers, belt harness, calf straps, toe straps and fasteners were noted as durability issues that are not tested under ISO 7176.

To identify new tests, a product testing matrix as shown in Table 4 was put together that lists failure modes of different parts and the applicability of ISO test methods for predicting each failure mode. Testing priority was assigned by consensus from experts based on parts that fail most often and make the wheelchair non-functional. The lack of standard test methods (ISO, ASTM and MIL-SPEC) for predicting most failure modes on wheelchair parts led ISWP-SWG to identify new test methods.

Castors and rear wheels were selected as crucial components that break down quickly in LREs and were prioritised for testing and test method development. Corrosion was identified as a variable that affects most wheelchair parts and was likewise prioritised for testing. Testing a complete wheelchair through simulated environmental conditions was considered as well.

As mentioned previously, there is little representation from LRE nations on the ISO testing committee, and consequently test methods do not completely reflect conditions in LREs. While WHO guidelines suggest using ISO 7176 as the basis to develop new standards for LREs (Borg & Khasnabis 2008), no new standards for this purpose have yet been proposed. Also, ISO standards testing has been carried out in resourced countries for more than 20 years for regulatory and research purposes, but no reports were found showing implementation of ISO 7176 in LREs thus far.

The ISO testing studies included in this review were conducted in an independent testing laboratory mostly on wheelchairs provided in REs. Results from Table 1 show that manual wheelchairs overall lack standard product quality, especially HWCs that resemble the majority of designs distributed in LREs (Constantine et al. 2006; Kim & Mulholland 1999; Pearlman et al. 2008; Zipfel et al. 2007). Around 70% – 90% of HWCs failed to pass minimum durability requirements (Cooper et al. 1996; Fitzgerald et al. 2001; Toro et al. 2013; Zipfel et al. 2007). Similar wheelchair designs produced in LREs (Toro et al. 2013; Zipfel et al. 2007) failed prematurely. Higher incidences of Class III failures with HWC designs indicate higher rates of breakdown and repairs during use, which is evident from anecdotal reports (Oderud 2014; Visagie et al. 2015a, 2015b) and reviewed field studies (Mukherjee & Samanta 2005; Saha et al. 1990; Toro et al. 2012, 2013). On the other hand, UWC designs were found to be durable and experienced fewer frame failures with ISO tests. This test outcome was predictable because UWCs are sophisticated wheelchair designs with superior quality materials that are designed for performance in developed environments and ISO durability tests subject wheelchairs to conditions that simulate such environments (Borg & Khasnabis 2008; Sheldon & Jacobs 2006). Field evidence with active users in REs has been reported with UWCs which shows positive satisfaction and fewer repairs in last six months of use (Fitzgerald et al. 2005). However, these designs are not suitable for LREs owing to high costs associated with their materials and manufacturing. Overall, it can be concluded that ISO durability tests are suitable to test wheelchair designs like HWCs that break prematurely and UWCs that are developed for performance in REs.

Field evaluation studies have been carried out with ISO-qualified wheelchairs appropriate for LREs. Four such field studies reported failures, repairs, replacements and missing parts over two weeks to eight months of field use (Armstrong et al. 2007; Rispin et al. 2012, 2013; Toro et al. 2016). Wheelchairs in two of these short-term studies were provided based on WHO guidelines (Borg & Khasnabis 2008; World Health Organization 2012, 2013), maintained frequently and favoured by the users (Armstrong et al. 2007; Toro et al. 2016). One study (Reese & Rispin 2015) assessed ISO-qualified appropriate wheelchairs after 1–2 years of use which were provided without user training and serviced occasionally. Several part failures were found that would require a technician’s attention (see Table 2). Findings from these studies demonstrate that failures occur on ISO-qualified models with everyday use especially with parts such as brakes, tyres, seat covers, castors, footrests and armrests. Field failures can be associated with product properties such as substandard material quality, poor parts selection, inappropriate design and manufacturing inconsistencies. These properties can vary with the locally produced versions of certain ISO-qualified wheelchairs like the Whirlwind Roughrider which makes them prone to early failure. Moreover, LRE environments are harsh and can degrade products rapidly. ISO test qualification is representative of 3–5 years of outdoor use (Cooper 1998; Hobson 1999) but apparently falls short of qualifying products for LRE use based on reviewed study results. Accurate prediction of life duration of certain wheelchair parts may not be guaranteed.

Failures seen with ISO testing in the laboratory were similar among wheelchair designs. Fractures with cross braces, side frames (at weld joints), backrests, castor spindles and footrests were found to be common in these studies. Failures were influenced by frame design, wheelchair material, screw holes, welding techniques and castor and tyre characteristics. However, failure modes observed with ISO testing are rare in the field based on field failure evidence gathered through literature review and failure evidence provided by ISWP-SWG members (see Figure 4). Dominant field failures found in LREs are flat and cracked tyres, wobbly rear wheels, bent frames, non-functional brakes, worn out bearings, damaged armrests, torn seat covers, loose upholstery, collapsed cushions and rusting and loosening of several parts. Any representation of these failures is not evident in ISO testing results which mostly produces fracture failures caused by impacts on MDT and CDT. This is because test conditions employed on ISO durability tests do not simulate environmental and demanding use conditions from LREs. To accurately predict failure modes and life duration of products for LREs, it is necessary to develop additional testing methods for LREs with relevant test conditions. ISWP-SWG experts echoed similar advice.

The product testing matrix developed through consensus of experts highlights the requisite test factors (conditions) for testing products for LREs. The matrix assisted in development of additional test methods for LREs. Based on availability of resources and capacity for development with ISWP partners in the SWG, four test methods were given high priority – castor durability testing, rolling resistance testing, corrosion testing and whole chair testing.

Castor failure was noted as a top concern in the field as per ISWP-SWG experts. Castors experience a variety of failure modes with tyres, bearings and stem hubs and ISO tests primarily subject castors to vertical loads and stresses. Hence, experts suggested that castor durability testing should be conducted separately. Incorporating amplified and angular loading patterns along with corrosive conditions and various types of simulated surfaces including sand, mud, gravel and stones is recommended for the new castor test method. Such testing is estimated to screen castor designs for greater durability, requiring less maintenance and incurring fewer repairs in LREs.

Corrosion of wheelchairs was observed as a critical concern because several wheelchair components are unable to operate after being rusted. Although ISO testing includes climatic testing of wheelchairs in hot and cold environments for power wheelchairs only, it does not simulate moisture and acidic exposure. It is known that corrosion adds to the effect of fatigue during field use for certain wheelchair parts like bearings. This calls for conducting fatigue and corrosion testing simultaneously. Experts recommended corrosion evaluation of the complete wheelchair similar to already established standards like ASTM B117.

Resistance to wheelchair rolling was also identified as a major performance issue in LREs. While resistance characteristics for rubber on different surfaces are known to an extent, propelling wheelchairs over a variety of surfaces requires a significant user effort (Armstrong et al. 2007; Oderud 2014; Rispin & Wee 2014). Wheels experience a range of rolling resistances based on variation of elastic rebound between the tyre and different surfaces, tyre tread design, type of tyre (pneumatic vs solid), camber angle, toe-in and toe-out alignment, type of spokes and characteristics of the axle hub bearings. Castors are also known to have greater rolling resistances based on tyre diameter, characteristics, surface, the type of materials used and bearing efficiency. Thus, testing to evaluate the rolling of wheels and castors, which is not a part of ISO 7176, is being considered in the new test methods. Comparing the rolling resistance of different types of available wheels and castors is critical so that performance products can be selected.

The ISWP-SWG recognises that the entire wheelchair suffers from different types of loads and effects of environmental factors causing wear (ultraviolet light, high temperature, dirt and dust) and corrosion (humidity and water exposure). None of the ISO tests subject wheelchairs to a combined effect of these test factors. Hence, testing the complete wheelchair simultaneously against relevant test factors to replicate failures as seen in the field is suggested.

Field studies that provided wheelchairs as per WHO guidelines (Armstrong et al. 2007; Toro et al. 2016) indicated that appropriate services, user training and regular maintenance are necessary to reduce the rate of field failures. However, LREs struggle with capacity for appropriate services. Provision of user training, funding and access to repair services is limited (Armstrong et al. 2007; Borg & Khasnabis 2008; Hof et al. 1993; Oderud 2014; Sheldon & Jacobs 2006; Visagie et al. 2013, 2015b) which was evident in field studies as well (Mukherjee & Samanta 2005; Reese & Rispin 2015; Rispin et al. 2012, 2013; Saha et al. 1990). In the wake of such concerns, international efforts focused on increasing capacity and improving service provision in LREs are ongoing (International Society of Wheelchair Professionals 2015). While such efforts are in progress, it is equally necessary to develop products with greater reliability and higher durability to reduce failure occurrences and prevent breakdowns. This perspective has been shared by the WHO guidelines as well which stress the parallel need for appropriate services and high-quality products (Borg & Khasnabis 2008; United Nations 2006). Development of durable, high-quality products, in turn, calls for development of rigorous test methods which were identified in this study.