The International Association of Classification Societies (IACS), that gathers the twelve largest ship classification societies worldwide covering more than 90% of the world’s cargo carrying tonnage, has recently published an updated version of Recommendation 42, “Guidelines for Use of Remote Inspection Techniques for surveys” (Rev.2 June 2016), where the conditions and procedures for the use of remote techniques for inspection of ships are addressed, and a non-exhaustive list of possible techniques that can be used to the purpose is given, including, among others, robotics platforms such as climbers and drones.
A “remote inspection technique” is intended in this context as an essential part of a survey, where the surveyor does not make personal direct sensory experience of the item inspected, nonetheless he obtains equivalent information by means of devices and/or other persons. An example of remote inspection technique already widely used is underwater inspections of hull in lieu of dry-docking, where authorized divers carry out the inspection by means of CCTV and the Class Surveyor is looking into the monitor and talking to the diver, giving him/her instructions.
The objective of a ship inspection is to verify the structural strength and integrity of essential parts of the ship’s hull and its appendages, and/or the reliability and function of the propulsion and steering systems, power generation and those other features and auxiliary systems that have been built into the ship.
It is usually neither possible nor expected that a survey is extended to a scrutiny of the entire structure of the vessel or its machinery: it usually involves a sampling, for which guidelines exist based upon empirical experience and the age of the vessel, which may indicate those parts of the vessel or its machinery that may be subject to corrosion, or are exposed to the highest incidence of stress, or may be likely to exhibit signs of fatigue or damage.
One of the key criteria contained in the IACS Recommendation 42 is that “the methods applied for remote inspection technique are to provide the survey results normally obtained for/by the Surveyor. The results of the surveys by remote inspection techniques when being used towards the crediting of surveys are to be acceptable to the attending Surveyor.” This criterion submits the applicability of ship inspections carried out by remote techniques to an evaluation of equivalence between the results obtained by personal, direct, sensory experience and the corresponding results obtained by other means like sensors on robotics platforms.
The overall concept of the ROBINS project is that equivalence between the outcomes of remote inspections carried out by means of Robotics and Autonomous Systems (RAS) and those obtained by traditional procedures can be achieved only if the following requirements are satisfied:
- Sensing and probing capabilities of RAS are at least as good as the corresponding human personal, direct, sensory experience, or provide equivalent or richer information;
- RAS have the capability to access and adequately explore confined spaces where the inspection is required or desirable, including the capability to detect or devise its own position and orientation, and hazardous, harsh or dirty conditions can be found;
- RAS performance in terms of safety, functionality, dependability and economic viability can be measured, assessed and verified in the required operational scenarios;
- The data collected during inspection activities can be securely acquired, transmitted, archived, maintained and used, with special concern to confidentiality, integrity and availability;
- The data collected during inspection activities, such as photographs, movies or thickness measurement data, are made available to end users by means of software tools capable of providing detailed information with adequate presentation and analysis tools.
The ROBINS project focuses on two types of RAS: aerial drones and crawlers. This choice derives from the assumption that a survey, including those for statutory and class certification, normally consists of an overall visual examination of the items of interest for the survey, and detailed checks of selected parts, e.g. measurement of thickness, on a sampling basis. Aerial drones are taken into consideration essentially for tasks related to visual inspection, while the crawler is used for thickness measurements and other close-up investigations.
For aerial drones, two main types of operational scenarios are considered in ROBINS:
- Wide volumes with a reduced number of obstacles and irregular surfaces, like cargo holds and cargo tanks;
- Very irregular, narrow, obstacle-rich spaces, like ballast tanks, forepeaks, cofferdams, etc.
The two types of operational scenarios are representative of the two environments where costs and risks connected to inspection activities are more significant:
- Wide volumes with significant heights, like bulk carrier’s cargo holds, require costly access means to reach high places; moreover, large unobstructed heights between levels imply severe consequences for the safety of surveyors in case of casualties.
- Narrow spaces, on the other side, pose hazards related to access, mobility, ventilation, cleanliness, toxic atmosphere, and so on.
In ROBINS, the requirements deriving from the two aforementioned operational scenarios are met by two different aerial drones with different features and capabilities:
- A collision-tolerant flying robot, designed for industrial inspection, allowing access to complex, cluttered indoor places;
- An advanced copter with rich equipment of sensors and software technology for highly autonomous unsupervised navigation, able to explore wide spaces according to complex path planning.
The crawler is used mainly in regular spaces, where its capabilities can be exploited at best, achieving good performance and cost effectiveness. The crawler is mainly used for thickness measurement and other probing activities in environments where the occurrence and type of obstacles are such that they can be overcome without the need of excessively time-consuming operations or frequent human intervention.
An estimation of the variety of applications and quantity of ships where the crawler could be used has led to consider bulk carrier cargo holds as the principal environment of operation for the crawler in ROBINS, also considering that similar environments can be found in other ship types (e.g. general cargos or containerships) and that the number of bulk carries is significantly high, approximately one fourth of the total world fleet.
As a common ground to all the RAS platforms used in ROBINS, suitable means for position detection and motion tracking are developed, leveraging several types of sensors and techniques.
The technological gap between the capabilities currently offered by the RAS platforms considered and the required or desirable capabilities is identified by means of an analysis of the needs and expectations of potential end users, including classification societies, ship owners, shipyards and service suppliers, compared to what the RAS platforms can actually do. The ROBINS project leverages the knowledge and experience of several stakeholders with heterogeneous technological background, and harmonizes different technological cultures: marine industry, robotics industry, CAD industry, classification societies.
The main idea of the ROBINS project is that an assessment of compliance of RAS platforms to requirements, regardless to the underlying technology adopted, can be achieved by means of the design, implementation and execution of a set of tests, carried out in a dedicated, specifically designed Testing Facility. To this purpose, test procedures, evaluation criteria, metrics and benchmarks are developed in the framework of the project.
The possibility to test RAS platforms in a controlled and easily accessible environment, where repeatable test protocols can be implemented, considerably increases the development opportunity for new platforms, significantly reducing the costs related to the assessment of their capabilities and to certification activities, if required.
The desired degrees of reliability, coverage and effectiveness of testing protocols implemented in the Testing Facility, that are essential factors in the process of development and assessment of RAS platforms, are achieved by means of field trials campaigns, comparing the results obtained in the Testing Facility with those obtained in equivalent trials on-board.
Possible differences between the two, or other unexpected behaviours of RAS platforms detected during the field trials, are used as a feedback to revise the test protocols and the Testing Facility itself, thus creating an iterative refining process that converges to a status where field trials can be reliably replaced by equivalent tests in the Testing Facility, or significantly reduced.
The possibility to replace, or at least reduce, field trails on-board by means of equivalent test protocols carried out in the Testing Facility, is one of the key concepts in ROBINS, and comes from the consideration that field trials onboard ships are generally very difficult to be arranged, can be quite expensive and usually cannot be executed in controlled and reproducible testing conditions: these are severe limitations for the development of RAS platforms for ship inspection, particularly for SMEs.
CAD and image processing software tools play a key role in the RAS-assisted inspection loop. The software is expected to:
- Devise a 3D numeric model of the confined space subject to inspection by means of image processing algorithms capable of combining together 2D pictures, and/or by means of meshing algorithms capable to devise textured meshes from point clouds and photographs;
- Provide virtual tours of the space subject to inspection. The user should be given the possibility to examine accurately the details of interest by moving in the 3D virtual space and setting the orientation of its viewpoint according to his need, and having always a detailed rendering of the surface observed consistent with the viewpoint;
- Provide the possibility to add hotspots and/or associate additional information to selected parts of the 3D model (augmented virtual reality model);
- Identify critical or suspect areas from the analysis of visual data acquired during the inspection and highlight such areas in order to provide a valuable guidance to the surveyor.
The level of realism in the representation of inspected spaces is to be such as to allow the user feel a satisfactory degree of presence in the virtual space. The quality of 3D models and augmented VR models, and hence the quality of the user’s virtual inspection, is strictly related to the quality of visual data taken by the aerial drone and additional data coming from probes and sensors carried by the drone or by the crawler. Whereas the quality of data does not allow a satisfactory reconstruction of the 3D model, additional data and/or images should be collected.
The ROBINS project also aims at integrating image-processing algorithms specifically developed for the recognition of ship hull’s critical or suspect areas in the software dedicated to virtual tours, thus creating a unified environment for virtual inspection. Dedicated software tools and algorithms for image processing will enhance the possibility to effectively and easily identify critical or suspect areas in inspected spaces.
The iterative approach adopted in the ROBINS project, based on several feedbacks and loops, is expected to significantly improve the quality of final results, giving valuable means to continuously monitor how effectively the solutions adopted meet the requirements.
The final framework of tests, procedures, criteria and metrics can be easily turned into a common, objective, widely accepted regulatory framework for the assessment of RAS for ship inspection, in accordance to the IACS Recommendation 42.
The ROBINS project aims at filling the technology and regulatory gaps that today still represent a barrier to the adoption of Robotics and Autonomous Systems (RAS) in activities related to inspection of ships, starting from understanding end-user’s actual needs and expectations as well as analyzing how existing or near-future technology can meet them.
Among the fields of application of the emerging robotics technology, inspection of ships promises many potential benefits, but it also poses particular challenges and requirements.
Several researches have provided a proof of concept for the application of RAS in the marine industry for the inspection of ships, and some industrial applications are already under development, also following similar experiences in other domains such as Energy and Oil and Gas industry. However, complete solutions ready for a massive adoption of Robotics and Autonomous Systems in this field are not yet available and technology, as well as rules and regulations, need to make one step forward and fill some gaps.
From the technological point of view, many progresses have already been made in the last years in a great variety of fields of application, and many more are expected in the near future. When applied to ship inspection, however, a technological gap is still to be filled to give final users suitable means to consider inspection operated through RAS equivalent to inspection carried out by direct human intervention and experience, at least in a selected set of operational scenarios.
In the ROBINS project, the focus is set mainly on operational scenarios that can be representative of a wide variety of applications. In particular, ROBINS focuses on inspections in bulk carrier holds and ballast tanks. In these environments, inspections are actually carried out by means of erection of scaffolds, portable ladders, cherry pickers, etc. that are time-consuming, expensive, may cause damage to coating, and expose the surveyor using such means of access at risk.
The challenges to be faced in the environments selected in ROBINS can also be met in other ship types, like general cargos, container ships and tankers, even if some differences and limitations are to be taken into account.
For what concerns robotics technology, the project has the following objectives:
- Improve the ability of RAS in sensing and probing;
- Improve capabilities in navigation and localization in confined spaces, access to and mobility within the environment;
- Improve safety and dependability of RAS in hazardous, harsh and dirty environments;
- Provide new tools, or improve existing tools for image and data processing based on data collected by advanced sensors, with special focus on the production of 3D models for virtual tours in augmented reality.
On the normative side, international standards, recognized and accepted by stakeholders and authorities, are still in progress and need to be finalized. Stable standards are a precondition to stimulate the European robotics industry and unleash the economic potential of new markets.
Concerning the regulatory aspects relevant to the use of RAS in ship inspection, the project has the following objectives:
- Provide a framework for the assessment of equivalence between the outcomes of RAS-assisted ship inspections and traditional inspection procedures, based on an evaluation of compliance of RAS platforms with a well-defined and limited set of requirements;
- Define criteria, testing procedures and metrics for the evaluation of RAS performance in terms of: safety, functionality, dependability, security, data quality, economic viability;
- Design, implement and assess a testing environment (Testing Facility) where repeatable tests and measurements can be performed for the evaluation of compliance of RAS with the requirements;
- By means of the Testing Facility, provide to SMEs operating in the robotics industry a low-cost alternative/complement to expensive and difficult on-board field trials, where extensive test campaigns can be carried out aimed at facilitating the developments in technology for ship inspection. Within the ROBINS project, test campaigns in the Testing Facility will be extended also to robotics platforms not participating to the project in other form.
The development of robust technical solutions for RAS-assisted ship inspection, based on the understanding of the challenges being faced by asset owners, is expected to streamline wide scale adoption of RAS technology in marine industry.
The developments in robotics technology and software tools aimed at filling technological gaps, and a comprehensive assessment of RAS platforms capabilities is expected to achieve and prove the equivalence between the outcomes of inspection activities carried out by means of RAS and those obtained directly by humans in selected operational scenarios. This, in turn, is expected to remove, or significantly reduce, existing barriers that still prevent the adoption of robotics technology in this domain. In particular, it is expected to allow RAS platforms to be accepted as means for remote inspections in Class and Statutory surveys as per IACS Recommendation 42.
The technological advances and the results relevant to the assessment of equivalence between RAS-assisted vs. traditional inspection outcomes are expected to encourage and support future modifications of the regulations in force that are a limitation to the adoption of robotics technology in ship inspection, e.g. limitations related to the current definition of “close-up survey” in IMO Resolution A.744(18).
The development of testing protocols to be implemented in a Testing Facility, aimed at reducing the need of onboard testing, is expected to reduce obstacles and costs for the development of new technologies and functionalities, increasing the opportunities to extend the range of application to new operational scenarios and to disclose new potential markets to robotics industry.
The testing protocols and the Testing Facility layouts developed in the ROBINS project can also be the basis for a proposal of international standards, recognized and accepted by stakeholders and authorities. Stable standards are a precondition to stimulate the European robotics industry and unleash the economic potential of new markets.
With an estimated total world fleet of over 36000 ships not older than 10 years (ref. 2016) and several hundreds of certified service suppliers operating in Europe and all over the world, the potential market for robotics technologies for ship inspection is significant.
The economic impact of ROBINS is expected to be manifold for equipment suppliers and robot industry, in particular for the development of a new supply chain and new potential markets, that could be particularly beneficial for SMEs. In Europe SMEs are particularly important in this industry and form a vital part of the robotics landscape in terms of establishing component supply chains, driving innovation.
The ROBINS project is expected to be beneficial also for ICT industry besides the robotics sector, allowing the development of new services and products for data processing and knowledge management specific to ships and marine industry.
The ROBINS project provides a concrete opportunity for SMEs to embrace cutting-edge technology to fill market niches with valuable products and services. This is crucial to expand, maintain competitiveness and create new job opportunities.
As highlighted above, the development of a testing protocol and a Testing Facility will be very beneficial for SMEs reducing the costs necessary for the development, verification and certification of the robotics platforms.