Article based on the report: Technology Evaluation of Augmented and Mixed Reality Systems in Shipbuilding Processes Preliminary Report.

Mari4_YARD is a European R&D project that connects real operational challenges at shipyards with modern digital solutions. TT PSC, with its skills and smart tools, had an opportunity to be directly involved in shipbuilding processes at Brodosplit in Croatia. At the same time, for Brodosplit, as for the end-users who are typically focused on their daily tasks, the project gave the chance to evaluate how new technologies could improve their work. Initiatives like Mari4_YARD bring these two sides together, ensuring that technological advancements address real industry needs.

The project’s main goal was to develop human-centric solutions that can preserve industry-specific knowledge, enhance the possibilities of present technologies, and improve work tasks’ performance.

Brodsplit, as an input provider of Mari4_YARD, tested Augmented Reality (AR) and Mixed Reality solutions in their shipbuilding processes. In this article, we present the highlights and results of testing of a head-mounted AR solution from TT PSC called SkillWorx Assisted Worker.

The system consists of a hands-free, voice-controlled head-mounted device (RealWear Navigator®) securely attached to a safety helmet. Equipped with wireless connectivity, a monocular camera, microphones, and advanced noise cancellation algorithms, the device enables engineers to operate entirely through voice commands, eliminating manual interaction. The device seamlessly integrates with  TT PSC SkillWorx™, leveraging computer vision and remote SLAM to overlay critical information onto the ship structure and shipyard equipment. This article presents a usage of the system in a medium-sized shipyard use case, showcasing how it enhances field engineers’ efficiency while always keeping their hands free.

Hardware & Software

The project tested the functionalities and capabilities of tools such as are compact wearable industrial headsets equipped with a mini display under one eye that provide work-related information such as instructions, orders, checklists, and standard operating procedures depending on the installed software. The device, combined with TT PSC SkillWorx™ allows users to tag objects viewed through the headset, enabling the attachment of text notes, alerts, or images for annotations and guidance.

TT PSC provided SkillWorx Platform, including SkillWorx Assisted Worker and SkillWorx Guided Worker apps, plus spatial tracking  technology, which made real-time tracking and tagging of objects possible during individual work and communication/cooperation with remote users

Tested use-cases

Object tracking, communication with outside users or mentors, real-time transcription/translation, and other functionalities of TT PSC SkillWorx™ were tested for three different use-cases:

  1. Training and educational tasks for operating the SAW (Submerged Arc Welding, SAW) welding tractor.
  2. Quality control for hot-works procedures and recording of pre-pass/inter-pass temperatures during welding.
  3. Commissioning in cooperation with the outfitting crew.
A set of screenshots from the TT PSC SkillWorx platform connected to a RealWear device. The screenshots are displayed in three columns and illustrate three use cases in which the solution was tested. The first one is about conducting inspections and commissioning processes remotely; the second one is about welding, a hot job, and the third one is about conducting training, instruction or onboarding remotely. All three screens are additionally accompanied by a photo of a person working in a shipyard, wearing a safety helmet with a screen attached to it.

Methodology

The methodology primarily involved comparing time, materials, and personnel requirements between conventional shipyard processes and those utilizing the device. Initially, each technology came with claims from its developers, which were later translated into measurable Key Performance Indicators (KPIs) by a third-party company, typically specializing in monitoring services. The KPIs for each technology had been established during earlier project meetings.

Testing

1. Training and educational tasks for operating the SAW optimization

The user received step-by-step instructions. Thanks to AR, specific objects were targeted for each of the steps, for example: “switch the polarity,” “engage clutch”, check voltage.” Furthermore, the voice-controlled actions were so intuitive that they eliminated the need for an additional workshops.

The graphic shows an employee wearing a safety helmet with a monitor attached to it, which allows for remote display of documents and the addition of information to the displayed image, e.g. names of the functionalities of the equipment the employee is looking at. Next to the employee's head, there is a display with an example of a welding machine control panel. The image shows the capabilities of the TT PSC SkillWorx solution in combination with the RealWear device.

Goal: to digitalize all necessary learning material, in this case, the operator manual for the submerged welding tractor.

Devices and applications:

  • RealWear device for hands-free assistance and guidance.
  • TT PSC SkillWorx™ (Assisted Worker) platform for remote cooperation and communication.
  • TT PSC SkillWorx™ AR tracking technology for overlaying contextual information in physical location.

How did it go?

TT PSC team received shortened and transformed into step-by-step phases of working with LT7 tractor to connect onto chosen devices and platforms. After an hour of necessary preparations, the solution was ready to perform.

One of the TT PSC’s specialists was asked to complete the procedure of preparing and starting the tractor. Even though he had no previous experience with Submerged Arc Mechanized Welding, he performed the procedure correctly the first time, and it took him 7 minutes.

The same result was reached when another employee, who also had no experience with SAW, tried to run the tractor. The only difference was the need for an introduction to the tool itself (RealWear) and the TT PSC SkillWorx™ platform, which took 15 minutes.

Such a solution is also great for remote communication between a mentor and an engineer when support is needed from a more experienced employee to perform some activity.

Results

Checked and confirmed claims:

  • Providing contextual information in front of the user.
  • Faster and more immersive.
  • AR tracking works, so it is precisely located.
  • Reduced training material.

Interested in how it looks? Watch demo:

2. Quality control for hot-works procedures

During multi-pass welding, maintaining WPS-compliant temperatures is essential. Traditional methods provide limited accuracy, while RealWear’s infrared camera offers precise, full-field temperature mapping after calibration. It also enables remote monitoring, mentoring, and documentation.

Screenshot from the TT PSC SkillWorx platform, showing the thermal imaging necessary for welding work. This is the view that the shipyard worker sees on the screen of the RealWear device connected to the safety helmet. SkillWorx software enhances the tool with advanced features to facilitate work in the shipyard.

Goal: to make temperature control during multi-pass welding more precise and easier to proceed.

Devices and apps:

RealWear with an infrared camera attached, coupled with TT PSC SkillWorx™Assisted Worker customized to include thermal camera streaming for welding quality control and registering safety issues.

How did it go?

Shipyard „hot works” procedure (may vary by shipyard but generally includes):

  1. Remove all flammable materials within 10m of welding.
  2. Ensure an additional person is present during welding.
  3. Fire Department personnel must confirm no residual heat or burning materials post-welding.

For shipyard „hot works,” the TT PSC SkillWorx™for RealWear streamlines safety checks. A Fire Department officer can verify post-welding safety with infrared imaging, reducing paperwork and improving efficiency.

Results

Checked and confirmed:

  • Real-time temperature data directly in the user’s view.
  • Skipping „hot works” paperwork speeds up the process.
  • Faster information flow once uploaded, ensuring claim validation.

3. Remote commissioning of ship systems

Ship components and systems must meet specific standards, verified by commissioning personnel through various tests, many of which are visual. SkillWorx and RealWear technologies could enable remote or recorded inspections, ensuring authenticity through real-time communication and providing video proof of compliance.

Screenshot from the TT PSC SkillWorx platform showing a view of some installation pipes; this is an illustration of a fragment of text about testing the TT PSC SkillWorx solution for remote commissioning of ship systems.

Goal: to verify if some of the commissioning tasks can be done remotely or just by recordings

Devices and apps

RealWear with TT PSC SkillWorx™Assisted Worker app dedicated to remote commissioning of ship systems.

How did it go?

Testing with Brodosplit Outfitting personnel confirmed its practicality. The main challenge was not technical feasibility but acceptance by external commissioners, influenced by test types, distance, and prior experience. A key limitation is weak connectivity in ship structures, which could be resolved with Wi-Fi routers or improved transmitter placement, benefiting broader Industry 4.0 projects.

Results

Checked and confirmed claims:

  • Live streaming allows engineers and external inspectors to follow the inspection in real time, give voice instructions, and interactively indicate checkpoints.
  • Automatic documentation—video recordings, photos and voice memos are uploaded to the SkillWorx cloud, creating a reliable audit trail.
  • No paper documentation reduces reporting time and speeds up the handover process.

Learn more about innovative manufacturing solutions:

Graphic presenting part of Transition Technologies PSC's offer. The two columns briefly describe the SkillWorx Guided Worker and SkillWorx Assisted Worker solutions.

Read also: Mari4_YARD project results: hand laser welding guide with HoloLens 2